Note the vast amount of information the system can provide – free space (green carpet), vehicle and pedestrian detection, traffic sign recognition, lane markings – for the vehicle to understand and negotiate the driving scene.

Mobileye is a global leader in the development of vision technology for Advanced Driver Assistance Systems (ADAS) and autonomous driving.

We have over 1,700 employees continuing our two-decade tradition of developing state-of-the-art technologies in support of automotive safety and autonomous driving solutions.

Mobileye is a Tier 2 automotive supplier working with major Tier 1 suppliers, covering the vast majority of the automotive market (programs with over 25 OEMs). These OEMs choose Mobileye, for its advanced technology, innovation culture, and agility. As a direct result, the robustness and performance of our technology have been battle-tested over millions of driving miles as part of the stringent validation processes of safety-critical automotive products.

From the beginning, Mobileye has developed hardware and software in-house. This has facilitated the strategic advantage of responsive and short development cycles of highly interdependent hardware, software and algorithmic stacks. This interdependence is key to producing high-performance and low power consumption products.

Mobileye’s system-on-chip (SoC) – the EyeQ® family – provides the processing power to support a comprehensive suite of ADAS functions based on a single camera sensor. In its fourth and fifth generations, EyeQ® will further support semi and fully autonomous driving, having the bandwidth/throughput to stream and process the full set of surround cameras, radars and LiDARs.


Advanced Driver Assistance Systems (ADAS) systems range on the spectrum of passive/active.

A passive system alerts the driver of a potentially dangerous situation so that the driver can take action to correct it. For example, Lane Departure Warning (LDW) alerts the driver of unintended/unindicated lane departure; Forward Collision Warning (FCW) indicates that under the current dynamics relative to the vehicle ahead, a collision is imminent. The driver then needs to brake in order to avoid the collision.

In contrast, active safety systems take action. Automatic Emergency Braking (AEB) identifies the imminent collision and brakes without any driver intervention. Other examples of active functions are Adaptive Cruise Control (ACC), Lane Keeping Assist (LKA), Lane Centering (LC), and Traffic Jam Assist (TJA).

ACC automatically adjusts the host vehicle speed from its pre-set value (as in standard cruise control) in case of a slower vehicle in its path. LKA and LC automatically steer the vehicle to stay within the lane boundaries. TJA is a combination of both ACC and LC under traffic jam conditions. It is these automated features which comprise the building blocks of semi/fully autonomous driving.

Mobileye supports a comprehensive suite of ADAS functions – AEB, LDW, FCW, LKA, LC, TJA, Traffic Sign Recognition (TSR), and Intelligent High-beam Control (IHC) – using a single camera mounted on the windshield, processed by a single EyeQ® chip.

In addition to the delivery of these ADAS products through integration with automotive OEMs, Mobileye offers an aftermarket warning-only system that can be retrofitted onto any existing vehicle. The Mobileye aftermarket product offers numerous life-saving warnings in a single bundle, protecting the driver against the dangers of distraction and fatigue. Find more on the aftermarket product here.

Computer Vision

From the outset, Mobileye’s philosophy has been that if a human can drive a car based on vision alone – so can a computer. Meaning, cameras are critical to allow an automated system to reach human-level perception/actuation: there is an abundant amount of information (explicit and implicit) that only camera sensors with full 360 degree coverage can extract, making it the backbone of any automotive sensing suite.

It is this early recognition – nearly two decades ago – of the camera sensor superiority and investment in its development, that led Mobileye to become a global leader in computer vision for automotive.

Mobileye’s approach to the development of camera capabilities has always been to first produce optimal, self-contained camera-only products, demonstrated and validated to serve all functional needs. As a showcase, our demonstration vehicle drives autonomously from Jerusalem to Tel Aviv and back relying on camera sensors alone, while series-production autonomous vehicles fuse-in additional sensors for delivering a robust, redundant solution based on multiple modalities (mainly radar and LiDAR).

From ADAS to Autonomous

The road from ADAS to full autonomy depends on mastering three technological pillars:

  • Sensing: robust and comprehensive human-level perception of the vehicle’s environment, and all actionable cues within it.
  • Mapping: as a means of path awareness and foresight, providing redundancy to the camera’s real-time path sensing.
  • Driving Policy: the decision-making layer which, given the Environmental Model – assesses threats, plans maneuvers, and negotiates the multi-agent game of traffic.

Only the combination of these three pillars will make fully autonomous driving a reality.

Note how the system detects all the necessary traffic light information in order to successfully navigate the intersection.

The Sensing Challenge

Perception of a comprehensive Environmental Model breaks down into four main challenges:

  • Freespace: determining the drivable area and its delimiters
  • Driving Paths: the geometry of the routes within the drivable area
  • Moving Objects: all road users within the drivable area or path
  • Scene Semantics: the vast vocabulary of visual cues (explicit and implicit) such as traffic lights and their color, traffic signs, turn indicators, pedestrian gaze direction, on-road markings, etc.

The Mapping Challenge

The need for a map to enable fully autonomous driving stems from the fact that functional safety standards require back-up sensors – “redundancy” – for all elements of the chain – from sensing to actuation. Within sensing, this applies to all four elements mentioned above.

While other sensors such as radar and LiDAR may provide redundancy for object detection – the camera is the only real-time sensor for driving path geometry and other static scene semantics (such as traffic signs, on-road markings, etc.). Therefore, for path sensing and foresight purposes, only a highly accurate map can serve as the source of redundancy.

In order for the map to be a reliable source of redundancy, it must be updated with an ultra-high refresh rate to secure its low Time to Reflect Reality (TTRR) qualities.

To address this challenge, Mobileye is paving the way for harnessing the power of the crowd: exploiting the proliferation of camera-based ADAS systems to build and maintain in near-real-time an accurate map of the environment.

Mobileye’s Road Experience Management (REMTM) is an end-to-end mapping and localization engine for full autonomy. The solution is comprised of three layers: harvesting agents (any camera-equipped vehicle), map aggregating server (cloud), and map-consuming agents (autonomous vehicle).

The harvesting agents collect and transmit data about the driving path’s geometry and stationary landmarks around it. Mobileye’s real-time geometrical and semantic analysis, implemented in the harvesting agent, allows it to compress the map-relevant information – facilitating very small communication bandwidth.

The relevant data is packed into small capsules called Road Segment Data (RSD) and sent to the cloud. The cloud server aggregates and reconciles the continuous stream of RSDs – a process resulting in a highly accurate and low TTRR map, called “Roadbook.”

The last link in the mapping chain is localization: in order for any map to be used by an autonomous vehicle, the vehicle must be able to localize itself within it. Mobileye software running within the map-consuming agent (the autonomous vehicle) automatically localizes the vehicle within the Roadbook by real-time detection of all landmarks stored in it.

Further, REMTM provides the technical and commercial conduit for cross-industry information sharing. REMTM is designed to allow different OEMs to take part in the construction of this AD-critical asset (Roadbook) while receiving adequate and proportionate compensation for their RSD contributions.

Driving Policy

Where sensing detects the present, driving policy plans for the future. Human drivers plan ahead by negotiating with other road users mainly using motion cues – the “desires” of giving-way and taking-way are communicated to other vehicles and pedestrians through steering, braking and acceleration. These “negotiations” take place all the time and are fairly complicated – which is one of the main reasons human drivers take many driving lessons and need an extended period of training until we master the art of driving. Moreover, the “norms” of negotiation vary from region to region as the code of driving in Massachusetts, for example, is quite different from that of California, even though the rules are identical.

The challenge behind making a robotic system control a car is that for the foreseeable future the “other” road users are likely to be human-driven, therefore in order not to obstruct traffic, the robotic car should display human negotiation skills but at the same time guarantee functional safety. In other words, we would like the robotic car to drive safely, yet conform to the driving norms of the region. Mobileye believes that the driving environment is too complex for hand-crafted rule-based decision making. Instead we adopt the use of machine learning to “learn” the decision making process through exposure to data.

Mobileye’s approach to this challenge is to employ what is called reinforcement learning algorithms trained through deep networks. This requires training the vehicle system through increasingly complex simulations by rewarding good behavior and punishing bad behavior. Our proprietary reinforcement learning algorithms add human-like driving skills to the vehicle system, in addition to the super-human sight and reaction times that our sensing and computing platforms provide. It also allows the system to negotiate with other human-driven vehicles in complex situations. Knowing how to do this well is one of the most critical enablers for safe autonomous driving.

For more details on the challenges of using reinforcement learning for driving policy and Mobileye’s approach to the problem, please see: S. Shalev-Shwartz, S. Shammah and A. Shashua. Safe, Multi-Agent, Reinforcement Learning for Autonomous Driving. NIPS Workshop on Learning, Inference and Control of Multi-Agent Systems: Dec., 2016.