The map data is projected onto the road – note how it is aligned with the lanes in the scene. On the right side we see the lanes mapped onto Google Earth.

The Mapping Challenge

The need for a map to enable fully autonomous driving stems from the fact that functional safety standards require back-up sensors – “redundancy” – for all elements of the chain – from sensing to actuation. Within sensing, this applies to all four elements requiring sensing: freespace, driving paths, moving objects, and scene semantics.

While other sensors such as radar and LiDAR may provide redundancy for object detection – the camera is the only real-time sensor for driving path geometry and other static scene semantics (such as traffic signs, on-road markings, etc.). Therefore, for path sensing and foresight purposes, only a highly accurate map can serve as the source of redundancy.

In order for the map to be a reliable source of redundancy, it must be updated with an ultra-high refresh rate to secure its low Time to Reflect Reality (TTRR) qualities.

To address this challenge, Mobileye is setting the path for harnessing the power of the crowd: exploiting the proliferation of camera-based ADAS systems to build and maintain in near-real-time an accurate map of the environment.

Mobileye’s Road Experience Management (REMTM) is an end-to-end mapping and localization engine for full autonomy. The solution is comprised of three layers: harvesting agents (any camera-equipped vehicle), map aggregating server (cloud), and map-consuming agents (autonomous vehicle).

The harvesting agents collect and transmit data about the driving path’s geometry and stationary landmarks around it. Mobileye’s real-time geometrical and semantic analysis, implemented in the harvesting agent, allows it to compress the map-relevant information – facilitating very small communication bandwidth (less than 10KB/km on average).

The relevant data is packed into small capsules called Road Segment Data (RSD) and sent to the cloud. The cloud server aggregates and reconciles the continuous stream of RSDs – a process resulting in a highly accurate and low TTRR map, called “Roadbook”.

The last link in the mapping chain is localization: in order for any map to be used by an autonomous vehicle, the vehicle must be able to localize itself within it. Mobileye software running within the map-consuming agent (the autonomous vehicle) automatically localizes the vehicle within the Roadbook by real-time detection of all landmarks stored in it.

Further, REMTM provides the technical and commercial conduit for cross-industry information sharing. REMTM is designed to allow different OEMs to take part in the construction of this AD-critical asset (Roadbook) while receiving adequate and proportionate compensation for their RSD contributions.