Radar Vision Fusion

Mobileye has implemented radar-vision fusion to provide enhanced ACC with Collision Mitigation by Braking functionality on multiple production vehicles since 2007.

In scenarios where a collision is imminent and the driver has not reacted, the Mobileye vehicle detection algorithms, supporting the radar system, provide the vehicle robust validated data which allows the radar-vision fusion system to pre-activate other vehicle systems to derive multiple potential functionalities. In the 2007 Volvo radar-vision fusion system the vehicle can autonomously brake up to 0.3g (30% of full braking force) and perform full emergency braking upon driver confirmation via the brake pedal.
In a Foward Collision Warning condition the system will additionally pre-charge the brakes in the event that driver then performs an emergency braking maneuver thus reducing braking system reaction time. Additionally the system will pre-condition secondary restraint systems (such as airbags or active seatbelts).

This feature was first launched in 2007, with Volvo Car Corporation and Delphi Corporation in the new  Volvo S80 and then from 2008 on the V70, XC70 and XC60.

From 2010 on the Volvo S60 the radar-vision fusion system will perform full autonomous braking on top of all previously mentioned features, as well as full emergency braking for pedestrians, in which situation the Vision-only system takes the lead.

Mobileye Series production for radar-vision fusion systems:
2007
- EyeQ Radar-Vision fusion Volvo S80
2008
- EyeQ Radar-Vision fusion V70, XC70, XC60
2010
- EyeQ2 Radar-Vision fusion Volvo S60

View Volvo S60 Demo Video on YouTube featuring Mobileye's Pedestrian Detection

Mobileye is also working in series development on advanced Vision-only vehicle detection systems to perform pre-conditioning of brakes and other vehicle systems and also Vision-only ACC with autonomous braking to 0.3g.

Vision and Radar have complimentary characteristics that naturally lead to the desire for sensor fusion. Radar has very accurate range and range rate (relative velocity measurement) but poorer angular resolution. The vision system has very good angle and angle rate accuracy and a wide Field of View (FoV). In some cases the vision range is less accurate than that of radar, but ongoing development at Mobileye ensures that it is now comparable to that of radar.

The key advantage of radar-vision fusion systems is that robust verified data is guaranteed and used by the algorithms in order to provide a solid foundation for the resulting applications. This is due to the fact that the data is derived from two separate sensors and fused, matched and then approved.

Controlling a vehicle based on radar-vision fusion requires zero tolerance sensing. Mobileye has been working on radar-vision fusion projects for many years and developed multiple technologies and testing methodologies to ensure that the product meets both the vehicle makers and the end-users requirements.

The above figure shows a typical road scene with the vehicles detected by the vision system (red and cyan squares) and the Radar targets (marked by vertical cyan bars). The right hand side of the figure shows an overhead view with the vision targets marked in solid rectangles and the Radar targets marked with empty rectangles. The Radar FOV is shown by the white ‘V’. The fusion information is marked by the white lines which can be seen linking the radar and vision targets.