Pedestrian Detection

View Pedestrian Protection Video

Pedestrians are the most vulnerable road users, whilst also being the most difficult to observe both in day and in night conditions. Pedestrians in the vehicle path or walking into the vehicle path are in danger of being hit causing severe injury both to the pedestrian and potentially also to the vehicle occupants.

Mobileye's pedestrian detection technology is based on EyeQ2 systems. Mobileye's unique approach to Pedestrian detection lies in the use of monocular cameras only, using advanced pattern recognition and classifiers with image processing and optic flow analysis. Both static and moving pedestrians can be detected to a range of around 30m using VGA resolution imagers. As higher resolution imagers become available range will scale with imager resolution, making detection ranges of up to 60m feasible.

View pedestrian protection development for Volvo City Safety, a new system developed by Volvo aiming to help avoid collisions with cars and pedestrians.

Mobileye's first start of production for Pedestrian detection systems is in 2009 on a range of Industrial Powered Vehicles. In this application 8 EyeQ2 based monocular cameras will provide a 360deg all-round Pedestrian Detection system to a range of 15m and will warn the vehicle operator via Audio/Visual warnings of pedestrian in the vehicles path. In late 2009 Mobileye will migrate Pedestrain Detection warning functions to the next generation of AWS consumer product line systems.
In mid 2010 Mobileye will launch a world's first application of full emergency braking for Collision Mitigation for pedestrians where Vision is the key technology for detecting pedestrians. This system is part of the next generation Volvo radar-vision fusion system which also provides Lane Departure Warning and Vehicle Detection with radar-vision fusion for an enhanced Collision mitigation by Braking System on the next Volvo S60 vehicle.

View Volvo S60 Demo Video on YouTube featuring Mobileye's Pedestrian Detection

Mobileye’s current Pedestrian Detection technology provides a solution for pedestrian detection in line with current OEM requirements for the lead production program. The sensor resolution and its FoV translate this requirement into the system detection envelope and hence sensor resolution improvements will increase detection ranges and effective FoV. Pedestrians that move laterally in the scene are of prime interest, as their motion trajectory can intersect with that of the vehicle. Their detection performance is virtually instantaneous due to the usage of inward motion cues. 

In the autonomous emergency braking application detected pedestrians are 'held' to the point of unavoidable impact. The new acquisition of targets is limited to fully visible pedestrians, and is currently extended to the ultra close range, where parts of the body are beyond the image boundaries. This is of particular importance for rear looking camera applications, and for Stop and Go forward applications.

There are four major challenges with pedestrian detection that required special technical development are as follows:
• Figure size:
  Far pedestrians appear very small in the image. For example, with VGA resolution and 36deg vertical FOV, the
  figure of a 1m height child at 30 meters is only 25 pixels long. The lateral figure dimension is even smaller.
• Fast dynamics:
  The detection latency must be small, and decisions must be obtained within a few frames.
• Heavy clutter:
  Pedestrian detection is typically taking place at urban scenes with a lot of background texture.
• Articulation: 
  Pedestrians are non-rigid objects, spanning high variability in appearance and cause tracking difficulties.

Although the first problem of image size is somewhat technical, Mobileye ascertained that real production programs always tend to push the detection requirements toward the sensor limit. Much effort was invested, therefore, to enable correct classification of very small image figures. In particular, part based classification approaches were abandoned, and a holistic full body approach was found to be suitable.

The fast dynamics and the heavy clutter challenges both require high classification precision. Intensive development effort led to dedicated pattern classifiers, which to our knowledge are the best among all published works. Local classification features are extracted from image intensities and derivatives, computed on a single pixel level or from small image patches. Global image features, which reflect scene context, are integrated into the classification process. For example, long image lines that pass through the region of interest provide a negative detection cue.

Early detection of people that run into the drive-path (“crossing pedestrians”) is associated with the fast dynamics challenge. Here Mobileye uses optical flow analysis, in order to distinguish the laterally moving objects from their background. Background optical flow, as seen by a forward moving camera, is always expanding and directed outward from the focus of expansion toward the image boundaries. Hence detecting inward optical flow is strong evidence to the existence of a moving object, which might be a crossing pedestrian.

Optical flow is used as a secondary detection cue for close stationary objects, where it is possible to distinguish the motion pattern of a solid object from that of the road plane. In this case the motion cue is not as strong as for crossing pedestrians, and hence for stationary object detection it is associated with a delay, and acts as a secondary mechanism.