Pedestrian Detection

Pedestrians are the most vulnerable road users, whilst also being the most difficult to observe both in day and in night conditions. Pedestrians in the vehicle path or walking into the vehicle path are in danger of being hit causing severe injury both to the pedestrian and potentially also to the vehicle occupants.

Mobileye’s pedestrian detection technology runs on EyeQ2 based systems and is currently the only mono-camera automotive pedestrian detection system in production globally. Mobileye’s unique approach to pedestrian detection lies in the use of monocular cameras only, using advanced pattern recognition and classifiers with image processing and optic flow analysis. Both static and moving pedestrians can be detected to a range of around 30m using VGA resolution imagers. As higher resolution imagers become available range will scale with imager resolution, making detection ranges of up to 60m feasible.

Mobileye’s first production for Pedestrian detection systems was in 2009 on a range of industrial powered vehicles where 8 EyeQ2 based monocular cameras  provide a 360deg all-round Pedestrian Detection system to a range of 15m and will warn the vehicle operator via Audio/Visual warnings of pedestrian in the vehicles path.

In late 2009 Mobileye added the Pedestrian detection warning functions to the next generation of consumer product line systems – called C2-270.

In mid 2010 Mobileye launched a world’s first application of full emergency braking for Collision Mitigation with pedestrians on the Volvo S60 and V60 vehicles. In this system Vision is the key technology  and lead sensor for pedestrians detection.

In the autonomous emergency braking application detected pedestrians are ‘held’ to the point of unavoidable impact. The new acquisition of targets is currently limited to fully visible pedestrians, but is currently being extended to detect pedestrians at  ultra close range, where parts of the body are beyond the image boundaries. This is of particular importance for rear looking camera applications, and for Stop and Go forward applications.

There are four major challenges with pedestrian detection that required special technical development are as follows:

  • Figure size:
    Far pedestrians appear very small in the image. For example, with VGA resolution and 36deg vertical FOV, the figure of a 1m height child at 30 meters is only 25 pixels long. The lateral figure dimension is even smaller.
  • Fast dynamics:
    The detection latency must be small, and decisions must be obtained within a few frames.
  • Heavy clutter:
    Pedestrian detection is typically taking place at urban scenes with a lot of background texture.
  • Articulation:
    Pedestrians are non-rigid objects, spanning high variability in appearance and cause tracking difficulties.

Although the first problem of image size is somewhat technical, Mobileye ascertained that real production programs always tend to push the detection requirements toward the sensor limit. Much effort was invested, therefore, to enable correct classification of very small image figures. In particular, part based classification approaches were abandoned, and a holistic full body approach was found to be suitable.

The fast dynamics and the heavy clutter challenges both require high classification precision. Intensive development effort led to dedicated pattern classifiers. Local classification features are extracted from image intensities and derivatives, computed on a single pixel level or from small image patches. Global image features, which reflect scene context, are integrated into the classification process. For example, long image lines that pass through the region of interest provide a negative detection cue.

Early detection of people that run into the drive-path (“crossing pedestrians”) is associated with the fast dynamics challenge. Here Mobileye uses optical flow analysis, in order to distinguish the laterally moving objects from their background. Background optical flow, as seen by a forward moving camera, is always expanding and directed outward from the focus of expansion toward the image boundaries. Hence detecting inward optical flow is strong evidence to the existence of a moving object, which might be a crossing pedestrian.

Optical flow is used as a secondary detection cue for close stationary objects, where it is possible to distinguish the motion pattern of a solid object from that of the road plane. In this case the motion cue is not as strong as for crossing pedestrians, and hence for stationary object detection it is associated with a delay, and acts as a secondary mechanism.

Today’s system is operation in Daytime only (based on Mobileye’s core day-night decision mechanism), but development is underway to extend this to dusk environments, and moving forward and based on NIR filtering on the imager, Mobileye is developing nigh time pedestrian detection for 2014 SOP.

Pedestrian Collision Warning


Send to a Friend:

Our Vision. Your Safety.