Multi-Modal Pedestrian Detection on the Move

Abstract

This paper presents an on-the-move pedestrian detection system that utilizes multiple sensor modalities to improve detection rates at deployable computational loads. The system was developed for a vehicle moving up to 40 kph that can detect moving pedestrians up to a distance of 50m, with support for both day and night operations. In the day, 3D pointclouds obtained from an 8-layer LIDAR sensor are processed to produce a labeling of the scene distinguishing ground, large structures, and potential pedestrians to produce reliable detections in the short range (up to 30m), while a stereo-based detection and classification system is used for ranges between 30-50m+. We describe the algorithms in detail and show that the combined system allows for reliable detection at faster frame-rates than when using each sensor or component individually. A second method for fusing two IR cameras with the LIDAR sensor is proposed for night operations, where LIDAR is used to produce multi-scale masks that define the search space for a HOG-based pedestrian classifier.