Multiple-Target Tracking Via Kinematics, Shape, and Appearance-Based Data Association

Abstract

This paper presents a real time system for tracking multiple ground moving targets in aerial video. The state of a target is described by its kinematics as well as shape and appearance features: the kinematics include location and velocity in an earth fixed coordinate system; the shape is described by the parameters of an ellipse; the appearance features consist of color histogram, color correlogram, edge matching and/or orientation correlation information. The target kinematics is represented by a constant velocity model and the shape and appearance features are represented by static models between two observation instances. The motion layers of elliptical shapes containing moving targets in stabilized video sequence are identified. The location and velocity in geospace and the corresponding covariances are computed for each target within a motion layer using the platform metadata. A k-best joint probabilistic data association (JPDA) algorithm updates the target kinematics, while an α-β filter updates the shape and appearance features. Additionally, the JPDA assignment cost matrix is formulated using the kinematics, the appearance features, and the target heading information. The k-best Hungarian algorithm is used to obtain the best assignments. The issues of target life cycle management and target splitting and merging are also addressed in our framework. The system has been tested and evaluated for vehicle tracking in sparse, medium, and dense traffic using aerial EO and IR videos.