Video Scene Classification

SRI has developed a scene segmentation and classification (SSC) process that operates in real time on live video streams. This capability enables the indexing and retrieval of individual shots of specific scenes in broadcast news videos, and permits the filtering or indexing of surveillance or training videos.

Our approach involves grouping a related sequence of individual frames with the same or similar camera views into shots, and grouping the shots into scenes. Currently, the broad classifications produced by the SCC module are “indoor”, “outdoor”, “graphic”, “0 persons”, “1 person”, and “> 1 person”. The primary indicators currently used include color distributions, average intensity, the presence or absence of faces, greenery, sky and horizon, and texture. To perform classification based on this diverse set, we have utilized a variety of techniques, including neural nets, nearest neighbor classifiers, and rule-based decision trees. Our approach is quite flexible and can be trained to detect additional categories.