AURORA: Content-Guided Search of Diverse Videos

Finding events of interest in large video collections is a current challenge to the multimedia research community. To speed up the accurate retrieval of relevant events, the Automated Low-Level Analysis and Description of Diverse Intelligence Video (ALADDIN) program of the Intelligence Advanced Research Projects Activity (IARPA) aims to develop revolutionary technologies for the efficient and automated analysis of massive amounts of open-source video. However, because these videos vary dramatically in quality and composition, similar activities of interest may appear very different, while very different events may share many common elements. 

To address this problem, SRI is developing AURORA—a prototype system consisting of four subsystems: 

  • Content Description Representation (CDR) Generator: The goal of the CDR Generator is to comprehensively extract and represent video content to enable the create-once-query-many-times mode for both known and ad hoc event queries 
  • CDR database: The indexed CDR database stores the results of the CDR Generator 
  • Event Agent Generator (EAG): The objective of the EAG system is to process an event query in the form of a collection of videos and supporting text, search the CDR database, and recount occurrences of the specified event in the retrieved videos 
  • Knowledge Base: The Knowledge Base is used by both the CDR Generator and the EAG 

The introduction of this novel technology is expected to find events of interest in very large video collections at a high throughput. 

Supported by the Intelligence Research Projects Activity (IARPA) via Department of Interior national Business Center contract number D11PC20066.  The U.S. Government is authorized to reproduce and distribute reprints for Governmental purposes notwithstanding any copyright annotation thereon. 

Disclaimer: The views and conclusions contained herein are those of the authors and should not be interpreted as necessarily representing the official policies or endorsements, either expressed or implied, of IARPA, DoI/NBC, or the U.S. Government. 

Read more from SRI

  • surgeons around a surgical robot

    The SRI research behind today’s surgical robotics

    Intuitive’s da Vinci 5 system represents a major leap in robotic-assisted medicine. It all started at SRI, which continues to advance teleoperation technologies.

  • a collage of digital graphs

    A banner year for quantum

    SRI-managed QED-C’s annual report on quantum trends captures an industry accelerating rapidly from technical promise toward major global impact.

  • ICE Cube containing SRI’s aerogel experiment, photographed prior to launch. Source: Aerospace Applications North America

    An SRI carbon capture experiment launches into space

    By synthesizing carbon-absorbing aerogels in microgravity, SRI research will give us a rare glimpse into how these materials could be radically improved.