Semantically Enabled Automated Assessment in Virtual Environments | SRI International

Toggle Menu

Semantically Enabled Automated Assessment in Virtual Environments

Automated performance assessment and content authoring tools to support training in virtual environments.

Virtual environments (VEs) for online training provide an appealing avenue for acquiring new skills, particularly when live, in-person training incurs significant time, expense, or risk. In homeland security or military training, for example, VEs can provide a learner with realistic, interactive scenarios accessed through immersive 3D worlds without the potential risk of live-action situations.

The value of VEs for training systems would be significantly enhanced by the ability to consistently assess the learner’s training performance. SRI is developing a framework called Semantically Enabled Automated Assessment in Virtual Environments (SAVE) that can observe a learner operating within an instrumented VE, assess his performance, and provide helpful feedback to improve his skills.

Understanding what the learner is doing is the key to enabling automated assessment. SRI’s approach involves a semantic characterization of the VE and the operations performed within it by the user. 3D models used in current-generation VEs lack this information, being limited to geometric meshes that define the basic spatial structure and visual appearance of objects. SAVE draws on ontologies to augment the core visual information with properties about individual objects and relationships between objects.

This ontological grounding bridges the gap between the graphical 3D models used in the VE and the instructional models used to drive learner assessment. The assessment component in SAVE analyzes semantic traces of learner actions within the VE to provide contextually relevant feedback for performance assessment.

In contrast to assessment tools that address “algorithmic” skills, which have a single or small number of acceptable responses, SAVE addresses more open-ended procedural skills that can have a range of acceptable solutions with significant variation among them.

Deploying SAVE in an online learning domain requires the creation of background models and content, including semantic overlays for 3D models and scene graphs (at the VE level) as well as characterizations of solutions to training exercises (at the instructional level). SAVE includes tools to support creators of online education systems in specifying this content, leveraging the visual nature of the VE to provide an intuitive 3D authoring environment.

Principal Investigators: Christian Greuel, Karen Myers

Acknowledgment of Support and Disclaimer: This material is based upon work supported by the United States Government under Contract No. W911QY-14-C-0023. Any opinions, findings and conclusions or recommendations expressed in this material are those of the author(s) and do not necessarily reflect the views of the Government.