Infrastructure Free 6 DOF Location and Pose Estimation for Mixed Reality Systems

Abstract

Mixed reality training systems using Head Mounted Displays (HMDs) require very high precision knowledge of the 3D location and 3D orientation of the user’s head. This is required by the system to know where to insert the synthetic actors and objects in the HMD. The inserted objects must appear stable and not jitter or drift. Moreover latency of less than 5 milliseconds for pose estimation is required for lag-free see-through HMD operation. We describe how to achieve this performance using a multi-camera based visual navigation system mounted on the HMD. A Kalman filter is used to integrate high rate estimates from an IMU with a visual odometry system and to predict head motion. Landmark matching and GPS when available are used to correct any drifts.