The 75 Years of Innovation series highlights the groundbreaking innovations spanning from SRI’s founding in 1946 to today. Each week, SRI will release an innovation, leading up to its 75th anniversary in November 2021.
The first mobile, precision, non-jitter, augmented reality binoculars.
Realer than real: augmented reality binoculars
Science fiction has brought some fantastic ideas to the world, humans being captivated by technologies that seem impossible, like Star Trek’s holodeck. But some of these technologies, which may seem outlandish and only achievable in the distant future, are becoming part of our everyday lives thanks to innovators in science and technology.
One such technology is augmented reality (AR). AR is not new — in fact, it first appeared in the 1960s. Its application and commercial use, however, have only recently been more fully explored.
In the early 2010s, SRI International’s Computer Vision Technologies Laboratory began exploring the potential applications of AR binoculars. In 2013, this team shared their research at an Institute for Electrical and Electronic Engineers (IEEE) conference on mixed and augmented reality and in the IEEE publication Transactions on Visualization and Computer Graphics. The team received best paper award for this work.
SRI International augmenting hand-held binoculars
SRI has a long history of developing cutting-edge technology. Even before the AR binoculars, SRI was a veteran of augmented reality, having developed the a prototype system for showing AR-based advertisements on live TV for baseball games, which led to “first-down yellow line” used in TV football broadcasts. The SRI team recognized that there was a compelling reason to add AR capability to binoculars, that such a device would be “handy to improve the experience of the large-scale outdoor augmented reality scenarios, as distant augmentations appear too small for the unaided eye.” The team also realized that AR binoculars would need to be much easier to use than traditional AR headsets and worked to ensure the binoculars would not require any bulky hardware or special glasses.
The original use case for AR binoculars was to meet the needs of military training and disaster relief. However, the team also foresaw applications in consumer situations such as bird watching and sports. One of the valuable features of AR is that information tags and labels can be inserted into the user’s view, and this information can also be shared with others. However, adding augmented reality to hand-held binoculars was not an easy task. Developing hand-held portable AR binoculars required SRI to deal with challenges like continuous hand and body movement to create a “precise jitter-free and drift-free insertion” of AR images into the eye-view. The binoculars also had to be compact and light enough that they could be used normally.
Precision vision with AR binoculars
The uniqueness in the design of SRI’s AR binoculars was to ensure that the augmentation — in the form of inserted objects — appeared stable in the display. Any jitter or drift as the binoculars were used normally would detract from the viewer’s experience and reduce the effectiveness of the augmentation. The AR binoculars were built from several components:
- Wide Field of View (FoV) Lens: used for context and to recover the 3D location and orientation of the binoculars
- Narrow FoV Lens: used for augmentation and improving tracking precision
- Inertial Measurement Unit (IMU): a microelectromechanical (MEM) system that tracks angular rate and acceleration
- Global Positioning System (GPS): a satellite-based location and navigation system for location estimation
- Magnetometer: measures the Earth’s magnetic field and used for measurements related to orientation estimation
- Barometric pressure sensor: measures atmospheric pressure and used for measurements related to altitude estimation
Two cameras were able to be seated in the lens bay of the binoculars. The design team created a navigation algorithm that combined data from the cameras and sensors. The team used a visual landmark matching and panorama mechanism to allow for rapid correction of orientation and minimize drift. The IMU and GPS provided the jitter-free, real-time pose estimation to deliver precise augmentation.
These AR binoculars could also share data. For example, the user could tag and label objects of interest in the AR view and share this information with other AR devices.
The team demonstrated the usefulness of the AR binoculars during multiple live simulated military-forward observer training exercises. The demonstration combined fixed rotary-wing aircraft, ground vehicles and weapon effects with real-world scenes.
The reality of augmented reality
AR can now be found in many places, from furniture store apps that show catalog items within a room to medical applications. SRI continues to work on military, industrial, educational and enterprise applications for AR, such as military defense tactical and training, AR for maintenance and repair, aides for construction, dyslexia training and more. SRI continues to push forward the technology needed to make AR a reality.
Resources
T. Oskiper, M. Sizintsev, V. Branzoi, S. Samarasekera and R. Kumar, “Augmented Reality Binoculars,” in IEEE Transactions on Visualization and Computer Graphics, vol. 21, no. 5, pp. 611–623, 1 May 2015, doi: 10.1109/TVCG.2015.2408612.
SRI 75 Years of Innovation: Virtual Advertising Insertion: https://sri.com/story/75-years-of-innovation-virtual-advertising-insertion-augmented-reality
