A research project that developed an enduring personalized cognitive assistant
The 75 Years of Innovation series highlights the groundbreaking innovations spanning from SRI’s founding in 1946 to today. Each week, SRI will release an innovation, leading up to its 75th anniversary in November 2021.
A little assistance goes a long way: How the CALO project created Siri and other smart digital assistants
One of the primary goals of computing today is the ability to learn in an unsupervised way. For computers to be effective in working with us and helping in our lives they need to understand how humans behave and to be able to communicate with us. A computer must use artificial intelligence (AI) to learn and use reason to make decisions.
The idea of a computer that uses AI to service human beings in their daily lives paved the way for the development of the ‘digital assistant’. The project that kick-started this was known as CALO, or the Cognitive Assistant that Learns and Organizes.
CALO is a story of collaboration, smart technology, and a new way to interact with computers.
A PAL named CALO
A device that organically interacts with humans has been a goal of humanity since the inception of the computer. Looking back through literature, countless examples of such digital assistants can be found. Many of them are embodied as robots. Some, like “HAL 9000” from 2001: A Space Odyssey, have a somewhat sinister side to them.
The CALO project would bring some of the more positive visions of cognitive digital agents to life.
CALO was an output of the PAL (Personal Assistant that Learns) framework from DARPA, a massive collaboration project led by SRI International. The program involved many institutions, including Carnegie Mellon University, the University of Massachusetts, and the Institute for Human and Machine Cognition. PAL focused on creating a framework for the use of cognitive systems that work alongside humans, learning from experience and applying logic and reason to commands from human operators.
The original PAL program worked closely with military partners. The U.S. Army’s Command Post of the Future (CPOF), continues to innovate using the PAL program to enable voice-based interaction with users to support military planning.
From the PAL program came CALO.
Hello CALO, can you sort out a meeting room for me?
CALO was a collaborative effort led by SRI International, involving more than 300 researchers from 22 premier research institutions over five years, beginning in 2003. One of the key design goals of the CALO team was to create a system that “will handle a broad range of interrelated decision-making tasks that have in the past been resistant to automation”.
In other words, facilitated autonomous learning that improved with use. This is still much of the remit of modern-day digital assistants.
To build a digital assistant with cognitive capabilities, CALO had to bring together an ecosystem of AI and traditional components. This type of ecosystem approach to advance technology is a strength of SRI, as the organization can leverage multiple disciplines. CALO composed several interoperable components, a baseline of which included:
- Ontology management
- Machine learning algorithms used in ‘Concept Learning’ to extend CALO’s ontology
- IRIS, an open source office desktop that interfaces with CALO and provides ontology management
Using the CALO ecosystem, CALO was able to:
- Organize and manage information
- Monitor and manage tasks
- Schedule and organize in time
- Prepare information products
- Acquire and allocate resources
- Observe and mediate interactions
From CALO to Siri
One of the offshoots of the CALO project was the meeting assistant CALO-MA (CALO Meeting Assistant System). CALO-MA was used not only to digitally assist with business meetings, but to also test out natural language and speech processing technologies. The nature of meetings, being multi-party and using domain language, made the system development challenging. The components of CALO-MA included speech recognition software based on the Hidden Markov model (HMM) — used in another of SRI’s inventions, the DECIPHER project.
Other products based on the research carried out by the CALO project included:
- Trapit — a news aggregation service
- Tempo AI — AI-enabled calendar for iOS
- Desti — AI-enabled travel guide app
Perhaps the most famous CALO descendent is the phone-based digital assistant, Siri, now part of Apple iOS, but originally another SRI International development.
The world of the digital assistant is not just about getting a computer to do your bidding — other considerations must be taken into account. During the development of CALO, SRI paid great attention to some of the thornier problems that the digital world has created; namely, data privacy and security. In building CALO, the team had to build a system that people trusted, ensuring privacy and security were by design and default. In doing so, CALO would become the guide for many smart technologies of the future.
Resources
SRI International, CALO Project: http://www.ai.sri.com/project/CALO
Sapiens, Homo Habilis: https://www.sapiens.org/evolution/homo-sapiens-and-tool-making/
Learning by Demonstration to Support Military Planning and Decision Making. — Scientific Figure on ResearchGate. Available from: https://www.researchgate.net/figure/PAL-CPOF-Architecture_fig2_221016446
PAL at SRI: https://pal.sri.com/
Press Release, SRI International is Awarded DARPA Contract to Develop New Cognitive Computing Software, 2003: https://www.businesswire.com/news/home/20030716005586/en/SRI-International-Awarded-DARPA-Contract-Develop-New
Interview with Adam Cheyer, Chief Architect in the CALO project: https://medium.com/swlh/the-story-behind-siri-fbeb109938b0
The CALO Meeting Assistant System: https://www.sri.com/wp-content/uploads/pdf/the_calo_meeting_assistant_system.pdf
The Dish, A Word in Your I/O Port: How Computers Were Made to DECIPHER Natural Human Speech: https://medium.com/dish/75-years-of-innovation-speech-recognition-c7f400c668f2
The Dish, Siri: https://medium.com/dish/75-years-of-innovation-siri-75244a25c741
