• Skip to primary navigation
  • Skip to main content
SRI logo
  • About
    • Press room
    • Our history
  • Expertise
    • Advanced imaging systems
    • Artificial intelligence
    • Biomedical R&D services
    • Biomedical sciences
    • Computer vision
    • Cyber & formal methods
    • Education and learning
    • Innovation strategy and policy
    • National security
    • Ocean & space
    • Quantum
    • Robotics, sensors & devices
    • Speech & natural language
    • Video test & measurement
  • Ventures
  • NSIC
  • Careers
  • Contact
  • 日本支社
Search
Close
Cyber & formal methods publications August 1, 1997

Multimodal Interfaces for Internet

Citation

Copy to clipboard


Julia, L., & Cheyer, A. Multimodal Interfaces for Internet.

Abstract

The World Wide Web frightened us: during the first two years of its popularity, we felt that human-computer interaction had been set back 30 years. Fortunately, Java expanded the possibilities for user interface design, providing a way to run complex programs over the net. In
this paper, we present a Java-enabled application with a multimodal (pen and voice) interface over the web.

Our implementation approach was to add Java to the set of languages accepted by the Open Agent Architecture (OAA), a framework for rapidly prototyping complex applications, and particularly suited to those with multimodal interfaces [1]. Given the OAA’s distributed nature, an OAA-based application can be run from a lightweight computer by downloading only the small user interface component, while the core of the application is implemented on a server composed of larger agents (e.g. speech recognition (SR), natural language, database) which cooperate and compete in parallel.

Despite the current lack of APIs for media input in Java, we chose voice entry as the primary input modality for the user [2], first by using a telephone to access a remote Nuance Speech Recognition server [3], and secondly by designing our own Java API for Speech Recognition using external native methods on the client.

ATIS [4], our first prototype application using SR over the telephone and the Java implementation of the OAA, has been publicly available on our web site for more than a year. Given the success of this first experiment (more than 4000 users), our next task was to bring multimodal concepts developed for several map-based applications [5] to the Web. In order to achieve our multimodal objectives, it was necessary to adapt the pen modalities (gestures and handwriting) to an Internet and multiplatform context, i.e. the pen had to be interchangeable with any pointing device. For portability, the gesture recognition algorithms [6] were recoded directly in Java. A trade-off was made for handwriting, integrating a Java-enabled version of JOT, a character by character recognizer developed by CIC [7].

↓ Download

Share this
Career call to action image

Work with us

Search jobs

How can we help?

Once you hit send…

We’ll match your inquiry to the person who can best help you.

Expect a response within 48 hours.

Our work

Case studies

Publications

Timeline of innovation

Areas of expertise

Institute

Leadership

Press room

Media inquiries

Compliance

Careers

Job listings

Contact

SRI Ventures

Our locations

Headquarters

333 Ravenswood Ave
Menlo Park, CA 94025 USA

+1 (650) 859-2000

Subscribe to our newsletter


日本支社
SRI International
  • Contact us
  • Privacy Policy
  • Cookies
  • DMCA
  • Copyright © 2023 SRI International
Manage Cookie Consent
To provide the best experiences, we use technologies like cookies to store and/or access device information. Consenting to these technologies will allow us to process data such as browsing behavior or unique IDs on this site. Not consenting or withdrawing consent, may adversely affect certain features and functions.
Functional Always active
The technical storage or access is strictly necessary for the legitimate purpose of enabling the use of a specific service explicitly requested by the subscriber or user, or for the sole purpose of carrying out the transmission of a communication over an electronic communications network.
Preferences
The technical storage or access is necessary for the legitimate purpose of storing preferences that are not requested by the subscriber or user.
Statistics
The technical storage or access that is used exclusively for statistical purposes. The technical storage or access that is used exclusively for anonymous statistical purposes. Without a subpoena, voluntary compliance on the part of your Internet Service Provider, or additional records from a third party, information stored or retrieved for this purpose alone cannot usually be used to identify you.
Marketing
The technical storage or access is required to create user profiles to send advertising, or to track the user on a website or across several websites for similar marketing purposes.
Manage options Manage services Manage {vendor_count} vendors Read more about these purposes
View preferences
{title} {title} {title}