SRI Blog

Deep Knowledge and the Rise of Virtual Specialists

woman using a credit card with a mobile phoneVirtual personal assistants like Siri are becoming widespread. They understand user intents expressed in natural language and respond by connecting users with services that hopefully fulfill those intents. But for many consumer needs such as banking, retail, and healthcare, just getting to the service isn’t enough. Consumers have difficulty getting things done in the service, especially when they are on a mobile device.

At SRI International, we are exploring a new generation of assistant, a virtual specialist, that helps consumers perform tasks that require deep knowledge of the service and of the consumer’s personal relationship with that service. For example, Kasisto, a recent SRI spin-off, has deep knowledge of the user’s banking world—the bank’s terminology and procedures, and the details of the user’s account. Kasisto is a virtual bank specialist, helping the consumer perform tasks that are currently difficult (or impossible) on a mobile device. In the mobile world, as in the real world, these tasks are best performed through conversation—but not just any conversation.

Siri can handle some conversation, but it is not designed to have deep knowledge of specific services like banking. If you ask Siri to find ATM’s near you, it will do a location-based search and suggest some ATM’s. But Siri doesn’t know much about banking. If you ask “how much do I have in my checking account," Siri directs you to the Wikipedia definition of a transactional account.

On the other hand, Kasisto knows a lot about banking, and is able to help users perform real banking tasks through conversations such as this one:

Consumer: I would like to make a payment on my credit card from my checking account.

Kasisto: The minimum payment amount on your credit card account (Gold Plus Credit Card *2234) is  $159.73.  Is this the amount you want to transfer?

Consumer: No, I would like to pay $2,000.

Kasisto: I am sorry you have only $1,544.78 in your checking account (Checking *9072). That's not enough to complete this transfer. Would you be interested in transferring $2,000 from savings account (Savings *9076) to your credit card (Gold Plus Credit Card *2234)?

Consumer: No, please pay $1,000 from checking today.

Kasisto: I can do that. To confirm, you would like to transfer $1,000 from checking account (Checking *9072) to credit card (Gold Plus Credit Card *2234). Is that correct?

Consumer: Yes, thank you.

This interaction, and almost all interactions within banking services (and retail, healthcare, etc. services) have an element of discovery: users refine their intents and learn more about the service through conversation. The virtual specialist must combine conversation with reasoning based on deep knowledge in order to fulfill the user’s intent.  For example, Kasisto needs to know that credit card bills have minimum payments and that when the consumer says “pay $2000," he or she is referring to the credit card bill (because bills are things that are paid and the only bill in the conversation is the credit card bill). Finally, in a real conversation both parties are proactive.  As a good virtual specialist, Kasisto proactively warns the consumer about insufficient funds and alternative payment options.

Proactivity is in fact a key characteristic of a specialist, indeed any assistant. We expect assistants to anticipate our needs, help us discover relevant new things, and work in the background on our behalf. Automated proactivity is a tricky business: the assistant needs to understand the user’s context and intent, and make only useful suggestions. Much of human behavior is routine, creating predictable context and intent (most people leave the same place at about the same time to go to work).  But we don’t need much proactive help with this part of our life. We want to be told about things that will disrupt our routine (unusual traffic situation, first meeting canceled, etc.), but the proactive help required in these situations is itself pretty routine and is becoming commoditized. Some assistants also proactively tell us “interesting” things. These can be useful (e.g., your boss’s birthday is tomorrow), but are often distracting, and are too often simply advertisements.

It is during non-routine behavior—when we are looking for something new, doing something unfamiliar, trying to fix a problem—that we most need proactive assistance. This is the realm of the specialist. Deep knowledge enables specialists to give much more targeted and useful proactive assistance. As we have seen, detailed knowledge of the consumer’s account and bank rules enables Kasisto to provide useful proactive guidance. A virtual healthcare specialist could learn from the content and style of a user’s conversation that a proactive suggestion about diet or medication is warranted. A virtual retail specialist can tell a consumer’s shopping expertise from their questions and vocabulary, enabling it to make proactive suggestions that the user will understand and appreciate.

This is only the beginning: much of what we do in the real world involves conversational interaction with specialists ranging from waiters to bank clerks to medical professionals. The rapid worldwide adoption of mobile devices requires us to move beyond virtual personal assistants to create the virtual specialists that will enable consumers to really get things done through their mobile devices.

Leave a Comment

Please join the discussion and comment using an authentic name. Spam, promotional, and derogatory comments will not be approved. Comments will appear on the live site within 24 hours.

The content of this field is kept private and will not be shown publicly.
  • Web page addresses and e-mail addresses turn into links automatically.
  • Allowed HTML tags: <a> <em> <strong> <cite> <code> <ul> <ol> <li> <dl> <dt> <dd>
  • Lines and paragraphs break automatically.
CAPTCHA
This question tests whether you are a human visitor so we can prevent automated spam submissions.
Image CAPTCHA
Enter the characters shown in the image.

2 Comments

Subscribe to comments

Mark Rupersburg

I would be interested in how this technology could assist warfighters. At GDLS, we make combat vehicles. It would be great to have an assistant to learn from the user and aid in decision making.

Jeff Nelson

Great ideas and productivity always coming out of SRI. Thank you, William. I love the possibilities in this field. Your example of banking is good and one could be created for many other industries and use-cases.

I have been thinking about scale and modularity in this growing world of smart agents with domain knowledge. How much sense does it make to create and interface with x^n front-end agents? Given individual language challenges, linking to a larger context of a person's schedule and other relevant data, or allowing a person to perform many tasks without switching assistants. This leads me to a service oriented architecture (think APIs) which can present domain information or semantic data to a front end agent to facilitate the interaction, so many services can be enabled by interfacing on the back end or simply loading the context of interaction.

Of course we need to see "wins" from leaders like Kasisto in terms of realized cost savings and better user experiences to drive scale to the point where this would be valuable, but I believe it is a matter of time. Thought provoking stuff. If anyone wants to talk about it, let's discuss.