Virtual Agents that Can Think!

IBM Watson and IBM Watson Engagement Advisor

If you’re a fan of Jeopardy! (The TV game show), then for sure you remember the IBM Challenge on February 14 – 16, 2011 when a supercomputer app from IBM named Watson (after Thomas J. Watson, IBM’s founder) played the game against its two biggest (multi-million dollar) winners and beat them handily. Watson delivered so many more correct responses so much faster than the former champions. It really was no contest. Check out this video if you don’t remember or if you’re not a Jeopardy! fan (

Well, IBM had made Watson a product—IBM Watson Engagement Advisor. While IBM positions and markets Watson Engagement Advisor more broadly, from our perspective, this very new offering can be the technology behind very useful and very intelligent virtual agents, virtual agents that can learn and (almost) think, virtual agents that can transform customer service. This week’s report is our evaluation of Watson Engagement Advisor against our Framework for Evaluating Virtual Assisted-Service (virtual agent) Products.

Cognitive Technology Makes the Difference

Cognitive technology is Watson Engagement Advisor’s most significant strength, advantage, and differentiator. Watson Engagement Advisor is the only customer service product that uses it. Watson Engagement Advisor’s approach to analyzing customers’ questions and matching them with knowledgebase answers uses a combination of cognitive technology, Natural Language Processing (NLP) technology, and machine learning technology. Similarly to alternative virtual agent approaches, Watson Engagement Advisor uses NLP technology to parse and understand the intent of customers’ questions. And, also similarly to alternative approaches, it uses machine learning technology to match canonical, representative, or expected forms of customer’s questions with knowledgebase answers. (Analysts train Watson’s machine learning model with question and answer pairs.) Cognitive technology uniquely enables Watson Engagement Advisor’s virtual agents to think. In analyzing customers’ questions, Watson’s cognitive technology 1) generates a number hypotheses, which are possible answers, 2) compares the language of the hypotheses with the language of the customer’s questions, and 3) scores each hypothesis for how well the question infers it. The hypotheses with the highest scores are delivered back to customers as the answers to their questions. By the way, Watson thinks fast. There’s no performance penalty for the “extra” work to perform hypothesis generation, comparison, and scoring. Watson won at Jeopardy! because it delivered correct answers faster than its human competitors.

Watson Engagement Advisor Can Answer Many Types of Customers’ Questions

Cognitive technology also enables Watson Engagement Advisor to answer many types of questions, another of its strengths and differentiators.

• Simple facts
• Definitions of terms
• Descriptions of topics
• Yes/no or true/false
• Steps in a procedure, or approaches to troubleshooting.

If analysts have includes the content that contains the answers to the questions in the knowledgebase and if they’ve trained their machine learning model with appropriate question and answer pairs, then Watson Engagement Advisor will deliver the correct answers. On the topic of knowledge, Watson Engagement Advisor has an excellent approach to knowledge management. Its knowledgebase is its corpus. Analysts create a corpus by uploading (existing) HTML, PDF, Word, or XML documents. Watson Engagement Advisor organizes, indexes, and manages this content as a knowledgebase. No authoring, editing, and managing knowledge items. No explicit indexing or categorization, either. Watson Engagement Advisor does the work. Pretty good, eh?

Work in Progress

As we mentioned above, Watson Engagement Advisor is a very new offering. While IBM Research developed (and continues to develop) Watson’s core NLP, cognitive, and machine learning technologies several years ago, Watson Engagement Advisor was introduced on May 21, 2013. It’s a bit immature and a bit incomplete. For example, the current version supports only English, does not support speech, does not have reporting capabilities, and does not integrate with external customer service apps. IBM told us that its product developers are working to deliver capabilities in all of these areas. Also, while approximately ten end-customers and ten partners have licensed Watson Engagement Advisor, none have yet deployed live apps. Remember, though, Watson Engagement Advisor is a new product but an offering from a very experienced and very established supplier. In fact, IBM has established the Watson Group to support the development and commercialization of cloud-delivered cognitive applications and announced that it would invest more than $1 billion in it, including $100 million available for venture investments to support its ecosystem of partners that are building and will be building “powered by Watson” cognitive apps. No question that IBM will deliver the missing pieces. No question that live powered by Watson apps will be coming soon.