Nuance Nina Virtual Assistants

We evaluated Nina, the virtual assistant offering from Nuance, for the third time, publishing our Product Evaluation Report on October 29, 2015. This Report covers both Nina Mobile and Nina Web.

Briefly, by way of background, Nina Mobile provides virtual assisted-service on mobile devices. Customers ask questions or request actions of Nina Mobile’s virtual assistants questions by speaking or typing them. Nina Mobile’s virtual assistants deliver answers in text. Nina Mobile was introduced in 2012. We estimate that approximately 15 Nina Mobile-based virtual assistants have been deployed in customer accounts.

Nina Web provides virtual assisted-service through web browsers on PCs and on mobile devices. Customers ask questions or requests actions of Nina Web’s virtual assistants questions by typing them into text boxes. Nina Web’s virtual assistants deliver answers or perform actions in text and/or in speech. Nina Web was introduced as VirtuOz Intelligent Virtual Agent in 2004. Nuance acquired VirtuOz in 2013. We estimate that approximately 35 Nina Web-based virtual assistants have been deployed in customer accounts.

The two products now have common technologies, tools, and a development and deployment platform. That’s a big deal. They had been separate and pretty much independent products, sharing little more than a brand. Nuance’s development team has been busy and productive. Nina also has many new and improved capabilities. Most significant are a new and additional toolset that supports key tasks in initial deployment and ongoing management, PCI (Payment Card Industry) certification, which means that Nina virtual assistants can perform ecommerce tasks for customers, support for additional languages, and packaged integrations with chat applications.

Nina Evaluation Process

We did not include an evaluation of Nina’s Ease of Evaluation. Our work on the Nina Product Evaluation Report was well underway before we added that criterion to our framework. So, we’ll offer that evaluation here.

For our evaluation, we used:

  • Product documentation, which was provided to us by Nuance under an NDA
  • Demonstrations, especially of new tools and functionality, conducted by Nuance product management staff
  • Web content of nuance.com
  • Online content of Nina deployments
  • Nuance’s SEC filings
  • Discussions with Nuance product management and product marketing staff
  • Thorough (and very much appreciated) review of report draft

We also leveraged our knowledge of Nina, knowledge that we acquired in our research for two previously published Product Evaluation Reports from July 2012 and January 2014. We know the product, the underlying technology, and the supplier. So we were able to focus our research on what was new and improved.

Product Documentation

Product documentation, the end user/admin manuals for Nina IQ Studio (NIQS) and the new Nuance Experience Studio (NES) toolsets, was they key source for our research. We found the manuals to be well written and reasonably easy to understand. Samples and examples illustrated simple use cases and supported descriptions very well. Showing more complex use cases, especially for customer/virtual assistant dialogs, would have been very helpful. Personalization facilities could be explained more thoroughly. Also, there’s a bit of inconsistency in terminology between the two toolsets and their documentation.

Nina Deployments

Online content of Nina deployments helped our research significantly. Within the report, we showed two examples of businesses that have licensed and deployed Nina Web are up2drive.com, the online auto loan site for BMW Financial Services NA, LLC and the Swedish language site for Swedbank, Sweden’s largest savings bank. The up2drive Assist box accesses the site’s Nina Web virtual assistant. We asked, “How to I qualify for the lowest rate new car rate?” See the Illustration just below.

up2drive

Online content of Nina Mobile deployments show how virtual assistants can perform actions for customers. For example, we showed how Dom, the Nina Mobile virtual assistant, could help you order pizza from Domino’s in our blog post of May 14, 2015. See https://www.youtube.com/watch?v=noVzvBG0GD0.

Take care when using virtual assistant deployments for evaluation and selection. They’re only as good as the deploying organization wants to make them. Their limitations are almost never the limitations of the virtual assistant software. Every virtual assistant software product that we’ve evaluated has the facilities to implement and deliver excellent customer service experience. Virtual assistant deployments, like all customer experience deployments, are limited by the deploying organization’s investment in them. The level of investment controls which questions they can answer, which actions they can perform, how well they can deal with vague or ambiguous questions and action requests, and their support for dialogs/conversations, personalization, and transactions.

No Trial/Test Drive

Note that Nuance did not provide us with a product trial/test drive of Nina. In fact, Nuance does not offer Nina trials/test drives to anyone. That’s typical of and common for virtual assistant software. Suppliers want easy and fast self-service trials that lead prospects to license their offerings. Virtual assistant software trials are not any of these things. They’re not designed for self-service deployment either for free or for fee.

Why not? Because virtual assistant software is complex. Even its simplest deployment requires building a knowledgebase of the answers to the typical and expected questions that customers ask, using virtual assistant facilities to deal with vague and ambiguous questions, engaging in a dialog/conversation, escalating to chat, or presenting a “no results found” message, for example, and using virtual assistant facilities to perform actions that customers request and deciding how to perform them. (Performing actions will likely require integration apps external to virtual assistant apps.) This is not the stuff of self-service trials and test-drives.

In addition, most virtual assistant suppliers have not yet invested in building tools that speed and simplify the work that organizations must perform for the initial deployment and ongoing management of virtual assistants software even after it has been licensed. Rather, suppliers offer their consulting services instead. (That’s changing for Nuance with toolsets like NES and for several other virtual assistant software suppliers and that’s certainly a topic for a later time.)

Thank You Very Much, Nuance

One more point about Ease of Evaluation. Our research goes into the details of customer service software. We publish in-depth Product Evaluation Reports. We demand a significant commitment from suppliers to support our work. Nuance certainly made that commitment and made Nina Easy to Evaluate for us. We so appreciate Nuance’s support and the time and effort taken by its staff.

Nina was very easy for us to evaluate. The product earns a grade of Exceeds Requirements in Ease of Evaluation.

Zendesk, Customer Service Software That’s Easy to Evaluate

Zendesk Product Evaluation

Zendesk is the customer service offering from Zendesk, Inc. a publicly held, San Francisco, CA based software supplier with 1,000 employees that was founded in 2004. The product provides cloud-based, cross-channel case management, knowledge management, communities and collaboration, and social customer service capabilities across assisted-service, self-service, and social customer service channels.

We evaluated Zendesk against our Evaluation Framework for Customer Service and published our Product Evaluation Report on October 22. Zendesk earned a very good Report Card—Exceeds Requirements grades in Product History and Strategy, Case Management, and Customer Service Integration, and Meets Requirements grades for all other criteria but one, Social Customer Service. Its Needs Improvement grade in Social Customer Service is less an issue with packaged capabilities than it is a requirement for a specialized external app designed for and positioned for wide and deep monitoring of social networks.

Evaluation Framework

Our Evaluation Framework considers an offering’s functionality and implementation, what a product does and how it does it. It also considers the supplier and the supplier’s product marketing (positioning, target markets, packaging and pricing, competition) and product management (release history and cycle, development approach, strategy and plans) for the offering.

We rely on the supplier for product marketing and product management information. First we gather that info from the supplier’s website and press releases and, if the supplier is publicly held, from the supplier’s SEC filings. We speak directly with the supplier for anything else in these areas.

For functionality and implementation, the supplier typically gives us (frequently under NDA) access to the product’s user and developer documentation, the manuals and help files that licensees get. In this era of cloud computing, we’ve been more and more frequently getting access to the product, itself, through online trials. We also read any supplier’s patents and patent applications to learn about the technology foundation of functionality and implementation.

In addition, we entertain the supplier’s presentations and demonstrations. They’re useful to get a feel for the style of the product and the supplier and to understand future capabilities. However, to really understand the product, there’s no substitute for actual usage (where we drive) and/or documentation.

Our research process includes insisting that the supplier reviews and provides feedback on a draft of the Product Evaluation Report. This review process ensures that we respect any NDA, improves the accuracy and usefulness of the information in the report, and prevents embarrassing the supplier and us.

Ease of Evaluation, a New Evaluation Criterion

Our frameworks have never had an Ease of Evaluation criterion. We’ve always figured that we’d do the work to make your evaluation and selection of products easier, faster, and less costly. Our evaluation of Zendesk has us rethinking that. We’ve learned that our Product Evaluation Reports can speed and shorten your evaluation and selection process but that your process doesn’t end with our reports. You do additional evaluation, modifying and extending our criteria or adding criteria for criteria to represent requirements specific to your organization, your business, and/or application for a product. Understanding Ease of Evaluation can further speed and shorten your evaluation and selection process.

So, beginning with our next Product Evaluation Report, you’ll find that Ease of Evaluation criterion in our framework.

Zendesk Was Very Easy to Evaluate

By the way, Zendesk would earn an Exceeds Requirements grade for Ease of Evaluation. We did a 30-day trial of the product. We signed-up for the trial online—no waiting. During the trial we submitted cases to Zendesk Support and we used the Zendesk community forums. In addition, Zendesk.com provided a wealth of detailed information about the product, including technical specifications and a published RESTful API.

Scroll down to the bottom of Zendesk.com’s home page to see a list of UNDER THE HOOD links.

under the hood

Looking at the UNDER THE HOOD links in a bit more detail:

  • Apps and integrations is a link to a marketplace for third party apps. Currently there are more than 300 of them.
  • Developer API is a link to the documentation of Zendesk’s RESTful, JavaScript API. It lists and comprehensively describes more than100 services.
  • Mobile SDK is a link to documentation for Android and iOS SDKs and for the Web Widget API. (The Web Widget embeds Zendesk functionality such as ticketing and knowledgebase search in a website.)
  • Security is a link to descriptions of security-related features descriptions lists of Zendesk’s security compliance certifications and memberships.
  • Tech Specs is a link to a comprehensive collection of documents that describe Zendesk’s functionality and implementation.
  • What’s new is a link to high-level descriptions of recently added capabilities
  • Uptime is a link to info and charts about the availability of Zendesk Inc.’s cloud computing infrastructure
  • Legal is a link to a description of the Terms of Service of the Zendesk offering

We spent considerable time in Tech Specs and Developer API. We found the content to be comprehensive, well organized and easy to access, and well written. The combination of the product trial and UNDER THE HOOD made Zendesk easy to evaluate. And, we did not have to sign an NDA for access to any of this information.

Many suppliers make their offerings as easy to evaluate as Zendesk, Inc. made Zendesk for us. On the other hand, many suppliers are not quite so willing to share detailed information about their products and, especially their underlying technologies. Products and technologies are, after all, software suppliers’ key IP. They have every right to protect this information. They don’t feel that patent protection is enough. Their offerings are much harder to evaluate at the level of our Product Evaluation Reports.

Consider Products That Are Easy to Evaluate

We feel as you should feel that in-depth evaluations are essential to the selection of customer service products. You’ll be spending very significant time and money to deploy and maintain these products. You should never rely on supplier presentations and demonstrations to justify those expenditures. Certainly rely on our reports and use them as the basis for your further, deeper evaluation, including our new Ease of Evaluation criterion. Put those suppliers that facilitate these evaluations on your short lists.