Omnichannel-AI Bots have become all the rage for delivering automated customer care. Unfortunately, with most bot tools on the market today, they're also an adjunct to the rest of the contact center and customer experience. Questions we typically hear from companies using generic Conversational AI products are

  • How do I know what it's doing?
  • Is it working?
  • What is my customer experience like?
  • Are customers hanging-up?
  • Are they having to repeat themselves?
  • Are there certain prompts that are failing more than others?
  • What has the impact been to my business?

If you're using one of these tools today, ask yourself "What's it like being one of our customers interacting with this bot?"If you can't answer that question, then this is for you.

What's Missing In the Bot World Today?

A critical deficiency with many Conversational-AI platforms, such as Amazon Lex, Google DailogueFlow, IBM Watson and Twilio Autopilot, is the lack of reporting and visibility into a given bot's performance and quantifying it's impact on Customer Experience.For platforms that do provide reporting, they're accessible only through their developer portal, and not generally available to your business users. Couple this with the metrics they do expose tend to be high-level and useless.The only other option for companies is to have a software developer build their own data pipelines, data normalization and CX dashboards in a third-party dashboard tool, such as Microsoft Power BI or Tableau.This approach requires close coordination between your bot developers, business intelligence team, and customer support organization. Elements such as intents, entities and slots are relatively new data types and in-house Data Science teams just aren't familiar with the business logic behind them.Multiexperience interactions that span multiple channels also break down existing reporting models. For example, if an interaction originates by voice call and then transfers over to SMS or perhaps starts with a web chat and results with a voice call, those interactions segments are difficult to stitch together into a full picture of the Customer Experience. This issue is compounded when multiple platforms are involved. Few vendors, save for Twilio and Amazon, have their own onboard Conversational AI or "Bot" engine. As a result, companies are left to build consolidated reporting on their own. Most organizations do not have the resources to get it done.As a result, bots tend to function as semi-autonomous blackboxes developed mostly in isolation with no way to bring that data back in the contact center.Until now.

Measuring Ourselves First

When we first built IVAn and delivered "Trashbot" for the City of Kansas City, MO, we knew that reporting and analytics were going to be critical to our success. As we run Conversational AI as a managed service for most of our clients, our team is accountable for monitoring performance and tuning each bot.To support our Human-in-the-Loop AI, we needed killer analytics and graphics. Much like you wouldn't put a new agent on the phones without being able to monitor and measure their performance, you shouldn't launch an intelligent virtual agent without being able monitor it's performance either. Spectra was the obvious choice to deliver the insights we needed, but the first dashboards were developed specifically for each customer, and therefore weren't repeatable or scalable.However, we quickly saw the value in the initial set of performance management and customer experience insights we developed. Subsequently, these are ultimately what led to us to shelve both Amazon Lex and Google Dialogueflow as our bot tools and build our own NLU engine instead.

We then challenged ourselves to imagine the art of the possible. And then we built it.

Customer Experience Metrics for Measuring Bot Performance

The first step in our design process was to identify what metrics and insights our customers need.As example, the key measure of a bot’s success today tends to be “containment rate”, or the percentage of calls that were handled by the bot or deflected from live agents. This is similar to how IVR performance has traditionally been measured. While that’s important, its more critical to be able to score the virtual assistant’s performance more akin to that of a live agent in terms of Customer Experience and Operational Performance. Call center managers are keenly focused on KPIs such as handle times, first call resolution and even sales conversion rates.Capturing the right KPIs and CX metrics is critical for Intelligent Virtual Assistants to move from being a stopgap for cost savings or absorbing call spikes to becoming a strategic organizational asset that customers want to interact with.

Introducing- Conversational Experience Analytics (CXA)

After months of development and collaboration with our customers, we're excited to launch Conversational Experience Analytics into Public Beta. What's more, we've opened it up to other bot platforms besides just our own. Xaqt’s new Conversational Experience Analytics (CXA) provides visibility and insights into any bot’s or Intelligent Virtual Agent’s performance and key metrics. CXA comes standard with all of Xaqt's Conversational-AI applications and Conversational Service Automation platform. It also integrates integrates with Google DialogueFlow and Amazon Lex to providing you with newfound insights into any bot or Intelligent Virtual Agent.CXA works as a module for Xaqt's Cognitive Insights Portal and is the most advanced Conversational and Bot Analytics suite available in the market today. With CXA, you no longer have to worry about how to access bot data. You're in control. As in full control.CXA frees your data from the grips of complex platforms and puts it in the hands of your business.Leveraging the power of Xaqt's Cognitive Insights Portal, CXA provides a rich set of enterprise class Business Intelligence features right out-of-the-box.

From APIs to Normalized Data Lake

Gone are the days of investing your software engineers' time in developing data extracts from proprietary APIs and then having to model the data for analysis.At the heart of CXA is a normalized data lake that connects to any bot platform and normalizes the data for analysis and visualization. For customers that already have a preferred BI engine, such as Power BI or Tableau, you can connect those tools directly the CXA data lake. We'll even give you the pre-built SQL Queries to kickstart your own insight development.CXA data is also available from a GraphQL API, which means your software engineers can embed data and insights throughout any application in your business.

Stunning Visualizations

Conversational Experience Analytics comes with over 30 standard insights and charts to accelerate your Bot Analytics journey.Leveraging the power of Cognitive Insights Portal, you can define Executive and Departmental level dashboards and tailor our standard metrics and charts, or create you own.

Multi-Tenant and Security First

CXA was designed to be Multi-tenant from the start, and employs a robust Data Governance and Security first infrastructure.This means BPOs, Call Center Outsourcers and Digital Agencies can provide these unprecedented insights to their customers without worrying about data governance or security.

Customer Intelligence

Every interaction with your customers, whether that's by phone, contactless or via bot, is an opportunity to build a relationship. Each of those interactions creates unique data that can be incorporated back into your enterprise to enhance future customer interactions.CXA captures data from every aspect of a bot's interaction and makes it immediately available for analysis. A few of these include:

  • Repeat Engagement and Cohort Analysis
  • Cohort Analysis
  • New contacts
  • Unique contacts
  • Returning contacts
  • Lifetime Contacts

Conversational Insights

Conversational AI and NLU creates new opportunities to understand your customers and their needs.With a traditional IVR, you can only capture general information based on the prompts a customer selects, such as "Sales" or "Support". But when customers are allowed to converse freely with an Intelligent Virtual Agent, you capture new insights into call drivers and potential customer issues.

Exception Management

Let's face it. Even the best of bots sometimes fail.With most Conversational AI platforms, it's nearly impossible to discern where bots fail or a customer interaction needs to escalate to a live agent.CXA takes the mystery out of bot failures and success and makes it easy to identify these exceptions. With CXA's Exception reporting and dashboards, you have instant visibility into customer needs that are not being met by your bot.

Conversation Detail & Transcripts

Every conversation transcript is stored along with the interaction details to provide the complete picture of your customer's journey.

Check it Out

Book a demo and see how Conversational Experience Analytics can support your existing or new Conversational AI initiatives.