Chatbots are a big thing right now, they are simultaneously extremely important and nothing new. At the same time, they’re confusing, really confusing. I’ve come up with a maturity model that I hope provides an aspect of categorization in an effort to clarify the conversation a bit. I intentionally named this post “A Chatbot Maturity Model” not “The Chatbot Maturity Model” because I’m trying to provide something useful, not something final. My maturity model focuses on big broad categorizations, and my perspective is naturally skewed to thinking of bots as support automation and content delivery systems. So, with that said, here is my model:
Going bottom up, I’m going to explain how I categorize each level of a bot. But before I do that, I want to reiterate the fact that I’m really focused on bots for support.
This post is part of our effort to bring better understanding to the topic of chatbots. Read more posts like this one and learn about how Heretto makes enterprise-grade chatbots a reality.
FAQBot
FAQBot, or Frequently Asked Questions Bot, is the most common form of support bot today. In some ways, I consider this to be the default. They’re fairly easy to implement and when done properly, they can be valuable. It’s common for FAQBots to be simply a different user experience for search. Zendesk’s Answer Bot is a great example of this (see How Dollar Shave Club Implemented AI With Zendesk). They call it a “bot” and say it has AI. I think both of those things are probably vaguely true, but they also demonstrate the flexibility of those labels. When you look at what Dollar Shave Club and Zendesk show in their webinar, it’s really just search matches being displayed for ticket deflection. None of this is to take away from the value it brought Dollar Shave Club. The two things that stuck with me after watching that webinar were 1) that’s a mighty liberal use of the term “bot” and 2) who cares if it provided the business value they’re claiming it provided.
Another example of a FAQBot can be found on the Allstate Business Insurance website. This chatbot takes the mixed interaction option of allowing a user to select from a list of common questions or enter their own. This chatbot has a well-defined mission, and it’s able to execute it in a fairly lean fashion because it’s backed by a team of people that know how to use the technology.
When considering a FAQBot, there are 3 common benefits:
- Compact, mobile-ready UX
- Scales nicely as new answers are added (that is, if you manage the content properly)
- Provides natural user feedback as questions are asked
You should consider a FAQBot when…
- You have too many FAQs to comfortably deliver as a list
- You want to gather new questions in a natural way
- You want to deliver answers on other platforms, like Facebook Messenger
When considering a FAQBot your primary cost factors are:
- Current state of your content
- Quantity of FAQs
- Average shelf life of your content
One last note on FAQBots, when considering a chatbot for a content delivery application (e.g. support), all bots have a FAQBot component, even if they’re much more than that. In that way, FAQBots form the base of your chatbot, and need careful consideration.
SkillsBot
From what I’ve seen, SkillsBots really represent the state of the art in chatbots. If you look through the examples provided by chatbot systems (Watson, Dialogflow), most of them are what I’d consider a SkillsBot.
All of these different Chatbots are specifically designed to handle different tasks. When thinking about implementing a chatbot, thinking of it as a conversational interface for different interactions is a useful way to plan. When a team sits down to build a chatbot and agrees they’re going to build a SkillsBot, the next part of the conversation is: “What skills do we want this to have?” That, in turn, generates a list that can be broken down, categorized, prioritized, and planned. Maybe this is obvious to most people, but this mental framework was extremely useful to me when I started diving into DITA.bot.
Upgrading from FAQBot to SkillsBot
FAQBots can provide a compact experience for users looking to answer questions, and in a way, this can be seen as a SkillsBot with just one skill, answering preloaded questions. In my opinion, crossing from FAQBot to SkillsBot is as much a mindset as it is a definition of functionality. An organization that is building a SkillsBot is dedicated to the concept that this is an ongoing project that will continually provide more capability and value. FAQBots can be built by writers with off the shelf technology, SkillsBots require a project approach and a developer mindset.
Capabilities of a SkillsBot
Where FAQBots are limited to answering questions that are preloaded into it, Skillsbots can be used to answer questions that are determined or calculated. A great example of this is the Valid Child Interaction we built for DITA.bot. Valid Child is a skill that can tell a user if a certain DITA element is a valid child of another DITA element (we’ll have an article describing this in more depth soon). This is still a question and answer interaction, but it’s not one that can be fulfilled with a FAQ approach because there are more than 140,000 different versions of this question for the standard DITA 1.2 specification. So, when we think about this question, we think of it as a set and we approach it using a determined response. In this case, the way it works is by using Heretto’s Bot Backend to analyze the DITA 1.2 content to provide an answer. This is possible because the DITA 1.2 spec is in a structured content format (XML) and Heretto knows how to analyze that format. This isn’t something you can do with Word documents, PDFs, or blog articles.
Skills can also be more general. One of the standard skills that comes with Heretto’s Bot Backend is a “HeresASearchFallback”. This skill can be used to provide search results back to the user when their question doesn’t have a good answer. Sometimes this is combined with another skill such as “CreateASupportTicketInteraction”. That way the chatbot can say something like “I’m not sure, but I’ve run a search, take a look at these results and let me know if any of them help. If not, I can create a support ticket for you, if you’d like.” This is a pretty good fallback option for a chatbot.
When considering a SkillsBot, there are 3 common benefits:
- The chatbot can use and impact other systems
- SkillsBots have finer parsing capabilities, large sets of similar questions are more easily handled
- SkillsBots open up a whole new level of analytics due to their connection with a logical backend
You should consider a SkillsBot when…
- You’re considering a chatbot and you need it to do more than provide human written answers
- You want a chatbot that can interact with other systems
- You have many questions that are similar
When considering a FAQBot, your primary cost factors are:
- Everything from FAQBot
- Scope of interactions
- APIs for connected systems
AIBot
I think one of the biggest issues with AI is that people mistake it for human-level intelligence, referred to as “General AI” in the industry. But in reality, AI is a very broad spectrum, and everyone claims to have some of it. For the purposes of my maturity model, I’m giving AI a straightforward definition:
“A bot that can interpret unstructured content, rather than just deliver it, and incorporates some degree of machine learning to improve its responses over time.”
Automation isn’t automatic.
These two qualities, interpret unstructured content and utilize machine learning, are pretty significant. They open up a lot of possibilities, but implementing and managing them is no small undertaking. In my conversations with people I’ve come to realize that many people think machine learning means “the machine will figure it out, so we don’t have to.” I’d like to suggest that this couldn’t be further from the truth. Automation isn’t automatic, just ask Elon Musk. Machine learning technologies have come a long way in the last 5-10 years, but they still don’t implement themselves. This means you should have good reason for implementing ML in your chatbot, the simple question “Do we really need this?” will go a long way.
In practice, this definition still covers a broad spectrum of implementations.
Upgrading from a SkillsBot to an AIBot
Just like going from FAQBot to SkillsBot is a mindset, SkillsBot to AIBot is as much about the project intention as it is any hard technology difference. The place to start with an AIBot is to tightly define a set of goals for the AI pieces. In the same way, as I think it’s important to break apart the capabilities of a SkillsBot, I think it’s important to break the machine learning usage into specific cases. The narrower your usage of machine learning, the easier it will be to tweak and improve it for each case.
In some ways, you can think of moving to an AIBot as attaching machine learning to specific skills to improve their overall quality.
As for interpreting unstructured content, there are many technologies out there, but no silver bullet. At their core, these tools are just textual analysis engines. Some of them focus on analyzing large sets of content and extracting meaning or replicating its patterns. Some of them focus on apply rules to extract meaning from specific blocks of text. Regardless of the technology, the one thing I have found them to have in common is that they’re all components. There is no sign a contract, upload some documents, and *poof* you have a solution. These system need to be incorporated into your larger project intelligently.
Capabilities of an AIBot
The increase in capabilities between a SkillsBot and an AIBot is pretty simple. It’s all about scalability. In (my) theory, an AIBot should be far more scalable than a SkillsBot. It should need less curation and programming to increase its value and scope.
When considering an AIBot, there are 2 common benefits:
- Quality of responses should automatically improve over time
- Greater capacity to utilize unstructured content
You should consider an AIBot when…
- There is an obvious case for machine learning
- You have unstructured or lightly-structured content and a strategy for using it
- You have a use case that can’t be handled with standard logic and you’ve found an AI system that effectively handles this case
When considering an AIBot, your primary cost factors are:
- Everything from SkillsBot
- Chatbot engine (and potentially other software)
- Datasets
- Training
TuringBot
This is a full human replacement. Named for the famous Turing Test (obviously), a TuringBot is not only just as good as a human at whatever task it is assigned, it is indistinguishable to the user. From what I know of the state of the art (as I write this in April of 2018), we’re a long, long way from a TuringBot for the purposes of generalized user support. What that means in years, I have no idea. And I’ll even go a step further and do something I always tell people not to do (“Never bet against technological progress” I always say), I’m not convinced we’ll ever get to a full TuringBot. To be clear, I’m not convinced that we won’t get there either, but I’ll stop there before I end up on a philosophical tangent.
How is this different than an AIBot? A TuringBot would be able to create content, understand the user, provide situationally appropriate rationale, etc. In other words, a TuringBot can think and reason, whereas an AIBot can only “learn” based on inputs or said a third way, AIBots are limited by the scope of knowledge they are provided.
Either way, we don’t have them, so while they’re fun to muse about, they’re not particularly useful from a planning perspective today.
Takeaways
- Be a bit skeptical of anyone who tells you their product includes “Machine Learning” or “AI”. The simple question “What does that mean?” goes a long way.
- Plan, spec, and document everything! Chatbots aren’t magical, they require engineering and content strategy to implement.
- Don’t build more bot than you need. If you just need to answer some canned questions that get asked thousands of times a day, build a simple FAQBot and take your quick win.
Want to learn more about chatbots? Find out about how Heretto makes enterprise-grade chatbots a reality.
