Knowledge Maintenance & Webchat Feedback – Mainstay

Overview

Knowledge Maintenance allows users to easily see instances where the bot response didn't meet the learner's need, and take corrective actions in the Knowledge Base if necessary. This tool is divided into four sections: All Messages, Missed Questions, Marked as Incorrect, and Webchat Feedback.

All Messages

This view shows all incoming messages from learners outside of a Campaign/Script, with information about the learner, the match, and the response provided.

Incoming Message: This column indicates the actual message received from the learner, in quotes. It also shows a relative timestamp (ie, how many days ago the message was received), and a link to the learner's conversation (as their name).
Matched Understanding: This column shows which Knowledge Base understanding the bot matched to, if any.
Bot Response: This column shows the response that the learner received. Above the actual text of the response is a tag indicating the type of response. For Static responses, a link to "Edit Answer" will appear below the response. For Interactive responses, a link to "View Script" will appear instead.
Sources: For KB generative responses and "Flash Responses" (generative fallbacks), this column will link to the Knowledge Sources (webpages and uploaded documents) that were used to generate the response.

Filtering

KM Webchat feedback updated filter view.png

The All Messages, Missed Questions, and Marked as Incorrect pages allow filtering in various ways:

Review Status: By default, resolved items are hidden, but this allows you to see them.
Message Type:
- For messages that matched to a KB understanding, differentiate between Generative Responses, Interactive Responses, and traditional Static Responses.
- For messages that did not match to a KB understanding, differentiate between Flash Responses and traditional Fallbacks.
Knowledge Sources Select one or more Knowledge Sources to view messages where the Knowledge Source(s) was used to generate a response.
Audiences: Filter the messages to only see those where learners below to particular Audiences.
Channels: Only see messages from SMS or Web-chat.
Date Received: Narrow the date range for which messages are displayed.

Missed Questions

This view includes all instances where your bot did not find a match in your Knowledge Base. This can happen when the Knowledge Base doesn’t have coverage for that incoming question, or when there is a good match, but the understanding is missing an approved answer. For more information on fallback responses or approving answers, please see Managing Fallback Responses or Approving Answers in Your Knowledge Base.

Contextual information

For partners with Flash Responses enabled, this tool will differentiate between traditional Fallbacks and AI-generated Flash Responses.

If the question was received during a Script or Campaign, the Notes column will indicate this, with a link to the Script and/or Campaign.

You can also add (and edit) a custom Note, such as helpful information for others investigating this case.

Knowledge Maintenance - received during script.png

Find an Understanding

When fallbacks get generated, our AI system is able to analyze the messages and identify potential close matches for future training. If no good matches exist in the current Knowledge Base, you are also able to create your own new understandings. After clicking Find an Understanding, there are three corrective workflows:

Add a new fuzzy question to an existing understanding. This will teach the bot a new way to ask a Summary question.
Approve an existing understanding to provide its response to future questions.
Create a new understanding if the question is not covered by any existing Knowledge Base content.

Add a Knowledge Source

For Generative Responses and Flash Responses, the corrective workflow may instead be to review the Knowledge Sources that are used to create these messages. For example, it may be necessary to remove outdated or irrelevant information, or restrict some sources to specific Audiences.

Another option is to add more relevant information. Click Add a Knowledge Source to select webpages, site sections, or a file to use going forward.

Review Special Instructions

For Generative Responses and Flash Responses, another option for improving future responses is adjusting the Special Instructions section of your Bot Personality Controls. For example, you may add a statement in the form of "If [situation], then [behavior]" to account for this particular scenario.

Note that these Special Instructions apply to the entire bot, so use caution before deleting any existing directives!

Marked as Incorrect

This view will show all messages that are manually marked as incorrect via the Conversations page, and added to the Knowledge Maintenance queue. Messages can be marked as incorrect when you find an understanding being used that needs an updated answer, or when your bot matched to the wrong understanding.

Marking messages as incorrect in Conversations

To mark a bot’s response to a learner's question as incorrect:

Click the menu button next to the learner's question, represented by the three dots.
Select Add to Knowledge Maintenance.
Choose Mark as Incorrect and add any notes you want to include for it.

Resolving a question that was marked as incorrect

When a bot response needs correcting, there are a few different actions you can take to fix the problem:

Edit an out-of-date answer.
Turn off fuzzy questions to prevent mismatches from happening.
Add fuzzy questions to encourage future bot matching to route to the a better, existing understanding.
Create a new understanding if the question is not covered by any existing Knowledge Base content.
As above, add or edit relevant Knowledge Sources.
As above, add or edit Special Instructions.

Webchat feedback

In addition to partners adding messages to Knowledge Maintenance for review, learners are able to leave feedback on how well the bot answered their questions.

During a webchat, contacts will be presented with an optional thumbs up / thumbs down indicator for every question the bot answers.

feedback

If the contact chooses, they can select a thumbs up or thumbs down, and leave a free-text comment on what they liked (or didn’t like) about the answer.