Knowledge Maintenance & Webchat Feedback

Overview

Knowledge Maintenance allows users to easily see instances where the bot response didn't meet the learner's need, and take corrective actions in the Knowledge Base if necessary. This tool is divided into three sections: Missed Questions, Marked as Incorrect, and Webchat Feedback.

Missed Questions

This view includes all instances where your bot did not find a match in your Knowledge Base. This can happen when the Knowledge Base doesn’t have coverage for that incoming question, or when there is a good match, but the understanding is missing an approved answer. For more information on fallback responses or approving answers, please see Managing Fallback Responses or Approving Answers in Your Knowledge Base.

Contextual information

For partners with Flash Responses enabled, this tool will differentiate between traditional Fallbacks and AI-generated Flash Responses.

If the question was received during a Script or Campaign, the Notes column will indicate this, with a link to the Script and/or Campaign.

Find an Understanding

When fallbacks get generated, our AI system is able to analyze the messages and identify potential close matches for future training. If no good matches exist in the current Knowledge Base, you are also able to create your own new understandings. 

Corrective Workflows 

When a fallback response is provided, there are four potential actions you can take within Knowledge Maintenance:

1. Add a new synonym question to an existing understanding. This will teach the bot a new way to ask a Summary question.
2. Approve an existing understanding to provide its response to future questions. 
3. Create a brand new understanding if the question is not covered by any existing Knowledge Base content.
4. Resolve any messages that require no action because it’s a non-relevant question (i.e. can you introduce me to Harry Potter).

To make sure your bot performs its best, there are best practices to follow when creating a brand new understanding. This includes creating additional Organization Questions to train your bot to recognize similar messages from learners. Please refer to our article on Understandings and Best Practices for more details.

Marked as Incorrect

This view will show all messages that are manually marked as incorrect via the Conversations page, and added to the Knowledge Maintenance queue. Messages can be marked as incorrect when you find an understanding being used that needs an updated answer, or when your bot matched to the wrong understanding.

Marking messages as incorrect in Conversations

To mark a bot’s response to a learner's question as incorrect:

1. Click the menu button next to the learner's question, represented by the three dots
2. Select add to ‘Knowledge Maintenance’
3. Choose ‘Mark as Incorrect’ and add any notes you want to include for it.

Resolving a question that was marked as incorrect

When a bot response needs correcting, there are a few different actions you can take to fix the problem: 

1. Edit an out-of-date answer
2. Turn off synonym questions to prevent mismatches from happening
3. Add synonym questions to encourage future bot matching to route to the a better, existing understanding
4. Create a brand new understanding if the question is not covered by any existing Knowledge Base content

Webchat feedback

In addition to partners adding messages to Knowledge Maintenance for review, learners are able to leave feedback on how well the bot answered their questions.

During a webchat, contacts will be presented with an optional thumbs up / thumbs down indicator for every question the bot answers.

If the contact chooses, they can select a thumbs up or thumbs down, and leave a free-text comment on what they liked (or didn’t like) about the answer.

This rating selected and comment will show up under the ‘Webchat Feedback’ section of Knowledge Maintenance, allowing you to identify other areas of Knowledge that you would like to improve.

You can also Download all webchat feedback for a given time range (based on the time of the bot's message). The download includes the following fields:

• Incoming Message At: timestamp of the learner's question
• Incoming Message Body: the text of the learner's question
• Outgoing Message At: timestamp of the bot's response
• Outgoing Message Body: the text of the bot's response
• Outgoing Source: indicates if the outgoing message came from the AI/Knowledge Base, a Script (such as the web-chat intro or a #command), or Live Chat
• Understanding Topic: if outgoing message came from the AI/Knowledge Base, the topic that was matched to
• Campaign ID: if the outgoing message came from a Campaign Script, the ID of that campaign
• Campaign Name: if the outgoing message came from a Campaign Script, the name of that campaign
• Rating: thumbs up or thumbs down
• Comment: the feedback left by the learner