Knowledge Maintenance & Webchat Feedback

Overview

Knowledge Maintenance is a powerful tool that will allow users to take corrective Knowledge Base actions based on direct feedback from contacts.

As a primer, please refer to Understandings and Best Practices to learn more about how Understandings work and how to optimize them.  

‘Missed Questions’ Section

This view automatically includes all instances where your bot responded with a fallback response. As a reminder, fallback responses are provided when your bot cannot confidently answer questions. This can happen when it doesn’t understand the incoming question, or when an understanding is missing an approved answer. For more information on fallback responses or approving answers, please see Managing Fallback Responses or Approving Answers in Your Knowledge Base.

Find an Understanding

When fallbacks get generated, our AI system is able to analyze the messages and identify potential close matches for future training. If no good matches exist in the current Knowledge Base, you are also able to create your own new understandings. 

Corrective Workflows 

When a fallback response is provided, there are four potential actions you can take within Knowledge Maintenance:

1. Add a new synonym question to an existing understanding. This will teach the bot a new way to ask a Summary question.
2. Approve an existing understanding to provide its response to future questions. 
3. Create a brand new understanding if the question is not covered by any existing Knowledge Base content.
4. Resolve any messages that require no action because it’s a non-relevant question (i.e. can you introduce me to Harry Potter).

To make sure your bot performs its best, there are best practices to follow when creating a brand new understanding. This includes creating additional Organization Questions to train your bot to recognize similar messages from learners. Please refer to our article on Understandings and Best Practices for more details.

‘Marked as Incorrect’ Section

This view will show all messages that are manually marked as incorrect via the Conversations page, and added to the Knowledge Maintenance queue. Messages can be marked as incorrect when you find an understanding being used that needs an updated answer, or when your bot matched to the wrong understanding.

Marking messages as incorrect in Conversations

To mark a bot’s response to a learner's question as incorrect:

1. Click the menu button next to the learner's question, represented by the three dots
2. Select add to ‘Knowledge Maintenance’
3. Choose ‘Mark as Incorrect’ and add any notes you want to include for it.

Resolving a question that was marked as incorrect

When a bot response needs correcting, there are a few different actions you can take to fix the problem: 

1. Edit an out-of-date answer
2. Turn off synonym questions to prevent mismatches from happening
3. Add synonym questions to encourage future bot matching to route to the a better, existing understanding
4. Create a brand new understanding if the question is not covered by any existing Knowledge Base content

Webchat feedback

In addition to partners adding messages to Knowledge Maintenance for review, learners are able to leave feedback on how well the bot answered their questions.

During a webchat, contacts will be presented with an optional thumbs up / thumbs down indicator for every question the bot answers.

If the contact chooses, they can select a thumbs up or thumbs down, and leave a free-text comment on what they liked (or didn’t like) about the answer.

This rating selected and comment will show up under the ‘Webchat Feedback’ section of Knowledge Maintenance, allowing you to identify other areas of Knowledge that you would like to improve.

You can also Download all webchat feedback for a given time range (based on the time of the bot's message). The download includes the following fields:

• Incoming Message At: timestamp of the learner's question
• Incoming Message Body: the text of the learner's question
• Outgoing Message At: timestamp of the bot's response
• Outgoing Message Body: the text of the bot's response
• Outgoing Source: indicates if the outgoing message came from the AI/Knowledge Base, a Script (such as the web-chat intro or a #command), or Live Chat
• Understanding Topic: if outgoing message came from the AI/Knowledge Base, the topic that was matched to
• Campaign ID: if the outgoing message came from a Campaign Script, the ID of that campaign
• Campaign Name: if the outgoing message came from a Campaign Script, the name of that campaign
• Rating: thumbs up or thumbs down
• Comment: the feedback left by the learner