Introduction

After you've added your content, tune the knowledge base for optimal performance:

  1. Perform a search using an authentic consumer query.
  2. Review the results: Check the titles and contents of the matched articles, as well as their scores, to make sure everything looks reasonable.
  3. Take steps to improve the performance:
    • Article content: Refine the article content to improve the article matching via AI Search. If you have a consumer query for which there isn’t a relevant article to serve as the answer, just add that article. While it’s unlikely that an existing, relevant article won’t yield results, it might happen. In this case, improve the article’s title and/or add tags to the article.
    • Associated intents: If you've tied the articles to intents, refine the intents to improve the article matching via intent matching. You do this by adjusting the training phrases for the intent.
    • Configuration: Adjust the answer threshold or number of results.
    • Prompt: If you're using Generative AI, refine the prompt that's provided to the LLM service in order to improve the response.

Answer Tester versus Test & Tune

There are 2 tools within KnowledgeAI that you can use to test the performance of a knowledge base:

  • Answer Tester: Use this when you want to search within a single knowledge base.
  • Test & Tune: Use this when you want to search across multiple knowledge bases.

The Answer Tester tool

The Test and Tune tool

Answer Tester and Test & Tune work the same, with two exceptions:

  • In Test & Tune, there’s a Knowledge Base field as an input criterion. You must use this to specify the knowledge bases to search. You can select up to five.
  • In Test & Tune, there’s a Language field as an input criterion. You can use this to filter the list of knowledge bases that are available for selection. For example, if you want to select one or more Spanish-language knowledge bases in the Knowledge Base field, select "Spanish" here.

Access Answer Tester

  1. Access KnowledgeAI.
  2. Click Knowledge Bases in the upper-left corner, and select the knowledge base.
  3. Click Articles in the upper-left corner.
  4. Scroll down to the Answer Tester on the right side.

Access Test & Tune

  1. Access KnowledgeAI.
  2. Click Test & Tune in the upper-left corner.
  3. Click the Test tab.

Testing Generative AI features

In the testing tools, there are several features that use Generative AI. To enable these options, activate our Generative AI features in the Management Console.

Enter a single utterance or set of messages

In the testing tool, you have the option of performing a knowledge base search using a single utterance (user query) or a set of messages from a conversation. Either way, enter something that’s authentic.

Use the Single Utterance tab if you want to search using a single query:

The Single Utterance tab in the testing tool, where you can search knowledge bases using a single utterance

The Conversation tab is useful in a few cases:

  • You want to test the knowledge base search using a set of messages from a conversation. In this case, only the customer’s last message is used to search the knowledge base, but you might be testing the rephrasing of that query using the conversation context. More on this below.
  • You want to test processing of the answers by an LLM, which often involves use of the conversation context. Such processing might be answer enrichment or custom processing. More on this below too.

The Conversation tab in the testing tool, where you can search knowledge bases using a series of messages in a conversation

Use the format shown in the image above. There are only two allowed actors: CUSTOMER and ASSIGNED_AGENT. Ensure the last message is from the CUSTOMER. Also ensure each message from an actor is on a new line.

If you import the messages from a conversation, the earliest N messages are imported. You can get a conversation’s ID from the Consumer Info widget in the Agent Workspace.

As mentioned farther above, the search is always performed using the last consumer message.

Specify search criteria

The Criteria section of the testing tool, which you can use to specify the search criteria

In the testing tool, specify how to perform the search:

  • Retrieve answers by: Select the type of search to perform.
  • Threshold: Select the confidence threshold that an article must meet for it to be returned as a result.
  • # of results (Answer Tester) or # of results per knowledge base (Test & Tune): Select how many results to return.
  • Article status: Select the status of the article, either Active, Inactive, or All. This option is only available for internal knowledge bases.
  • Language (Test & Tune only): Use this to filter the list of knowledge bases that are available for selection. For example, if you want to select one or more Spanish-language knowledge bases in the Knowledge base field, select "Spanish" here.
  • Knowledge base (Test & Tune only): Select the knowledge bases to search. You can select up to five.

Process user’s query (Conversation Assist only)

These options are only supported by Conversation Assist. They’re made available in the KnowledgeAI testing tool, so brands with Conversation Assist solutions can perform testing.

The section of the testing tool that makes available options for processing the user's query for better search results

  • Enhance (rephrase) using conversation context (LivePerson SLM): Often the user’s query doesn’t include enough context to retrieve a high-quality answer from a knowledge base. To solve this, you can turn on this setting so that the system gathers additional conversation context (conversation turns) and uses it to rephrase the user’s query before searching the knowledge base. Rephrasing is done using KnowledgeAI’s Query Contextualization feature.
  • Custom processing via Generative AI: Turn on this setting (and select a prompt) to perform any custom processing of the user query that you require. For example, if you’re supporting cross-lingual queries (say the query is in Spanish, but your knowledge base is in English), you might want to translate the query into the language of the knowledge base that will be searched for answers. Doing so can improve the search results significantly.

Process answers

The section of the testing tool that makes available options for processing the answers for better quality

Testing and debugging

  1. Enter the search criteria, and click Get Answers.

  2. Review the results under Matched answers.

    The answers matched to the search

  3. Click the article to view its details:

    Detailed article info on an answer matched to the search

  4. Click JSON to view the JSON:

    The section of the testing tool that displays the JSON for a matched article

    If you need to troubleshoot an error, return to the results, and click the Debug tab:

    The section of the testing tool that makes available the debug logs for the seach process