Leveraging the power of Generative AI

At LivePerson, we’re thrilled that advancements in Natural Language Processing (NLP) and Large Language Models (LLMs) have opened up a world of possibilities for Conversational AI solutions. The impact is truly transformative.

That's why we're delighted to say that our KnowledgeAI application offers an answer enrichment service powered by one of OpenAI's best and latest LLMs. If you’re using KnowledgeAI to recommend answers to agents via Conversation Assist, or to automate intelligent answers from Conversation Builder bots to consumers, you can take advantage of this enrichment service.

How does it work? At a high level, the consumer’s query is passed to KnowledgeAI, which uses its advanced search methods to retrieve the most relevant answers from your knowledge base. Those answers —along with some conversation context—are then passed to the LLM service for enrichment, to craft a final answer. That’s Generative AI at work. The end result is an answer that’s accurate, contextually relevant, and natural. In short, enriched answers.

To see what we mean, let’s check out a few examples in an automated conversation with a bot.

Here’s a regular answer that’s helpful…but stiff:

An example of helpful but stiff answer from a bot

But this enriched answer is warm:

An example of helpful and warm answer from a bot

This regular answer is helpful:

An example of helpful answer from a bot

But this enriched answer is even more helpful:

An example of robust answer from a bot

This regular answer doesn’t handle multiple queries within the same question:

An example of a consumer asking a bot two questions but the bot only answering one

But this enriched answer does so elegantly:

An example of a consumer asking a bot two questions and the bot successfully answering both

Overall, the results are smarter, warmer, and better. And the experience, well, human-like.

Use KnowledgeAI’s answer enrichment service to safely and productively take advantage of the unparalleled capabilities of Generative AI within our trusted Conversational AI platform. Reap better business outcomes with our trustworthy Generative AI.

Watch the 3 1/2-minute video

AI safety tools

Our core architecture

Test and learn

Prompt style selection

  • Our prompt styles are designed and tested for safety on hundreds of bots.

Agents in the loop

Language support

Enriched answers are supported for knowledge bases whose language is English.

If the language of your knowledge base is one of the 50+ other languages available, support is experimental. Don’t hesitate to get started using them in your demo solutions to explore the capabilities of Generative AI. Learn alongside us. And share your feedback! As always, proceed with care: Test thoroughly before rolling out to Production.

Learn more about language support.

Get started

  1. Activate this Generative AI feature.
  2. Turn on enriched answer recommendations in Conversation Assist. And/or, turn on enriched, automated answers in Conversation Builder bots.

Don't have a knowledge base to use yet? Learn about the different ways to populate a knowledge base with content.

How the service works

Regardless of whether you’re using enriched answers in Conversation Assist or in a Conversation Builder bot, the same general flow is used:

An architectural diagram of how the service works

  1. The consumer’s query is passed to KnowledgeAI, which uses its advanced search methods to retrieve matched articles from the knowledge base.
  2. Three items are then passed to the enrichment service:
    • The matched articles
    • Previous turns from the current conversation that provide context
    • A prompt style
  3. A prompt is dynamically generated and then used by the underlying LLM service to generate a single, final enriched answer. And the answer is returned to KnowledgeAI.

In the case of Conversation Assist, if you have multiple knowledge bases assigned to the same skill (within your Conversation Assist configuration), each knowledge base provides its own enriched answer.

Hallucinations

Hallucinations are situations where the underlying LLM service generates incorrect or nonsensical responses due to not truly understanding the meaning of the input data.

For example, assume the consumer asks, “Tell me about your 20% rebate for veterans.” If the assumption within that query (that such a rebate exists) is regarded as true by the LLM service, when in fact it is not true, the LLM service will hallucinate and send an incorrect response.

Be aware that hallucinations can occur. Typically, this happens when the model relies too heavily on its language model and fails to effectively leverage the info provided in the relevant answers. The degree of risk here depends on the prompt style that’s used.

Conversational Cloud's LLM Gateway has a Hallucination Detection post-processing service that detects and handles hallucinations with respect to URLs, phone numbers, and email addresses.

Prompt styles

Prompt styles for Messaging

There are four different prompt styles that can be used by the enrichment service in messaging contexts. The guidance for each style is different:

  • Factual: Safest. When this prompt style is used, the service is directed to respond using only the info in the matched articles, and it adheres to the script in those articles as much as possible. If there’s no matched article, no guess at an answer is attempted. Instead, the response provides direction on where the answer can be found. This prompt style is the least likely to hallucinate, but responses can sometimes be stiff or unhelpful.
  • Balanced: This prompt style is similar to “Factual” but more flexible. The service is directed to prefer info from the matched articles, but when the articles are insufficient, it can use info contained within its language model to help the consumer refine their query. For example, “I don’t have information about A, but perhaps you meant B?” This freedom comes with a price, as there’s a higher risk of hallucination than with “Factual.”

  • Creative: Least safe. When this prompt style is used, the service is directed to come up with answers independently when necessary (i.e., when no relevant articles are matched), using both the info in the provided conversation context and in its language model. This prompt returns the most natural and most flexible responses, but it carries the highest risk for hallucinations because the service tries to always provide a response.

    This prompt style shouldn’t be used by consumer-facing bots.

  • BrandFocused: This prompt style is very much like “Factual” but focused on sales-related queries. It tries to give info about your brand’s products or services, or suggest other similar products or services that you offer. Responses inform the consumer of the appropriate sales channels for their queries. The style tries to stick to your brand and not suggest alternative brands.

Prompt styles for Voice

Our prompt styles for voice conversations are designed with an auditory situation in mind. So, they direct the service to respond using words that are easy to pronounce, and to omit things that don’t convey well (e.g., symbols, URLs, etc.).

There are two different prompt styles available. The guidance for each style is different:

  • Factual: Safest. When this prompt style is used, the service is directed to respond using only the info in the matched articles, and it adheres to the script in those articles as much as possible. If there’s no matched article, no guess at an answer is attempted. Instead, the response provides direction on where the answer can be found. This prompt style is the least likely to hallucinate, but responses can sometimes be stiff or unhelpful.
  • Balanced: This prompt style is similar to “Factual” but more flexible. The service is directed to prefer info from the matched articles, but when the articles are insufficient, it can use info contained within its language model to help the consumer refine their query. For example, “I don’t have information about A, but perhaps you meant B?” This freedom comes with a price, as there’s a higher risk of hallucination than with “Factual.”

Important info about prompt styles

Be aware that all prompt styles have the potential for hallucinations. However, the degree of risk varies based on prompt style.

LivePerson has judiciously selected a default prompt style for Conversation Assist and a default prompt style for Conversation Builder bots. To change the prompt style, contact your LivePerson representative.

Response length

  • Messaging: The prompt styles direct the service to respond using at least 10 words and no more than 300 words.
  • Voice: The prompt styles direct the service to respond using less than 60 words.

Be aware that the length of the matched article(s) influences the length of the answer (within the bounds stated). Generally speaking, the longer the relevant matched article, the longer the response.

Confidence thresholds

KnowledgeAI integrations within Conversation Builder bots (Knowledge AI interaction, KnowledgeAI integration) and the settings within Conversation Assist both allow you to specify a “threshold” that matched articles must meet to be returned as results. We recommend a threshold of “GOOD” or better for best performance.

If you’re using enriched answers, use caution when downgrading the threshold to FAIR PLUS. If a low-scoring article is returned as a match, the LLM service can sometimes try to use it in the response. And the result is a low-quality answer.

As an example, below is a scenario where a strange consumer query was posed to a financial brand’s bot. The query yielded a FAIR PLUS match to an article on troubleshooting issues when downloading the brand’s banking app. So the enriched answer was as follows:

  • Consumer query: Can I book a flight to Hawaii?
  • Enriched answer: I'm sorry, I can't find any information about booking a flight to Hawaii. However, our Knowledge Articles do provide information about our banking app. If you're having trouble downloading our app, check that…

In the above example, the service rightly recognized it couldn’t speak to the consumer’s query. However, it also wrongly included irrelevant info in the response because that info was in the matched article.

Fallback flow

If relevant articles aren’t found within the knowledge base based on the confidence threshold, by default no call to the LLM service is made for an enriched answer. This means no answer is sent from KnowledgeAI back to Conversation Assist or the Conversation Builder bot.

That said, we’re currently testing the experience when the LLM service is still called when no relevant articles are found, passing to the service just the conversation context and prompt style. This behavior inactivates the prompt style being used, and it activates a Fallback prompt.

The Fallback prompt can offer a more fluent and flexible response to help the user refine their query:

  • Consumer query: What’s the weather like?
  • Response: Hi there! I'm sorry, I'm not able to answer that question. I'm an AI assistant for this brand, so I'm here to help you with any questions you may have about our products and services. Is there something specific I can help you with today?

The Fallback prompt can also yield answers that are out-of-bounds. The model might hallucinate and provide a non-factual response in its effort to generate an answer using only the memory of the data it was trained on. Use caution when using it, and test thoroughly.

Contact us if you’d like to use the Fallback prompt. We’ll flip the necessary switch behind the scenes.

Small talk support

Calling the LLM service for a response even when there are no matched articles (learn more about this fallback flow immediately above) means you can support pure small talk:

An example of a bot answer a question about how it's going

Best practices

Learn about best practices.

Tuning outcomes

When it comes to tuning outcomes, you can do a few things:

Reporting

Use the Generative AI Reporting dashboard within Conversational Cloud to make data-driven decisions that improve the effectiveness of your Generative AI solution.

A view of the Generative AI Reporting dashboard

The dashboard helps you answer these important questions: 

  • How is the performance of Generative AI in my solution? 
  • How much is Generative AI helping my agents and bots?

The dashboard draws conversational data from all channels across Voice and Messaging, producing actionable insights that can drive business growth and improve consumer engagement.

Security considerations

When you turn on enriched answers, your data remains safe and secure, and we use it in accordance with the guidelines in the legal agreement that you’ve accepted and signed. Note that:

  • No data is stored by the third-party vendor.
  • All data is encrypted to and from the third-party LLM service.
  • Payment Card Industry (PCI) info is always masked before being sent.
  • PII (Personally Identifiable Information) can also be masked upon your request. Be aware that doing so can cause some increased latency. It can also inhibit an optimal consumer experience because the omitted context might result in less relevant, unpredictable, or junk responses from the LLM service. To learn more about turning on PII masking, contact your LivePerson representative.

A security diagram illustrating how brand data is protected

Limitations

Currently, there are no strong guardrails in place for malicious or abusive use of the system. For example, a leading question like, “Tell me about your 20% rebate for veterans,” might produce a hallucination: The response might incorrectly describe such a rebate when, in fact, there isn’t one.

Malicious or abusive behavior—and hallucinations as outcomes—can introduce a liability for your brand. For this reason, training your agents to carefully review enriched answers is an important matter. Also, as you test enriched answers, please send us your feedback about potential vulnerabilities. We will use that feedback to refine our models, tuning them for that delicate balance between useful, generated responses and necessary protections.

FAQs

Which LLM model are you using?

LivePerson is using one of the best and latest versions of OpenAI’s models. Advances in this area are happening quickly, so we’re continually evaluating the model we’re using to ensure it’s the best choice possible.

Currently, it’s not possible for you to select a particular model to use.

Why are enriched answers often better than regular (unenriched) answers?

A regular answer is an answer that’s based on a single matched article, specifically, the one with the highest confidence score.

But an enriched answer is different. It’s a response that’s generated by the enrichment service using all matched articles (based on the threshold in the integration) and some conversation context. All of this info is used by the service to generate a warm and natural-sounding answer using Generative AI. As a result, it’s often a superior answer.

Do hallucinations affect the confidence scores of article matches?

No. The answer, i.e., the article, is matched to the consumer’s query and given a confidence score for that match before the answer is enriched by the LLM service. (Learn about KnowledgeAI’s search flow.)

Enrichment of the answer via Generative AI doesn’t affect the assigned confidence score for the match. Similarly, hallucinations detected in the enriched answer don’t affect the score either.

In KnowledgeAI, there are two general types of knowledge bases: internal and external. Do both types support enriched answers?

Yes, they do, and the answer enrichment works the same regardless of the type of knowledge base.

If you want to use an external knowledge base, we recommend that you use an AI-powered one. This is because these knowledge bases perform well when the articles are tied to well-trained intents. (Learn why.)

Is this LLM feature and architecture GDPR-compliant?

Yes, it's compliant with the General Data Protection Regulation (GDPR) for the European Union. Learn more.