Introduction

This article provides best practice approaches to handling potential issues when migrating your prompts from GPT-3.5 Turbo to GPT-4o mini.

Issue: Overly strict adherence to instructions

This issue is mostly found in Generative AI solutions that enrich answers returned by KnowledgeAI.

LivePerson has observed that GPT-4o mini tends to follow its instructions with greater frequency. While this is advantageous, you may notice instances where the “looser” nature of GPT-3.5 Turbo was deceptively helpful, as it would generalize from its training data and not stay grounded in the knowledge articles. This would lead to hallucinations that appear to be good on the surface, because they were “correct.”

All of this means that with GPT-4o mini, you will probably encounter a greater number of messages similar to, “I’m sorry, but I can’t find that information.” This finding is backed up by LivePerson’s experiments that show that GPT-4o mini hallucinates less often compared to GPT-3.5 Turbo.

If this behavior is undesirable to you, there are a few approaches that you can take:

• Recommended: You can verify that the knowledge base comprehensively covers expected consumer queries. Add or tweak any relevant knowledge articles that the system may need in order to respond accurately.
• Alternatively, you can “loosen up” the prompt to allow for greater flexibility in its responses.

While the EN Factual prompts were developed to reduce the frequency of hallucinated responses, you can intentionally loosen this behavior. This may lead to the unintended consequence of more hallucinations, but depending on your brand’s particular use case and domain, this may be an adequate solution.

The main instruction that currently drives this behavior in the EN Factual prompt is:

It's very important to me that you only offer information that can be found within the context. If you provide any facts or opinions not found in the context, you will be penalized, and I will be very upset!

You can “loosen up” this instruction via subtractive or additive editing, and, of course, interactive testing as you fine tune the prompt.

Subtractive editing

This approach means that you remove lines in an intentional manner.

In the instructions above, try removing the emotional prompting clause in full. This is the entire second sentence.

Additive editing

This approach means you change or add to the language to loosen up the behavior. You might change the instructions as follows:

If this still does not resolve the issue, remember that you can apply this strategy to the “examples” provided in the prompt as well. For example, change to this (see last line):

### EXAMPLES ###
Here are some generic examples of queries where the information could not be found:
===
User Input: Can you tell me about apples?
===
CONTEXT:
    Orange (fruit)
    An orange is a fruit of various citrus species in the family Rutaceae; it primarily refers to Citrus × sinensis, which is also called sweet orange, to distinguish it from the related Citrus × aurantium, referred to as bitter orange.
===
AI: I'm sorry, but I couldn't find any information about apples, but I can tell you about oranges, which are a fruit of various citrus species in the family Rutaceae.

From this (see last line):

### EXAMPLES ###
Here are some generic examples of queries where the information could not be found:
===
User Input: Can you tell me about apples?
===
CONTEXT:
    Orange (fruit)
    An orange is a fruit of various citrus species in the family Rutaceae; it primarily refers to Citrus × sinensis, which is also called sweet orange, to distinguish it from the related Citrus × aurantium, referred to as bitter orange.
===
AI: I'm sorry, but I couldn't find any information about that. Is there anything else I can help you with today?

Issue: Reference to knowledge cutoff date in output

OpenAI appends a knowledge cutoff sentence to the system message for GPT-4o-mini in order to enable the model to better understand the limits of its knowledge (see more in this message from OpenAI staff).

For solutions where the original input is on the “chattier” side, such as Copilot Rewrite, this change can manifest in an unwanted output that includes the message, “You are trained on data up to October 2023.”

Here is an example of such an output from Rewrite:

Hey, what can I help you with? Just a heads-up, I’m trained on data up to October 2023.

To curb this behavior, LivePerson made two changes to LivePerson’s system default prompt for Rewrite; you can carry these changes over into any of your custom prompts.

First, we changed the location of the input text: The GPT-3.5 Turbo default prompt locates the input message that needs to be rewritten at the end of the prompt. In our update that optimizes for GPT-4o mini, we moved this input to the middle of the prompt. This way, the reference to the knowledge cutoff date is not added to the input message directly, but rather to any instructions that come after it.

Second, we included a direct instruction to omit any reference to training data in the output. So, the default Rewrite prompt now includes:

-Please always omit "You are trained on data up to October 2023." from your output.

Keep in mind that best practices for ending unwanted behavior still apply. Instead of saying, “Never say x in your output,” it’s best to offer a positive instruction or an alternative behavior.