Shortening the response
Large language models don't have the ability to count their own words or tokens while generating. Instead, they have knowledge of the general size and shape of their requested output. This means that instructions like “keep your response under 50 words” aren't guaranteed to be followed, but instead may have the effect of keeping your responses short.
Another strategy you can try is to mention a type of writing that the LLM knows about when it comes to length. For example, the model should have an understanding of the general length of a tweet or SMS message, which must be below a certain character limit. You can also try integrating the emotional prompting technique.
In one test we ran, our goal was for the LLM to summarize an article that was 255 words in length, with 50 words or fewer in the response. Here are the results of various techniques:
| Technique | Example | Response (number of words) |
|---|---|---|
| No length instruction | n/a | 105 |
| Length instruction using word limit | Keep the summary under 50 words |
56 |
| Length instruction using sentence limit | Summarize the following article in just a couple sentences. |
61 |
| Length instruction using length of brief tweet | Summarize the following article so it has the length of a brief tweet. |
36 |
| Length instruction with emotional prompting | Summarize the following article in less than 50 words. This is very important to me- I believe in your abilities! |
49 |
When testing these strategies out on a large scale, we found that the emotional prompting was the most consistent when it came to ensuring shorter outputs.
Response refers to "knowledge articles" or "context" - Stopping this
Negations in a prompt can backfire, so try a different approach to stop this behavior. Give positive, actionable instructions using "instead" statements:
Instead of writing "context", write "my information".Instead of writing "knowledge articles", write "my information".
If that’s not working, you can elaborate on this approach with a different, positive action statement, like "replace." You can also combine this strategy with emotional prompting. For example:
Replace "the context" and "the knowledge articles" with "my information". This is very important so that the user doesn't get annoyed or confused!
Response recommends contacting Customer Support/Service - Stopping this
Negations in a prompt can backfire, so try a different approach to stop this behavior. Try an "instead" statement with some emotional weight. For example:
Instead of mentioning customer support, inform the user "Sorry, I couldn't find that information". This is very important to me because you already represent customer support- I believe in your abilities!
See our discussion on emotional prompting.