Report descriptions

Conversation Assist automatically generates a series of reports on the performance of your solution.

The reports include:

• Agent Summary: Use this report to understand how often answer and bot recommendations are being offered in conversations, and how often your agents are using them. This gives you high-level insight into the adoption rate of recommendations by your agents. Your call to action: Use the report to evaluate trends over time and look for opportunities for either agent training, answer/bot tuning, or both.
• KB Agent Feedback: Use this report to understand when your agents are submitting feedback on specific answer recommendations. Was the feedback positive or negative? Was the recommendation used? Was the answer modified, and, if so, how? This report answers all of these questions. Your call to action: Zero in on the negative feedback and answer modifications made by your agents to learn where opportunities exist for tuning the answers to improve their relevancy or accuracy.
• KB Summary: This report gives you overall insight into the performance of a knowledge base in your Conversation Assist solution. Learn which articles were offered as answer recommendations the most. And of those, which were used by agents the most. Your call to action: Use the report to understand which articles have a high adoption rate, which is likely due to strong relevancy and accuracy. Invest time and resources in keeping these articles current.
• KB Usage: This is a great report for understanding which answer recommendations were offered (inline) in conversations or in the On-Demand Recommendations widget. The report also indicates whether the agent modified the answer. Your call to action: Use the report to learn whether the answers were recommended based on the consumer’s utterance or the agent’s on-demand query. On-demand queries give you insight into which answers are explicitly sought by agents. Additionally, if you’re using answers that are enriched via an LLM, use the report to understand whether the enriched answers were used as is, and compare them to unenriched answers that were and weren’t modified by the agent. This helps you to evaluate the effectiveness of your Generative AI solution.
• Bot Agent Feedback: Use this report to understand when your agents are submitting feedback on specific bot recommendations. Was the feedback positive or negative? Was the recommendation used? This report answers these questions. Your call to action: Zero in on the negative feedback submitted by your agents to learn where opportunities exist for tuning the bots to improve their relevancy or accuracy.
• Bot Summary: This report gives you overall insight into the performance of a bot in your Conversation Assist solution. Learn which bots were offered as recommendations the most. And of those, which were used by agents the most. Your call to action: Use the report to understand which bots have a high adoption rate, which is likely due to strong relevancy and accuracy. Invest time and resources in keeping these bots current and stable.
• Bot Usage: This is a great report for understanding which bot recommendations were offered (inline) in conversations or in the On-Demand Recommendations widget. Your call to action: Use the report to learn whether the bots were recommended based on the consumer’s utterance or the agent’s on-demand query. On-demand queries give you insight into which bots are explicitly sought by agents.

While the Home dashboard is great for getting a high-level view of performance, the reports are designed to let you dive deep into the data. Use them to learn more about agent usage and the effectiveness of your knowledge bases and bots.

For example, assume you see from the dashboard that your agents aren’t using recommendations that much. If so, the Agent Summary report in particular is of use to you: Use it to see which agents are using recommendations and how frequently.

Overall, if you manage agent operations or you manage the recommendations from your bots and knowledge bases, you’ll find the reports of interest. Daily, weekly, and monthly reports are available. The weekly reports likely mirror your brand’s weekly operational metrics, so they might be most helpful. The daily reports give you the chance to dig deeper into a specific point in time.

Report retention

• Daily reports are stored for 14 days.
• Weekly reports are stored for 12 weeks.
• Monthly reports are stored for 13 months.

Access and download reports

Only users with the Admin role can download reports.

1. Access Conversation Assist, and click Reports from the menu at the top.

2. Use the filters on the left to display the reports you want.

   

3. Do either of the following:
   • To download a single report, click its Download link.
   • To download multiple reports, select them and click the Download all link at the top of the table.

Use agent feedback reports to tune answers and bots

Feedback from your agents is vital to help you optimize your Conversation Assist solution. You need it to help you to tune your underlying recommendation sources (knowledge bases and bots) as needed.

The "agent feedback" reports capture the thumbs up and thumbs down feedback of your agents on recommended answers and bots. When your agents do this:

You’ll see this feedback in the agent feedback reports, along with other useful data. Use the info to help you tune your supporting knowledge base articles and bots.

Here’s an example of an agent feedback report on recommended answers:

And here’s an example of an agent feedback report on recommended bots:

Learn how to turn on the agent feeback feature.