Scale AI agents for high availability and load

If you're testing your AI agents used in Production, it's important to ensure your AI agent can effectively handle the total conversation load, accounting for both live customer traffic and the additional load generated by simulations.

For LivePerson AI agents (built and deployed in LivePerson Conversation Builder), the primary scaling mechanism is the agent connector.

For third-party AI agents used and managed through LivePerson’s Conversational Cloud, similar scaling principles apply. In this case, a connector shouldn't handle more than 250 conversations, so add additional connectors as needed.

For third-party AI agents used and managed through other contact center solutions, similar scaling principles also apply. Refer to the documentation for the contact center solution for guidance.

Optionally test AI agent transfer paths without triggering transfer

Sometimes you want the AI agents involved in simulations to run through their conversational journeys end to end, including transfers to other bots or live agents.

Other times you don't; you want to test just the AI agent itself. It's enough to just display info about the transfer that would have occurred without actually performing the hand-off.

To test only the AI agent and not the end-to-end journey of a LivePerson Conversation Builder AI agent, follow our best practice. If the AI agent is third-party, the solution in your vendor's bot building tool will be similar.

Manage human agent load via CCaaS tools

The introduction of simulations shouldn't overwhelm your human agents. Rely on the built-in workload management tools within your CCaaS solution to manage their load. These features allow you to set strict concurrency limits, ensuring agents only handle a manageable number of conversations at once. By automating these caps, you protect your team from burnout while maintaining high-quality service for every customer, whether real or synthetic.

Blind versus open testing

When using Syntrix to evaluate human agents, you must choose between blind testing (where your agents may be unaware they are interacting with a synthetic customer) and open testing (where the simulation is disclosed).

While blind testing provides the most authentic assessment regarding an agent’s natural performance, it carries cultural risks regarding surveillance and employee trust.

Conversely, open testing fosters a psychological "safe zone" ideal for coaching and upskilling without performance anxiety.

We recommend that you prioritize open testing for training purposes to build confidence, and reserve blind testing for compliance audits, ensuring that agents are informed beforehand that synthetic quality assurance checks are a part of standard operations.

When you create a simulation

Focus on objectives, not numbers when choosing personas

Instead of focusing on the number of personas to include in a simulation, prioritize the specific results that you want to achieve. To find the right balance, define your main testing goal:

Testing goal Number of personas
Agent readiness across customer types 5-10 personas. Expose your agents to the full spectrum during training.
A specific edge case or failure mode 1-2 specific personas that reliably trigger the behavior you're investigating.
A specific AI agent interaction (repeatedly and reliably) 1 persona + 1 scenario. This minimizes variables, so you can isolate the flow being tested.
AI agent quality and helpfulness (for example, an FAQ agent) 5-10 personas. Test that the bot handles different communication styles well.
Comprehensive QA As many distinct personas as practical. Maximize variance for realistic coverage.

Key principle: Prioritize unique personas over a high volume of similar ones. While more personas increase variety and realism, adding those with overlapping traits provides little extra value.

Logically align scenarios and personas

Pair personas with scenarios that naturally align. Avoid unlikely combinations—such as a hostile customer calling to join a positive promotion—as they produce unrealistic data and lack real-world utility.

Validate your scenario and persona strategy

Ensure every combination is realistic and directly supports the specific testing results you aim to achieve.

  • Negative scenarios: Pair disputes, outages, and complaints with any persona, from friendly to irate. Use the persona to dictate the intensity of the frustration rather than the situation itself.
  • Positive scenarios: Match sign-ups and inquiries with neutral or positive personas. Reserve hostile pairings for rare edge-case testing rather than standard simulations.
  • Complex scenarios: Pair technical or multi-step tasks with confused or impatient personas. This stress-tests an agent’s ability to guide difficult customers through challenging processes.

Ignore response speed

  • When training human agents: Turn this setting off, so the synthetic customer respects each persona's configured response speed. Realistic reply pacing is part of what makes the training experience representative of real customer interactions.
  • When testing AI agents: Turn this setting off if you have AI agent or bot behaviors that are responsive to different response speeds. But when the response time of the AI agent isn't meaningful, turn it on so that the run completes faster.

Total conversations

To ensure your performance data is meaningful, aim for at least 5–10 conversations per agent. Calculate your total test volume by scaling this target against your active agent count.

Testing/training goal Recommended value for "Total conversations"
Quick validation or smoke test 3-5 conversations is enough to validate flows are working before committing to a full run.
Agent training Scale to your agent count to ensure everyone gets meaningful exposure.
Bot testing Scale volume to bot complexity. Simple flows might require fewer conversations to surface issues.

Max. concurrent conversations

Getting this right is critical.

  • When training human agents: Set concurrency slightly above total agent capacity (agents × conversations per agent) to keep a small queue. This ensures agents always have work without a long backlog. For example, 5 agents at 3 concurrent each = 15 capacity, so 18–20 concurrency works well.
  • When testing AI agents: Bots can handle much higher concurrency. Start at 10–15 and increase to stress-test throughput, but monitor for rate-limit errors.

Max. conversation turns

This is the safety net that prevents conversations from looping indefinitely.

When this limit is reached, the conversation is closed automatically by the system. This can negatively affect assessment scores, so choose a limit that allows legitimate conversation flows to finish.

  • General recommendation: 20–30 turns for typical conversations.
  • When testing AI agents: Set lower (10–15) to detect infinite loops quickly. If a bot can't resolve an issue in 15 turns, that's a finding in itself.
  • For complex, multi-step scenarios: Increase to 30 if the scenario requires extensive back-and-forth, for example, technical troubleshooting with multiple steps.

Before you run a simulation

  • Ensure agent availability. Confirm your human agents are logged in—or your AI agents are deployed and online—for all routing targets in the scenarios. If a conversation cannot route, it will eventually time out and consume your allotment of total conversations in the simulation.
  • Pilot your test. Run 3–5 conversations before starting a full-scale test. Use these initial runs to catch configuration errors, such as incorrect routing targets.

During a run

Monitor the conversations in the agent workspace

Watch the first 5–10 conversations closely; look for:

  • In-progress conversations that have with no turns progressing (routing issue)
  • Rapid closures with very few conversation turns (bot immediately closing, or agent rejecting)

Stop early if something is wrong

If you see systematic errors (wrong routing target, broken bot, agent confusion, etc.), stop the simulation run instead of letting bad conversations run their course. Fix the issue and re-run the simulation. Allowing flawed runs to continue wastes valuable agent time and compromises the integrity of your reports.

After a run

Analyze at three levels

  • Macro (overall simulation): Start with the summary. Did the simulation succeed? Look for broad patterns and system-wide trends.
  • Mid (agent performance): Drill into individual results to identify exactly who needs coaching and on which specific skills, and which AI agents need tuning.
  • Micro (conversation deep-dive): Review specific failures. Use these transcripts as concrete, real-world examples for training sessions.

Track long-term progress

Run the same simulations periodically to measure ROI. Consistent benchmarking is the only way to verify if your bot updates, process changes, or agent coaching programs are actually working.

Close the feedback loop

Use your results to sharpen your scenarios and personas:

  • Too easy? If everyone passes, increase the scenario difficulty or tighten your agent goals.
  • Universal failures? If most agents fail the same metric, decide if the goal is unrealistic or if you've found a critical training gap.
  • Repetitive outcomes? If multiple personas always lead to the same result, your personas are too generic. Sharpen their unique traits to ensure they sound like distinct individuals.