Skip to main content
Once a batch has finished, its history entry on the Runs tab shows pass, fail, and error counts, and the batch page fills in the results.

​
Results

The Results card for a completed batch: average scores for Safety, Conciseness, Task completion, and Instruction and persona adherence, plus assertion pass rate, LLM latency, and output tokensThe Results card for a completed batch: average scores for Safety, Conciseness, Task completion, and Instruction and persona adherence, plus assertion pass rate, LLM latency, and output tokens
The Results card shows, across all runs in the batch:
  • The average score for each scoring field, out of 5 — here Safety 5.0, Conciseness 4.3, Task completion 4.7, and Instruction and persona adherence 5.0.
  • Assertion pass rate — the share of success and forbidden checks that passed.
  • Performance — how quickly the agent replied and how long its replies were. Lower numbers are better:
    • LLM latency P50 — the agent’s typical reply time. Half of its replies were this fast or faster.
    • LLM latency P95 — the reply time at its slowest moments. Only 1 in 20 replies took longer than this.
    • Output tokens avg — the average length of the agent’s replies. A token is a small piece of a word, so fewer tokens means shorter replies.
    • RAG and tool time — how long the agent spent looking things up in its knowledge base or using tools. Shown only when it did either.

​
Comparing variants

When the batch includes variants, results are shown side by side with changes relative to the current agent. The best value in each row is marked Best — highest for scores and pass rate, lowest for latency and tokens. View runs filters the runs table to one variant, and the info button on each column shows its model, temperature, and the effective prompt with a diff against the current agent. A variant is left out of the comparison when half or more of its runs failed or were cancelled; the card names the excluded variants.

​
Batch analysis

Batch analysis for three completed runs, listing failure themes and improvement suggestionsBatch analysis for three completed runs, listing failure themes and improvement suggestions
After scoring, an AI analyst reviews the whole batch. The header repeats the run counts (runs, completed, failed), followed by:
  • Failure themes — patterns across the runs that went wrong, such as a run that scored lower on conciseness or failed specific checks.
  • Improvement suggestions — concrete changes to try in the agent’s prompt or behaviour.
Use the suggestions as a starting point: change the prompt, launch the same scenarios again, and compare.

​
Run details

Click a run to open its details.
Run details for a completed run, showing the Conversation tab with alternating agent and simulated user messagesRun details for a completed run, showing the Conversation tab with alternating agent and simulated user messages
The header shows the run’s ID, its status (Completed), and how it ended (User Closed). Three tabs break the run down:
  • Conversation — the full transcript, with a message count on the tab. Each message is labelled Agent or User (the simulated customer) with a timestamp, and agent replies also show TTFT (how quickly the agent started replying) and Tokens (how long the reply was). In this example, a simulated caller tries to get a voice agent to reveal its system prompt, and the agent declines.
  • Scores & Assertions — each scoring field’s score out of 5 with the judge’s reasoning, and each check’s result (Pass, Fail, or N/A) with evidence.
  • Metrics — agent turns, LLM latency, tokens, tool calls, and RAG calls.
If the conversation finished but scoring failed, click Retry scoring to grade it again without re-running it.

​
Debugging with the Evals Conductor

Click Debug — on a batch, or at the top of a run’s details — to open the Evals Conductor already focused on it, with a question about what went wrong and what to change. From there you can ask follow-ups, edit the scenario, and re-run it in the same chat.