Simulate Conversation

The Playground grid tests single-shot answers; Simulate conversation (below its Run button) tests multi-turn behavior. A simulated user - a persona and a goal you write, played by its own model - converses with your current prompt, model, and tools for up to a turn cap (default 6 - the dialog prefills 5 - engine maximum 10):

The simulated user opens the conversation, reacts in character to each agent reply, and ends the conversation itself: GOAL ACHIEVED when the agent genuinely got them there, USER GAVE UP when the persona would realistically walk away, or TURN LIMIT REACHED.
The agent side of each turn is exactly a Playground run - same model settings, same tools, including simulated tool results for schema-only tools.
With a judge scorer starred in the Playground’s Scorers panel, the finished transcript is scored 0-10 against its criteria (reference-free - a simulated conversation has no expected answer); the result names the judge that scored it.
Turns stream into the transcript in real time as each exchange completes, and every finished simulation is kept in the Playground’s History (Runs | Simulations) for later review or re-running.

The Python SDK’s simulate_conversation sends its own default of 5 turns when max_turns is omitted. The same simulation runs from the SDK - useful for scripted persona sweeps (each call blocks for the whole conversation, and each turn is at least two LLM calls - one for the simulated user, one for the agent - plus extra agent calls for any tool rounds, plus the closing judgment). The simulated user runs on its own model, defaulting to the engine’s default judge model:

result = client.evaluations.simulate_conversation(
    model="gpt-4o-mini",
    system_prompt=SYSTEM_PROMPT,
    persona="An impatient customer whose order arrived three weeks late...",
    goal="Get written confirmation of the exact refund amount.",
    max_turns=5,
    tools=[LOOKUP_TOOL],          # optional; simulated results unless an endpoint is set
    agent_name="sim-support-agent",
)
print(result["outcome"], result["sessionId"])   # "goal_achieved" / "gave_up" / "max_turns" / "error"

(The dialog renders those wire values as GOAL ACHIEVED, USER GAVE UP, and TURN LIMIT REACHED.) Recorded as a real session. Unless you untick “Record as a session”, every turn is written through the normal ingest path under one sim-<id> session with metadata.simulated: true - each turn a real span tree (the model call and any tool executions as timed child spans, so the trace dialog’s Execution Timeline works). That means the simulation shows up in Observe → Sessions like any conversation: the built-in Session Baseline Judge (once you’ve enabled it) judges it after it goes idle, session-scoped evaluators score it, and Add to dataset works from the session view - so a bad simulated conversation converts directly into regression test cases.

Pre-Deployment Quality Gate

Build & Run

Datasets & RAG

Playground

Framework Examples

Simulate Conversation