Simulate Conversation

The Playground grid tests single-shot answers; Simulate conversation (below its Run button) tests multi-turn behavior. A simulated user - a persona and a goal you write, played by its own model - converses with your current prompt, model, and tools for up to a turn cap (default 6, max 10):

The simulated user opens the conversation, reacts in character to each agent reply, and ends the conversation itself: GOAL ACHIEVED when the agent genuinely got them there, USER GAVE UP when the persona would realistically walk away, or TURN LIMIT REACHED.
The agent side of each turn is exactly a Playground run - same model settings, same tools, including simulated tool results for schema-only tools.
With an Evaluator config selected (“Scored with judge” in the Playground), the finished transcript is scored 0-10 against its criteria (reference-free - a simulated conversation has no expected answer); the result names the judge that scored it.
Turns stream into the transcript in real time as each exchange completes, and every finished simulation is kept in the Playground’s History (Runs | Simulations) for later review or re-running.

The same simulation runs from the SDK - useful for scripted persona sweeps (each call blocks for the whole conversation, one LLM call per turn plus the closing judgment):

result = client.evaluations.simulate_conversation(
    model="gpt-4o-mini",
    system_prompt=SYSTEM_PROMPT,
    persona="An impatient customer whose order arrived three weeks late...",
    goal="Get written confirmation of the exact refund amount.",
    max_turns=5,
    tools=[LOOKUP_TOOL],          # optional; simulated results unless an endpoint is set
    agent_name="sim-support-agent",
)
print(result["outcome"], result["sessionId"])   # GOAL_ACHIEVED / USER_GAVE_UP / TURN_LIMIT

Recorded as a real session. Unless you untick “Record as a session”, every turn is written through the normal ingest path under one sim-<id> session with metadata.simulated: true - each turn a real span tree (the model call and any tool executions as timed child spans, so the trace dialog’s Execution Timeline works). That means the simulation shows up in Observe → Sessions like any conversation: the built-in Session Baseline Judge judges it once idle, session-scoped evaluators score it, and Add to dataset works from the session view - so a bad simulated conversation converts directly into regression test cases.