Fastest way to run test on different models against your datasets with any prompt, tools, and MCPs
- Edit a multi-turn prompt - system message, optional few-shot user/assistant example turns - freely, or load a saved Prompt Registry version as a starting point.
- Select one or more OpenAI, Anthropic, and/or Gemini models, from the same dashboard-managed model catalog Model portability uses.
- Pick an existing dataset and check which of its questions to include.
- Run grid fires the whole (question × model) grid for real - each cell independently shows the actual output, latency, tokens, and estimated cost, plus a judge rating whenever that question has an expected answer, and each of the dataset’s enabled code scorers’ results alongside it. At most 4 cells run at once (a fixed concurrency cap, so a large grid doesn’t trip your provider’s rate limits) - the rest queue and fill in as earlier ones finish.
Tools
The Playground tests prompt and tools together. Add a tool ad hoc, or pull one straight from the Tool Schemas registry with the From Tool Schemas picker - name, description, and parameter schema come in as the registry’s current published version. The endpoint URL is optional:- Blank (simulated): the tool is still sent to the model, and its calls return a canned
{simulated: true}result - so whether the prompt/model choose the tool and form valid arguments is fully testable with zero infrastructure. The cell’s tool-call trace shows the arguments and marks the result simulated, so nothing pretends to be a real lookup. - Set: each call POSTs
{tool, arguments}to your real service and feeds the actual response back to the model - the full loop, including how the prompt handles real tool outputs.