tools=[...]) are the other half, and a vague parameter description causes malformed tool calls the same way a vague prompt causes bad answers. Tools & MCPs in the sidebar’s Manage section gives them the same registry treatment as prompts: version-scoped, edited in a schema-aware JSON editor, with full version history.
Suggest improvement works the same way as a prompt’s, fed by tool-specific evidence: recorded tool calls that failed (success: false, the same field tracer.trace_tool_call captures automatically when your tool raises) and low-rated online-evaluator traces that used the tool. The judge proposes a rewritten definition - typically tightening parameter descriptions and enums against the failures it saw - shown as an editable diff, and nothing publishes until you approve it. Your agent’s code then pulls the published definition instead of hardcoding it, the same source-of-truth pattern as the prompt registry.
The Playground can also add a tool straight from this registry, so you can test a proposed definition against real models before publishing it.
The whole loop is scriptable via client.evaluations.tool_schemas - the tool-definition
analog of the prompt registry’s SDK path:
Tools you haven’t registered yet
Failure classification was never registry-gated:agent-tool-failure:<name> events tally for any traced tool (an operational outcome, not a scorer - it feeds the KPI metrics and this evidence loop directly, without raising triage signals). What registration unlocks is the improvement loop - so the Tool Schemas view has a Registered / Unregistered toggle, with live counts on each side. Unregistered lists tool names observed in recent traces that aren’t in the registry yet, with call/failure counts and a drafted definition ready to review - verbatim from the trace’s metadata.tools when available, otherwise inferred from the arguments the model actually formed (marked as inferred).
The SDK’s raw-client integrations (patch_openai_client, patch_anthropic_client, the LiteLLM logger, Google GenAI) and the LangChain/LangGraph handler capture the request’s tools=[...] definitions automatically and attach them as metadata.tools - so any tool your agent actually passed to the model surfaces here with its real name, description, and parameter schema, ready to register unchanged. Manual tracing can do the same with tracer.trace(..., metadata={"tools": my_tools_array}). Click Register, review the draft, save - and because the failure history was keyed by the tool’s name all along, the accumulated evidence feeds Suggest improvement immediately.
Test endpoint
A registered tool can optionally carry a test endpoint - an http(s) URL the Playground’s “From Tool Schemas” picker uses as the tool’s endpoint default, so a real-loop test of a registered tool needs no re-typing. Calls POST{tool, arguments} and expect {result} back. It is only ever called from a Playground run or a conversation simulation you start: the registry itself stays execution-free, and production traffic always runs your agent’s own real tool. Set or clear it any time via PATCH /evaluate/tool-schemas/:id/test-endpoint.