curl -X POST http://localhost:4700/api/v1/ingest/traces/mJ3vQ8pTr2LqYw6bZk9Xd/evaluate \
-H "x-api-key: agtx_local_0f3c9a17d2b84e6a5c01b9f4e7d8a2c6431b5f97a0e2d4c8" \
-H "Content-Type: application/json" \
-d '{ "datasetId": "dS4tG7hNb2VxZ8kQ5wMyA" }'
result = client.tracer.evaluate_trace(
trace_id="mJ3vQ8pTr2LqYw6bZk9Xd",
dataset_id="dS4tG7hNb2VxZ8kQ5wMyA",
)
print(result["rating"])
print(result["justification"])
{
"run_id": "rK7dP2qWx9TzB4mV6nJcE",
"trace_id": "mJ3vQ8pTr2LqYw6bZk9Xd",
"rating": 8,
"justification": "The agent correctly described the password reset flow and provided accurate step-by-step instructions.",
"status": "completed"
}
{
"error": "Trace not found"
}
Tracing
Evaluate Trace
Score a recorded trace against a dataset or grading config without re-running the agent
POST
/
api
/
v1
/
ingest
/
traces
/
{traceId}
/
evaluate
curl -X POST http://localhost:4700/api/v1/ingest/traces/mJ3vQ8pTr2LqYw6bZk9Xd/evaluate \
-H "x-api-key: agtx_local_0f3c9a17d2b84e6a5c01b9f4e7d8a2c6431b5f97a0e2d4c8" \
-H "Content-Type: application/json" \
-d '{ "datasetId": "dS4tG7hNb2VxZ8kQ5wMyA" }'
result = client.tracer.evaluate_trace(
trace_id="mJ3vQ8pTr2LqYw6bZk9Xd",
dataset_id="dS4tG7hNb2VxZ8kQ5wMyA",
)
print(result["rating"])
print(result["justification"])
{
"run_id": "rK7dP2qWx9TzB4mV6nJcE",
"trace_id": "mJ3vQ8pTr2LqYw6bZk9Xd",
"rating": 8,
"justification": "The agent correctly described the password reset flow and provided accurate step-by-step instructions.",
"status": "completed"
}
{
"error": "Trace not found"
}
Grades the trace’s recorded input/output against a dataset’s (or standalone grading config’s)
criteria, as a real one-result evaluation run. The agent is not called again. The run is
stored with
runSource: "trace-eval" and shows up in Evaluate > Runs like any other, but it is
never used as a no-regression baseline for full runs.
Scoring uses the full run-scoring stack: the LLM judge (which needs a judge provider key
configured), the trace’s recorded tool trajectory, retrieval context, and any code scorers on
the config.
Need a
traceId? Submit Trace returns one on every
call. In the Python SDK, only client.tracer.trace(..., sync=True) (context manager) gets it
back synchronously; the decorator form is fire-and-forget and never returns an id.Authentication
string
required
Project API key.
Path Parameters
string
required
ID of the trace to evaluate, returned by
POST /ingest/traces.Body
string
required
ID of the dataset or standalone evaluation-settings config to score against. Datasets
grade with their own criteria; a grading config brings its own judge prompt/model.
The Python SDK’s
question_index argument is sent on the wire but ignored by the
self-host engine - the trace is scored against the config’s general criteria, not one
question’s expectedResults.Response
This response keeps its historical snake_case keys (
run_id, trace_id) - they are the
actual wire keys, matching what the Python SDK’s evaluate_trace() reads.string
ID of the created one-result evaluation run.
string
Echoed back for confirmation.
number | null
Score from 0 (poor) to 10 (excellent).
null when the judge could not score (for example,
no judge LLM key is configured) - the run’s result row records the reason.string | null
LLM-generated explanation of the score.
string
Always
"completed" on success.Errors
| Status | Body | Meaning |
|---|---|---|
400 | { "error": "traceId and datasetId are required" } | Missing path or body parameter |
404 | { "error": "Trace not found" } | No such trace in this project |
404 | { "error": "Dataset or evaluator config not found" } | datasetId matches neither a dataset nor a grading config |
curl -X POST http://localhost:4700/api/v1/ingest/traces/mJ3vQ8pTr2LqYw6bZk9Xd/evaluate \
-H "x-api-key: agtx_local_0f3c9a17d2b84e6a5c01b9f4e7d8a2c6431b5f97a0e2d4c8" \
-H "Content-Type: application/json" \
-d '{ "datasetId": "dS4tG7hNb2VxZ8kQ5wMyA" }'
result = client.tracer.evaluate_trace(
trace_id="mJ3vQ8pTr2LqYw6bZk9Xd",
dataset_id="dS4tG7hNb2VxZ8kQ5wMyA",
)
print(result["rating"])
print(result["justification"])
{
"run_id": "rK7dP2qWx9TzB4mV6nJcE",
"trace_id": "mJ3vQ8pTr2LqYw6bZk9Xd",
"rating": 8,
"justification": "The agent correctly described the password reset flow and provided accurate step-by-step instructions.",
"status": "completed"
}
{
"error": "Trace not found"
}