Errors

Response format

All error responses follow the same shape:

{
  "status": "error",
  "statusCode": 400,
  "message": "Human-readable error message",
  "errors": []
}

errors is an array of field-level validation errors when applicable (e.g. request body validation failures). It is empty for most errors.

HTTP status codes

Code	Name	When it occurs
`400`	Bad Request	Missing required field, invalid ObjectId, batch too large, invalid parameter value
`401`	Unauthorized	Missing or invalid API key / session cookie
`403`	Forbidden	API key doesn’t have access to the workspace; agent not in workspace
`404`	Not Found	Dataset, run, trace, or agent not found
`409`	Conflict	Run is already in a terminal state (`completed` or `failed`); analysis already running
`429`	Too Many Requests	Rate limit exceeded (see Rate Limits)
`500`	Internal Server Error	Unexpected server error

Common errors

401 Unauthorized

On x-api-key-authenticated routes (SDK, CI/CD, evaluations):

{ "status": "error", "statusCode": 401, "message": "Unauthorized - Missing API key", "errors": [] }
{ "status": "error", "statusCode": 401, "message": "Unauthorized - Invalid API key", "errors": [] }

Check that:

Your x-api-key header is present and matches a valid workspace API key
The key hasn’t been revoked in Settings → API Keys

Session-cookie routes (browser UI, e.g. GET /ingest/traces, tracked-agent management) return a bare 401 with no JSON body when the session is missing or expired.

403 Forbidden

{ "status": "error", "statusCode": 403, "message": "Could not resolve a valid workspace", "errors": [] }
{ "status": "error", "statusCode": 403, "message": "Unauthorized workspace access", "errors": [] }
{ "status": "error", "statusCode": 403, "message": "Agent not found in workspace or insufficient permissions", "errors": [] }

The caller is authenticated, but doesn’t have the required access. Verify the workspaceId belongs to the same workspace as your API key or session.

404 Not Found

{ "status": "error", "statusCode": 404, "message": "Dataset not found", "errors": [] }
{ "status": "error", "statusCode": 404, "message": "Run not found", "errors": [] }

The requested resource doesn’t exist or belongs to a different workspace.

409 Conflict: terminal state

{ "status": "error", "statusCode": 409, "message": "Run is already in a terminal state", "errors": [] }

Returned by POST /results when the run has already been finalized or failed. Check the run status with GET /runs/:id before submitting more results.

400 Bad Request

{ "status": "error", "statusCode": 400, "message": "batchId is required", "errors": [] }
{ "status": "error", "statusCode": 400, "message": "results must be a non-empty array", "errors": [] }
{ "status": "error", "statusCode": 400, "message": "Batch size must not exceed 10", "errors": [] }
{ "status": "error", "statusCode": 400, "message": "Each question must have main_question.query", "errors": [] }
{ "status": "error", "statusCode": 400, "message": "CI/CD is not enabled for this dataset. Enable it in the dataset settings.", "errors": [] }

Rate limits

There is currently no enforced per-key or per-route rate limit on the ingest, CI/CD, or custom evaluation APIs. Don’t rely on this. Design your integration to be resilient to a limit being introduced later, and avoid tight retry loops.

Retrying

Safe to retry on 500 and 429. Not safe to retry POST /results without idempotency keys. Always set idempotencyKey on every result to avoid duplicate scoring on retry.

Get Started

Self-Hosting

Python SDK

Response format

HTTP status codes

Common errors

401 Unauthorized

403 Forbidden

404 Not Found

409 Conflict: terminal state

400 Bad Request

Rate limits

Retrying

​Response format

​HTTP status codes

​Common errors

​401 Unauthorized

​403 Forbidden

​404 Not Found

​409 Conflict: terminal state

​400 Bad Request

​Rate limits

​Retrying

Response format

HTTP status codes

Common errors

401 Unauthorized

403 Forbidden

404 Not Found

409 Conflict: terminal state

400 Bad Request

Rate limits

Retrying