> ## Documentation Index
> Fetch the complete documentation index at: https://developers.agentx.so/llms.txt
> Use this file to discover all available pages before exploring further.

# Submit Results

> Submit a batch of agent outputs for synchronous judge scoring

Submits up to 10 agent outputs for a run. Each new result is scored **synchronously inside this
request** - a judge LLM call per result, plus any enabled similarity metrics and code scorers -
so allow a generous client timeout for judged batches. Submissions are idempotent: resubmitting
an `idempotencyKey` the run has already accepted returns the stored score without re-scoring.

<Note>
  Send results as you collect them rather than accumulating everything client-side: every batch
  response carries `liveStatistics`, so the run's average rating is observable while the run is
  still executing.
</Note>

## Authentication

<ParamField header="x-api-key" type="string" required>
  Project API key.
</ParamField>

## Path Parameters

<ParamField path="runId" type="string" required>
  Run ID returned by `POST /runs`.
</ParamField>

## Body

<ParamField body="batchId" type="string" required>
  Identifier for this batch, stored on each result row.
</ParamField>

<ParamField body="results" type="array" required>
  Array of 1-10 result objects. Each result has:

  <Expandable title="Result object fields">
    <ParamField body="idempotencyKey" type="string" required>
      Unique key for this result within the run. Prevents duplicate scoring on retry (the
      engine also enforces uniqueness on `(runId, idempotencyKey)` at insert, so a racing
      duplicate counts as a duplicate, never an error). Recommended format:
      `{runId}:{caseId}:run-{runNumber}`. A result without one is counted in
      `failedValidation` and dropped.
    </ParamField>

    <ParamField body="questionIndex" type="number" default="0">
      0-based index into the dataset's `questions` array. Selects the case's
      `expectedResults`, `judgeGuideline`, and other per-case grading inputs.
    </ParamField>

    <ParamField body="runNumber" type="number" default="1">
      Which repetition this is (1 through `numberOfRequests`).
    </ParamField>

    <ParamField body="caseId" type="string">
      Case identifier. Defaults to `case-{questionIndex}`.
    </ParamField>

    <ParamField body="input" type="object">
      Agent input: `{ "query": string }`.
    </ParamField>

    <ParamField body="output" type="object">
      Agent output: `{ "text": string }`. Required unless `error` is set - a result with
      neither is counted in `failedValidation` and dropped.
    </ParamField>

    <ParamField body="error" type="object">
      Error that occurred: `{ "type": string, "message": string }`. When set, the result is
      stored with status `"failed"` and rating 0; no judge call is made.
    </ParamField>

    <ParamField body="traceId" type="string">
      ID of a trace previously ingested via
      [Submit Trace](/api-reference/tracing/submit-trace) (the Python SDK's
      `client.tracer.trace(..., sync=True)` returns it as `span.trace_id`). Linking a trace
      unlocks trajectory-aware judging (the judge sees the actual tool-call path), the
      trajectory and context match scorers, tool definitions for `toolContext: "detailed"`
      configs, and the dashboard's View Trace on the result.
    </ParamField>

    <ParamField body="retrievalContext" type="string | string[]">
      What the agent actually retrieved for this case - a string or a list of chunk strings.
      Feeds `{context}`-referencing judge prompts (the RAG metric pack) and the context match
      scorer. When absent, the engine falls back to the linked trace's recorded retrieval
      spans, then the case's pinned `retrievalContext`.
    </ParamField>

    <ParamField body="timings" type="object">
      Performance metrics: `{ "latencyMs": number, "inputTokens": number, "outputTokens": number }`.
    </ParamField>

    <ParamField body="isSmokeTestVariant" type="boolean" default="false">
      Marks this row as a paraphrase variant from the run's `smokeTestVariants`. Variant rows
      are excluded from every rating aggregate - `liveStatistics`, the run's `averageRating`
      and `scorerBreakdown`, case statistics, low-scoring cases, and CI-gate aggregates - but
      still count toward the run's `resultCount`.
    </ParamField>

    <ParamField body="smokeTestVariantText" type="string">
      The variant phrasing that was actually asked, when `isSmokeTestVariant` is `true`.
    </ParamField>
  </Expandable>

  <Note>
    The hosted platform's `observableTrace` and `metadata` result fields are ignored by the
    self-host engine - they are accepted without error but not stored. Link a real trace via
    `traceId` instead; it powers strictly more (trajectory judging, timeline, scorers).
  </Note>
</ParamField>

## Response

<ResponseField name="runId" type="string">
  Echoed back.
</ResponseField>

<ResponseField name="batchId" type="string">
  Echoed back.
</ResponseField>

<ResponseField name="accepted" type="number">
  New results scored and stored in this call.
</ResponseField>

<ResponseField name="duplicates" type="number">
  Results skipped because their `idempotencyKey` already existed on this run.
</ResponseField>

<ResponseField name="failedValidation" type="number">
  Results dropped (missing `idempotencyKey`, or both `output.text` and `error` absent).
</ResponseField>

<ResponseField name="status" type="string">
  The run's status, e.g. `"in_progress"`.
</ResponseField>

<ResponseField name="scoredResults" type="array">
  Per-result scores, in submission order (validation-dropped results are omitted):
  `[{ idempotencyKey, rating, justification, status, vectorSimilarity, jaccardSimilarity, bleuScore, rougeScore, codeScorerResults, judgeScorerResults, deduped? }]`.
  `judgeScorerResults` carries one `{ scorerId, name, rating, justification }` entry per
  additional judge scorer (or scorer-group judge member) on the run, `null` when there are
  none.
  `status` is `"scored"`, `"skipped"` (judge could not score - the reason is in
  `justification`), or `"failed"` (the result carried an `error`). Duplicates echo the stored
  first-submission payload with `deduped: true`.
</ResponseField>

<ResponseField name="liveStatistics" type="object">
  Rating aggregate recomputed after this batch:
  `{ averageRating, minRating, maxRating, ratedCount, skippedCount, failedCount }`. Available
  on every batch response, no [Analyze Run](/api-reference/custom-eval/analyze-run) required;
  the Python SDK exposes the same numbers as `run.average_rating` and friends.
</ResponseField>

## Errors

| Status | Body                                                                | Meaning                                                                                                                   |
| ------ | ------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------- |
| `400`  | `{ "error": "batchId must be a non-empty string" }`                 | Missing or invalid `batchId`                                                                                              |
| `400`  | `{ "error": "results must be a non-empty array" }`                  | Missing or empty `results`                                                                                                |
| `400`  | `{ "error": "Batch size must not exceed 10" }`                      | More than 10 results in one call                                                                                          |
| `400`  | `{ "error": "Every results entry must be an object" }`              | A `results` element is `null`, an array, or not an object                                                                 |
| `404`  | `{ "error": "Run not found" }`                                      | No run with this id in the project                                                                                        |
| `409`  | `{ "error": "Run is already in a terminal state" }`                 | The run is `"completed"` or `"failed"`; no more results are accepted                                                      |
| `409`  | `{ "error": "The scorer group grading this run no longer exists" }` | The run's scorer group was deleted mid-run; results are refused rather than silently regraded on the dataset's own rubric |

<RequestExample>
  ```bash cURL theme={null}
  curl -X POST http://localhost:4700/api/v1/custom-agent-evaluations/runs/rK7dP2qWx9TzB4mV6nJcE/results \
    -H "x-api-key: agtx_local_0f3c9a17d2b84e6a5c01b9f4e7d8a2c6431b5f97a0e2d4c8" \
    -H "Content-Type: application/json" \
    -d '{
      "batchId": "batch-001",
      "results": [
        {
          "idempotencyKey": "rK7dP2qWx9TzB4mV6nJcE:case-0:run-1",
          "questionIndex": 0,
          "runNumber": 1,
          "caseId": "case-0",
          "input": { "query": "How do I reset my password?" },
          "output": { "text": "Click Forgot Password on the login screen." },
          "timings": { "latencyMs": 1240, "inputTokens": 150, "outputTokens": 45 },
          "traceId": "mJ3vQ8pTr2LqYw6bZk9Xd"
        }
      ]
    }'
  ```
</RequestExample>

<ResponseExample>
  ```json 200 OK theme={null}
  {
    "runId": "rK7dP2qWx9TzB4mV6nJcE",
    "batchId": "batch-001",
    "accepted": 1,
    "duplicates": 0,
    "failedValidation": 0,
    "status": "in_progress",
    "scoredResults": [
      {
        "idempotencyKey": "rK7dP2qWx9TzB4mV6nJcE:case-0:run-1",
        "rating": 8,
        "justification": "The agent correctly identified the password reset flow and gave clear step-by-step instructions.",
        "status": "scored",
        "vectorSimilarity": null,
        "jaccardSimilarity": 0.62,
        "bleuScore": null,
        "rougeScore": null,
        "codeScorerResults": null,
        "judgeScorerResults": null
      }
    ],
    "liveStatistics": {
      "averageRating": 8,
      "minRating": 8,
      "maxRating": 8,
      "ratedCount": 1,
      "skippedCount": 0,
      "failedCount": 0
    }
  }
  ```

  ```json 409 Terminal state theme={null}
  {
    "error": "Run is already in a terminal state"
  }
  ```
</ResponseExample>