> ## Documentation Index
> Fetch the complete documentation index at: https://developers.agentx.so/llms.txt
> Use this file to discover all available pages before exploring further.

# Get Missing Results

> Resume support: the idempotency keys a run has already accepted

Returns the idempotency keys this run has already stored, so a client resuming after a crash or
network failure can skip re-running (and re-paying for) already-submitted cases. The Python
SDK's `execute()` derives its keys deterministically as `{runId}:{caseId}:run-{runNumber}` and
uses exactly this endpoint to resume.

<Note>
  The engine cannot know your full case list, so it reports what was **submitted**, not what is
  missing: the legacy `missing` field is always an empty array. Compute missing cases
  client-side as your own expected keys minus `submittedKeys`.
</Note>

## Authentication

<ParamField header="x-api-key" type="string" required>
  Project API key.
</ParamField>

## Path Parameters

<ParamField path="runId" type="string" required>
  Run ID.
</ParamField>

## Response

<ResponseField name="runId" type="string">
  Echoed back.
</ResponseField>

<ResponseField name="submittedKeys" type="array">
  The `idempotencyKey` of every result this run has accepted.
</ResponseField>

<ResponseField name="submittedCount" type="number">
  `submittedKeys.length`.
</ResponseField>

<ResponseField name="missing" type="array">
  Always `[]`. Legacy field kept for older SDK versions; the client computes missing cases
  from `submittedKeys`.
</ResponseField>

## Errors

| Status | Body                           | Meaning                            |
| ------ | ------------------------------ | ---------------------------------- |
| `404`  | `{ "error": "Run not found" }` | No run with this id in the project |

<RequestExample>
  ```bash cURL theme={null}
  curl "http://localhost:4700/api/v1/custom-agent-evaluations/runs/rK7dP2qWx9TzB4mV6nJcE/missing-results" \
    -H "x-api-key: agtx_local_0f3c9a17d2b84e6a5c01b9f4e7d8a2c6431b5f97a0e2d4c8"
  ```

  ```python Resume pattern (Python SDK) theme={null}
  from agentx.evaluations.models import EvaluationResult

  submitted = set(client.evaluations.get_submitted_keys(run_id))

  for i, question in enumerate(questions):
      key = f"{run_id}:case-{i}:run-1"
      if key in submitted:
          continue  # already scored - skip re-running and re-paying
      output = my_agent(question)
      client.evaluations.append_results(run_id, batch_id=f"resume-{run_id}", results=[
          EvaluationResult(
              case_id=f"case-{i}",
              question_index=i,
              run_number=1,
              idempotency_key=key,
              input={"query": question},
              output={"text": output},
          )
      ])
  ```
</RequestExample>

<ResponseExample>
  ```json 200 OK theme={null}
  {
    "runId": "rK7dP2qWx9TzB4mV6nJcE",
    "submittedKeys": [
      "rK7dP2qWx9TzB4mV6nJcE:case-0:run-1",
      "rK7dP2qWx9TzB4mV6nJcE:case-1:run-1"
    ],
    "submittedCount": 2,
    "missing": []
  }
  ```

  ```json 404 Not found theme={null}
  {
    "error": "Run not found"
  }
  ```
</ResponseExample>