> ## Documentation Index
> Fetch the complete documentation index at: https://developers.agentx.so/llms.txt
> Use this file to discover all available pages before exploring further.

# CI/CD Overview

> Gate merges and deploys on evaluation quality with the run gate endpoint

CI/CD evaluation on AgentX is an ordinary
[custom evaluation run](/api-reference/custom-eval/overview) plus one extra call: after the run
finalizes, `GET /runs/:id/gate` turns its ratings into a binary **passed / failed** verdict
your pipeline can exit on. Your agent runs in your infrastructure; the engine scores results
and computes the gate.

<Note>
  There is no separate CI run type or `/ci-runs` endpoint family on the self-host engine - a
  CI run is a normal evaluation run, so it appears in Evaluate → Runs with full results,
  analysis, and history like any other. (The hosted platform additionally offers a lightweight
  CI Ingest API, which the SDK's `run_eval()` wraps - see [SDK CI/CD](/sdk/ci-cd).) The
  [Python SDK](/sdk/ci-cd) wraps the whole self-host flow too, including `report.gate(...)`
  and the pytest helper `agentx.testing.assert_evaluation`.
</Note>

## How it works

```
CI pipeline
    │
    ├─ POST /custom-agent-evaluations/runs               ← create the run
    │
    ├─ (run your agent per dataset case)
    │
    ├─ POST /custom-agent-evaluations/runs/:id/results   ← submit + judge-score each batch
    │
    ├─ POST /custom-agent-evaluations/runs/:id/finalize  ← close the run
    │
    └─ GET  /custom-agent-evaluations/runs/:id/gate      ← verdict: passed true/false
```

## Gate checks

The gate runs one or both of these checks (at least one is required) and passes only when
**every requested check passes**:

1. **Absolute floor** (`failUnder=<0-10>`): the run's average rating must be at or above the
   floor. A run with no rated results fails this check.
2. **No regression** (`noRegression=true`): the average must not have dropped more than
   `tolerance` (default `0.5`) below the baseline run's average. The baseline is the dataset's
   most recent earlier completed rated run **with the same grading identity** - same grading
   config (`evaluationSettingsId`), same scorer group, and same split - so switching how a run
   is graded starts a fresh baseline rather than comparing incomparable scores. Judge scores
   are noisy, so an exact comparison would flake builds on variance; tune `tolerance` to your
   dataset. With no matching prior run, the check passes explicitly ("nothing to regress
   against"). Single-trace evaluations are never used as the baseline.

The verdict is computed fresh from stored ratings on every call, so re-running a failed CI job
re-evaluates against current state.

## Gate history and alerting

`record=true` appends the verdict to the project's gate history - what the dashboard's CI page
lists - along with an optional `caller` label (e.g. `"github-actions"`). A recorded **failed**
gate also fires the webhook channels configured on the project's monitoring profiles, so a
gate blocking a merge can reach Slack. Preview calls omit `record` and stay compute-only.

## Smoke splits

Tag a subset of dataset cases with a named split (`main_question.splits: ["smoke"]`) and create
the run with `"split": "smoke"` to gate PRs on a fast subset while nightly jobs run the full
dataset. Case indexes are preserved, so per-case comparisons line up across split and full
runs.

## Endpoints

| Method | Path                                          | Description                                                          |
| ------ | --------------------------------------------- | -------------------------------------------------------------------- |
| `POST` | `/custom-agent-evaluations/runs`              | [Create the run](/api-reference/ci-cd/create-run)                    |
| `GET`  | `/custom-agent-evaluations/datasets/:id`      | [Read the dataset's test cases](/api-reference/ci-cd/get-test-cases) |
| `POST` | `/custom-agent-evaluations/runs/:id/results`  | [Submit results](/api-reference/ci-cd/submit-result)                 |
| `POST` | `/custom-agent-evaluations/runs/:id/finalize` | [Finalize the run](/api-reference/ci-cd/finalize-run)                |
| `GET`  | `/custom-agent-evaluations/runs/:id/gate`     | [Compute the gate verdict](/api-reference/ci-cd/get-gate)            |

All endpoints share the engine's global error envelope for authentication (`401`) and rate
limiting (`429`) - see [Errors](/errors).

## Minimal pipeline example

```bash theme={null}
BASE=http://localhost:4700/api/v1/custom-agent-evaluations
KEY="x-api-key: $AGENTX_API_KEY"

RUN_ID=$(curl -s -X POST "$BASE/runs" -H "$KEY" -H "Content-Type: application/json" \
  -d '{"datasetId":"'"$DATASET_ID"'","runSource":"sdk"}' | jq -r .runId)

# ... run your agent per case and POST batches to $BASE/runs/$RUN_ID/results ...

curl -s -X POST "$BASE/runs/$RUN_ID/finalize" -H "$KEY" > /dev/null

PASSED=$(curl -s "$BASE/runs/$RUN_ID/gate?failUnder=7&noRegression=true&record=true&caller=github-actions" \
  -H "$KEY" | jq -r .passed)

[ "$PASSED" = "true" ] || { echo "CI gate FAILED"; exit 1; }
```