> ## Documentation Index
> Fetch the complete documentation index at: https://developers.agentx.so/llms.txt
> Use this file to discover all available pages before exploring further.

# Webhooks

> Receive CI gate results via HTTP callback

Configure a webhook URL on a dataset to have AgentX POST the gate result to your server immediately after a CI run is finalized.

## Configuration

Set `webhookUrl` on the dataset in **Settings → Datasets → \[dataset] → CI/CD**, or include it when creating a dataset via the API:

```json theme={null}
{
  "ci": {
    "enabled": true,
    "webhookUrl": "https://your-server.com/hooks/agentx"
  }
}
```

## Delivery

* Sent as a single `POST` request with `Content-Type: application/json`
* **Fire-and-forget**: AgentX does not retry on failure or timeout
* Delivery is best-effort; build idempotency into your handler using `run_id`

## Payload

```json theme={null}
{
  "event": "ci_run.finalized",
  "run_id": "6876ccc111aaa222bbb333dd",
  "dataset_id": "6876ddd222bbb333ccc444ee",
  "gate": "pass",
  "pass_rate": 0.875,
  "git_context": {
    "branch": "feat/new-retrieval",
    "commitSha": "a1b2c3d4e5f6",
    "prNumber": 42,
    "repoUrl": "https://github.com/org/repo",
    "triggeredBy": "github-actions"
  },
  "scores": [
    {
      "question_index": 0,
      "rating": 4,
      "justification": "The agent correctly described the password reset flow.",
      "passed": true,
      "input": "How do I reset my password?",
      "output": "Click Forgot Password on the login screen."
    },
    {
      "question_index": 1,
      "rating": 2,
      "justification": "The agent gave an incorrect billing answer.",
      "passed": false,
      "input": "Why was I charged twice?",
      "output": "..."
    }
  ],
  "finalized_at": "2026-07-06T11:00:00.000Z"
}
```

### Fields

| Field          | Type                 | Description                                                |
| -------------- | -------------------- | ---------------------------------------------------------- |
| `event`        | string               | Always `"ci_run.finalized"`                                |
| `run_id`       | string               | CI run ID                                                  |
| `dataset_id`   | string               | Dataset (EvaluationSettings) ID                            |
| `gate`         | `"pass"` \| `"fail"` | Gate result                                                |
| `pass_rate`    | number               | Fraction of questions that passed (0.0–1.0)                |
| `git_context`  | object \| null       | Branch, commit SHA, PR number, etc. (from `create_ci_run`) |
| `scores`       | array                | Per-question scores (see below)                            |
| `finalized_at` | string               | ISO 8601 finalization timestamp                            |

### `scores[n]` fields

| Field            | Type           | Description                                           |
| ---------------- | -------------- | ----------------------------------------------------- |
| `question_index` | number         | 0-based question index                                |
| `rating`         | number         | LLM score, 0–10                                       |
| `justification`  | string         | LLM explanation                                       |
| `passed`         | boolean        | Whether the question passed all threshold gates       |
| `input`          | string \| null | Question text (null when `exposeTestInputs` is false) |
| `output`         | string \| null | Agent's response text                                 |

## Example receiver

```python theme={null}
from flask import Flask, request, jsonify

app = Flask(__name__)

@app.route("/hooks/agentx", methods=["POST"])
def handle_agentx_webhook():
    payload = request.get_json()

    if payload.get("event") != "ci_run.finalized":
        return jsonify({"ok": True})

    run_id = payload["run_id"]
    gate = payload["gate"]
    pass_rate = payload["pass_rate"]
    branch = (payload.get("git_context") or {}).get("branch", "unknown")

    print(f"[{branch}] CI run {run_id}: {gate.upper()} ({pass_rate:.0%})")

    if gate == "fail":
        # notify Slack, update GitHub status, etc.
        notify_team(run_id, payload["scores"])

    return jsonify({"ok": True}), 200
```

```typescript theme={null}
import express from "express";

const app = express();
app.use(express.json());

app.post("/hooks/agentx", (req, res) => {
  const { event, run_id, gate, pass_rate, git_context, scores } = req.body;

  if (event !== "ci_run.finalized") return res.json({ ok: true });

  const branch = git_context?.branch ?? "unknown";
  console.log(`[${branch}] CI run ${run_id}: ${gate.toUpperCase()} (${Math.round(pass_rate * 100)}%)`);

  if (gate === "fail") {
    const failing = scores.filter((s: any) => !s.passed);
    console.log(`Failing questions: ${failing.map((s: any) => s.question_index).join(", ")}`);
  }

  res.json({ ok: true });
});
```

## Verifying delivery

Because AgentX does not sign webhook payloads currently, restrict your endpoint to known AgentX IP ranges or use the `run_id` to fetch the authoritative result from the API:

```python theme={null}
import requests

@app.route("/hooks/agentx", methods=["POST"])
def handle():
    payload = request.get_json()
    run_id = payload["run_id"]

    # Verify with the API
    result = requests.get(
        f"https://api.agentx.so/api/v1/ingest/ci-runs/{run_id}",
        headers={"x-api-key": API_KEY},
    ).json()

    assert result["gate"] == payload["gate"]
    return "", 200
```