> ## Documentation Index
> Fetch the complete documentation index at: https://developers.agentx.so/llms.txt
> Use this file to discover all available pages before exploring further.

# GitHub Actions

> Gate PRs with AgentX CI/CD evaluation

Block merges and deploys when your agent's pass rate drops below your configured threshold — using GitHub Actions as the runner.

## Prerequisites

1. A dataset with **CI/CD enabled** in AgentX settings.
2. Your `AGENTX_API_KEY` stored as a GitHub Actions secret.
3. Your `AGENTX_DATASET_ID` — visible in the dataset settings page URL.

## Workflow template

Save as `.github/workflows/eval.yml` in your repository:

```yaml theme={null}
name: AgentX Evaluation Gate

on:
  pull_request:
    branches: [main]

jobs:
  eval:
    name: Evaluate agent
    runs-on: ubuntu-latest

    steps:
      - uses: actions/checkout@v4

      - uses: actions/setup-python@v5
        with:
          python-version: "3.11"

      - name: Install dependencies
        run: pip install agentx-python

      - name: Run AgentX evaluation gate
        env:
          AGENTX_API_KEY: ${{ secrets.AGENTX_API_KEY }}
        run: |
          python -c "
          import os, sys
          from agentx import AgentX, CIGateFailure

          # Import your agent
          from your_package import my_agent

          client = AgentX.from_env()

          try:
              result = client.tracer.run_eval(
                  dataset_id=os.environ['AGENTX_DATASET_ID'],
                  agent_fn=my_agent,
                  agent_name='my-agent',
                  git_context={
                      'branch': os.environ.get('GITHUB_HEAD_REF', ''),
                      'commit_sha': os.environ.get('GITHUB_SHA', ''),
                      'pr_number': os.environ.get('GITHUB_REF', '').split('/')[-2],
                      'repo_url': 'https://github.com/' + os.environ.get('GITHUB_REPOSITORY', ''),
                      'triggered_by': 'github-actions',
                  },
                  fail_on_gate=True,
              )
              print(f'Gate PASSED — {result.pass_rate:.0%} passed')
          except CIGateFailure as e:
              print(f'Gate FAILED — {e.result.pass_rate:.0%} passed ({e.result.passed_questions}/{e.result.total_questions})')
              sys.exit(1)
          "
        env:
          AGENTX_DATASET_ID: ${{ secrets.AGENTX_DATASET_ID }}
```

## Using a dedicated eval script

For larger test setups, use a separate script:

```python theme={null}
# scripts/eval_gate.py
import os
import sys
from agentx import AgentX, CIGateFailure
from your_package import my_agent  # import your agent

client = AgentX.from_env()

try:
    result = client.tracer.run_eval(
        dataset_id=os.environ["AGENTX_DATASET_ID"],
        agent_fn=my_agent,
        agent_name="my-agent",
        concurrency=4,
        git_context={
            "branch": os.environ.get("GITHUB_HEAD_REF", ""),
            "commit_sha": os.environ.get("GITHUB_SHA", ""),
            "pr_number": os.environ.get("GITHUB_REF", "").split("/")[-2],
            "repo_url": "https://github.com/" + os.environ.get("GITHUB_REPOSITORY", ""),
            "triggered_by": "github-actions",
        },
        fail_on_gate=True,
    )
    print(f"Gate PASSED — {result.pass_rate:.0%} ({result.passed_questions}/{result.total_questions})")

except CIGateFailure as e:
    r = e.result
    print(f"Gate FAILED — {r.pass_rate:.0%} ({r.passed_questions}/{r.total_questions})")
    for v in r.violations:
        print(f"  Q{v.question_index}: {v.metric} = {v.actual:.2f} (threshold {v.threshold:.2f})")
    sys.exit(1)
```

In the workflow:

```yaml theme={null}
- name: Run AgentX evaluation gate
  run: python scripts/eval_gate.py
  env:
    AGENTX_API_KEY: ${{ secrets.AGENTX_API_KEY }}
    AGENTX_DATASET_ID: ${{ secrets.AGENTX_DATASET_ID }}
```

## Adding required status checks

To **block merges** when the gate fails:

1. Go to your GitHub repo → **Settings → Branches**
2. Add a branch protection rule for `main`
3. Enable **Require status checks to pass before merging**
4. Add `eval / Evaluate agent` as a required check

## Viewing results

Every CI run is recorded in the **CI Runs tab** in your AgentX workspace, showing:

* Gate result (PASS / FAIL) badge
* Per-question scores and justifications
* Branch, commit SHA, and PR number
* Threshold violations

## Environment variables

| Variable            | Description                             |
| ------------------- | --------------------------------------- |
| `AGENTX_API_KEY`    | Your workspace API key                  |
| `AGENTX_DATASET_ID` | Evaluation dataset ID with CI enabled   |
| `GITHUB_HEAD_REF`   | PR branch name (set by GitHub Actions)  |
| `GITHUB_SHA`        | Commit SHA (set by GitHub Actions)      |
| `GITHUB_REF`        | Full ref string (set by GitHub Actions) |
| `GITHUB_REPOSITORY` | `org/repo` (set by GitHub Actions)      |