> ## Documentation Index
> Fetch the complete documentation index at: https://developers.agentx.so/llms.txt
> Use this file to discover all available pages before exploring further.

# Automation Rules

> Route matching traffic to human review, into a dataset, or out to a webhook

A rule watches incoming traces and routes the ones that match somewhere useful. Three things it
can do, one per rule:

| Action              | What lands where                                                                            |
| ------------------- | ------------------------------------------------------------------------------------------- |
| **Send to review**  | The trace joins the [human review queue](/monitor/review-queue) for a good/bad label.       |
| **Add to dataset**  | The trace becomes a dataset case, with the expected result left blank for a human to write. |
| **Post to webhook** | A small JSON payload (Slack-compatible) with the rule name and trace id.                    |

Rules live under **Manage → Rules**.

## Rules route, scorers score

This is the distinction worth getting right before you build one.

A **scorer** scores traffic, and owns its own sampling, because what a judge costs is a scorer
question. A **rule** routes traffic, and never scores anything. (A third thing notifies on
**aggregates** rather than on single traces - failure rate, p95 latency, spend - and pages an
on-call channel: that is an [alert rule](/monitor/alert-rules), which lives on this same page
below the routing rules.)

The consequence people care about: **enabling a rule cannot change what your judges cost or what
verdicts they produce.** Routing is cheap - no LLM call - so the only reason to sample below 100%
of whatever matches the filter is queue volume, not the bill. (The dashboard starts a new rule at
10% for exactly that reason; a rule created via the API with no `sampleRate` runs at 100%.) Dial
the rule down when the queue fills up.

## Building one

A rule is a filter, a sample rate, and an action.

The filter has typed fields rather than an expression language:

* **Any trace** or **errored only**
* **Contains** - text that must appear in the input or output
* **Model** - exact match

(An agent scope - all agents, or a chosen few - also exists on the wire as `scopeMode`/`agentIds`
and is enforced at run time, but the rule editor doesn't expose a picker for it yet.)

This is a deliberate limit. A rule whose filter cannot match anything is visibly wrong in the
editor, instead of parsing cleanly into "match nothing" and looking healthy while doing nothing.

The sample rate then applies to whatever survived the filter: `10%` of matching traces, not 10%
of all traffic.

## Honest activity

Every rule shows either `fired 24x · last 3 minutes ago` or, plainly, **never fired**. A rule that
has never matched anything looks different from one working hard, because the most common failure
here is not a broken rule - it is a rule that quietly matches nothing while everyone assumes
coverage exists.

## What a rule never sees

Two kinds of traffic skip rules entirely at ingest: traces sent with `monitor=False`, and
traces with `source="eval-run"` (unless they explicitly set `monitor=True`) - the same opt-outs
that skip every other ingest-time check.

And two matches deflect silently instead of firing. A **dataset** action does nothing when the
case would be a duplicate or the dataset is already at its case cap; a **review** action does
nothing when the trace is already queued or the queue is at its 200-item pending cap. Neither
increments the rule's fired count - so **never fired** can also mean "always deflected", not
just "never matched". The deflections are logged engine-side.

## Two rules worth having

<CardGroup cols={2}>
  <Card title="Sample ordinary traffic" icon="filter">
    10% of everything → **Send to review**. This is what keeps judge calibration measured on
    normal traffic, not only on what got flagged.
  </Card>

  <Card title="Harvest failures" icon="bug">
    Errored only → **Add to dataset**. Real production failures become regression cases. The
    expected answer stays blank on purpose: what the agent said is what happened, not what
    should have happened.
  </Card>
</CardGroup>

## API

```bash theme={null}
curl -X POST "$AGENTX_URL/api/v1/agent-monitoring/rules" \
  -H "x-api-key: $AGENTX_API_KEY" -H "content-type: application/json" \
  -d '{
        "name": "Sample support traffic",
        "action": "review",
        "sampleRate": 0.1,
        "filter": {"status": "any", "contains": "order"}
      }'
```

Update and delete use the same path with the rule's id: `PUT /agent-monitoring/rules/:id`
(partial - only the fields you send change) and `DELETE /agent-monitoring/rules/:id`.

The Python SDK wraps all of it:

```python theme={null}
rule = client.monitor.rules.create(
    "Harvest failures", "dataset",
    filter={"status": "error"},
    action_config={"datasetId": dataset_id},   # required for "dataset" and "webhook" actions
)
client.monitor.rules.update(rule.id, sample_rate=0.05)
client.monitor.rules.delete(rule.id)
```

An action missing its configuration is refused at creation - `dataset` without an
`action_config` `datasetId`, `webhook` without a `url` - rather than stored as a rule that can
never do anything. A rule action that fails at run time is logged and isolated, so one broken
rule never stops the others and never fails an ingest.

Rules are included in [backups](/self-host/backup).