> ## Documentation Index
> Fetch the complete documentation index at: https://developers.agentx.so/llms.txt
> Use this file to discover all available pages before exploring further.

# Auto-improve

> Confirmed production failures become an improvement report, and the auto-improve skill applies the fixes to your source

Online evaluation exists to catch your agent failing in production. **Auto-improve** is what
happens after the catch: the failures a human *confirmed* are spent - as one body of evidence -
on an improvement report, and the report is applied to your agent's actual source code by a
coding agent. The loop runs entirely on **online evidence**: live-traffic verdicts a person
vouched for, never offline dataset runs.

## 1. Confirm failures (Review)

In **Review → Review signals**, passing a **Confirm** verdict on a signal - a low
judge score or a matched failure pattern - automatically lands the confirmed occurrence in the
**pending batch** ("Confirmed failures"). Confirm *is* the accumulation gesture; there is no
second button, and declining is choosing **Ignore**. The evidence (input, output,
judge rationale, score) is snapshotted at confirm time, so later trace pruning cannot hollow it
out. Re-confirming the same verdict after a reopen never double-counts.

What makes this evidence unusual: every item was flagged by a machine **and** confirmed by a
person. It is the high-precision slice of production traffic - the opposite of auto-selected
"worst examples".

## 2. Generate a report (Insights → Auto-improve)

**Insights → Auto-improve** shows the pending batch and its count. **Generate
improvement report** runs one LLM pass (explicit and billed - never implicit) that clusters the
confirmed failures into issues, each with a title, description, recommendation, and its
evidence inline. Generating **seals the batch**: the group becomes that report's permanent
source-case set (renamed with its seal time) and the pending accumulator is cleared - the next
Confirm starts a fresh batch, and the next generate produces a new report from a new group. One
report per group, one group per report. The report is stored under its own id, shown with a
copy button. A mis-confirmed item can be pruned from the batch before generating with
`client.monitor.improvement_groups.remove_member(group_id, member_id)`.

From the SDK:

```python theme={null}
groups = client.monitor.improvement_groups.list()
report = client.monitor.improvement_groups.generate_report(groups[0]["_id"])
print(report["_id"], len(report["issues"]))
```

## 3. Apply the fixes (the auto-improve skill)

The report id is the hand-off to the [AgentX Agent Skill
plugin](https://github.com/AgentX-ai/AgentX-Eval-Skill). In your agent's repo, tell your coding
agent:

```
Use the agentx auto-improve skill on report <report-id>.
```

The skill fetches the report from the engine (`GET
/agent-monitoring/improvement-reports/:id`), then **triages** each recommendation against your
real source rather than applying it literally - the recommendations were written by a judge
that never saw the code, so they are hypotheses, not instructions. Each issue gets a verdict
(apply / already handled / reject with a reason), and the surviving fixes are made in place.

## Verifying the fix

The same scorers keep watching production. A fix that holds stops re-raising its signals; a
signal you resolved as **Fixed** that fires again reopens as a regression - the engine telling
you the fix did not hold. For a pre-deploy check, curate the confirmed failures into a dataset
and run an offline evaluation against it.

<Note>
  Auto-improve is self-host only, and the report generator uses the engine's default judge
  model (override with `generate_report(group_id, model="...")` or the route's `model` field).
</Note>