Skip to main content
AgentX’s hosted platform has an “autotune” workflow: propose an instruction change, run it on a candidate branch of the agent’s config, compare against the current version, merge if it wins. Self-host can’t do the branch/merge/apply part - there’s no agent config here for AgentX to own or edit, your agent’s code lives entirely outside AgentX. What is portable is the comparison: run your agent twice against the same dataset via the SDK - once as-is, once with a change you made yourself - tag each run with a version label, and let self-host tell you which one scored higher.
No SDK changes needed - version just needs to be a key inside metadata, or a top-level version key on the subject itself (checked first when both are present).

​
Reading the verdict

From the dashboard: the sidebar’s Manage → Datasets → a dataset’s row menu → Compare versions. It shows every version that’s been run against that dataset (run count, rated count, average rating) and a headline verdict comparing the two most recent versions, the same candidateAvg >= baselineAvg check autotune’s own validation step uses. The per-version average is a true average over every rated result in that version’s runs - not an average of per-run averages, which would misweight versions whose runs have different result counts - and single-trace evaluations are excluded from the version buckets entirely. Nothing gets merged or shipped automatically - you read the verdict and go edit your own code. The dialog also includes a Case by case drill-down: each version’s latest run diffed per question, with rating deltas, regressions highlighted, and both outputs (plus the judge’s justification) one click away - the averages say whether a version regressed, this says where. To gate a deploy on this same comparison automatically, see the self-host CI gate’s no_regression check.