- Engine - a TypeScript HTTP API (the same Trace/Evaluate/Monitor logic the hosted product runs), compiled to a single native binary so you don’t need Node or Bun installed to run it.
- CLI (
agentx/agentx-server) - a small Go launcher: starts the engine, opens the dashboard, forwards shutdown signals. - Dashboard - the real AgentX Governance UI, not a rebuild: AgentX’s own frontend, built in a self-host mode that swaps the login/workspace switcher for a synthetic local user, so it’s genuinely the same interface the hosted product’s Governance page uses rather than a lighter clone.
What the dashboard covers
Self-host covers Governance’s Observe tab (traces and sessions, Monitor signals, pattern CRUD, signal triage/feedback), Agents tab (per-agent monitoring profiles, health rate), Overview tab (KPIs strip, trend chart, top-failing breakdown, topic map, Model Comparison, and Judge Calibration), Evaluate’s Runs, Datasets, and Evaluator sub-views (dataset/config CRUD including version history and custom code scorers, viewing run results and judge scores), Monitor’s Online Evaluators, Custom Evaluators, and Topics sub-views (self-host-only, alongside Signals/Patterns, since the backend they call doesn’t exist on the hosted platform), the Improve tab, which shows entirely different content per mode (hosted’s native autotune proposals stay hosted-only, self-host gets Prompt Management and Tool Schemas instead), and a self-host-only Playground tab (see Interactive Playground) for testing prompts/models against a dataset interactively. The two LLM-assisted Monitor features (regex generation, feedback drafting), “create evaluator from signal,” Evaluate’s judge scoring, online evaluators, Topics, a prompt’s or tool schema’s “Suggest improvement,” and Playground all need a provider key set (
OPENAI_API_KEY/ANTHROPIC_API_KEY/GEMINI_API_KEY) - code scorers don’t, since they run as plain JavaScript, not an LLM call. The Governance header’s “New evaluation” button is hidden on self-host - it’s built around picking a native AgentX Agent/Team to run, which self-host has no registry for.Open source scope
The engine, CLI, and SDK are all open source (AgentX-trace-eval, Apache-2.0). The dashboard is the exception: it’s built from AgentX’s own frontend, which stays closed-source. Rather than maintaining a second, lighter dashboard implementation to keep open source, only its compiled build output is published - as a versioned asset attached to AgentX-trace-eval’s own releases - so nobody installing or building self-host, including outside contributors, ever needs access to that private repo. Hosted’s autotune/“Improve” proposal workflow (candidate branch creation, evaluation, merging) follows the same boundary: it’s fundamentally tied to AgentX’s native agent config-branching system, the same “agent building” surface that’s out of scope for self-host - not a missing feature to build later, an explicit non-goal. The Improve tab itself isn’t hidden on self-host, though: it shows Prompt Management instead, self-host’s own answer to “how do I make my agent better.”Roadmap
- SDK support for creating/editing code scorers (dashboard-only today)
- SDK support for creating/editing custom evaluators (dashboard-only today)
- SDK support for the Tool Schema registry (dashboard-only today)
- Guardrail (doesn’t exist on the hosted product yet either)
Tracing
What gets recorded on every traced call
Monitor
Automatic detection, patterns, and signals
Custom agent evaluations
LLM-as-a-judge scoring for any agent
AgentX-trace-eval on GitHub
Source, issues, and the full README