# AgentX

## Docs

- [Introduction](https://developers.agentx.so/introduction.md): Evaluate, monitor, and gate AI agents in production
- [Quickstart](https://developers.agentx.so/quickstart.md): Send your first trace in under 5 minutes
- [Authentication](https://developers.agentx.so/authentication.md): API keys and session authentication
- [Concepts](https://developers.agentx.so/concepts.md): Core vocabulary for working with the AgentX API
- [Errors](https://developers.agentx.so/errors.md): HTTP error codes and error response format
- [Self-Host Overview](https://developers.agentx.so/self-host/overview.md): Run the full Trace, Evaluate, Monitor, and Improve stack on your own machine - no account, bring your own LLM keys
- [Installation](https://developers.agentx.so/self-host/installation.md): curl | bash, the Python SDK launcher, Docker, or from source - then point the SDK or OTel at it
- [Configuration](https://developers.agentx.so/self-host/configuration.md): Environment variables, Postgres, and the end-to-end smoke test
- [Python SDK Overview](https://developers.agentx.so/sdk/overview.md): AI agent Tracing and CI/CD evaluation
- [Python SDK Reference](https://developers.agentx.so/sdk/python-reference.md): Complete API reference for agentx-python
- [Tracing](https://developers.agentx.so/sdk/tracing.md): Record agent runs from any Python framework
- [OpenTelemetry](https://developers.agentx.so/trace/opentelemetry.md): Send traces from any OTel-instrumented app - no AgentX SDK required
- [LangChain](https://developers.agentx.so/sdk/integrations/langchain.md): Auto-trace every LangChain chain and agent run with AgentXCallbackHandler
- [CrewAI](https://developers.agentx.so/sdk/integrations/crewai.md): Trace CrewAI crew runs with AgentXCrewObserver
- [OpenAI Agents SDK](https://developers.agentx.so/sdk/integrations/openai-agents.md): Trace OpenAI agent runs with AgentXTracingProcessor
- [OpenAI](https://developers.agentx.so/sdk/integrations/openai.md): Auto-trace raw OpenAI SDK calls with patch_openai_client
- [Anthropic](https://developers.agentx.so/sdk/integrations/anthropic.md): Auto-trace Anthropic SDK calls with patch_anthropic_client
- [Google](https://developers.agentx.so/sdk/integrations/google.md): Trace Google ADK agents and Gemini API calls
- [LiteLLM](https://developers.agentx.so/sdk/integrations/litellm.md): Auto-trace every LiteLLM completion call with AgentXLiteLLMLogger
- [LlamaIndex](https://developers.agentx.so/sdk/integrations/llamaindex.md): Auto-trace query engines, chat engines, and agents with AgentXLlamaIndexHandler
- [AutoGen](https://developers.agentx.so/sdk/integrations/autogen.md): Trace Microsoft AutoGen agent and team runs with AgentXAutoGenObserver
- [Monitor](https://developers.agentx.so/sdk/monitor.md): Automatic production monitoring that turns your traces into detected issues, no dashboard setup required
- [Patterns](https://developers.agentx.so/monitor/patterns.md): Rule-based failure detection - phrase, regex, and semantic conditions that turn matching traces into signals
- [Online Evaluators](https://developers.agentx.so/monitor/online-evaluators.md): A real LLM judge scoring live traffic continuously, per trace or per session
- [Custom Evaluators](https://developers.agentx.so/monitor/custom-evaluators.md): Delegate the verdict to your own HTTP endpoint
- [Topics](https://developers.agentx.so/monitor/topics.md): What your agents are actually being asked, clustered
- [Outcomes & Judge Calibration](https://developers.agentx.so/monitor/outcomes.md): Report what actually happened, and measure the judges against it
- [Model Comparison](https://developers.agentx.so/monitor/model-comparison.md): Per-model quality, cost, and latency aggregated from real traffic
- [Custom Agent Evaluations](https://developers.agentx.so/sdk/evaluations/overview.md): Score any agent from any framework or provider with AgentX's evaluation backend
- [Installation](https://developers.agentx.so/sdk/evaluations/installation.md): Install the AgentX Python SDK and the framework extras you need
- [Quick Start](https://developers.agentx.so/sdk/evaluations/quickstart.md): Run your first custom agent evaluation in a few lines
- [Build Dataset](https://developers.agentx.so/sdk/evaluations/build-dataset.md): Create evaluation datasets and test cases with the Python SDK
- [Evaluation Settings](https://developers.agentx.so/sdk/evaluations/evaluation-settings.md): Build a standalone, reusable grading config decoupled from any one dataset
- [EvaluationSubject Fields](https://developers.agentx.so/sdk/evaluations/evaluation-subject.md): Describe the agent being evaluated so analysis can check instruction adherence
- [AI Analysis Report](https://developers.agentx.so/sdk/evaluations/analysis-report.md): What .analyze() returns, strengths, weaknesses, instruction adherence, and recommendations
- [Datasets from Production](https://developers.agentx.so/evaluation/datasets-from-production.md): Turn a real trace, session, or signal into a golden dataset case in two clicks
- [Code Scorers](https://developers.agentx.so/evaluation/code-scorers.md): Grade eval results with exact JavaScript, alongside the judge and similarity metrics
- [Version History](https://developers.agentx.so/evaluation/version-history.md): Every dataset and evaluator edit, saved as a browsable version
- [Playground](https://developers.agentx.so/evaluation/playground.md): Test prompts, models, and tools against real dataset cases interactively
- [Model Portability](https://developers.agentx.so/evaluation/model-portability.md): Replay a captured trace against alternative models for a cost/latency/quality estimate
- [Examples](https://developers.agentx.so/sdk/evaluations/examples/overview.md): The common pattern behind every framework integration
- [OpenAI](https://developers.agentx.so/sdk/evaluations/examples/openai.md): Evaluate a plain OpenAI chat completions agent
- [OpenAI Agents SDK](https://developers.agentx.so/sdk/evaluations/examples/openai-agents.md): Evaluate an agent built on the OpenAI Agents SDK (the `agents` / `openai-agents` package)
- [Anthropic Claude](https://developers.agentx.so/sdk/evaluations/examples/anthropic.md): Evaluate an agent built on the Anthropic Messages API
- [LangChain](https://developers.agentx.so/sdk/evaluations/examples/langchain.md): Evaluate a LangChain chain or agent
- [CrewAI](https://developers.agentx.so/sdk/evaluations/examples/crewai.md): Evaluate a CrewAI crew
- [AutoGen / AG2](https://developers.agentx.so/sdk/evaluations/examples/autogen.md): Evaluate a conversation between AutoGen (or AG2) ConversableAgents
- [LlamaIndex](https://developers.agentx.so/sdk/evaluations/examples/llamaindex.md): Evaluate a LlamaIndex RAG query engine
- [HTTP Endpoint](https://developers.agentx.so/sdk/evaluations/examples/http-endpoint.md): Evaluate an agent exposed as a web service, without writing a Python callable
- [Submit Pre-Defined Results](https://developers.agentx.so/sdk/evaluations/examples/precomputed-results.md): Score outputs you already generated, without re-running the agent
- [Prompt Management](https://developers.agentx.so/improve/prompt-management.md): A version-scoped prompt registry with evidence-fed improvement proposals
- [Tool Schemas](https://developers.agentx.so/improve/tool-schemas.md): A version-scoped registry for tool definitions, improved from real failures
- [Validating Proposals](https://developers.agentx.so/improve/validating-proposals.md): Every proposed prompt or tool-schema rewrite ships with a measured verdict before a human publishes it
- [Comparing Versions](https://developers.agentx.so/improve/comparing-versions.md): Run your agent twice, tag each run, let AgentX tell you which version won
- [Improve Prompts with Claude Code](https://developers.agentx.so/improve/claude-code.md): Drive the prompt-improvement loop from Claude Code, no engine-side LLM key needed
- [CI/CD Evaluation](https://developers.agentx.so/sdk/ci-cd.md): Gate agent releases with eval test sets
- [Self-Host CI Gate](https://developers.agentx.so/integrations/self-host-ci.md): Block a merge when your agent's eval score drops - an exit-code contract over a normal eval run
- [GitHub Actions](https://developers.agentx.so/integrations/github-actions.md): Gate PRs with AgentX CI/CD evaluation
- [Webhooks](https://developers.agentx.so/integrations/webhook.md): Receive CI gate results via HTTP callback
- [Submit Trace](https://developers.agentx.so/api-reference/tracing/submit-trace.md): Record a single agent run from an external framework
- [List Traces](https://developers.agentx.so/api-reference/tracing/list-traces.md): Paginated feed of SDK and native agent traces
- [Evaluate Trace](https://developers.agentx.so/api-reference/tracing/evaluate-trace.md): Score a recorded trace against a dataset without re-running the agent
- [Add Tracked Agent](https://developers.agentx.so/api-reference/tracked-agents/add.md): Start tracking a Robot's native traces
- [List Tracked Agents](https://developers.agentx.so/api-reference/tracked-agents/list.md): Get all tracked agents in a workspace
- [Remove Tracked Agent](https://developers.agentx.so/api-reference/tracked-agents/remove.md): Stop tracking a Robot's native traces
- [Custom Evaluations Overview](https://developers.agentx.so/api-reference/custom-eval/overview.md): Run evaluations programmatically against any external agent
- [Create Dataset](https://developers.agentx.so/api-reference/custom-eval/create-dataset.md): Create an evaluation dataset with questions and scoring configuration
- [Get Dataset](https://developers.agentx.so/api-reference/custom-eval/get-dataset.md): Get a single evaluation dataset
- [List Datasets](https://developers.agentx.so/api-reference/custom-eval/list-datasets.md): List evaluation datasets in a workspace
- [Create Run](https://developers.agentx.so/api-reference/custom-eval/create-run.md): Start a new evaluation run against a dataset
- [Get Run](https://developers.agentx.so/api-reference/custom-eval/get-run.md): Get a single evaluation run
- [List Runs](https://developers.agentx.so/api-reference/custom-eval/list-runs.md): List evaluation runs in a workspace
- [Submit Results](https://developers.agentx.so/api-reference/custom-eval/submit-results.md): Submit a batch of agent outputs for scoring
- [Get Missing Results](https://developers.agentx.so/api-reference/custom-eval/missing-results.md): Check which questions still need a result submitted
- [Finalize Run](https://developers.agentx.so/api-reference/custom-eval/finalize-run.md): Mark a run as complete
- [Analyze Run](https://developers.agentx.so/api-reference/custom-eval/analyze-run.md): Trigger AI analysis of a finalized run
- [Get Report](https://developers.agentx.so/api-reference/custom-eval/get-report.md): Get the full evaluation report including scores, analysis, and recommendations
- [List Models](https://developers.agentx.so/api-reference/custom-eval/list-models.md): List supported LLM models available for evaluation scoring
- [CI/CD Overview](https://developers.agentx.so/api-reference/ci-cd/overview.md): Gate agent releases with eval datasets
- [Create CI Run](https://developers.agentx.so/api-reference/ci-cd/create-run.md): Start a CI run and receive the test cases
- [Get Test Cases](https://developers.agentx.so/api-reference/ci-cd/get-test-cases.md): Preview test cases for a CI-enabled dataset
- [Submit Result](https://developers.agentx.so/api-reference/ci-cd/submit-result.md): Submit and score one question result
- [Finalize CI Run](https://developers.agentx.so/api-reference/ci-cd/finalize-run.md): Compute the final gate result for the run
- [Get CI Run](https://developers.agentx.so/api-reference/ci-cd/get-run.md): Poll the status and result of a CI run