Python SDK Reference

`AgentX`

The top-level client. Create one per process.

from agentx import AgentX

client = AgentX(api_key="a1b2c3d4e5f60718293a4b5c6d7e8f9012345678", base_url="https://api.agentx.so")
# or from environment:
client = AgentX.from_env()   # reads AGENTX_API_KEY, AGENTX_API_BASE_URL

Constructor

Parameter	Type	Default	Description
`api_key`	`str`	`AGENTX_API_KEY` env var	Workspace API key
`base_url`	`str`	`AGENTX_API_BASE_URL` or SDK default	Override API base URL (self-hosted)
`workspace_id`	`str`	`AGENTX_WORKSPACE_ID` env var	Explicit workspace ID, required when the API key belongs to a different user than the workspace being targeted

Class methods

Method	Returns	Description
`AgentX.from_env()`	`AgentX`	Construct from `AGENTX_API_KEY` environment variable

Attributes

Attribute	Type	Description
`client.tracer`	`Tracer`	Tracer and CI/CD evaluation
`client.evaluations`	`EvaluationsRunner`	Dataset-driven evaluation runner
`client.monitor`	`MonitorClient`	Pattern CRUD for Monitor (`client.monitor.patterns`)

`Tracer`

Accessed via client.tracer. Handles both tracing and CI/CD evaluation.

`trace()`

tracer.trace(
    name: str,
    *,
    input: Any = None,
    metadata: dict | None = None,
    framework: str | None = None,
    model: str | None = None,
    session_id: str | None = None,
    sync: bool = False,
    monitor: bool | None = None,
    pattern_ids: list[str] | None = None,
) -> _TraceSpan

Returns a _TraceSpan that works as a decorator or context manager.

Parameter	Description
`name`	Agent or operation label shown in the UI
`input`	Initial input (context manager only; overrides captured function args)
`metadata`	Arbitrary key-value metadata (not indexed, max 16 KB)
`framework`	`"langchain"`, `"crewai"`, `"openai-agents"`, `"anthropic"`, or custom
`model`	LLM model name, e.g. `"gpt-4o"`
`session_id`	Groups traces from the same session or conversation thread
`sync`	Context manager only. Sends synchronously (blocking) instead of the default fire-and-forget queue, so `span.trace_id` is populated once the block exits. See Linking a trace to an evaluation result.
`monitor`	`True`: check this trace against Monitor patterns immediately. `False`: opt out of every ingest-time check (patterns, online evaluators, topics) - for traces made inside evaluation runs. `None` (default): normal engine behavior.
`pattern_ids`	With `monitor=True`, restrict detection to exactly these pattern ids (built-ins skipped).

`flush()`

tracer.flush(timeout: float = 5.0) -> None

Blocks until all queued traces have been delivered. Call before process exit in scripts or one-shot jobs.

`use_span()`

tracer.use_span(span: _TraceSpan) -> ContextManager[_TraceSpan]

Makes span (created on another thread) the active span for the duration of the block, on the calling thread. The active-span stack is thread-local, so a span opened with tracer.trace(...) on the main thread isn’t automatically visible inside a ThreadPoolExecutor worker (or any other thread). Wrap the worker’s body in use_span() to attach its LLM and tool calls to the parent span instead of starting an independent trace.

with tracer.trace("orchestrator") as span:
    def worker():
        with tracer.use_span(span):
            chain.invoke(query, config={"callbacks": [handler]})

    with ThreadPoolExecutor(max_workers=2) as ex:
        ex.submit(worker).result()

Safe to call concurrently from multiple threads for the same span: each thread pushes and pops on its own stack.

`run_eval()`

tracer.run_eval(
    dataset_id: str,
    agent_fn: Callable[[str], str],
    *,
    agent_name: str | None = None,
    pass_rate_threshold: float | None = None,
    git_context: dict | None = None,
    concurrency: int = 1,
    fail_on_gate: bool = False,
    timeout_per_question: float | None = None,
) -> CIRunResult

High-level CI/CD evaluation: creates a run, calls agent_fn for every test case, submits results, finalizes, and returns the gate decision. Raises CIGateFailure (containing the CIRunResult) when fail_on_gate=True and the gate is "fail".

`create_ci_run()`

tracer.create_ci_run(
    dataset_id: str,
    *,
    agent_name: str | None = None,
    pass_rate_threshold: float | None = None,
    git_context: dict | None = None,
    workspace_id: str | None = None,
) -> CIRun

`submit_result()`

tracer.submit_result(
    run_id: str,
    question_index: int,
    output: Any,
    *,
    input: Any = None,
    latency_ms: int | None = None,
) -> CIQuestionScore

`finalize_ci_run()`

tracer.finalize_ci_run(run_id: str) -> CIRunResult

`get_ci_run()`

tracer.get_ci_run(run_id: str) -> CIRunStatus

`evaluate_trace()`

tracer.evaluate_trace(
    trace_id: str,
    dataset_id: str,
    *,
    question_index: int | None = None,
) -> dict

Score a previously-recorded trace against a dataset. The agent is not re-run. Returns { run_id, trace_id, rating, justification, status }. trace_id comes from a prior tracer.trace(..., sync=True) call’s span.trace_id.

`_TraceSpan`

Returned by tracer.trace(). Can be used as a decorator or context manager.

Attributes

Attribute	Type	Description
`span.input`	`Any`	Agent input. Assign in context manager to override captured args.
`span.output`	`Any`	Agent output. Must be set manually in context manager mode.
`span.tool_calls`	`list`	Accumulated tool calls. Append with `add_tool_call()`.
`span.trace_id`	`str \| None`	The ingested trace’s id, readable once the block exits. Only populated when opened with `sync=True`; `None` otherwise, since the default mode queues the send and never learns the id.

Methods

`add_tool_call()`

span.add_tool_call(
    name: str,
    *,
    input: Any = None,
    output: Any = None,
    latency_ms: int | None = None,
) -> None

Records a tool call made during this span. Safe to call multiple times.

`set_error()`

span.set_error(message: str) -> None

Marks this span as failed. Takes precedence over any exception automatically caught by __exit__.

Usage patterns

@tracer.trace("my-agent", framework="langchain", model="gpt-4o")
def run(query: str) -> str:
    return chain.invoke(query)
# input = captured from function args
# output = return value

with tracer.trace("my-agent") as span:
    span.input = {"query": query, "user_id": user_id}
    kb = search_kb(query)
    span.add_tool_call("search_kb", input=query, output=kb, latency_ms=180)
    span.output = llm.invoke(f"Context: {kb}\n\n{query}")

with tracer.trace("my-agent", sync=True) as span:
    span.output = call_agent(query)

print(span.trace_id)  # None unless sync=True

@tracer.trace("async-agent", framework="openai-agents")
async def run(query: str) -> str:
    result = await runner.run(agent, query)
    return result.final_output

with tracer.trace("my-agent") as span:
    try:
        span.output = call_agent(query)
    except Exception as e:
        span.set_error(str(e))
        raise

CI/CD Dataclasses

`CIRun`

Returned by create_ci_run().

@dataclass
class CIRun:
    run_id: str
    dataset_id: str
    total_questions: int
    test_cases: list[CITestCase]
    expires_at: str          # ISO 8601, run expires if not finalized within 2 hours

`CITestCase`

@dataclass
class CITestCase:
    index: int
    query: str | None        # None when ci.exposeTestInputs is False

`CIQuestionScore`

Returned by submit_result().

@dataclass
class CIQuestionScore:
    question_index: int
    rating: int              # 0–10
    justification: str
    passed: bool
    gate_fired: bool         # True when failFast fired, stop submitting
    input: Any
    output: Any

`CIRunResult`

Returned by finalize_ci_run() and run_eval().

@dataclass
class CIRunResult:
    run_id: str
    gate: Literal["pass", "fail"]
    pass_rate: float                      # 0.0 – 1.0
    total_questions: int
    passed_questions: int
    scores: list[CIQuestionScore]
    violations: list[ThresholdViolation]
    git_context: dict | None
    finalized_at: str | None

`CIRunStatus`

Returned by get_ci_run().

@dataclass
class CIRunStatus:
    run_id: str
    status: Literal["in_progress", "completed", "failed"]
    gate: Literal["pass", "fail"] | None  # None while in_progress
    results_submitted: int
    total_questions: int
    created_at: str
    expires_at: str
    finalized_at: str | None
    git_context: dict | None

`ThresholdViolation`

@dataclass
class ThresholdViolation:
    question_index: int
    metric: str              # e.g. "rating"
    threshold: float
    actual: float
    question_text: str

Exceptions

All exceptions inherit from AgentXError.

from agentx import (
    AgentXError,
    AgentXAuthError,
    AgentXAPIError,
    DatasetNotFound,
    CINotEnabled,
    CIRunExpired,
    CIGateFailure,
)

Exception	When raised
`AgentXError`	Base class for all SDK errors
`AgentXAuthError`	Invalid or missing API key
`AgentXAPIError`	Unexpected API error. Has `.status_code: int \| None` attribute.
`DatasetNotFound`	Dataset ID does not exist or is not accessible
`CINotEnabled`	Dataset exists but `ci.enabled` is `False`
`CIRunExpired`	CI run was not finalized within 2 hours
`CIGateFailure`	Gate is `"fail"` and `fail_on_gate=True`. Has `.result: CIRunResult` attribute.

`CIGateFailure`

try:
    result = client.tracer.run_eval(
        dataset_id=dataset_id,
        agent_fn=my_agent,
        fail_on_gate=True,
    )
except CIGateFailure as e:
    print(f"Pass rate: {e.result.pass_rate:.0%}")
    for v in e.result.violations:
        print(f"  Q{v.question_index}: {v.metric} = {v.actual:.2f} (threshold {v.threshold:.2f})")
    sys.exit(1)

Environment variables

Variable	Description
`AGENTX_API_KEY`	Workspace API key (required)
`AGENTX_API_BASE_URL`	Override API base URL for self-hosted deployments
`AGENTX_WORKSPACE_ID`	Explicit workspace ID, injected into every request automatically

Get Started

Self-Hosting

Python SDK

Python SDK Reference

`AgentX`

Constructor

Class methods

Attributes

`Tracer`

`trace()`

`flush()`

`use_span()`

`run_eval()`

`create_ci_run()`

`submit_result()`

`finalize_ci_run()`

`get_ci_run()`

`evaluate_trace()`

`_TraceSpan`

Attributes

Methods

`add_tool_call()`

`set_error()`

Usage patterns

CI/CD Dataclasses

`CIRun`

`CITestCase`

`CIQuestionScore`

`CIRunResult`

`CIRunStatus`

`ThresholdViolation`

Exceptions

`CIGateFailure`

Environment variables

​AgentX

​Constructor

​Class methods

​Attributes

​Tracer

​trace()

​flush()

​use_span()

​run_eval()

​create_ci_run()

​submit_result()

​finalize_ci_run()

​get_ci_run()

​evaluate_trace()

​_TraceSpan

​Attributes

​Methods

​add_tool_call()

​set_error()

​Usage patterns

​CI/CD Dataclasses

​CIRun

​CITestCase

​CIQuestionScore

​CIRunResult

​CIRunStatus

​ThresholdViolation

​Exceptions

​CIGateFailure

​Environment variables

`AgentX`

Constructor

Class methods

Attributes

`Tracer`

`trace()`

`flush()`

`use_span()`

`run_eval()`

`create_ci_run()`

`submit_result()`

`finalize_ci_run()`

`get_ci_run()`

`evaluate_trace()`

`_TraceSpan`

Attributes

Methods

`add_tool_call()`

`set_error()`

Usage patterns

CI/CD Dataclasses

`CIRun`

`CITestCase`

`CIQuestionScore`

`CIRunResult`

`CIRunStatus`

`ThresholdViolation`

Exceptions

`CIGateFailure`

Environment variables