Python SDK Reference

`AgentX`

The top-level client. Create one per process.

from agentx import AgentX

client = AgentX(api_key="ax_live_...", base_url="https://api.agentx.so")
# or from environment:
client = AgentX.from_env()   # reads AGENTX_API_KEY, AGENTX_API_BASE_URL

Constructor

Parameter	Type	Default	Description
`api_key`	`str`	`AGENTX_API_KEY` env var	Workspace API key
`base_url`	`str`	`AGENTX_API_BASE_URL` or SDK default	Override API base URL (self-hosted)
`workspace_id`	`str`	`AGENTX_WORKSPACE_ID` env var	Explicit workspace ID — required when the API key belongs to a different user than the workspace being targeted

Class methods

Method	Returns	Description
`AgentX.from_env()`	`AgentX`	Construct from `AGENTX_API_KEY` environment variable

Attributes

Attribute	Type	Description
`client.tracer`	`Tracer`	Production tracer and CI/CD evaluation
`client.evaluations`	`EvaluationsRunner`	Dataset-driven evaluation runner

`Tracer`

Accessed via client.tracer. Handles both production tracing and CI/CD evaluation.

`trace()`

tracer.trace(
    name: str,
    *,
    input: Any = None,
    metadata: dict | None = None,
    framework: str | None = None,
    model: str | None = None,
    session_id: str | None = None,
) -> _TraceSpan

Returns a _TraceSpan that works as a decorator or context manager.

Parameter	Description
`name`	Agent or operation label shown in the UI
`input`	Initial input (context manager only; overrides captured function args)
`metadata`	Arbitrary key-value metadata (not indexed, max 16 KB)
`framework`	`"langchain"`, `"crewai"`, `"openai-agents"`, `"anthropic"`, or custom
`model`	LLM model name, e.g. `"gpt-4o"`
`session_id`	Groups traces from the same session or conversation thread

`flush()`

tracer.flush(timeout: float = 5.0) -> None

Blocks until all queued traces have been delivered. Call before process exit in scripts or one-shot jobs.

`run_eval()`

tracer.run_eval(
    dataset_id: str,
    agent_fn: Callable[[str], str],
    *,
    agent_name: str | None = None,
    pass_rate_threshold: float | None = None,
    git_context: dict | None = None,
    concurrency: int = 1,
    fail_on_gate: bool = False,
    timeout_per_question: float | None = None,
) -> CIRunResult

High-level CI/CD evaluation — creates a run, calls agent_fn for every test case, submits results, finalizes, and returns the gate decision. Raises CIGateFailure (containing the CIRunResult) when fail_on_gate=True and the gate is "fail".

`create_ci_run()`

tracer.create_ci_run(
    dataset_id: str,
    *,
    agent_name: str | None = None,
    pass_rate_threshold: float | None = None,
    git_context: dict | None = None,
    workspace_id: str | None = None,
) -> CIRun

`submit_result()`

tracer.submit_result(
    run_id: str,
    question_index: int,
    output: Any,
    *,
    input: Any = None,
    latency_ms: int | None = None,
) -> CIQuestionScore

`finalize_ci_run()`

tracer.finalize_ci_run(run_id: str) -> CIRunResult

`get_ci_run()`

tracer.get_ci_run(run_id: str) -> CIRunStatus

`evaluate_trace()`

tracer.evaluate_trace(
    trace_id: str,
    dataset_id: str,
    *,
    question_index: int | None = None,
) -> dict

Score a previously-recorded production trace against a dataset. The agent is not re-run. Returns { run_id, trace_id, rating, justification, status }.

`_TraceSpan`

Returned by tracer.trace(). Can be used as a decorator or context manager.

Attributes

Attribute	Type	Description
`span.input`	`Any`	Agent input. Assign in context manager to override captured args.
`span.output`	`Any`	Agent output. Must be set manually in context manager mode.
`span.tool_calls`	`list`	Accumulated tool calls. Append with `add_tool_call()`.

Methods

`add_tool_call()`

span.add_tool_call(
    name: str,
    *,
    input: Any = None,
    output: Any = None,
    latency_ms: int | None = None,
) -> None

Records a tool call made during this span. Safe to call multiple times.

`set_error()`

span.set_error(message: str) -> None

Marks this span as failed. Takes precedence over any exception automatically caught by __exit__.

Usage patterns

@tracer.trace("my-agent", framework="langchain", model="gpt-4o")
def run(query: str) -> str:
    return chain.invoke(query)
# input = captured from function args
# output = return value

CI/CD Dataclasses

`CIRun`

Returned by create_ci_run().

@dataclass
class CIRun:
    run_id: str
    dataset_id: str
    total_questions: int
    test_cases: list[CITestCase]
    expires_at: str          # ISO 8601 — run expires if not finalized within 2 hours

`CITestCase`

@dataclass
class CITestCase:
    index: int
    query: str | None        # None when ci.exposeTestInputs is False

`CIQuestionScore`

Returned by submit_result().

@dataclass
class CIQuestionScore:
    question_index: int
    rating: int              # 1–5
    justification: str
    passed: bool
    gate_fired: bool         # True when failFast fired — stop submitting
    input: Any
    output: Any

`CIRunResult`

Returned by finalize_ci_run() and run_eval().

@dataclass
class CIRunResult:
    run_id: str
    gate: Literal["pass", "fail"]
    pass_rate: float                      # 0.0 – 1.0
    total_questions: int
    passed_questions: int
    scores: list[CIQuestionScore]
    violations: list[ThresholdViolation]
    git_context: dict | None
    finalized_at: str | None

`CIRunStatus`

Returned by get_ci_run().

@dataclass
class CIRunStatus:
    run_id: str
    status: Literal["in_progress", "completed", "failed"]
    gate: Literal["pass", "fail"] | None  # None while in_progress
    results_submitted: int
    total_questions: int
    created_at: str
    expires_at: str
    finalized_at: str | None
    git_context: dict | None

`ThresholdViolation`

@dataclass
class ThresholdViolation:
    question_index: int
    metric: str              # e.g. "rating"
    threshold: float
    actual: float
    question_text: str

Exceptions

All exceptions inherit from AgentXError.

from agentx import (
    AgentXError,
    AgentXAuthError,
    AgentXAPIError,
    DatasetNotFound,
    CINotEnabled,
    CIRunExpired,
    CIGateFailure,
)

Exception	When raised
`AgentXError`	Base class for all SDK errors
`AgentXAuthError`	Invalid or missing API key
`AgentXAPIError`	Unexpected API error. Has `.status_code: int \| None` attribute.
`DatasetNotFound`	Dataset ID does not exist or is not accessible
`CINotEnabled`	Dataset exists but `ci.enabled` is `False`
`CIRunExpired`	CI run was not finalized within 2 hours
`CIGateFailure`	Gate is `"fail"` and `fail_on_gate=True`. Has `.result: CIRunResult` attribute.

`CIGateFailure`

try:
    result = client.tracer.run_eval(
        dataset_id=dataset_id,
        agent_fn=my_agent,
        fail_on_gate=True,
    )
except CIGateFailure as e:
    print(f"Pass rate: {e.result.pass_rate:.0%}")
    for v in e.result.violations:
        print(f"  Q{v.question_index}: {v.metric} = {v.actual:.2f} (threshold {v.threshold:.2f})")
    sys.exit(1)

Environment variables

Variable	Description
`AGENTX_API_KEY`	Workspace API key (required)
`AGENTX_API_BASE_URL`	Override API base URL for self-hosted deployments
`AGENTX_WORKSPACE_ID`	Explicit workspace ID — injected into every request automatically

​AgentX

​Constructor

​Class methods

​Attributes

​Tracer

​trace()

​flush()

​run_eval()

​create_ci_run()

​submit_result()

​finalize_ci_run()

​get_ci_run()

​evaluate_trace()

​_TraceSpan

​Attributes

​Methods

​add_tool_call()

​set_error()

​Usage patterns

​CI/CD Dataclasses

​CIRun

​CITestCase

​CIQuestionScore

​CIRunResult

​CIRunStatus

​ThresholdViolation

​Exceptions

​CIGateFailure

​Environment variables

`AgentX`

Constructor

Class methods

Attributes

`Tracer`

`trace()`

`flush()`

`run_eval()`

`create_ci_run()`

`submit_result()`

`finalize_ci_run()`

`get_ci_run()`

`evaluate_trace()`

`_TraceSpan`

Attributes

Methods

`add_tool_call()`

`set_error()`

Usage patterns

CI/CD Dataclasses

`CIRun`

`CITestCase`

`CIQuestionScore`

`CIRunResult`

`CIRunStatus`

`ThresholdViolation`

Exceptions

`CIGateFailure`

Environment variables