> ## Documentation Index
> Fetch the complete documentation index at: https://developers.agentx.so/llms.txt
> Use this file to discover all available pages before exploring further.

# Python SDK Reference

> Complete API reference for agentx-python

## `AgentX`

The top-level client. Create one per process.

```python theme={null}
from agentx import AgentX

client = AgentX(api_key="ax_live_...", base_url="https://api.agentx.so")
# or from environment:
client = AgentX.from_env()   # reads AGENTX_API_KEY, AGENTX_API_BASE_URL
```

### Constructor

| Parameter      | Type  | Default                              | Description                                                                                                     |
| -------------- | ----- | ------------------------------------ | --------------------------------------------------------------------------------------------------------------- |
| `api_key`      | `str` | `AGENTX_API_KEY` env var             | Workspace API key                                                                                               |
| `base_url`     | `str` | `AGENTX_API_BASE_URL` or SDK default | Override API base URL (self-hosted)                                                                             |
| `workspace_id` | `str` | `AGENTX_WORKSPACE_ID` env var        | Explicit workspace ID — required when the API key belongs to a different user than the workspace being targeted |

### Class methods

| Method              | Returns  | Description                                          |
| ------------------- | -------- | ---------------------------------------------------- |
| `AgentX.from_env()` | `AgentX` | Construct from `AGENTX_API_KEY` environment variable |

### Attributes

| Attribute            | Type                | Description                            |
| -------------------- | ------------------- | -------------------------------------- |
| `client.tracer`      | `Tracer`            | Production tracer and CI/CD evaluation |
| `client.evaluations` | `EvaluationsRunner` | Dataset-driven evaluation runner       |

***

## `Tracer`

Accessed via `client.tracer`. Handles both production tracing and CI/CD evaluation.

### `trace()`

```python theme={null}
tracer.trace(
    name: str,
    *,
    input: Any = None,
    metadata: dict | None = None,
    framework: str | None = None,
    model: str | None = None,
    session_id: str | None = None,
) -> _TraceSpan
```

Returns a `_TraceSpan` that works as a **decorator** or **context manager**.

| Parameter    | Description                                                            |
| ------------ | ---------------------------------------------------------------------- |
| `name`       | Agent or operation label shown in the UI                               |
| `input`      | Initial input (context manager only; overrides captured function args) |
| `metadata`   | Arbitrary key-value metadata (not indexed, max 16 KB)                  |
| `framework`  | `"langchain"`, `"crewai"`, `"openai-agents"`, `"anthropic"`, or custom |
| `model`      | LLM model name, e.g. `"gpt-4o"`                                        |
| `session_id` | Groups traces from the same session or conversation thread             |

### `flush()`

```python theme={null}
tracer.flush(timeout: float = 5.0) -> None
```

Blocks until all queued traces have been delivered. Call before process exit in scripts or one-shot jobs.

### `run_eval()`

```python theme={null}
tracer.run_eval(
    dataset_id: str,
    agent_fn: Callable[[str], str],
    *,
    agent_name: str | None = None,
    pass_rate_threshold: float | None = None,
    git_context: dict | None = None,
    concurrency: int = 1,
    fail_on_gate: bool = False,
    timeout_per_question: float | None = None,
) -> CIRunResult
```

High-level CI/CD evaluation — creates a run, calls `agent_fn` for every test case, submits results, finalizes, and returns the gate decision.

Raises `CIGateFailure` (containing the `CIRunResult`) when `fail_on_gate=True` and the gate is `"fail"`.

### `create_ci_run()`

```python theme={null}
tracer.create_ci_run(
    dataset_id: str,
    *,
    agent_name: str | None = None,
    pass_rate_threshold: float | None = None,
    git_context: dict | None = None,
    workspace_id: str | None = None,
) -> CIRun
```

### `submit_result()`

```python theme={null}
tracer.submit_result(
    run_id: str,
    question_index: int,
    output: Any,
    *,
    input: Any = None,
    latency_ms: int | None = None,
) -> CIQuestionScore
```

### `finalize_ci_run()`

```python theme={null}
tracer.finalize_ci_run(run_id: str) -> CIRunResult
```

### `get_ci_run()`

```python theme={null}
tracer.get_ci_run(run_id: str) -> CIRunStatus
```

### `evaluate_trace()`

```python theme={null}
tracer.evaluate_trace(
    trace_id: str,
    dataset_id: str,
    *,
    question_index: int | None = None,
) -> dict
```

Score a previously-recorded production trace against a dataset. The agent is not re-run. Returns `{ run_id, trace_id, rating, justification, status }`.

***

## `_TraceSpan`

Returned by `tracer.trace()`. Can be used as a decorator or context manager.

### Attributes

| Attribute         | Type   | Description                                                       |
| ----------------- | ------ | ----------------------------------------------------------------- |
| `span.input`      | `Any`  | Agent input. Assign in context manager to override captured args. |
| `span.output`     | `Any`  | Agent output. Must be set manually in context manager mode.       |
| `span.tool_calls` | `list` | Accumulated tool calls. Append with `add_tool_call()`.            |

### Methods

#### `add_tool_call()`

```python theme={null}
span.add_tool_call(
    name: str,
    *,
    input: Any = None,
    output: Any = None,
    latency_ms: int | None = None,
) -> None
```

Records a tool call made during this span. Safe to call multiple times.

#### `set_error()`

```python theme={null}
span.set_error(message: str) -> None
```

Marks this span as failed. Takes precedence over any exception automatically caught by `__exit__`.

### Usage patterns

<CodeGroup>
  ```python Decorator theme={null}
  @tracer.trace("my-agent", framework="langchain", model="gpt-4o")
  def run(query: str) -> str:
      return chain.invoke(query)
  # input = captured from function args
  # output = return value
  ```

  ```python Context manager theme={null}
  with tracer.trace("my-agent") as span:
      span.input = {"query": query, "user_id": user_id}
      kb = search_kb(query)
      span.add_tool_call("search_kb", input=query, output=kb, latency_ms=180)
      span.output = llm.invoke(f"Context: {kb}\n\n{query}")
  ```

  ```python Async decorator theme={null}
  @tracer.trace("async-agent", framework="openai-agents")
  async def run(query: str) -> str:
      result = await runner.run(agent, query)
      return result.final_output
  ```

  ```python Error handling theme={null}
  with tracer.trace("my-agent") as span:
      try:
          span.output = call_agent(query)
      except Exception as e:
          span.set_error(str(e))
          raise
  ```
</CodeGroup>

***

## CI/CD Dataclasses

### `CIRun`

Returned by `create_ci_run()`.

```python theme={null}
@dataclass
class CIRun:
    run_id: str
    dataset_id: str
    total_questions: int
    test_cases: list[CITestCase]
    expires_at: str          # ISO 8601 — run expires if not finalized within 2 hours
```

### `CITestCase`

```python theme={null}
@dataclass
class CITestCase:
    index: int
    query: str | None        # None when ci.exposeTestInputs is False
```

### `CIQuestionScore`

Returned by `submit_result()`.

```python theme={null}
@dataclass
class CIQuestionScore:
    question_index: int
    rating: int              # 1–5
    justification: str
    passed: bool
    gate_fired: bool         # True when failFast fired — stop submitting
    input: Any
    output: Any
```

### `CIRunResult`

Returned by `finalize_ci_run()` and `run_eval()`.

```python theme={null}
@dataclass
class CIRunResult:
    run_id: str
    gate: Literal["pass", "fail"]
    pass_rate: float                      # 0.0 – 1.0
    total_questions: int
    passed_questions: int
    scores: list[CIQuestionScore]
    violations: list[ThresholdViolation]
    git_context: dict | None
    finalized_at: str | None
```

### `CIRunStatus`

Returned by `get_ci_run()`.

```python theme={null}
@dataclass
class CIRunStatus:
    run_id: str
    status: Literal["in_progress", "completed", "failed"]
    gate: Literal["pass", "fail"] | None  # None while in_progress
    results_submitted: int
    total_questions: int
    created_at: str
    expires_at: str
    finalized_at: str | None
    git_context: dict | None
```

### `ThresholdViolation`

```python theme={null}
@dataclass
class ThresholdViolation:
    question_index: int
    metric: str              # e.g. "rating"
    threshold: float
    actual: float
    question_text: str
```

***

## Exceptions

All exceptions inherit from `AgentXError`.

```python theme={null}
from agentx import (
    AgentXError,
    AgentXAuthError,
    AgentXAPIError,
    DatasetNotFound,
    CINotEnabled,
    CIRunExpired,
    CIGateFailure,
)
```

| Exception         | When raised                                                                     |
| ----------------- | ------------------------------------------------------------------------------- |
| `AgentXError`     | Base class for all SDK errors                                                   |
| `AgentXAuthError` | Invalid or missing API key                                                      |
| `AgentXAPIError`  | Unexpected API error. Has `.status_code: int \| None` attribute.                |
| `DatasetNotFound` | Dataset ID does not exist or is not accessible                                  |
| `CINotEnabled`    | Dataset exists but `ci.enabled` is `False`                                      |
| `CIRunExpired`    | CI run was not finalized within 2 hours                                         |
| `CIGateFailure`   | Gate is `"fail"` and `fail_on_gate=True`. Has `.result: CIRunResult` attribute. |

### `CIGateFailure`

```python theme={null}
try:
    result = client.tracer.run_eval(
        dataset_id=dataset_id,
        agent_fn=my_agent,
        fail_on_gate=True,
    )
except CIGateFailure as e:
    print(f"Pass rate: {e.result.pass_rate:.0%}")
    for v in e.result.violations:
        print(f"  Q{v.question_index}: {v.metric} = {v.actual:.2f} (threshold {v.threshold:.2f})")
    sys.exit(1)
```

***

## Environment variables

| Variable              | Description                                                       |
| --------------------- | ----------------------------------------------------------------- |
| `AGENTX_API_KEY`      | Workspace API key (required)                                      |
| `AGENTX_API_BASE_URL` | Override API base URL for self-hosted deployments                 |
| `AGENTX_WORKSPACE_ID` | Explicit workspace ID — injected into every request automatically |