> ## Documentation Index
> Fetch the complete documentation index at: https://developers.agentx.so/llms.txt
> Use this file to discover all available pages before exploring further.

# Examples

> The common pattern behind every framework integration

All examples follow the same pattern: wrap your framework's output in a function that accepts an `EvaluationCase` and returns a `str`, `dict`, or `EvaluationResult`.

```python theme={null}
from agentx.evaluations.models import EvaluationCase

def my_agent(case: EvaluationCase) -> str:
    return f"Answer to: {case.query}"

run_context = (
    client.evaluations
    .run(dataset_id="...", subject={"kind": "custom_agent", "displayName": "My Bot", "framework": "raw_python"})
    .execute(my_agent)
    .finalize()
)
print(f"Average rating: {run_context.average_rating:.2f}")  # no .analyze() needed for this
```

`EvaluationCase` gives your function:

| Field                   | Type          | Description                                                                                    |
| ----------------------- | ------------- | ---------------------------------------------------------------------------------------------- |
| `case.query`            | `str`         | The test question's text                                                                       |
| `case.case_id`          | `str`         | Stable id for this case, e.g. `"case-0"`                                                       |
| `case.question_index`   | `int`         | 0-based index into the dataset's `questions` array                                             |
| `case.run_number`       | `int`         | Which repetition this is (1 through the config's `number_of_requests`)                         |
| `case.expected_results` | `str \| None` | The question's expected-answer text, if the dataset defines one                                |
| `case.model`            | `str \| None` | Set only when Sovereignty & Portability is enabled: which comparison model this run should use |

What you do inside the function is entirely up to you: call an LLM directly, invoke a LangChain chain, kick off a CrewAI crew, or hit an HTTP endpoint. What you return determines what AgentX sees:

| Return type        | Behavior                                                                                                                                                                                                       |
| ------------------ | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| `str`              | Used directly as the response text                                                                                                                                                                             |
| `dict`             | `output`/`text`/`response` for the text, plus optional `metadata`, `input_tokens`, `output_tokens`, `trace_id`, `retrieval_context` (what the agent retrieved - feeds [RAG judging](/evaluation/rag)), `error` |
| `EvaluationResult` | Full manual control: set every field yourself, including `observable_trace`                                                                                                                                    |

<Tip>
  Returning a plain `str` is enough to get scored. Add `metadata: {"model": ...}` when you want the Sovereignty & Portability breakdown, or `trace_id` when you want a full [Execution Timeline](/sdk/tracing#linking-a-trace-to-an-evaluation-result) attached to the result. Neither is required.
</Tip>

Pick your framework:

<CardGroup cols={2}>
  <Card title="OpenAI" icon="brain" href="/sdk/evaluations/examples/openai">
    Plain OpenAI chat completions
  </Card>

  <Card title="OpenAI Agents SDK" icon="brain" href="/sdk/evaluations/examples/openai-agents">
    Agent/Runner, tool calling, handoffs
  </Card>

  <Card title="Anthropic Claude" icon="brain" href="/sdk/evaluations/examples/anthropic">
    Claude via the Anthropic SDK
  </Card>

  <Card title="LangChain" icon="link" href="/sdk/evaluations/examples/langchain">
    Prompt templates and chains
  </Card>

  <Card title="CrewAI" icon="users" href="/sdk/evaluations/examples/crewai">
    Multi-agent crews
  </Card>

  <Card title="AutoGen / AG2" icon="messages" href="/sdk/evaluations/examples/autogen">
    Conversable agents
  </Card>

  <Card title="LlamaIndex" icon="database" href="/sdk/evaluations/examples/llamaindex">
    RAG query engines
  </Card>

  <Card title="HTTP Endpoint" icon="globe" href="/sdk/evaluations/examples/http-endpoint">
    Any agent exposed as a web service
  </Card>

  <Card title="Pre-Defined Results" icon="upload" href="/sdk/evaluations/examples/precomputed-results">
    Score outputs you already generated
  </Card>
</CardGroup>

A complete, runnable version of each example lives in the [AgentX-Python repository](https://github.com/AgentX-ai/AgentX-Python/tree/main/examples/evaluations).