HTTP Endpoint - AgentX

No install needed beyond agentx-python itself. HttpEndpointAdapter uses requests, already a core dependency. Use it when your agent runs behind an HTTP service (FastAPI, LangServe, Flask, an n8n webhook, or anything else) instead of writing a Python function.

Usage

from agentx import AgentX
from agentx.evaluations.adapters.http_endpoint import HttpEndpointAdapter

client = AgentX.from_env()

adapter = HttpEndpointAdapter(
    url="http://localhost:8000/agent/invoke",
    headers={"Authorization": "Bearer your-token"},
    timeout=30,
)

run_context = (
    client.evaluations
    .run(dataset_id="...", subject={"kind": "custom_agent", "displayName": "My API Agent", "framework": "other"})
    .execute(adapter)
    .finalize()
)
print("Average rating:", run_context.average_rating)

Endpoint contract

For every test case, the adapter sends:

{method} {url}
Content-Type: application/json

{
  "query": "How do I reset my password?",
  "case_id": "case-0",
  "question_index": 0,
  "run_number": 1
}

Your service must respond with a 2xx and a JSON body. A non-2xx status, a request timeout, or an empty output is recorded as a failed result for that case, not a hard error that stops the run - and a failed row is stored with rating 0, so it pulls the run’s average down.

Response field	Required	Description
`output` (or `text`, or `response`)	Yes	The agent’s response text
`metadata`	No	Arbitrary key-value data. Include `model` to record which model produced the response on the result row
`trace`	No	`{ "events": [...] }`, a lightweight reasoning/tool-call summary shown in AI analysis. For a full browsable Execution Timeline instead, have your service submit a real trace itself (`POST /ingest/traces`, or the tracing SDK) and return `"trace_id"` here
`retrieval_context`	No	String or chunk list: what the agent actually retrieved for this case, feeds RAG judging and expected-retrieval-context matching
`input_tokens` / `output_tokens`	No	Token counts, shown in the result’s timing detail

{
  "output": "Click Forgot Password on the login screen.",
  "metadata": { "model": "gpt-4o-mini" },
  "trace_id": "mJ3vQ8pTr2LqYw6bZk9Xd"
}

`HttpEndpointAdapter` parameters

HttpEndpointAdapter(
    url: str,
    headers: dict | None = None,
    timeout: int = 30,
    method: str = "POST",
)

Parameter	Type	Default	Description
`url`	`str`	required	Endpoint to call for each test case
`headers`	`dict`	`None`	Extra headers, e.g. an `Authorization` bearer token
`timeout`	`int`	`30`	Seconds to wait for a response before recording the case as failed
`method`	`str`	`"POST"`	HTTP method to use, override if your service expects something other than `POST`

The SDK (running locally) makes the request directly to your service. The AgentX API never touches your endpoint. Your endpoint must be reachable from wherever the script runs, not from AgentX’s servers.

A complete working example is available as http_endpoint_eval.py in the AgentX-Python repository.