EvaluationSubject Fields - AgentX

subject (passed to client.evaluations.run()) tells AgentX what it’s evaluating. It’s used by AI analysis to check instruction adherence and to power the Sovereignty & Portability matrix; it never changes how scoring itself runs.

report = (
    client.evaluations
    .run(
        dataset_id="...",
        subject={
            "kind": "custom_agent",
            "displayName": "Support Bot",
            "framework": "langchain",
            "runtime": "local",
            "agentInstructions": "You are a helpful support agent...",
        },
    )
    .execute(my_agent)
    .finalize()
    .analyze()
)

Fields

Field	Values	Description
`kind`	`"custom_agent"`	Always `"custom_agent"` for externally-owned agents evaluated via this SDK
`displayName`	`str`	Human-readable identifier shown in the dashboard
`framework`	`"raw_python"` \| `"openai"` \| `"anthropic"` \| `"google"` \| `"langchain"` \| `"llamaindex"` \| `"crewai"` \| `"autogen"` \| `"n8n"` \| `"flowise"` \| `"other"`	Underlying technology stack
`runtime`	`"local"` \| `"ci"` \| `"customer_hosted"` \| `"low_code"`	Where the agent is running
`version`	`str`	Optional version identifier
`endpoint`	`str`	Optional URL, for HTTP-based agents
`agentInstructions`	`str`	Optional system prompt / instructions, checked against actual behavior during analysis

framework and runtime are the only fields required beyond kind and displayName. Set them to "other" and "local" if nothing else fits.

Return values from your agent function

Whatever callable you pass to .execute() can return:

A string: used directly as the output text.
A dict with an "output" key: the output text is extracted; a "trace_id" key links the result to a recorded trace (enabling View trace, trajectory-aware judging, and expected-trajectory matching); a "retrieval_context" key (string or chunk list) feeds RAG judging; everything else in the dict is retained as metadata.
An EvaluationResult: full control over output, metadata, observable trace, and timing:

from agentx.evaluations.models import EvaluationResult

def my_agent(case):
    return EvaluationResult(
        output="Click Forgot Password on the login screen.",
        metadata={"model": "gpt-4o"},
        latency_ms=1240,
        input_tokens=150,
        output_tokens=45,
    )

Sensitive values are redacted automatically before anything is uploaded: API keys (sk-...), bearer tokens, authorization headers, and password-like fields. Raw prompts are never uploaded; only the response text and metadata you explicitly include are sent.