Score outputs you already generated, without re-running the agent
No install needed beyond agentx-python itself. Use PrecomputedAdapter when you already have your agent’s outputs (from a batch job, a low-code tool like n8n, or a previous run) and just want them scored, without invoking a live callable or HTTP endpoint.
Usage
from agentx import AgentXfrom agentx.evaluations.adapters.precomputed import PrecomputedAdapterclient = AgentX.from_env()outputs = { "case-0": "To reset your password, go to Login → Forgot Password.", "case-1": { "output": "We accept Visa, Mastercard, PayPal, and bank transfers.", "metadata": {"source": "n8n-export", "model": "gpt-4o"}, },}adapter = PrecomputedAdapter(outputs)run_context = ( client.evaluations .run(dataset_id="...", subject={"kind": "custom_agent", "displayName": "n8n Batch", "framework": "n8n", "runtime": "low_code"}) .execute(adapter) .finalize())print("Average rating:", run_context.average_rating)
A dict keyed by case_id (the format shown above): "case-{questionIndex}" for a dataset’s Nth question, e.g. "case-0", "case-1". This is the recommended form.
A plain list, positionally matched to question index: outputs[0] is the answer for question 0, and so on. Internally this is just sugar for {"0": outputs[0], "1": outputs[1], ...}.
Each value is either the output text directly (a str), or a dict for full control, using the same shape as any other adapter’s return value: output/text, plus optional metadata, input_tokens, output_tokens, trace_id. Every case in the dataset must have a matching key: a missing key becomes a failed result (empty output), stored with rating 0 and counted into the run’s average and any CI gate.
case_id doesn’t vary by repetition. If the dataset’s config runs each question more than once (number_of_requests > 1), every repetition of a question looks up the same key and gets the same precomputed output submitted for all of them. PrecomputedAdapter is built for the common case of one precomputed answer per question; if you have distinct precomputed outputs per repetition, submit results directly via the REST results endpoint instead (POST /custom-agent-evaluations/runs/:id/results), setting a distinct runNumber and idempotencyKey per call.
{"docsConfig":{"theme":"mint","$schema":"https://mintlify.com/docs.json","name":"AgentX","colors":{"primary":"#2A5699","light":"#5C8DCC","dark":"#1A3F73"},"logo":{"light":"https://mintcdn.com/agentx-ffe8d995/ZTzZ3HDQJ-RJxymw/logo/light.png?fit=max\u0026auto=format\u0026n=ZTzZ3HDQJ-RJxymw\u0026q=85\u0026s=c1e30c949dc1cc2aaa3a137c1ee5c77f","dark":"https://mintcdn.com/agentx-ffe8d995/ZTzZ3HDQJ-RJxymw/logo/dark.png?fit=max\u0026auto=format\u0026n=ZTzZ3HDQJ-RJxymw\u0026q=85\u0026s=35a521e090f7875c15bef7ae392af788"},"favicon":"https://mintcdn.com/agentx-ffe8d995/FRrvmlkH0m2BUb9c/logo/favicon.png?fit=max\u0026auto=format\u0026n=FRrvmlkH0m2BUb9c\u0026q=85\u0026s=e87d636d38c8e6e187b4be187e72381d","api":{"mdx":{"auth":{"method":"key","name":"x-api-key"},"server":"http://localhost:4700"}},"navbar":{"primary":{"type":"github","href":"https://github.com/AgentX-ai/AgentX-trace-eval"}},"footer":{"socials":{"website":"https://www.agentx.so/mcp/ai-evaluation","github":"https://github.com/AgentX-ai/AgentX-trace-eval"}},"styling":{"latex":true},"navigation":{"pages":[]}},"deployment":{"deploymentMetadataValue":{"subdomain":"agentx-ffe8d995","hasAudioNarration":false,"actualSubdomain":"agentx-ffe8d995","customDomains":["developers.agentx.so"],"gitSource":{"type":"github","owner":"agentx-ai","repo":"agentx-mintlify","deployBranch":"main","contentDirectory":"","isPrivate":true,"mountPath":"","label":""},"entitlements":{"AI_CHAT":{"status":"DISABLED"}},"buildId":"6ac683381cb9dd011c327d04","clientVersion":"0.0.4270","searchFilterCounts":[{"language":"default","tag":"API Reference","count":25},{"language":"default","tag":"Offline Evaluation","count":24},{"language":"default","tag":"Trace","count":17},{"language":"default","tag":"Scorers \u0026 Monitoring","count":16},{"language":"default","tag":"Custom Evaluations","count":13},{"language":"default","tag":"Documentation","count":13},{"language":"default","tag":"Framework Integrations","count":12},{"language":"default","tag":"Framework Examples","count":10},{"language":"default","tag":"Tracing","count":8},{"language":"default","tag":"Live Traffic Scoring","count":7}]},"assistantConfig":{"enableAskAiButton":false,"enableStarterQuestions":false,"enableContextualStarterQuestions":false,"enableAskAiSearchBar":true},"basePath":"","hasChatPermissions":false,"hasWebMcpPermissions":false,"toggles":[{"name":"kill-isr","enabled":true,"variant":{"name":"disabled","enabled":false,"feature_enabled":true},"impressionData":false},{"name":"custom-rbac-roles","enabled":true,"variant":{"name":"disabled","enabled":false,"feature_enabled":true},"impressionData":false},{"name":"agent-mintlify-slack","enabled":true,"variant":{"name":"disabled","enabled":false,"feature_enabled":true},"impressionData":false},{"name":"agent-integrations-settings","enabled":true,"variant":{"name":"disabled","enabled":false,"feature_enabled":true},"impressionData":false},{"name":"rbac-expansion","enabled":true,"variant":{"name":"disabled","enabled":false,"feature_enabled":true},"impressionData":false},{"name":"automations-pr-grouping","enabled":true,"variant":{"name":"disabled","enabled":false,"feature_enabled":true},"impressionData":false},{"name":"custom-prefix","enabled":true,"variant":{"name":"disabled","enabled":false,"feature_enabled":true},"impressionData":false},{"name":"analytics-v7","enabled":true,"variant":{"name":"disabled","enabled":false,"feature_enabled":true},"impressionData":false},{"name":"user-flows-analytics","enabled":true,"variant":{"name":"disabled","enabled":false,"feature_enabled":true},"impressionData":false},{"name":"no-publish-flow","enabled":true,"variant":{"name":"disabled","enabled":false,"feature_enabled":true},"impressionData":false},{"name":"workos-onboarding","enabled":true,"variant":{"name":"disabled","enabled":false,"feature_enabled":true},"impressionData":false},{"name":"snapshot-serving-live","enabled":true,"variant":{"name":"disabled","enabled":false,"feature_enabled":true},"impressionData":false},{"name":"stripTemperatureMiddleware","enabled":true,"variant":{"name":"disabled","enabled":false,"feature_enabled":true},"impressionData":false},{"name":"support-platform-pylon-zendesk","enabled":true,"variant":{"name":"disabled","enabled":false,"feature_enabled":true},"impressionData":false},{"name":"snapshot-serving","enabled":true,"variant":{"name":"disabled","enabled":false,"feature_enabled":true},"impressionData":false},{"name":"instant-rollback","enabled":true,"variant":{"name":"disabled","enabled":false,"feature_enabled":true},"impressionData":false},{"name":"granular-notifications-server","enabled":true,"variant":{"name":"disabled","enabled":false,"feature_enabled":true},"impressionData":false},{"name":"granular-notifications","enabled":true,"variant":{"name":"disabled","enabled":false,"feature_enabled":true},"impressionData":false},{"name":"automations-fable","enabled":true,"variant":{"name":"disabled","enabled":false,"feature_enabled":true},"impressionData":false},{"name":"fixed-ai-action-pricing","enabled":true,"variant":{"name":"disabled","enabled":false,"feature_enabled":true},"impressionData":false},{"name":"site-roles","enabled":true,"variant":{"name":"disabled","enabled":false,"feature_enabled":true},"impressionData":false},{"name":"automation-resolve-merge-conflict","enabled":true,"variant":{"name":"disabled","enabled":false,"feature_enabled":true},"impressionData":false},{"name":"scoped-publishing","enabled":true,"variant":{"name":"disabled","enabled":false,"feature_enabled":true},"impressionData":false},{"name":"mintql","enabled":true,"variant":{"name":"disabled","enabled":false,"feature_enabled":true},"impressionData":false},{"name":"page-level-analytics","enabled":true,"variant":{"name":"disabled","enabled":false,"feature_enabled":true},"impressionData":false},{"name":"editor-version-history-v2","enabled":true,"variant":{"name":"disabled","enabled":false,"feature_enabled":true},"impressionData":false},{"name":"billing-page-v2","enabled":true,"variant":{"name":"disabled","enabled":false,"feature_enabled":true},"impressionData":false},{"name":"full-page-search","enabled":true,"variant":{"name":"disabled","enabled":false,"feature_enabled":true},"impressionData":false},{"name":"workspace-projects","enabled":true,"variant":{"name":"disabled","enabled":false,"feature_enabled":true},"impressionData":false},{"name":"assistant-v5","enabled":true,"variant":{"name":"disabled","enabled":false,"feature_enabled":true},"impressionData":false},{"name":"fork-pr-previews","enabled":true,"variant":{"name":"disabled","enabled":false,"feature_enabled":true},"impressionData":false},{"name":"revalidation-jwt","enabled":true,"variant":{"name":"disabled","enabled":false,"feature_enabled":true},"impressionData":false},{"name":"workos-sso-scim","enabled":true,"variant":{"name":"disabled","enabled":false,"feature_enabled":true},"impressionData":false},{"name":"hobby-abuse-scan","enabled":true,"variant":{"name":"disabled","enabled":false,"feature_enabled":true},"impressionData":false},{"name":"dashboard-inbox","enabled":true,"variant":{"name":"disabled","enabled":false,"feature_enabled":true},"impressionData":false},{"name":"workos-signup","enabled":true,"variant":{"name":"disabled","enabled":false,"feature_enabled":true},"impressionData":false},{"name":"turbopuffer-dual-write","enabled":true,"variant":{"name":"disabled","enabled":false,"feature_enabled":true},"impressionData":false},{"name":"workos-enabled","enabled":true,"variant":{"name":"disabled","enabled":false,"feature_enabled":true},"impressionData":false},{"name":"approve-review-flow","enabled":true,"variant":{"name":"disabled","enabled":false,"feature_enabled":true},"impressionData":false},{"name":"editor-metadata","enabled":true,"variant":{"name":"disabled","enabled":false,"feature_enabled":true},"impressionData":false},{"name":"sso-setup-deadline-banner","enabled":true,"variant":{"name":"disabled","enabled":false,"feature_enabled":true},"impressionData":false},{"name":"turbopuffer-serve","enabled":true,"variant":{"name":"disabled","enabled":false,"feature_enabled":true},"impressionData":false},{"name":"live-agents","enabled":true,"variant":{"name":"disabled","enabled":false,"feature_enabled":true},"impressionData":false},{"name":"starter-pricing","enabled":true,"variant":{"name":"control","enabled":true,"payload":{"type":"string","value":"control"},"feature_enabled":true},"impressionData":false},{"name":"llms-txt-v2","enabled":true,"variant":{"name":"disabled","enabled":false,"feature_enabled":true},"impressionData":false},{"name":"wildcard-email-auth","enabled":true,"variant":{"name":"disabled","enabled":false,"feature_enabled":true},"impressionData":false}],"googleFontsInlined":false},"customCss":[]}
{"page":{"pagePath":"sdk/evaluations/examples/precomputed-results","slug":"/sdk/evaluations/examples/precomputed-results","pageMetadata":{"title":"Submit Pre-Defined Results","description":"Score outputs you already generated, without re-running the agent","href":"/sdk/evaluations/examples/precomputed-results"},"description":{"compiledSource":"\"use strict\";\nconst {jsx: _jsx} = arguments[0];\nconst {useMDXComponents: _provideComponents} = arguments[0];\nfunction _createMdxContent(props) {\n const _components = {\n p: \"p\",\n ..._provideComponents(),\n ...props.components\n };\n return _jsx(_components.p, {\n children: \"Score outputs you already generated, without re-running the agent\"\n });\n}\nfunction MDXContent(props = {}) {\n const {wrapper: MDXLayout} = {\n ..._provideComponents(),\n ...props.components\n };\n return MDXLayout ? _jsx(MDXLayout, {\n ...props,\n children: _jsx(_createMdxContent, {\n ...props\n })\n }) : _createMdxContent(props);\n}\nreturn {\n default: MDXContent\n};\n","frontmatter":{},"scope":{}},"pageType":"default","mdx":{"mdxSource":{"compiledSource":"\"use strict\";const{Fragment:_Fragment,jsx:_jsx,jsxs:_jsxs}=arguments[0];const{useMDXComponents:_provideComponents}=arguments[0];function _createMdxContent(props){const _components={a:`a`,code:`code`,li:`li`,p:`p`,pre:`pre`,span:`span`,strong:`strong`,ul:`ul`,..._provideComponents(),...props.components},{CodeBlock,Heading,Note}=_components;if(!CodeBlock)_missingMdxReference(`CodeBlock`,true);if(!Heading)_missingMdxReference(`Heading`,true);if(!Note)_missingMdxReference(`Note`,true);return _jsxs(_Fragment,{children:[_jsxs(_components.p,{children:[`No install needed beyond `,_jsx(_components.code,{children:`agentx-python`}),` itself. Use `,_jsx(_components.code,{children:`PrecomputedAdapter`}),` when you already have your agent’s outputs (from a batch job, a low-code tool like n8n, or a previous run) and just want them scored, without invoking a live callable or HTTP endpoint.`]}),`\n`,_jsx(Heading,{level:`2`,id:`usage`,children:`Usage`}),`\n`,_jsx(CodeBlock,{filename:``,numberOfLines:`22`,language:`python`,children:_jsx(_components.pre,{className:`shiki shiki-themes github-light-default dark-plus`,style:{backgroundColor:`#ffffff`,\"--shiki-dark-bg\":`#0B0C0E`,color:`#1f2328`,\"--shiki-dark\":`#D4D4D4`},language:`python`,children:_jsxs(_components.code,{language:`python`,numberOfLines:`22`,children:[_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#CF222E`,\"--shiki-dark\":`#C586C0`},children:`from`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:` agentx `}),_jsx(_components.span,{style:{color:`#CF222E`,\"--shiki-dark\":`#C586C0`},children:`import`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:` AgentX`})]}),`\n`,_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#CF222E`,\"--shiki-dark\":`#C586C0`},children:`from`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:` agentx.evaluations.adapters.precomputed `}),_jsx(_components.span,{style:{color:`#CF222E`,\"--shiki-dark\":`#C586C0`},children:`import`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:` PrecomputedAdapter`})]}),`\n`,_jsx(_components.span,{className:`line`}),`\n`,_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:`client `}),_jsx(_components.span,{style:{color:`#CF222E`,\"--shiki-dark\":`#D4D4D4`},children:`=`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:` AgentX.from_env()`})]}),`\n`,_jsx(_components.span,{className:`line`}),`\n`,_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:`outputs `}),_jsx(_components.span,{style:{color:`#CF222E`,\"--shiki-dark\":`#D4D4D4`},children:`=`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:` {`})]}),`\n`,_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#0A3069`,\"--shiki-dark\":`#CE9178`},children:` \"case-0\"`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:`: `}),_jsx(_components.span,{style:{color:`#0A3069`,\"--shiki-dark\":`#CE9178`},children:`\"To reset your password, go to Login → Forgot Password.\"`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:`,`})]}),`\n`,_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#0A3069`,\"--shiki-dark\":`#CE9178`},children:` \"case-1\"`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:`: {`})]}),`\n`,_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#0A3069`,\"--shiki-dark\":`#CE9178`},children:` \"output\"`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:`: `}),_jsx(_components.span,{style:{color:`#0A3069`,\"--shiki-dark\":`#CE9178`},children:`\"We accept Visa, Mastercard, PayPal, and bank transfers.\"`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:`,`})]}),`\n`,_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#0A3069`,\"--shiki-dark\":`#CE9178`},children:` \"metadata\"`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:`: {`}),_jsx(_components.span,{style:{color:`#0A3069`,\"--shiki-dark\":`#CE9178`},children:`\"source\"`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:`: `}),_jsx(_components.span,{style:{color:`#0A3069`,\"--shiki-dark\":`#CE9178`},children:`\"n8n-export\"`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:`, `}),_jsx(_components.span,{style:{color:`#0A3069`,\"--shiki-dark\":`#CE9178`},children:`\"model\"`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:`: `}),_jsx(_components.span,{style:{color:`#0A3069`,\"--shiki-dark\":`#CE9178`},children:`\"gpt-4o\"`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:`},`})]}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:` },`})}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:`}`})}),`\n`,_jsx(_components.span,{className:`line`}),`\n`,_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:`adapter `}),_jsx(_components.span,{style:{color:`#CF222E`,\"--shiki-dark\":`#D4D4D4`},children:`=`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:` PrecomputedAdapter(outputs)`})]}),`\n`,_jsx(_components.span,{className:`line`}),`\n`,_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:`run_context `}),_jsx(_components.span,{style:{color:`#CF222E`,\"--shiki-dark\":`#D4D4D4`},children:`=`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:` (`})]}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:` client.evaluations`})}),`\n`,_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:` .run(`}),_jsx(_components.span,{style:{color:`#953800`,\"--shiki-dark\":`#9CDCFE`},children:`dataset_id`}),_jsx(_components.span,{style:{color:`#CF222E`,\"--shiki-dark\":`#D4D4D4`},children:`=`}),_jsx(_components.span,{style:{color:`#0A3069`,\"--shiki-dark\":`#CE9178`},children:`\"...\"`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:`, `}),_jsx(_components.span,{style:{color:`#953800`,\"--shiki-dark\":`#9CDCFE`},children:`subject`}),_jsx(_components.span,{style:{color:`#CF222E`,\"--shiki-dark\":`#D4D4D4`},children:`=`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:`{`}),_jsx(_components.span,{style:{color:`#0A3069`,\"--shiki-dark\":`#CE9178`},children:`\"kind\"`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:`: `}),_jsx(_components.span,{style:{color:`#0A3069`,\"--shiki-dark\":`#CE9178`},children:`\"custom_agent\"`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:`, `}),_jsx(_components.span,{style:{color:`#0A3069`,\"--shiki-dark\":`#CE9178`},children:`\"displayName\"`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:`: `}),_jsx(_components.span,{style:{color:`#0A3069`,\"--shiki-dark\":`#CE9178`},children:`\"n8n Batch\"`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:`, `}),_jsx(_components.span,{style:{color:`#0A3069`,\"--shiki-dark\":`#CE9178`},children:`\"framework\"`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:`: `}),_jsx(_components.span,{style:{color:`#0A3069`,\"--shiki-dark\":`#CE9178`},children:`\"n8n\"`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:`, `}),_jsx(_components.span,{style:{color:`#0A3069`,\"--shiki-dark\":`#CE9178`},children:`\"runtime\"`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:`: `}),_jsx(_components.span,{style:{color:`#0A3069`,\"--shiki-dark\":`#CE9178`},children:`\"low_code\"`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:`})`})]}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:` .execute(adapter)`})}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:` .finalize()`})}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:`)`})}),`\n`,_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#0550AE`,\"--shiki-dark\":`#DCDCAA`},children:`print`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:`(`}),_jsx(_components.span,{style:{color:`#0A3069`,\"--shiki-dark\":`#CE9178`},children:`\"Average rating:\"`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:`, run_context.average_rating)`})]}),`\n`]})})}),`\n`,_jsx(Heading,{level:`3`,id:`precomputedadapter-outputs`,children:_jsx(_components.code,{children:`PrecomputedAdapter(outputs)`})}),`\n`,_jsx(CodeBlock,{filename:``,numberOfLines:`1`,language:`python`,children:_jsx(_components.pre,{className:`shiki shiki-themes github-light-default dark-plus`,style:{backgroundColor:`#ffffff`,\"--shiki-dark-bg\":`#0B0C0E`,color:`#1f2328`,\"--shiki-dark\":`#D4D4D4`},language:`python`,children:_jsxs(_components.code,{language:`python`,numberOfLines:`1`,children:[_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:`PrecomputedAdapter(outputs: list[Any] `}),_jsx(_components.span,{style:{color:`#CF222E`,\"--shiki-dark\":`#D4D4D4`},children:`|`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:` dict[`}),_jsx(_components.span,{style:{color:`#0550AE`,\"--shiki-dark\":`#4EC9B0`},children:`str`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:`, Any])`})]}),`\n`]})})}),`\n`,_jsxs(_components.p,{children:[_jsx(_components.code,{children:`outputs`}),` is either:`]}),`\n`,_jsxs(_components.ul,{children:[`\n`,_jsxs(_components.li,{children:[_jsxs(_components.strong,{children:[`A `,_jsx(_components.code,{children:`dict`}),` keyed by `,_jsx(_components.code,{children:`case_id`})]}),` (the format shown above): `,_jsx(_components.code,{children:`\"case-{questionIndex}\"`}),` for a dataset’s `,_jsx(_components.code,{children:`N`}),`th question, e.g. `,_jsx(_components.code,{children:`\"case-0\"`}),`, `,_jsx(_components.code,{children:`\"case-1\"`}),`. This is the recommended form.`]}),`\n`,_jsxs(_components.li,{children:[_jsxs(_components.strong,{children:[`A plain `,_jsx(_components.code,{children:`list`})]}),`, positionally matched to question index: `,_jsx(_components.code,{children:`outputs[0]`}),` is the answer for question 0, and so on. Internally this is just sugar for `,_jsx(_components.code,{children:`{\"0\": outputs[0], \"1\": outputs[1], ...}`}),`.`]}),`\n`]}),`\n`,_jsxs(_components.p,{children:[`Each value is either the output text directly (a `,_jsx(_components.code,{children:`str`}),`), or a `,_jsx(_components.code,{children:`dict`}),` for full control, using the same shape as any other adapter’s return value: `,_jsx(_components.code,{children:`output`}),`/`,_jsx(_components.code,{children:`text`}),`, plus optional `,_jsx(_components.code,{children:`metadata`}),`, `,_jsx(_components.code,{children:`input_tokens`}),`, `,_jsx(_components.code,{children:`output_tokens`}),`, `,_jsx(_components.code,{children:`trace_id`}),`. Every case in the dataset must have a matching key: a missing key becomes a failed result (empty output), stored with rating 0 and counted into the run’s average and any CI gate.`]}),`\n`,_jsx(Note,{children:_jsxs(_components.p,{children:[_jsx(_components.code,{children:`case_id`}),` doesn’t vary by repetition. If the dataset’s config runs each question more than once (`,_jsx(_components.code,{children:`number_of_requests > 1`}),`), every repetition of a question looks up the `,_jsx(_components.strong,{children:`same`}),` key and gets the `,_jsx(_components.strong,{children:`same`}),` precomputed output submitted for all of them. `,_jsx(_components.code,{children:`PrecomputedAdapter`}),` is built for the common case of one precomputed answer per question; if you have distinct precomputed outputs per repetition, submit results directly via the REST results endpoint instead (`,_jsx(_components.code,{children:`POST /custom-agent-evaluations/runs/:id/results`}),`), setting a distinct `,_jsx(_components.code,{children:`runNumber`}),` and `,_jsx(_components.code,{children:`idempotencyKey`}),` per call.`]})}),`\n`,_jsxs(_components.p,{children:[`A complete working example is available as `,_jsx(_components.a,{href:`https://github.com/AgentX-ai/AgentX-Python/blob/main/examples/evaluations/precomputed_results_eval.py`,children:_jsx(_components.code,{children:`precomputed_results_eval.py`})}),` in the AgentX-Python repository, or `,_jsx(_components.a,{href:`https://github.com/AgentX-ai/AgentX-Python/blob/main/examples/evaluations/csv_import_eval.py`,children:_jsx(_components.code,{children:`csv_import_eval.py`})}),` for loading the dataset itself from a CSV file first (see `,_jsx(_components.a,{href:`/sdk/evaluations/build-dataset#csv-import`,children:`Build Dataset`}),`).`]})]})}function MDXContent(props={}){const{wrapper:MDXLayout}={..._provideComponents(),...props.components};return MDXLayout?_jsx(MDXLayout,{...props,children:_jsx(_createMdxContent,{...props})}):_createMdxContent(props)}return{default:MDXContent};function _missingMdxReference(id,component){throw new Error(`Expected `+(component?`component`:`object`)+\" `\"+id+\"` to be defined: you likely forgot to import, pass, or provide it.\")}","frontmatter":{},"scope":{"config":{},"pageMetadata":{"title":"Submit Pre-Defined Results","description":"Score outputs you already generated, without re-running the agent","href":"/sdk/evaluations/examples/precomputed-results"}}},"mdxSourceWithNoJs":null},"mdxExtracts":{"tableOfContents":[{"title":"Usage","slug":"usage","depth":2,"children":[{"title":"PrecomputedAdapter(outputs)","slug":"precomputedadapter-outputs","depth":3,"children":[]}]}],"codeExamples":{}},"jsLinks":[],"subdomain":"agentx-ffe8d995","siteName":"AgentX","title":"Submit Pre-Defined Results - AgentX","navMaps":{"firstHrefInVersion":{},"firstHrefInLanguage":{}},"lastModified":"2026-09-14T02:27:26.010Z","originalFileLocation":"sdk/evaluations/examples/precomputed-results.mdx"},"headHtml":"\u003cscript data-mint-head>(function(){var d=document.documentElement,p=null;if(!p){try{p=localStorage.getItem(\"isDarkMode\")}catch(e){}if(p==='true')p='dark';else if(p==='false')p='light';if(p!=='light'\u0026\u0026p!=='dark'\u0026\u0026p!=='system')p=\"system\"}var r=p==='system'?(matchMedia(\"(prefers-color-scheme: dark)\").matches?'dark':'light'):p;d.classList.remove(r==='dark'?'light':'dark');d.classList.add(r);d.setAttribute(\"data-theme-preference\",p)})();\u003c/script>\u003cmeta data-mint-head name=\"description\" content=\"Score outputs you already generated, without re-running the agent\">\u003cmeta data-mint-head name=\"application-name\" content=\"AgentX\">\u003cmeta data-mint-head name=\"generator\" content=\"Mintlify\">\u003clink data-mint-head rel=\"icon\" type=\"image/png\" sizes=\"192x192\" href=\"/mintlify-assets/_mintlify/favicons/agentx-ffe8d995/sticQuUfAA1h_xwo/_generated/favicon/android-chrome-192x192.png\" media=\"(prefers-color-scheme: light)\">\u003clink data-mint-head rel=\"apple-touch-icon\" type=\"image/png\" sizes=\"180x180\" href=\"/mintlify-assets/_mintlify/favicons/agentx-ffe8d995/sticQuUfAA1h_xwo/_generated/favicon/apple-touch-icon.png\">\u003clink data-mint-head rel=\"icon\" type=\"image/png\" sizes=\"16x16\" href=\"/mintlify-assets/_mintlify/favicons/agentx-ffe8d995/sticQuUfAA1h_xwo/_generated/favicon/favicon-16x16.png\" media=\"(prefers-color-scheme: light)\">\u003clink data-mint-head rel=\"icon\" type=\"image/png\" sizes=\"32x32\" href=\"/mintlify-assets/_mintlify/favicons/agentx-ffe8d995/sticQuUfAA1h_xwo/_generated/favicon/favicon-32x32.png\" media=\"(prefers-color-scheme: light)\">\u003clink data-mint-head rel=\"shortcut icon\" type=\"image/x-icon\" href=\"/mintlify-assets/_mintlify/favicons/agentx-ffe8d995/sticQuUfAA1h_xwo/_generated/favicon/favicon.ico\" media=\"(prefers-color-scheme: light)\">\u003clink data-mint-head rel=\"icon\" type=\"image/png\" sizes=\"192x192\" href=\"/mintlify-assets/_mintlify/favicons/agentx-ffe8d995/sticQuUfAA1h_xwo/_generated/favicon-dark/android-chrome-192x192.png\" media=\"(prefers-color-scheme: dark)\">\u003clink data-mint-head rel=\"icon\" type=\"image/png\" sizes=\"16x16\" href=\"/mintlify-assets/_mintlify/favicons/agentx-ffe8d995/sticQuUfAA1h_xwo/_generated/favicon-dark/favicon-16x16.png\" media=\"(prefers-color-scheme: dark)\">\u003clink data-mint-head rel=\"icon\" type=\"image/png\" sizes=\"32x32\" href=\"/mintlify-assets/_mintlify/favicons/agentx-ffe8d995/sticQuUfAA1h_xwo/_generated/favicon-dark/favicon-32x32.png\" media=\"(prefers-color-scheme: dark)\">\u003clink data-mint-head rel=\"shortcut icon\" type=\"image/x-icon\" href=\"/mintlify-assets/_mintlify/favicons/agentx-ffe8d995/sticQuUfAA1h_xwo/_generated/favicon-dark/favicon.ico\" media=\"(prefers-color-scheme: dark)\">\u003cmeta data-mint-head property=\"og:title\" content=\"Submit Pre-Defined Results - AgentX\">\u003cmeta data-mint-head property=\"og:description\" content=\"Score outputs you already generated, without re-running the agent\">\u003cmeta data-mint-head property=\"og:site_name\" content=\"AgentX\">\u003cmeta data-mint-head property=\"og:url\" content=\"https://developers.agentx.so/sdk/evaluations/examples/precomputed-results\">\u003cmeta data-mint-head property=\"og:type\" content=\"website\">\u003cmeta data-mint-head property=\"og:image\" content=\"https://agentx-ffe8d995.mintlify.site/_mintlify/api/og/v2?division=Framework+Examples\u0026amp;title=Submit+Pre-Defined+Results\u0026amp;description=Score+outputs+you+already+generated%2C+without+re-running+the+agent\u0026amp;theme=2ebdc38e645cd0f197f78a50\u0026amp;signature=76757410fdce745ec845d53cc2b472c89bc932c17d45c52881dad9a6bc6d82ca\">\u003cmeta data-mint-head property=\"og:image:width\" content=\"1200\">\u003cmeta data-mint-head property=\"og:image:height\" content=\"630\">\u003cmeta data-mint-head name=\"twitter:card\" content=\"summary_large_image\">\u003cmeta data-mint-head name=\"twitter:title\" content=\"Submit Pre-Defined Results - AgentX\">\u003cmeta data-mint-head name=\"twitter:description\" content=\"Score outputs you already generated, without re-running the agent\">\u003cmeta data-mint-head name=\"twitter:image\" content=\"https://agentx-ffe8d995.mintlify.site/_mintlify/api/og/v2?division=Framework+Examples\u0026amp;title=Submit+Pre-Defined+Results\u0026amp;description=Score+outputs+you+already+generated%2C+without+re-running+the+agent\u0026amp;theme=2ebdc38e645cd0f197f78a50\u0026amp;signature=76757410fdce745ec845d53cc2b472c89bc932c17d45c52881dad9a6bc6d82ca\">\u003cmeta data-mint-head name=\"twitter:image:width\" content=\"1200\">\u003cmeta data-mint-head name=\"twitter:image:height\" content=\"630\">\u003cmeta data-mint-head name=\"msapplication-config\" content=\"/mintlify-assets/_mintlify/favicons/agentx-ffe8d995/sticQuUfAA1h_xwo/_generated/favicon/browserconfig.xml\">\u003cmeta data-mint-head name=\"apple-mobile-web-app-title\" content=\"AgentX\">\u003cmeta data-mint-head name=\"msapplication-TileColor\" content=\"#2A5699\">\u003clink data-mint-head rel=\"canonical\" href=\"https://developers.agentx.so/sdk/evaluations/examples/precomputed-results\">\u003clink data-mint-head rel=\"alternate\" type=\"application/xml\" href=\"/sitemap.xml\">\u003clink data-mint-head rel=\"alternate\" type=\"text/markdown\" href=\"/sdk/evaluations/examples/precomputed-results.md\">\u003cscript data-mint-head type=\"application/ld+json\">{\"@context\":\"https://schema.org\",\"@graph\":[{\"@type\":\"Organization\",\"@id\":\"https://developers.agentx.so/#organization\",\"name\":\"AgentX\",\"url\":\"https://developers.agentx.so\",\"logo\":{\"@type\":\"ImageObject\",\"url\":\"https://mintcdn.com/agentx-ffe8d995/ZTzZ3HDQJ-RJxymw/logo/light.png?fit=max\u0026auto=format\u0026n=ZTzZ3HDQJ-RJxymw\u0026q=85\u0026s=c1e30c949dc1cc2aaa3a137c1ee5c77f\"}},{\"@type\":\"WebSite\",\"@id\":\"https://developers.agentx.so#website\",\"name\":\"AgentX\",\"url\":\"https://developers.agentx.so\",\"publisher\":{\"@id\":\"https://developers.agentx.so/#organization\"}},{\"@type\":\"WebPage\",\"@id\":\"https://developers.agentx.so/sdk/evaluations/examples/precomputed-results#webpage\",\"url\":\"https://developers.agentx.so/sdk/evaluations/examples/precomputed-results\",\"name\":\"Submit Pre-Defined Results\",\"description\":\"Score outputs you already generated, without re-running the agent\",\"dateModified\":\"2026-09-14T02:27:26.010Z\",\"isPartOf\":{\"@id\":\"https://developers.agentx.so#website\"},\"breadcrumb\":{\"@id\":\"https://developers.agentx.so/sdk/evaluations/examples/precomputed-results#breadcrumb\"}},{\"@type\":\"BreadcrumbList\",\"@id\":\"https://developers.agentx.so/sdk/evaluations/examples/precomputed-results#breadcrumb\",\"itemListElement\":[{\"@type\":\"ListItem\",\"position\":1,\"name\":\"Framework Examples\",\"item\":\"https://developers.agentx.so/sdk/evaluations/examples/overview\"},{\"@type\":\"ListItem\",\"position\":2,\"name\":\"Submit Pre-Defined Results\",\"item\":\"https://developers.agentx.so/sdk/evaluations/examples/precomputed-results\"}]},{\"@type\":[\"Article\",\"TechArticle\"],\"@id\":\"https://developers.agentx.so/sdk/evaluations/examples/precomputed-results#article\",\"headline\":\"Submit Pre-Defined Results\",\"name\":\"Submit Pre-Defined Results\",\"description\":\"Score outputs you already generated, without re-running the agent\",\"url\":\"https://developers.agentx.so/sdk/evaluations/examples/precomputed-results\",\"mainEntityOfPage\":{\"@id\":\"https://developers.agentx.so/sdk/evaluations/examples/precomputed-results#webpage\"},\"dateModified\":\"2026-09-14T02:27:26.010Z\",\"publisher\":{\"@id\":\"https://developers.agentx.so/#organization\"},\"isPartOf\":{\"@id\":\"https://developers.agentx.so#website\"}}]}\u003c/script>\u003cscript data-mint-head type=\"application/ld+json\">{\"@context\":\"https://schema.org\",\"@type\":\"WebSite\",\"name\":\"AgentX\",\"publisher\":{\"@type\":\"Organization\",\"name\":\"AgentX\"},\"provider\":{\"@type\":\"Organization\",\"name\":\"Mintlify\",\"url\":\"https://mintlify.com\"}}\u003c/script>","agentPointerHref":"/sdk/evaluations/examples/precomputed-results.md","siteName":"AgentX","nav":{"global":null,"tabs":[{"tab":"Documentation","pages":[{"title":"Documentation","href":"/introduction"}]},{"tab":"Trace","pages":[{"title":"Trace","href":"/sdk/tracing"}]},{"tab":"Scorers \u0026 Monitoring","pages":[{"title":"Scorers \u0026 Monitoring","href":"/monitor/scorers"}]},{"tab":"Offline Evaluation","groups":[{"group":"Pre-Deployment Quality Gate","pages":[{"title":"Offline Evaluation","href":"/sdk/evaluations/overview"},{"title":"Installation","href":"/sdk/evaluations/installation"},{"title":"Quick Start","href":"/sdk/evaluations/quickstart"}]},{"group":"Build \u0026 Run","pages":[{"title":"Build Dataset","href":"/sdk/evaluations/build-dataset"},{"title":"Evaluation Settings","href":"/sdk/evaluations/evaluation-settings"},{"title":"EvaluationSubject Fields","href":"/sdk/evaluations/evaluation-subject"},{"title":"AI Analysis Report","href":"/sdk/evaluations/analysis-report"},{"title":"Head-to-Head Comparison","href":"/evaluation/head-to-head"}]},{"group":"Datasets \u0026 RAG","pages":[{"title":"Datasets from Production","href":"/evaluation/datasets-from-production"},{"title":"RAG Evaluation","href":"/evaluation/rag"},{"title":"Version History","href":"/evaluation/version-history"},{"title":"Dataset Coverage","href":"/evaluation/dataset-coverage"}]},{"group":"Playground","pages":[{"title":"Playground","href":"/evaluation/playground"},{"title":"Simulate Conversation","href":"/evaluation/simulate-conversation"}]},{"group":"Framework Examples","pages":[{"title":"Examples","href":"/sdk/evaluations/examples/overview"},{"title":"OpenAI","href":"/sdk/evaluations/examples/openai"},{"title":"OpenAI Agents SDK","href":"/sdk/evaluations/examples/openai-agents"},{"title":"Anthropic Claude","href":"/sdk/evaluations/examples/anthropic"},{"title":"LangChain","href":"/sdk/evaluations/examples/langchain"},{"title":"CrewAI","href":"/sdk/evaluations/examples/crewai"},{"title":"AutoGen / AG2","href":"/sdk/evaluations/examples/autogen"},{"title":"LlamaIndex","href":"/sdk/evaluations/examples/llamaindex"},{"title":"HTTP Endpoint","description":"Evaluate an agent exposed as a web service, without writing a Python callable","href":"/sdk/evaluations/examples/http-endpoint"},{"title":"Submit Pre-Defined Results","href":"/sdk/evaluations/examples/precomputed-results"}]}]},{"tab":"Improve","pages":[{"title":"Improve","href":"/improve/tool-schemas"}]},{"tab":"CI/CD","pages":[{"title":"CI/CD","href":"/integrations/self-host-ci"}]},{"tab":"API Reference","pages":[{"title":"API Reference","href":"/api-reference/tracing/submit-trace"}]}]},"basePath":"","title":"Submit Pre-Defined Results - AgentX","deploymentId":"dpl_DyXGnKeCjZyA3sUdfMJ7FjkdBefg","buildId":"dpl_DyXGnKeCjZyA3sUdfMJ7FjkdBefg","dynamics":[],"chunks":[]}