Submits up to 10 agent outputs for a run. Each new result is scored synchronously inside this
request - a judge LLM call per result, plus any enabled similarity metrics and code scorers -
so allow a generous client timeout for judged batches. Submissions are idempotent: resubmitting
an idempotencyKey the run has already accepted returns the stored score without re-scoring.
Send results as you collect them rather than accumulating everything client-side: every batch
response carries liveStatistics, so the run’s average rating is observable while the run is
still executing.
Authentication
string
required
Project API key.
Path Parameters
string
required
Run ID returned by POST /runs.
Body
string
required
Identifier for this batch, stored on each result row.
array
required
Array of 1-10 result objects. Each result has:
Show Result object fields
string
required
Unique key for this result within the run. Prevents duplicate scoring on retry (the
engine also enforces uniqueness on (runId, idempotencyKey) at insert, so a racing
duplicate counts as a duplicate, never an error). Recommended format:
{runId}:{caseId}:run-{runNumber}. A result without one is counted in
failedValidation and dropped.
number
default:"0"
0-based index into the dataset’s questions array. Selects the case’s
expectedResults, judgeGuideline, and other per-case grading inputs.
number
default:"1"
Which repetition this is (1 through numberOfRequests).
string
Case identifier. Defaults to case-{questionIndex}.
object
Agent input: { "query": string }.
object
Agent output: { "text": string }. Required unless error is set - a result with
neither is counted in failedValidation and dropped.
object
Error that occurred: { "type": string, "message": string }. When set, the result is
stored with status "failed" and rating 0; no judge call is made.
string
ID of a trace previously ingested via
Submit Trace (the Python SDK’s
client.tracer.trace(..., sync=True) returns it as span.trace_id). Linking a trace
unlocks trajectory-aware judging (the judge sees the actual tool-call path), the
trajectory and context match scorers, tool definitions for toolContext: "detailed"
configs, and the dashboard’s View Trace on the result.
string | string[]
What the agent actually retrieved for this case - a string or a list of chunk strings.
Feeds {context}-referencing judge prompts (the RAG metric pack) and the context match
scorer. When absent, the engine falls back to the linked trace’s recorded retrieval
spans, then the case’s pinned retrievalContext.
object
Performance metrics: { "latencyMs": number, "inputTokens": number, "outputTokens": number }.
boolean
default:"false"
Marks this row as a paraphrase variant from the run’s smokeTestVariants. Variant rows
are excluded from every rating aggregate - liveStatistics, the run’s averageRating
and scorerBreakdown, case statistics, low-scoring cases, and CI-gate aggregates - but
still count toward the run’s resultCount.
string
The variant phrasing that was actually asked, when isSmokeTestVariant is true.
The hosted platform’s observableTrace and metadata result fields are ignored by the
self-host engine - they are accepted without error but not stored. Link a real trace via
traceId instead; it powers strictly more (trajectory judging, timeline, scorers).
Response
string
Echoed back.
string
Echoed back.
number
New results scored and stored in this call.
number
Results skipped because their idempotencyKey already existed on this run.
number
Results dropped (missing idempotencyKey, or both output.text and error absent).
string
The run’s status, e.g. "in_progress".
array
Per-result scores, in submission order (validation-dropped results are omitted):
[{ idempotencyKey, rating, justification, status, vectorSimilarity, jaccardSimilarity, bleuScore, rougeScore, codeScorerResults, judgeScorerResults, deduped? }].
judgeScorerResults carries one { scorerId, name, rating, justification } entry per
additional judge scorer (or scorer-group judge member) on the run, null when there are
none.
status is "scored", "skipped" (judge could not score - the reason is in
justification), or "failed" (the result carried an error). Duplicates echo the stored
first-submission payload with deduped: true.
object
Rating aggregate recomputed after this batch:
{ averageRating, minRating, maxRating, ratedCount, skippedCount, failedCount }. Available
on every batch response, no Analyze Run required;
the Python SDK exposes the same numbers as run.average_rating and friends.
Errors
Status
Body
Meaning
400
{ "error": "batchId must be a non-empty string" }
Missing or invalid batchId
400
{ "error": "results must be a non-empty array" }
Missing or empty results
400
{ "error": "Batch size must not exceed 10" }
More than 10 results in one call
400
{ "error": "Every results entry must be an object" }
A results element is null, an array, or not an object
404
{ "error": "Run not found" }
No run with this id in the project
409
{ "error": "Run is already in a terminal state" }
The run is "completed" or "failed"; no more results are accepted
409
{ "error": "The scorer group grading this run no longer exists" }
The run’s scorer group was deleted mid-run; results are refused rather than silently regraded on the dataset’s own rubric
{"docsConfig":{"theme":"mint","$schema":"https://mintlify.com/docs.json","name":"AgentX","colors":{"primary":"#2A5699","light":"#5C8DCC","dark":"#1A3F73"},"logo":{"light":"https://mintcdn.com/agentx-ffe8d995/ZTzZ3HDQJ-RJxymw/logo/light.png?fit=max\u0026auto=format\u0026n=ZTzZ3HDQJ-RJxymw\u0026q=85\u0026s=c1e30c949dc1cc2aaa3a137c1ee5c77f","dark":"https://mintcdn.com/agentx-ffe8d995/ZTzZ3HDQJ-RJxymw/logo/dark.png?fit=max\u0026auto=format\u0026n=ZTzZ3HDQJ-RJxymw\u0026q=85\u0026s=35a521e090f7875c15bef7ae392af788"},"favicon":"https://mintcdn.com/agentx-ffe8d995/FRrvmlkH0m2BUb9c/logo/favicon.png?fit=max\u0026auto=format\u0026n=FRrvmlkH0m2BUb9c\u0026q=85\u0026s=e87d636d38c8e6e187b4be187e72381d","api":{"mdx":{"auth":{"method":"key","name":"x-api-key"},"server":"http://localhost:4700"}},"navbar":{"primary":{"type":"github","href":"https://github.com/AgentX-ai/AgentX-trace-eval"}},"footer":{"socials":{"website":"https://www.agentx.so/mcp/ai-evaluation","github":"https://github.com/AgentX-ai/AgentX-trace-eval"}},"styling":{"latex":true},"navigation":{"pages":[]}},"deployment":{"deploymentMetadataValue":{"subdomain":"agentx-ffe8d995","hasAudioNarration":false,"actualSubdomain":"agentx-ffe8d995","customDomains":["developers.agentx.so"],"gitSource":{"type":"github","owner":"agentx-ai","repo":"agentx-mintlify","deployBranch":"main","contentDirectory":"","isPrivate":true,"mountPath":"","label":""},"entitlements":{"AI_CHAT":{"status":"DISABLED"}},"buildId":"6ac683381cb9dd011c327d04","clientVersion":"0.0.4270","searchFilterCounts":[{"language":"default","tag":"API Reference","count":25},{"language":"default","tag":"Offline Evaluation","count":24},{"language":"default","tag":"Trace","count":17},{"language":"default","tag":"Scorers \u0026 Monitoring","count":16},{"language":"default","tag":"Custom Evaluations","count":13},{"language":"default","tag":"Documentation","count":13},{"language":"default","tag":"Framework Integrations","count":12},{"language":"default","tag":"Framework Examples","count":10},{"language":"default","tag":"Tracing","count":8},{"language":"default","tag":"Live Traffic Scoring","count":7}]},"assistantConfig":{"enableAskAiButton":false,"enableStarterQuestions":false,"enableContextualStarterQuestions":false,"enableAskAiSearchBar":true},"basePath":"","hasChatPermissions":false,"hasWebMcpPermissions":false,"toggles":[{"name":"kill-isr","enabled":true,"variant":{"name":"disabled","enabled":false,"feature_enabled":true},"impressionData":false},{"name":"custom-rbac-roles","enabled":true,"variant":{"name":"disabled","enabled":false,"feature_enabled":true},"impressionData":false},{"name":"agent-mintlify-slack","enabled":true,"variant":{"name":"disabled","enabled":false,"feature_enabled":true},"impressionData":false},{"name":"agent-integrations-settings","enabled":true,"variant":{"name":"disabled","enabled":false,"feature_enabled":true},"impressionData":false},{"name":"rbac-expansion","enabled":true,"variant":{"name":"disabled","enabled":false,"feature_enabled":true},"impressionData":false},{"name":"automations-pr-grouping","enabled":true,"variant":{"name":"disabled","enabled":false,"feature_enabled":true},"impressionData":false},{"name":"custom-prefix","enabled":true,"variant":{"name":"disabled","enabled":false,"feature_enabled":true},"impressionData":false},{"name":"analytics-v7","enabled":true,"variant":{"name":"disabled","enabled":false,"feature_enabled":true},"impressionData":false},{"name":"user-flows-analytics","enabled":true,"variant":{"name":"disabled","enabled":false,"feature_enabled":true},"impressionData":false},{"name":"no-publish-flow","enabled":true,"variant":{"name":"disabled","enabled":false,"feature_enabled":true},"impressionData":false},{"name":"workos-onboarding","enabled":true,"variant":{"name":"disabled","enabled":false,"feature_enabled":true},"impressionData":false},{"name":"snapshot-serving-live","enabled":true,"variant":{"name":"disabled","enabled":false,"feature_enabled":true},"impressionData":false},{"name":"stripTemperatureMiddleware","enabled":true,"variant":{"name":"disabled","enabled":false,"feature_enabled":true},"impressionData":false},{"name":"support-platform-pylon-zendesk","enabled":true,"variant":{"name":"disabled","enabled":false,"feature_enabled":true},"impressionData":false},{"name":"snapshot-serving","enabled":true,"variant":{"name":"disabled","enabled":false,"feature_enabled":true},"impressionData":false},{"name":"instant-rollback","enabled":true,"variant":{"name":"disabled","enabled":false,"feature_enabled":true},"impressionData":false},{"name":"granular-notifications-server","enabled":true,"variant":{"name":"disabled","enabled":false,"feature_enabled":true},"impressionData":false},{"name":"granular-notifications","enabled":true,"variant":{"name":"disabled","enabled":false,"feature_enabled":true},"impressionData":false},{"name":"automations-fable","enabled":true,"variant":{"name":"disabled","enabled":false,"feature_enabled":true},"impressionData":false},{"name":"fixed-ai-action-pricing","enabled":true,"variant":{"name":"disabled","enabled":false,"feature_enabled":true},"impressionData":false},{"name":"site-roles","enabled":true,"variant":{"name":"disabled","enabled":false,"feature_enabled":true},"impressionData":false},{"name":"automation-resolve-merge-conflict","enabled":true,"variant":{"name":"disabled","enabled":false,"feature_enabled":true},"impressionData":false},{"name":"scoped-publishing","enabled":true,"variant":{"name":"disabled","enabled":false,"feature_enabled":true},"impressionData":false},{"name":"mintql","enabled":true,"variant":{"name":"disabled","enabled":false,"feature_enabled":true},"impressionData":false},{"name":"page-level-analytics","enabled":true,"variant":{"name":"disabled","enabled":false,"feature_enabled":true},"impressionData":false},{"name":"editor-version-history-v2","enabled":true,"variant":{"name":"disabled","enabled":false,"feature_enabled":true},"impressionData":false},{"name":"billing-page-v2","enabled":true,"variant":{"name":"disabled","enabled":false,"feature_enabled":true},"impressionData":false},{"name":"full-page-search","enabled":true,"variant":{"name":"disabled","enabled":false,"feature_enabled":true},"impressionData":false},{"name":"workspace-projects","enabled":true,"variant":{"name":"disabled","enabled":false,"feature_enabled":true},"impressionData":false},{"name":"assistant-v5","enabled":true,"variant":{"name":"disabled","enabled":false,"feature_enabled":true},"impressionData":false},{"name":"fork-pr-previews","enabled":true,"variant":{"name":"disabled","enabled":false,"feature_enabled":true},"impressionData":false},{"name":"revalidation-jwt","enabled":true,"variant":{"name":"disabled","enabled":false,"feature_enabled":true},"impressionData":false},{"name":"workos-sso-scim","enabled":true,"variant":{"name":"disabled","enabled":false,"feature_enabled":true},"impressionData":false},{"name":"hobby-abuse-scan","enabled":true,"variant":{"name":"disabled","enabled":false,"feature_enabled":true},"impressionData":false},{"name":"dashboard-inbox","enabled":true,"variant":{"name":"disabled","enabled":false,"feature_enabled":true},"impressionData":false},{"name":"workos-signup","enabled":true,"variant":{"name":"disabled","enabled":false,"feature_enabled":true},"impressionData":false},{"name":"turbopuffer-dual-write","enabled":true,"variant":{"name":"disabled","enabled":false,"feature_enabled":true},"impressionData":false},{"name":"workos-enabled","enabled":true,"variant":{"name":"disabled","enabled":false,"feature_enabled":true},"impressionData":false},{"name":"approve-review-flow","enabled":true,"variant":{"name":"disabled","enabled":false,"feature_enabled":true},"impressionData":false},{"name":"editor-metadata","enabled":true,"variant":{"name":"disabled","enabled":false,"feature_enabled":true},"impressionData":false},{"name":"sso-setup-deadline-banner","enabled":true,"variant":{"name":"disabled","enabled":false,"feature_enabled":true},"impressionData":false},{"name":"turbopuffer-serve","enabled":true,"variant":{"name":"disabled","enabled":false,"feature_enabled":true},"impressionData":false},{"name":"live-agents","enabled":true,"variant":{"name":"disabled","enabled":false,"feature_enabled":true},"impressionData":false},{"name":"starter-pricing","enabled":true,"variant":{"name":"control","enabled":true,"payload":{"type":"string","value":"control"},"feature_enabled":true},"impressionData":false},{"name":"llms-txt-v2","enabled":true,"variant":{"name":"disabled","enabled":false,"feature_enabled":true},"impressionData":false},{"name":"wildcard-email-auth","enabled":true,"variant":{"name":"disabled","enabled":false,"feature_enabled":true},"impressionData":false}],"googleFontsInlined":false},"customCss":[]}
{"page":{"pagePath":"api-reference/custom-eval/submit-results","slug":"/api-reference/custom-eval/submit-results","pageMetadata":{"title":"Submit Results","api":"POST /api/v1/custom-agent-evaluations/runs/{runId}/results","description":"Submit a batch of agent outputs for synchronous judge scoring","href":"/api-reference/custom-eval/submit-results"},"description":{"compiledSource":"\"use strict\";\nconst {jsx: _jsx} = arguments[0];\nconst {useMDXComponents: _provideComponents} = arguments[0];\nfunction _createMdxContent(props) {\n const _components = {\n p: \"p\",\n ..._provideComponents(),\n ...props.components\n };\n return _jsx(_components.p, {\n children: \"Submit a batch of agent outputs for synchronous judge scoring\"\n });\n}\nfunction MDXContent(props = {}) {\n const {wrapper: MDXLayout} = {\n ..._provideComponents(),\n ...props.components\n };\n return MDXLayout ? _jsx(MDXLayout, {\n ...props,\n children: _jsx(_createMdxContent, {\n ...props\n })\n }) : _createMdxContent(props);\n}\nreturn {\n default: MDXContent\n};\n","frontmatter":{},"scope":{}},"pageType":"default","mdx":{"mdxSource":{"compiledSource":"\"use strict\";const{Fragment:_Fragment,jsx:_jsx,jsxs:_jsxs}=arguments[0];const{useMDXComponents:_provideComponents}=arguments[0];function _createMdxContent(props){const _components={a:`a`,code:`code`,p:`p`,strong:`strong`,tbody:`tbody`,td:`td`,th:`th`,thead:`thead`,tr:`tr`,..._provideComponents(),...props.components},{Expandable,Heading,Note,ParamField,ResponseField,Table}=_components;if(!Expandable)_missingMdxReference(`Expandable`,true);if(!Heading)_missingMdxReference(`Heading`,true);if(!Note)_missingMdxReference(`Note`,true);if(!ParamField)_missingMdxReference(`ParamField`,true);if(!ResponseField)_missingMdxReference(`ResponseField`,true);if(!Table)_missingMdxReference(`Table`,true);return _jsxs(_Fragment,{children:[_jsxs(_components.p,{children:[`Submits up to 10 agent outputs for a run. Each new result is scored `,_jsx(_components.strong,{children:`synchronously inside this\nrequest`}),` - a judge LLM call per result, plus any enabled similarity metrics and code scorers -\nso allow a generous client timeout for judged batches. Submissions are idempotent: resubmitting\nan `,_jsx(_components.code,{children:`idempotencyKey`}),` the run has already accepted returns the stored score without re-scoring.`]}),`\n`,_jsx(Note,{children:_jsxs(_components.p,{children:[`Send results as you collect them rather than accumulating everything client-side: every batch\nresponse carries `,_jsx(_components.code,{children:`liveStatistics`}),`, so the run’s average rating is observable while the run is\nstill executing.`]})}),`\n`,_jsx(Heading,{level:`2`,id:`authentication`,children:`Authentication`}),`\n`,_jsx(ParamField,{header:`x-api-key`,type:`string`,required:true,id:`param-x-api-key`,children:_jsx(_components.p,{children:`Project API key.`})}),`\n`,_jsx(Heading,{level:`2`,id:`path-parameters`,children:`Path Parameters`}),`\n`,_jsx(ParamField,{path:`runId`,type:`string`,required:true,id:`param-run-id`,children:_jsxs(_components.p,{children:[`Run ID returned by `,_jsx(_components.code,{children:`POST /runs`}),`.`]})}),`\n`,_jsx(Heading,{level:`2`,id:`body`,children:`Body`}),`\n`,_jsx(ParamField,{body:`batchId`,type:`string`,required:true,id:`param-batch-id`,children:_jsx(_components.p,{children:`Identifier for this batch, stored on each result row.`})}),`\n`,_jsxs(ParamField,{body:`results`,type:`array`,required:true,id:`param-results`,children:[_jsx(_components.p,{children:`Array of 1-10 result objects. Each result has:`}),_jsxs(Expandable,{title:`Result object fields`,children:[_jsx(ParamField,{body:`idempotencyKey`,type:`string`,required:true,id:`param-idempotency-key`,children:_jsxs(_components.p,{children:[`Unique key for this result within the run. Prevents duplicate scoring on retry (the\nengine also enforces uniqueness on `,_jsx(_components.code,{children:`(runId, idempotencyKey)`}),` at insert, so a racing\nduplicate counts as a duplicate, never an error). Recommended format:\n`,_jsx(_components.code,{children:`{runId}:{caseId}:run-{runNumber}`}),`. A result without one is counted in\n`,_jsx(_components.code,{children:`failedValidation`}),` and dropped.`]})}),_jsx(ParamField,{body:`questionIndex`,type:`number`,default:`0`,id:`param-question-index`,children:_jsxs(_components.p,{children:[`0-based index into the dataset’s `,_jsx(_components.code,{children:`questions`}),` array. Selects the case’s\n`,_jsx(_components.code,{children:`expectedResults`}),`, `,_jsx(_components.code,{children:`judgeGuideline`}),`, and other per-case grading inputs.`]})}),_jsx(ParamField,{body:`runNumber`,type:`number`,default:`1`,id:`param-run-number`,children:_jsxs(_components.p,{children:[`Which repetition this is (1 through `,_jsx(_components.code,{children:`numberOfRequests`}),`).`]})}),_jsx(ParamField,{body:`caseId`,type:`string`,id:`param-case-id`,children:_jsxs(_components.p,{children:[`Case identifier. Defaults to `,_jsx(_components.code,{children:`case-{questionIndex}`}),`.`]})}),_jsx(ParamField,{body:`input`,type:`object`,id:`param-input`,children:_jsxs(_components.p,{children:[`Agent input: `,_jsx(_components.code,{children:`{ \"query\": string }`}),`.`]})}),_jsx(ParamField,{body:`output`,type:`object`,id:`param-output`,children:_jsxs(_components.p,{children:[`Agent output: `,_jsx(_components.code,{children:`{ \"text\": string }`}),`. Required unless `,_jsx(_components.code,{children:`error`}),` is set - a result with\nneither is counted in `,_jsx(_components.code,{children:`failedValidation`}),` and dropped.`]})}),_jsx(ParamField,{body:`error`,type:`object`,id:`param-error`,children:_jsxs(_components.p,{children:[`Error that occurred: `,_jsx(_components.code,{children:`{ \"type\": string, \"message\": string }`}),`. When set, the result is\nstored with status `,_jsx(_components.code,{children:`\"failed\"`}),` and rating 0; no judge call is made.`]})}),_jsx(ParamField,{body:`traceId`,type:`string`,id:`param-trace-id`,children:_jsxs(_components.p,{children:[`ID of a trace previously ingested via\n`,_jsx(_components.a,{href:`/api-reference/tracing/submit-trace`,children:`Submit Trace`}),` (the Python SDK’s\n`,_jsx(_components.code,{children:`client.tracer.trace(..., sync=True)`}),` returns it as `,_jsx(_components.code,{children:`span.trace_id`}),`). Linking a trace\nunlocks trajectory-aware judging (the judge sees the actual tool-call path), the\ntrajectory and context match scorers, tool definitions for `,_jsx(_components.code,{children:`toolContext: \"detailed\"`}),`\nconfigs, and the dashboard’s View Trace on the result.`]})}),_jsx(ParamField,{body:`retrievalContext`,type:`string | string[]`,id:`param-retrieval-context`,children:_jsxs(_components.p,{children:[`What the agent actually retrieved for this case - a string or a list of chunk strings.\nFeeds `,_jsx(_components.code,{children:`{context}`}),`-referencing judge prompts (the RAG metric pack) and the context match\nscorer. When absent, the engine falls back to the linked trace’s recorded retrieval\nspans, then the case’s pinned `,_jsx(_components.code,{children:`retrievalContext`}),`.`]})}),_jsx(ParamField,{body:`timings`,type:`object`,id:`param-timings`,children:_jsxs(_components.p,{children:[`Performance metrics: `,_jsx(_components.code,{children:`{ \"latencyMs\": number, \"inputTokens\": number, \"outputTokens\": number }`}),`.`]})}),_jsx(ParamField,{body:`isSmokeTestVariant`,type:`boolean`,default:`false`,id:`param-is-smoke-test-variant`,children:_jsxs(_components.p,{children:[`Marks this row as a paraphrase variant from the run’s `,_jsx(_components.code,{children:`smokeTestVariants`}),`. Variant rows\nare excluded from every rating aggregate - `,_jsx(_components.code,{children:`liveStatistics`}),`, the run’s `,_jsx(_components.code,{children:`averageRating`}),`\nand `,_jsx(_components.code,{children:`scorerBreakdown`}),`, case statistics, low-scoring cases, and CI-gate aggregates - but\nstill count toward the run’s `,_jsx(_components.code,{children:`resultCount`}),`.`]})}),_jsx(ParamField,{body:`smokeTestVariantText`,type:`string`,id:`param-smoke-test-variant-text`,children:_jsxs(_components.p,{children:[`The variant phrasing that was actually asked, when `,_jsx(_components.code,{children:`isSmokeTestVariant`}),` is `,_jsx(_components.code,{children:`true`}),`.`]})})]}),_jsx(Note,{children:_jsxs(_components.p,{children:[`The hosted platform’s `,_jsx(_components.code,{children:`observableTrace`}),` and `,_jsx(_components.code,{children:`metadata`}),` result fields are ignored by the\nself-host engine - they are accepted without error but not stored. Link a real trace via\n`,_jsx(_components.code,{children:`traceId`}),` instead; it powers strictly more (trajectory judging, timeline, scorers).`]})})]}),`\n`,_jsx(Heading,{level:`2`,id:`response`,children:`Response`}),`\n`,_jsx(ResponseField,{name:`runId`,type:`string`,id:`param-run-id-1`,children:_jsx(_components.p,{children:`Echoed back.`})}),`\n`,_jsx(ResponseField,{name:`batchId`,type:`string`,id:`param-batch-id-1`,children:_jsx(_components.p,{children:`Echoed back.`})}),`\n`,_jsx(ResponseField,{name:`accepted`,type:`number`,id:`param-accepted`,children:_jsx(_components.p,{children:`New results scored and stored in this call.`})}),`\n`,_jsx(ResponseField,{name:`duplicates`,type:`number`,id:`param-duplicates`,children:_jsxs(_components.p,{children:[`Results skipped because their `,_jsx(_components.code,{children:`idempotencyKey`}),` already existed on this run.`]})}),`\n`,_jsx(ResponseField,{name:`failedValidation`,type:`number`,id:`param-failed-validation`,children:_jsxs(_components.p,{children:[`Results dropped (missing `,_jsx(_components.code,{children:`idempotencyKey`}),`, or both `,_jsx(_components.code,{children:`output.text`}),` and `,_jsx(_components.code,{children:`error`}),` absent).`]})}),`\n`,_jsx(ResponseField,{name:`status`,type:`string`,id:`param-status`,children:_jsxs(_components.p,{children:[`The run’s status, e.g. `,_jsx(_components.code,{children:`\"in_progress\"`}),`.`]})}),`\n`,_jsx(ResponseField,{name:`scoredResults`,type:`array`,id:`param-scored-results`,children:_jsxs(_components.p,{children:[`Per-result scores, in submission order (validation-dropped results are omitted):\n`,_jsx(_components.code,{children:`[{ idempotencyKey, rating, justification, status, vectorSimilarity, jaccardSimilarity, bleuScore, rougeScore, codeScorerResults, judgeScorerResults, deduped? }]`}),`.\n`,_jsx(_components.code,{children:`judgeScorerResults`}),` carries one `,_jsx(_components.code,{children:`{ scorerId, name, rating, justification }`}),` entry per\nadditional judge scorer (or scorer-group judge member) on the run, `,_jsx(_components.code,{children:`null`}),` when there are\nnone.\n`,_jsx(_components.code,{children:`status`}),` is `,_jsx(_components.code,{children:`\"scored\"`}),`, `,_jsx(_components.code,{children:`\"skipped\"`}),` (judge could not score - the reason is in\n`,_jsx(_components.code,{children:`justification`}),`), or `,_jsx(_components.code,{children:`\"failed\"`}),` (the result carried an `,_jsx(_components.code,{children:`error`}),`). Duplicates echo the stored\nfirst-submission payload with `,_jsx(_components.code,{children:`deduped: true`}),`.`]})}),`\n`,_jsx(ResponseField,{name:`liveStatistics`,type:`object`,id:`param-live-statistics`,children:_jsxs(_components.p,{children:[`Rating aggregate recomputed after this batch:\n`,_jsx(_components.code,{children:`{ averageRating, minRating, maxRating, ratedCount, skippedCount, failedCount }`}),`. Available\non every batch response, no `,_jsx(_components.a,{href:`/api-reference/custom-eval/analyze-run`,children:`Analyze Run`}),` required;\nthe Python SDK exposes the same numbers as `,_jsx(_components.code,{children:`run.average_rating`}),` and friends.`]})}),`\n`,_jsx(Heading,{level:`2`,id:`errors`,children:`Errors`}),`\n`,_jsxs(Table,{children:[_jsx(_components.thead,{children:_jsxs(_components.tr,{children:[_jsx(_components.th,{children:`Status`}),_jsx(_components.th,{children:`Body`}),_jsx(_components.th,{children:`Meaning`})]})}),_jsxs(_components.tbody,{children:[_jsxs(_components.tr,{children:[_jsx(_components.td,{\"data-numeric\":`true`,children:_jsx(_components.code,{children:`400`})}),_jsx(_components.td,{children:_jsx(_components.code,{children:`{ \"error\": \"batchId must be a non-empty string\" }`})}),_jsxs(_components.td,{children:[`Missing or invalid `,_jsx(_components.code,{children:`batchId`})]})]}),_jsxs(_components.tr,{children:[_jsx(_components.td,{\"data-numeric\":`true`,children:_jsx(_components.code,{children:`400`})}),_jsx(_components.td,{children:_jsx(_components.code,{children:`{ \"error\": \"results must be a non-empty array\" }`})}),_jsxs(_components.td,{children:[`Missing or empty `,_jsx(_components.code,{children:`results`})]})]}),_jsxs(_components.tr,{children:[_jsx(_components.td,{\"data-numeric\":`true`,children:_jsx(_components.code,{children:`400`})}),_jsx(_components.td,{children:_jsx(_components.code,{children:`{ \"error\": \"Batch size must not exceed 10\" }`})}),_jsx(_components.td,{children:`More than 10 results in one call`})]}),_jsxs(_components.tr,{children:[_jsx(_components.td,{\"data-numeric\":`true`,children:_jsx(_components.code,{children:`400`})}),_jsx(_components.td,{children:_jsx(_components.code,{children:`{ \"error\": \"Every results entry must be an object\" }`})}),_jsxs(_components.td,{children:[`A `,_jsx(_components.code,{children:`results`}),` element is `,_jsx(_components.code,{children:`null`}),`, an array, or not an object`]})]}),_jsxs(_components.tr,{children:[_jsx(_components.td,{\"data-numeric\":`true`,children:_jsx(_components.code,{children:`404`})}),_jsx(_components.td,{children:_jsx(_components.code,{children:`{ \"error\": \"Run not found\" }`})}),_jsx(_components.td,{children:`No run with this id in the project`})]}),_jsxs(_components.tr,{children:[_jsx(_components.td,{\"data-numeric\":`true`,children:_jsx(_components.code,{children:`409`})}),_jsx(_components.td,{children:_jsx(_components.code,{children:`{ \"error\": \"Run is already in a terminal state\" }`})}),_jsxs(_components.td,{children:[`The run is `,_jsx(_components.code,{children:`\"completed\"`}),` or `,_jsx(_components.code,{children:`\"failed\"`}),`; no more results are accepted`]})]}),_jsxs(_components.tr,{children:[_jsx(_components.td,{\"data-numeric\":`true`,children:_jsx(_components.code,{children:`409`})}),_jsx(_components.td,{children:_jsx(_components.code,{children:`{ \"error\": \"The scorer group grading this run no longer exists\" }`})}),_jsx(_components.td,{children:`The run’s scorer group was deleted mid-run; results are refused rather than silently regraded on the dataset’s own rubric`})]})]})]})]})}function MDXContent(props={}){const{wrapper:MDXLayout}={..._provideComponents(),...props.components};return MDXLayout?_jsx(MDXLayout,{...props,children:_jsx(_createMdxContent,{...props})}):_createMdxContent(props)}return{default:MDXContent};function _missingMdxReference(id,component){throw new Error(`Expected `+(component?`component`:`object`)+\" `\"+id+\"` to be defined: you likely forgot to import, pass, or provide it.\")}","frontmatter":{},"scope":{"config":{},"pageMetadata":{"title":"Submit Results","api":"POST /api/v1/custom-agent-evaluations/runs/{runId}/results","description":"Submit a batch of agent outputs for synchronous judge scoring","href":"/api-reference/custom-eval/submit-results"}}},"mdxSourceWithNoJs":{"compiledSource":"\"use strict\";const{Fragment:_Fragment,jsx:_jsx,jsxs:_jsxs}=arguments[0];const{useMDXComponents:_provideComponents}=arguments[0];function _createMdxContent(props){const _components={a:`a`,code:`code`,p:`p`,pre:`pre`,span:`span`,strong:`strong`,tbody:`tbody`,td:`td`,th:`th`,thead:`thead`,tr:`tr`,..._provideComponents(),...props.components},{CodeBlock,Expandable,Heading,Note,ParamField,RequestExample,ResponseExample,ResponseField,Table}=_components;if(!CodeBlock)_missingMdxReference(`CodeBlock`,true);if(!Expandable)_missingMdxReference(`Expandable`,true);if(!Heading)_missingMdxReference(`Heading`,true);if(!Note)_missingMdxReference(`Note`,true);if(!ParamField)_missingMdxReference(`ParamField`,true);if(!RequestExample)_missingMdxReference(`RequestExample`,true);if(!ResponseExample)_missingMdxReference(`ResponseExample`,true);if(!ResponseField)_missingMdxReference(`ResponseField`,true);if(!Table)_missingMdxReference(`Table`,true);return _jsxs(_Fragment,{children:[_jsxs(_components.p,{children:[`Submits up to 10 agent outputs for a run. Each new result is scored `,_jsx(_components.strong,{children:`synchronously inside this\nrequest`}),` - a judge LLM call per result, plus any enabled similarity metrics and code scorers -\nso allow a generous client timeout for judged batches. Submissions are idempotent: resubmitting\nan `,_jsx(_components.code,{children:`idempotencyKey`}),` the run has already accepted returns the stored score without re-scoring.`]}),`\n`,_jsx(Note,{children:_jsxs(_components.p,{children:[`Send results as you collect them rather than accumulating everything client-side: every batch\nresponse carries `,_jsx(_components.code,{children:`liveStatistics`}),`, so the run’s average rating is observable while the run is\nstill executing.`]})}),`\n`,_jsx(Heading,{level:`2`,id:`authentication`,children:`Authentication`}),`\n`,_jsx(ParamField,{header:`x-api-key`,type:`string`,required:true,id:`param-x-api-key`,children:_jsx(_components.p,{children:`Project API key.`})}),`\n`,_jsx(Heading,{level:`2`,id:`path-parameters`,children:`Path Parameters`}),`\n`,_jsx(ParamField,{path:`runId`,type:`string`,required:true,id:`param-run-id`,children:_jsxs(_components.p,{children:[`Run ID returned by `,_jsx(_components.code,{children:`POST /runs`}),`.`]})}),`\n`,_jsx(Heading,{level:`2`,id:`body`,children:`Body`}),`\n`,_jsx(ParamField,{body:`batchId`,type:`string`,required:true,id:`param-batch-id`,children:_jsx(_components.p,{children:`Identifier for this batch, stored on each result row.`})}),`\n`,_jsxs(ParamField,{body:`results`,type:`array`,required:true,id:`param-results`,children:[_jsx(_components.p,{children:`Array of 1-10 result objects. Each result has:`}),_jsxs(Expandable,{title:`Result object fields`,children:[_jsx(ParamField,{body:`idempotencyKey`,type:`string`,required:true,id:`param-idempotency-key`,children:_jsxs(_components.p,{children:[`Unique key for this result within the run. Prevents duplicate scoring on retry (the\nengine also enforces uniqueness on `,_jsx(_components.code,{children:`(runId, idempotencyKey)`}),` at insert, so a racing\nduplicate counts as a duplicate, never an error). Recommended format:\n`,_jsx(_components.code,{children:`{runId}:{caseId}:run-{runNumber}`}),`. A result without one is counted in\n`,_jsx(_components.code,{children:`failedValidation`}),` and dropped.`]})}),_jsx(ParamField,{body:`questionIndex`,type:`number`,default:`0`,id:`param-question-index`,children:_jsxs(_components.p,{children:[`0-based index into the dataset’s `,_jsx(_components.code,{children:`questions`}),` array. Selects the case’s\n`,_jsx(_components.code,{children:`expectedResults`}),`, `,_jsx(_components.code,{children:`judgeGuideline`}),`, and other per-case grading inputs.`]})}),_jsx(ParamField,{body:`runNumber`,type:`number`,default:`1`,id:`param-run-number`,children:_jsxs(_components.p,{children:[`Which repetition this is (1 through `,_jsx(_components.code,{children:`numberOfRequests`}),`).`]})}),_jsx(ParamField,{body:`caseId`,type:`string`,id:`param-case-id`,children:_jsxs(_components.p,{children:[`Case identifier. Defaults to `,_jsx(_components.code,{children:`case-{questionIndex}`}),`.`]})}),_jsx(ParamField,{body:`input`,type:`object`,id:`param-input`,children:_jsxs(_components.p,{children:[`Agent input: `,_jsx(_components.code,{children:`{ \"query\": string }`}),`.`]})}),_jsx(ParamField,{body:`output`,type:`object`,id:`param-output`,children:_jsxs(_components.p,{children:[`Agent output: `,_jsx(_components.code,{children:`{ \"text\": string }`}),`. Required unless `,_jsx(_components.code,{children:`error`}),` is set - a result with\nneither is counted in `,_jsx(_components.code,{children:`failedValidation`}),` and dropped.`]})}),_jsx(ParamField,{body:`error`,type:`object`,id:`param-error`,children:_jsxs(_components.p,{children:[`Error that occurred: `,_jsx(_components.code,{children:`{ \"type\": string, \"message\": string }`}),`. When set, the result is\nstored with status `,_jsx(_components.code,{children:`\"failed\"`}),` and rating 0; no judge call is made.`]})}),_jsx(ParamField,{body:`traceId`,type:`string`,id:`param-trace-id`,children:_jsxs(_components.p,{children:[`ID of a trace previously ingested via\n`,_jsx(_components.a,{href:`/api-reference/tracing/submit-trace`,children:`Submit Trace`}),` (the Python SDK’s\n`,_jsx(_components.code,{children:`client.tracer.trace(..., sync=True)`}),` returns it as `,_jsx(_components.code,{children:`span.trace_id`}),`). Linking a trace\nunlocks trajectory-aware judging (the judge sees the actual tool-call path), the\ntrajectory and context match scorers, tool definitions for `,_jsx(_components.code,{children:`toolContext: \"detailed\"`}),`\nconfigs, and the dashboard’s View Trace on the result.`]})}),_jsx(ParamField,{body:`retrievalContext`,type:`string | string[]`,id:`param-retrieval-context`,children:_jsxs(_components.p,{children:[`What the agent actually retrieved for this case - a string or a list of chunk strings.\nFeeds `,_jsx(_components.code,{children:`{context}`}),`-referencing judge prompts (the RAG metric pack) and the context match\nscorer. When absent, the engine falls back to the linked trace’s recorded retrieval\nspans, then the case’s pinned `,_jsx(_components.code,{children:`retrievalContext`}),`.`]})}),_jsx(ParamField,{body:`timings`,type:`object`,id:`param-timings`,children:_jsxs(_components.p,{children:[`Performance metrics: `,_jsx(_components.code,{children:`{ \"latencyMs\": number, \"inputTokens\": number, \"outputTokens\": number }`}),`.`]})}),_jsx(ParamField,{body:`isSmokeTestVariant`,type:`boolean`,default:`false`,id:`param-is-smoke-test-variant`,children:_jsxs(_components.p,{children:[`Marks this row as a paraphrase variant from the run’s `,_jsx(_components.code,{children:`smokeTestVariants`}),`. Variant rows\nare excluded from every rating aggregate - `,_jsx(_components.code,{children:`liveStatistics`}),`, the run’s `,_jsx(_components.code,{children:`averageRating`}),`\nand `,_jsx(_components.code,{children:`scorerBreakdown`}),`, case statistics, low-scoring cases, and CI-gate aggregates - but\nstill count toward the run’s `,_jsx(_components.code,{children:`resultCount`}),`.`]})}),_jsx(ParamField,{body:`smokeTestVariantText`,type:`string`,id:`param-smoke-test-variant-text`,children:_jsxs(_components.p,{children:[`The variant phrasing that was actually asked, when `,_jsx(_components.code,{children:`isSmokeTestVariant`}),` is `,_jsx(_components.code,{children:`true`}),`.`]})})]}),_jsx(Note,{children:_jsxs(_components.p,{children:[`The hosted platform’s `,_jsx(_components.code,{children:`observableTrace`}),` and `,_jsx(_components.code,{children:`metadata`}),` result fields are ignored by the\nself-host engine - they are accepted without error but not stored. Link a real trace via\n`,_jsx(_components.code,{children:`traceId`}),` instead; it powers strictly more (trajectory judging, timeline, scorers).`]})})]}),`\n`,_jsx(Heading,{level:`2`,id:`response`,children:`Response`}),`\n`,_jsx(ResponseField,{name:`runId`,type:`string`,id:`param-run-id-1`,children:_jsx(_components.p,{children:`Echoed back.`})}),`\n`,_jsx(ResponseField,{name:`batchId`,type:`string`,id:`param-batch-id-1`,children:_jsx(_components.p,{children:`Echoed back.`})}),`\n`,_jsx(ResponseField,{name:`accepted`,type:`number`,id:`param-accepted`,children:_jsx(_components.p,{children:`New results scored and stored in this call.`})}),`\n`,_jsx(ResponseField,{name:`duplicates`,type:`number`,id:`param-duplicates`,children:_jsxs(_components.p,{children:[`Results skipped because their `,_jsx(_components.code,{children:`idempotencyKey`}),` already existed on this run.`]})}),`\n`,_jsx(ResponseField,{name:`failedValidation`,type:`number`,id:`param-failed-validation`,children:_jsxs(_components.p,{children:[`Results dropped (missing `,_jsx(_components.code,{children:`idempotencyKey`}),`, or both `,_jsx(_components.code,{children:`output.text`}),` and `,_jsx(_components.code,{children:`error`}),` absent).`]})}),`\n`,_jsx(ResponseField,{name:`status`,type:`string`,id:`param-status`,children:_jsxs(_components.p,{children:[`The run’s status, e.g. `,_jsx(_components.code,{children:`\"in_progress\"`}),`.`]})}),`\n`,_jsx(ResponseField,{name:`scoredResults`,type:`array`,id:`param-scored-results`,children:_jsxs(_components.p,{children:[`Per-result scores, in submission order (validation-dropped results are omitted):\n`,_jsx(_components.code,{children:`[{ idempotencyKey, rating, justification, status, vectorSimilarity, jaccardSimilarity, bleuScore, rougeScore, codeScorerResults, judgeScorerResults, deduped? }]`}),`.\n`,_jsx(_components.code,{children:`judgeScorerResults`}),` carries one `,_jsx(_components.code,{children:`{ scorerId, name, rating, justification }`}),` entry per\nadditional judge scorer (or scorer-group judge member) on the run, `,_jsx(_components.code,{children:`null`}),` when there are\nnone.\n`,_jsx(_components.code,{children:`status`}),` is `,_jsx(_components.code,{children:`\"scored\"`}),`, `,_jsx(_components.code,{children:`\"skipped\"`}),` (judge could not score - the reason is in\n`,_jsx(_components.code,{children:`justification`}),`), or `,_jsx(_components.code,{children:`\"failed\"`}),` (the result carried an `,_jsx(_components.code,{children:`error`}),`). Duplicates echo the stored\nfirst-submission payload with `,_jsx(_components.code,{children:`deduped: true`}),`.`]})}),`\n`,_jsx(ResponseField,{name:`liveStatistics`,type:`object`,id:`param-live-statistics`,children:_jsxs(_components.p,{children:[`Rating aggregate recomputed after this batch:\n`,_jsx(_components.code,{children:`{ averageRating, minRating, maxRating, ratedCount, skippedCount, failedCount }`}),`. Available\non every batch response, no `,_jsx(_components.a,{href:`/api-reference/custom-eval/analyze-run`,children:`Analyze Run`}),` required;\nthe Python SDK exposes the same numbers as `,_jsx(_components.code,{children:`run.average_rating`}),` and friends.`]})}),`\n`,_jsx(Heading,{level:`2`,id:`errors`,children:`Errors`}),`\n`,_jsxs(Table,{children:[_jsx(_components.thead,{children:_jsxs(_components.tr,{children:[_jsx(_components.th,{children:`Status`}),_jsx(_components.th,{children:`Body`}),_jsx(_components.th,{children:`Meaning`})]})}),_jsxs(_components.tbody,{children:[_jsxs(_components.tr,{children:[_jsx(_components.td,{\"data-numeric\":`true`,children:_jsx(_components.code,{children:`400`})}),_jsx(_components.td,{children:_jsx(_components.code,{children:`{ \"error\": \"batchId must be a non-empty string\" }`})}),_jsxs(_components.td,{children:[`Missing or invalid `,_jsx(_components.code,{children:`batchId`})]})]}),_jsxs(_components.tr,{children:[_jsx(_components.td,{\"data-numeric\":`true`,children:_jsx(_components.code,{children:`400`})}),_jsx(_components.td,{children:_jsx(_components.code,{children:`{ \"error\": \"results must be a non-empty array\" }`})}),_jsxs(_components.td,{children:[`Missing or empty `,_jsx(_components.code,{children:`results`})]})]}),_jsxs(_components.tr,{children:[_jsx(_components.td,{\"data-numeric\":`true`,children:_jsx(_components.code,{children:`400`})}),_jsx(_components.td,{children:_jsx(_components.code,{children:`{ \"error\": \"Batch size must not exceed 10\" }`})}),_jsx(_components.td,{children:`More than 10 results in one call`})]}),_jsxs(_components.tr,{children:[_jsx(_components.td,{\"data-numeric\":`true`,children:_jsx(_components.code,{children:`400`})}),_jsx(_components.td,{children:_jsx(_components.code,{children:`{ \"error\": \"Every results entry must be an object\" }`})}),_jsxs(_components.td,{children:[`A `,_jsx(_components.code,{children:`results`}),` element is `,_jsx(_components.code,{children:`null`}),`, an array, or not an object`]})]}),_jsxs(_components.tr,{children:[_jsx(_components.td,{\"data-numeric\":`true`,children:_jsx(_components.code,{children:`404`})}),_jsx(_components.td,{children:_jsx(_components.code,{children:`{ \"error\": \"Run not found\" }`})}),_jsx(_components.td,{children:`No run with this id in the project`})]}),_jsxs(_components.tr,{children:[_jsx(_components.td,{\"data-numeric\":`true`,children:_jsx(_components.code,{children:`409`})}),_jsx(_components.td,{children:_jsx(_components.code,{children:`{ \"error\": \"Run is already in a terminal state\" }`})}),_jsxs(_components.td,{children:[`The run is `,_jsx(_components.code,{children:`\"completed\"`}),` or `,_jsx(_components.code,{children:`\"failed\"`}),`; no more results are accepted`]})]}),_jsxs(_components.tr,{children:[_jsx(_components.td,{\"data-numeric\":`true`,children:_jsx(_components.code,{children:`409`})}),_jsx(_components.td,{children:_jsx(_components.code,{children:`{ \"error\": \"The scorer group grading this run no longer exists\" }`})}),_jsx(_components.td,{children:`The run’s scorer group was deleted mid-run; results are refused rather than silently regraded on the dataset’s own rubric`})]})]})]}),`\n`,_jsx(RequestExample,{children:_jsx(CodeBlock,{filename:`cURL`,numberOfLines:`18`,language:`shellscript`,children:_jsx(_components.pre,{className:`shiki shiki-themes github-light-default dark-plus`,style:{backgroundColor:`#ffffff`,\"--shiki-dark-bg\":`#0B0C0E`,color:`#1f2328`,\"--shiki-dark\":`#D4D4D4`},language:`shellscript`,children:_jsxs(_components.code,{language:`shellscript`,numberOfLines:`18`,children:[_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#953800`,\"--shiki-dark\":`#DCDCAA`},children:`curl`}),_jsx(_components.span,{style:{color:`#0550AE`,\"--shiki-dark\":`#569CD6`},children:` -X`}),_jsx(_components.span,{style:{color:`#0A3069`,\"--shiki-dark\":`#CE9178`},children:` POST`}),_jsx(_components.span,{style:{color:`#0A3069`,\"--shiki-dark\":`#CE9178`},children:` http://localhost:4700/api/v1/custom-agent-evaluations/runs/rK7dP2qWx9TzB4mV6nJcE/results`}),_jsx(_components.span,{style:{color:`#CF222E`,\"--shiki-dark\":`#D7BA7D`},children:` \\\\`})]}),`\n`,_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#0550AE`,\"--shiki-dark\":`#569CD6`},children:` -H`}),_jsx(_components.span,{style:{color:`#0A3069`,\"--shiki-dark\":`#CE9178`},children:` \"x-api-key: agtx_local_0f3c9a17d2b84e6a5c01b9f4e7d8a2c6431b5f97a0e2d4c8\"`}),_jsx(_components.span,{style:{color:`#CF222E`,\"--shiki-dark\":`#D7BA7D`},children:` \\\\`})]}),`\n`,_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#0550AE`,\"--shiki-dark\":`#569CD6`},children:` -H`}),_jsx(_components.span,{style:{color:`#0A3069`,\"--shiki-dark\":`#CE9178`},children:` \"Content-Type: application/json\"`}),_jsx(_components.span,{style:{color:`#CF222E`,\"--shiki-dark\":`#D7BA7D`},children:` \\\\`})]}),`\n`,_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#0550AE`,\"--shiki-dark\":`#569CD6`},children:` -d`}),_jsx(_components.span,{style:{color:`#0A3069`,\"--shiki-dark\":`#CE9178`},children:` '{`})]}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{style:{color:`#0A3069`,\"--shiki-dark\":`#CE9178`},children:` \"batchId\": \"batch-001\",`})}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{style:{color:`#0A3069`,\"--shiki-dark\":`#CE9178`},children:` \"results\": [`})}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{style:{color:`#0A3069`,\"--shiki-dark\":`#CE9178`},children:` {`})}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{style:{color:`#0A3069`,\"--shiki-dark\":`#CE9178`},children:` \"idempotencyKey\": \"rK7dP2qWx9TzB4mV6nJcE:case-0:run-1\",`})}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{style:{color:`#0A3069`,\"--shiki-dark\":`#CE9178`},children:` \"questionIndex\": 0,`})}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{style:{color:`#0A3069`,\"--shiki-dark\":`#CE9178`},children:` \"runNumber\": 1,`})}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{style:{color:`#0A3069`,\"--shiki-dark\":`#CE9178`},children:` \"caseId\": \"case-0\",`})}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{style:{color:`#0A3069`,\"--shiki-dark\":`#CE9178`},children:` \"input\": { \"query\": \"How do I reset my password?\" },`})}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{style:{color:`#0A3069`,\"--shiki-dark\":`#CE9178`},children:` \"output\": { \"text\": \"Click Forgot Password on the login screen.\" },`})}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{style:{color:`#0A3069`,\"--shiki-dark\":`#CE9178`},children:` \"timings\": { \"latencyMs\": 1240, \"inputTokens\": 150, \"outputTokens\": 45 },`})}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{style:{color:`#0A3069`,\"--shiki-dark\":`#CE9178`},children:` \"traceId\": \"mJ3vQ8pTr2LqYw6bZk9Xd\"`})}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{style:{color:`#0A3069`,\"--shiki-dark\":`#CE9178`},children:` }`})}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{style:{color:`#0A3069`,\"--shiki-dark\":`#CE9178`},children:` ]`})}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{style:{color:`#0A3069`,\"--shiki-dark\":`#CE9178`},children:` }'`})}),`\n`]})})})}),`\n`,_jsxs(ResponseExample,{children:[_jsx(CodeBlock,{filename:`200 OK`,numberOfLines:`30`,language:`json`,children:_jsx(_components.pre,{className:`shiki shiki-themes github-light-default dark-plus`,style:{backgroundColor:`#ffffff`,\"--shiki-dark-bg\":`#0B0C0E`,color:`#1f2328`,\"--shiki-dark\":`#D4D4D4`},language:`json`,children:_jsxs(_components.code,{language:`json`,numberOfLines:`30`,children:[_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:`{`})}),`\n`,_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#116329`,\"--shiki-dark\":`#9CDCFE`},children:` \"runId\"`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:`: `}),_jsx(_components.span,{style:{color:`#0A3069`,\"--shiki-dark\":`#CE9178`},children:`\"rK7dP2qWx9TzB4mV6nJcE\"`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:`,`})]}),`\n`,_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#116329`,\"--shiki-dark\":`#9CDCFE`},children:` \"batchId\"`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:`: `}),_jsx(_components.span,{style:{color:`#0A3069`,\"--shiki-dark\":`#CE9178`},children:`\"batch-001\"`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:`,`})]}),`\n`,_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#116329`,\"--shiki-dark\":`#9CDCFE`},children:` \"accepted\"`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:`: `}),_jsx(_components.span,{style:{color:`#0550AE`,\"--shiki-dark\":`#B5CEA8`},children:`1`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:`,`})]}),`\n`,_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#116329`,\"--shiki-dark\":`#9CDCFE`},children:` \"duplicates\"`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:`: `}),_jsx(_components.span,{style:{color:`#0550AE`,\"--shiki-dark\":`#B5CEA8`},children:`0`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:`,`})]}),`\n`,_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#116329`,\"--shiki-dark\":`#9CDCFE`},children:` \"failedValidation\"`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:`: `}),_jsx(_components.span,{style:{color:`#0550AE`,\"--shiki-dark\":`#B5CEA8`},children:`0`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:`,`})]}),`\n`,_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#116329`,\"--shiki-dark\":`#9CDCFE`},children:` \"status\"`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:`: `}),_jsx(_components.span,{style:{color:`#0A3069`,\"--shiki-dark\":`#CE9178`},children:`\"in_progress\"`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:`,`})]}),`\n`,_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#116329`,\"--shiki-dark\":`#9CDCFE`},children:` \"scoredResults\"`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:`: [`})]}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:` {`})}),`\n`,_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#116329`,\"--shiki-dark\":`#9CDCFE`},children:` \"idempotencyKey\"`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:`: `}),_jsx(_components.span,{style:{color:`#0A3069`,\"--shiki-dark\":`#CE9178`},children:`\"rK7dP2qWx9TzB4mV6nJcE:case-0:run-1\"`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:`,`})]}),`\n`,_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#116329`,\"--shiki-dark\":`#9CDCFE`},children:` \"rating\"`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:`: `}),_jsx(_components.span,{style:{color:`#0550AE`,\"--shiki-dark\":`#B5CEA8`},children:`8`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:`,`})]}),`\n`,_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#116329`,\"--shiki-dark\":`#9CDCFE`},children:` \"justification\"`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:`: `}),_jsx(_components.span,{style:{color:`#0A3069`,\"--shiki-dark\":`#CE9178`},children:`\"The agent correctly identified the password reset flow and gave clear step-by-step instructions.\"`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:`,`})]}),`\n`,_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#116329`,\"--shiki-dark\":`#9CDCFE`},children:` \"status\"`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:`: `}),_jsx(_components.span,{style:{color:`#0A3069`,\"--shiki-dark\":`#CE9178`},children:`\"scored\"`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:`,`})]}),`\n`,_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#116329`,\"--shiki-dark\":`#9CDCFE`},children:` \"vectorSimilarity\"`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:`: `}),_jsx(_components.span,{style:{color:`#0550AE`,\"--shiki-dark\":`#569CD6`},children:`null`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:`,`})]}),`\n`,_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#116329`,\"--shiki-dark\":`#9CDCFE`},children:` \"jaccardSimilarity\"`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:`: `}),_jsx(_components.span,{style:{color:`#0550AE`,\"--shiki-dark\":`#B5CEA8`},children:`0.62`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:`,`})]}),`\n`,_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#116329`,\"--shiki-dark\":`#9CDCFE`},children:` \"bleuScore\"`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:`: `}),_jsx(_components.span,{style:{color:`#0550AE`,\"--shiki-dark\":`#569CD6`},children:`null`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:`,`})]}),`\n`,_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#116329`,\"--shiki-dark\":`#9CDCFE`},children:` \"rougeScore\"`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:`: `}),_jsx(_components.span,{style:{color:`#0550AE`,\"--shiki-dark\":`#569CD6`},children:`null`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:`,`})]}),`\n`,_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#116329`,\"--shiki-dark\":`#9CDCFE`},children:` \"codeScorerResults\"`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:`: `}),_jsx(_components.span,{style:{color:`#0550AE`,\"--shiki-dark\":`#569CD6`},children:`null`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:`,`})]}),`\n`,_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#116329`,\"--shiki-dark\":`#9CDCFE`},children:` \"judgeScorerResults\"`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:`: `}),_jsx(_components.span,{style:{color:`#0550AE`,\"--shiki-dark\":`#569CD6`},children:`null`})]}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:` }`})}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:` ],`})}),`\n`,_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#116329`,\"--shiki-dark\":`#9CDCFE`},children:` \"liveStatistics\"`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:`: {`})]}),`\n`,_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#116329`,\"--shiki-dark\":`#9CDCFE`},children:` \"averageRating\"`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:`: `}),_jsx(_components.span,{style:{color:`#0550AE`,\"--shiki-dark\":`#B5CEA8`},children:`8`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:`,`})]}),`\n`,_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#116329`,\"--shiki-dark\":`#9CDCFE`},children:` \"minRating\"`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:`: `}),_jsx(_components.span,{style:{color:`#0550AE`,\"--shiki-dark\":`#B5CEA8`},children:`8`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:`,`})]}),`\n`,_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#116329`,\"--shiki-dark\":`#9CDCFE`},children:` \"maxRating\"`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:`: `}),_jsx(_components.span,{style:{color:`#0550AE`,\"--shiki-dark\":`#B5CEA8`},children:`8`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:`,`})]}),`\n`,_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#116329`,\"--shiki-dark\":`#9CDCFE`},children:` \"ratedCount\"`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:`: `}),_jsx(_components.span,{style:{color:`#0550AE`,\"--shiki-dark\":`#B5CEA8`},children:`1`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:`,`})]}),`\n`,_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#116329`,\"--shiki-dark\":`#9CDCFE`},children:` \"skippedCount\"`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:`: `}),_jsx(_components.span,{style:{color:`#0550AE`,\"--shiki-dark\":`#B5CEA8`},children:`0`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:`,`})]}),`\n`,_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#116329`,\"--shiki-dark\":`#9CDCFE`},children:` \"failedCount\"`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:`: `}),_jsx(_components.span,{style:{color:`#0550AE`,\"--shiki-dark\":`#B5CEA8`},children:`0`})]}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:` }`})}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:`}`})}),`\n`]})})}),_jsx(CodeBlock,{filename:`409 Terminal state`,numberOfLines:`3`,language:`json`,children:_jsx(_components.pre,{className:`shiki shiki-themes github-light-default dark-plus`,style:{backgroundColor:`#ffffff`,\"--shiki-dark-bg\":`#0B0C0E`,color:`#1f2328`,\"--shiki-dark\":`#D4D4D4`},language:`json`,children:_jsxs(_components.code,{language:`json`,numberOfLines:`3`,children:[_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:`{`})}),`\n`,_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#116329`,\"--shiki-dark\":`#9CDCFE`},children:` \"error\"`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:`: `}),_jsx(_components.span,{style:{color:`#0A3069`,\"--shiki-dark\":`#CE9178`},children:`\"Run is already in a terminal state\"`})]}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:`}`})}),`\n`]})})})]})]})}function MDXContent(props={}){const{wrapper:MDXLayout}={..._provideComponents(),...props.components};return MDXLayout?_jsx(MDXLayout,{...props,children:_jsx(_createMdxContent,{...props})}):_createMdxContent(props)}return{default:MDXContent};function _missingMdxReference(id,component){throw new Error(`Expected `+(component?`component`:`object`)+\" `\"+id+\"` to be defined: you likely forgot to import, pass, or provide it.\")}","frontmatter":{},"scope":{"config":{},"pageMetadata":{"title":"Submit Results","api":"POST /api/v1/custom-agent-evaluations/runs/{runId}/results","description":"Submit a batch of agent outputs for synchronous judge scoring","href":"/api-reference/custom-eval/submit-results"}}}},"mdxExtracts":{"tableOfContents":[{"title":"Authentication","slug":"authentication","depth":2,"children":[]},{"title":"Path Parameters","slug":"path-parameters","depth":2,"children":[]},{"title":"Body","slug":"body","depth":2,"children":[]},{"title":"Response","slug":"response","depth":2,"children":[]},{"title":"Errors","slug":"errors","depth":2,"children":[]}],"codeExamples":{"request":{"type":"RequestExample","children":[{"filename":"cURL","html":"\u003cpre class=\"shiki shiki-themes github-light-default dark-plus\" style=\"background-color:#ffffff;--shiki-dark-bg:#0B0C0E;color:#1f2328;--shiki-dark:#D4D4D4\" language=\"shellscript\">\u003ccode language=\"shellscript\" numberOfLines=\"18\">\u003cspan class=\"line\">\u003cspan style=\"color:#953800;--shiki-dark:#DCDCAA\">curl\u003c/span>\u003cspan style=\"color:#0550AE;--shiki-dark:#569CD6\"> -X\u003c/span>\u003cspan style=\"color:#0A3069;--shiki-dark:#CE9178\"> POST\u003c/span>\u003cspan style=\"color:#0A3069;--shiki-dark:#CE9178\"> http://localhost:4700/api/v1/custom-agent-evaluations/runs/rK7dP2qWx9TzB4mV6nJcE/results\u003c/span>\u003cspan style=\"color:#CF222E;--shiki-dark:#D7BA7D\"> \\\u003c/span>\u003c/span>\n\u003cspan class=\"line\">\u003cspan style=\"color:#0550AE;--shiki-dark:#569CD6\"> -H\u003c/span>\u003cspan style=\"color:#0A3069;--shiki-dark:#CE9178\"> \"x-api-key: agtx_local_0f3c9a17d2b84e6a5c01b9f4e7d8a2c6431b5f97a0e2d4c8\"\u003c/span>\u003cspan style=\"color:#CF222E;--shiki-dark:#D7BA7D\"> \\\u003c/span>\u003c/span>\n\u003cspan class=\"line\">\u003cspan style=\"color:#0550AE;--shiki-dark:#569CD6\"> -H\u003c/span>\u003cspan style=\"color:#0A3069;--shiki-dark:#CE9178\"> \"Content-Type: application/json\"\u003c/span>\u003cspan style=\"color:#CF222E;--shiki-dark:#D7BA7D\"> \\\u003c/span>\u003c/span>\n\u003cspan class=\"line\">\u003cspan style=\"color:#0550AE;--shiki-dark:#569CD6\"> -d\u003c/span>\u003cspan style=\"color:#0A3069;--shiki-dark:#CE9178\"> '{\u003c/span>\u003c/span>\n\u003cspan class=\"line\">\u003cspan style=\"color:#0A3069;--shiki-dark:#CE9178\"> \"batchId\": \"batch-001\",\u003c/span>\u003c/span>\n\u003cspan class=\"line\">\u003cspan style=\"color:#0A3069;--shiki-dark:#CE9178\"> \"results\": [\u003c/span>\u003c/span>\n\u003cspan class=\"line\">\u003cspan style=\"color:#0A3069;--shiki-dark:#CE9178\"> {\u003c/span>\u003c/span>\n\u003cspan class=\"line\">\u003cspan style=\"color:#0A3069;--shiki-dark:#CE9178\"> \"idempotencyKey\": \"rK7dP2qWx9TzB4mV6nJcE:case-0:run-1\",\u003c/span>\u003c/span>\n\u003cspan class=\"line\">\u003cspan style=\"color:#0A3069;--shiki-dark:#CE9178\"> \"questionIndex\": 0,\u003c/span>\u003c/span>\n\u003cspan class=\"line\">\u003cspan style=\"color:#0A3069;--shiki-dark:#CE9178\"> \"runNumber\": 1,\u003c/span>\u003c/span>\n\u003cspan class=\"line\">\u003cspan style=\"color:#0A3069;--shiki-dark:#CE9178\"> \"caseId\": \"case-0\",\u003c/span>\u003c/span>\n\u003cspan class=\"line\">\u003cspan style=\"color:#0A3069;--shiki-dark:#CE9178\"> \"input\": { \"query\": \"How do I reset my password?\" },\u003c/span>\u003c/span>\n\u003cspan class=\"line\">\u003cspan style=\"color:#0A3069;--shiki-dark:#CE9178\"> \"output\": { \"text\": \"Click Forgot Password on the login screen.\" },\u003c/span>\u003c/span>\n\u003cspan class=\"line\">\u003cspan style=\"color:#0A3069;--shiki-dark:#CE9178\"> \"timings\": { \"latencyMs\": 1240, \"inputTokens\": 150, \"outputTokens\": 45 },\u003c/span>\u003c/span>\n\u003cspan class=\"line\">\u003cspan style=\"color:#0A3069;--shiki-dark:#CE9178\"> \"traceId\": \"mJ3vQ8pTr2LqYw6bZk9Xd\"\u003c/span>\u003c/span>\n\u003cspan class=\"line\">\u003cspan style=\"color:#0A3069;--shiki-dark:#CE9178\"> }\u003c/span>\u003c/span>\n\u003cspan class=\"line\">\u003cspan style=\"color:#0A3069;--shiki-dark:#CE9178\"> ]\u003c/span>\u003c/span>\n\u003cspan class=\"line\">\u003cspan style=\"color:#0A3069;--shiki-dark:#CE9178\"> }'\u003c/span>\u003c/span>\n\u003c/code>\u003c/pre>"}],"attributes":[]},"response":{"type":"ResponseExample","children":[{"filename":"200 OK","html":"\u003cpre class=\"shiki shiki-themes github-light-default dark-plus\" style=\"background-color:#ffffff;--shiki-dark-bg:#0B0C0E;color:#1f2328;--shiki-dark:#D4D4D4\" language=\"json\">\u003ccode language=\"json\" numberOfLines=\"30\">\u003cspan class=\"line\">\u003cspan style=\"color:#1F2328;--shiki-dark:#D4D4D4\">{\u003c/span>\u003c/span>\n\u003cspan class=\"line\">\u003cspan style=\"color:#116329;--shiki-dark:#9CDCFE\"> \"runId\"\u003c/span>\u003cspan style=\"color:#1F2328;--shiki-dark:#D4D4D4\">: \u003c/span>\u003cspan style=\"color:#0A3069;--shiki-dark:#CE9178\">\"rK7dP2qWx9TzB4mV6nJcE\"\u003c/span>\u003cspan style=\"color:#1F2328;--shiki-dark:#D4D4D4\">,\u003c/span>\u003c/span>\n\u003cspan class=\"line\">\u003cspan style=\"color:#116329;--shiki-dark:#9CDCFE\"> \"batchId\"\u003c/span>\u003cspan style=\"color:#1F2328;--shiki-dark:#D4D4D4\">: \u003c/span>\u003cspan style=\"color:#0A3069;--shiki-dark:#CE9178\">\"batch-001\"\u003c/span>\u003cspan style=\"color:#1F2328;--shiki-dark:#D4D4D4\">,\u003c/span>\u003c/span>\n\u003cspan class=\"line\">\u003cspan style=\"color:#116329;--shiki-dark:#9CDCFE\"> \"accepted\"\u003c/span>\u003cspan style=\"color:#1F2328;--shiki-dark:#D4D4D4\">: \u003c/span>\u003cspan style=\"color:#0550AE;--shiki-dark:#B5CEA8\">1\u003c/span>\u003cspan style=\"color:#1F2328;--shiki-dark:#D4D4D4\">,\u003c/span>\u003c/span>\n\u003cspan class=\"line\">\u003cspan style=\"color:#116329;--shiki-dark:#9CDCFE\"> \"duplicates\"\u003c/span>\u003cspan style=\"color:#1F2328;--shiki-dark:#D4D4D4\">: \u003c/span>\u003cspan style=\"color:#0550AE;--shiki-dark:#B5CEA8\">0\u003c/span>\u003cspan style=\"color:#1F2328;--shiki-dark:#D4D4D4\">,\u003c/span>\u003c/span>\n\u003cspan class=\"line\">\u003cspan style=\"color:#116329;--shiki-dark:#9CDCFE\"> \"failedValidation\"\u003c/span>\u003cspan style=\"color:#1F2328;--shiki-dark:#D4D4D4\">: \u003c/span>\u003cspan style=\"color:#0550AE;--shiki-dark:#B5CEA8\">0\u003c/span>\u003cspan style=\"color:#1F2328;--shiki-dark:#D4D4D4\">,\u003c/span>\u003c/span>\n\u003cspan class=\"line\">\u003cspan style=\"color:#116329;--shiki-dark:#9CDCFE\"> \"status\"\u003c/span>\u003cspan style=\"color:#1F2328;--shiki-dark:#D4D4D4\">: \u003c/span>\u003cspan style=\"color:#0A3069;--shiki-dark:#CE9178\">\"in_progress\"\u003c/span>\u003cspan style=\"color:#1F2328;--shiki-dark:#D4D4D4\">,\u003c/span>\u003c/span>\n\u003cspan class=\"line\">\u003cspan style=\"color:#116329;--shiki-dark:#9CDCFE\"> \"scoredResults\"\u003c/span>\u003cspan style=\"color:#1F2328;--shiki-dark:#D4D4D4\">: [\u003c/span>\u003c/span>\n\u003cspan class=\"line\">\u003cspan style=\"color:#1F2328;--shiki-dark:#D4D4D4\"> {\u003c/span>\u003c/span>\n\u003cspan class=\"line\">\u003cspan style=\"color:#116329;--shiki-dark:#9CDCFE\"> \"idempotencyKey\"\u003c/span>\u003cspan style=\"color:#1F2328;--shiki-dark:#D4D4D4\">: \u003c/span>\u003cspan style=\"color:#0A3069;--shiki-dark:#CE9178\">\"rK7dP2qWx9TzB4mV6nJcE:case-0:run-1\"\u003c/span>\u003cspan style=\"color:#1F2328;--shiki-dark:#D4D4D4\">,\u003c/span>\u003c/span>\n\u003cspan class=\"line\">\u003cspan style=\"color:#116329;--shiki-dark:#9CDCFE\"> \"rating\"\u003c/span>\u003cspan style=\"color:#1F2328;--shiki-dark:#D4D4D4\">: \u003c/span>\u003cspan style=\"color:#0550AE;--shiki-dark:#B5CEA8\">8\u003c/span>\u003cspan style=\"color:#1F2328;--shiki-dark:#D4D4D4\">,\u003c/span>\u003c/span>\n\u003cspan class=\"line\">\u003cspan style=\"color:#116329;--shiki-dark:#9CDCFE\"> \"justification\"\u003c/span>\u003cspan style=\"color:#1F2328;--shiki-dark:#D4D4D4\">: \u003c/span>\u003cspan style=\"color:#0A3069;--shiki-dark:#CE9178\">\"The agent correctly identified the password reset flow and gave clear step-by-step instructions.\"\u003c/span>\u003cspan style=\"color:#1F2328;--shiki-dark:#D4D4D4\">,\u003c/span>\u003c/span>\n\u003cspan class=\"line\">\u003cspan style=\"color:#116329;--shiki-dark:#9CDCFE\"> \"status\"\u003c/span>\u003cspan style=\"color:#1F2328;--shiki-dark:#D4D4D4\">: \u003c/span>\u003cspan style=\"color:#0A3069;--shiki-dark:#CE9178\">\"scored\"\u003c/span>\u003cspan style=\"color:#1F2328;--shiki-dark:#D4D4D4\">,\u003c/span>\u003c/span>\n\u003cspan class=\"line\">\u003cspan style=\"color:#116329;--shiki-dark:#9CDCFE\"> \"vectorSimilarity\"\u003c/span>\u003cspan style=\"color:#1F2328;--shiki-dark:#D4D4D4\">: \u003c/span>\u003cspan style=\"color:#0550AE;--shiki-dark:#569CD6\">null\u003c/span>\u003cspan style=\"color:#1F2328;--shiki-dark:#D4D4D4\">,\u003c/span>\u003c/span>\n\u003cspan class=\"line\">\u003cspan style=\"color:#116329;--shiki-dark:#9CDCFE\"> \"jaccardSimilarity\"\u003c/span>\u003cspan style=\"color:#1F2328;--shiki-dark:#D4D4D4\">: \u003c/span>\u003cspan style=\"color:#0550AE;--shiki-dark:#B5CEA8\">0.62\u003c/span>\u003cspan style=\"color:#1F2328;--shiki-dark:#D4D4D4\">,\u003c/span>\u003c/span>\n\u003cspan class=\"line\">\u003cspan style=\"color:#116329;--shiki-dark:#9CDCFE\"> \"bleuScore\"\u003c/span>\u003cspan style=\"color:#1F2328;--shiki-dark:#D4D4D4\">: \u003c/span>\u003cspan style=\"color:#0550AE;--shiki-dark:#569CD6\">null\u003c/span>\u003cspan style=\"color:#1F2328;--shiki-dark:#D4D4D4\">,\u003c/span>\u003c/span>\n\u003cspan class=\"line\">\u003cspan style=\"color:#116329;--shiki-dark:#9CDCFE\"> \"rougeScore\"\u003c/span>\u003cspan style=\"color:#1F2328;--shiki-dark:#D4D4D4\">: \u003c/span>\u003cspan style=\"color:#0550AE;--shiki-dark:#569CD6\">null\u003c/span>\u003cspan style=\"color:#1F2328;--shiki-dark:#D4D4D4\">,\u003c/span>\u003c/span>\n\u003cspan class=\"line\">\u003cspan style=\"color:#116329;--shiki-dark:#9CDCFE\"> \"codeScorerResults\"\u003c/span>\u003cspan style=\"color:#1F2328;--shiki-dark:#D4D4D4\">: \u003c/span>\u003cspan style=\"color:#0550AE;--shiki-dark:#569CD6\">null\u003c/span>\u003cspan style=\"color:#1F2328;--shiki-dark:#D4D4D4\">,\u003c/span>\u003c/span>\n\u003cspan class=\"line\">\u003cspan style=\"color:#116329;--shiki-dark:#9CDCFE\"> \"judgeScorerResults\"\u003c/span>\u003cspan style=\"color:#1F2328;--shiki-dark:#D4D4D4\">: \u003c/span>\u003cspan style=\"color:#0550AE;--shiki-dark:#569CD6\">null\u003c/span>\u003c/span>\n\u003cspan class=\"line\">\u003cspan style=\"color:#1F2328;--shiki-dark:#D4D4D4\"> }\u003c/span>\u003c/span>\n\u003cspan class=\"line\">\u003cspan style=\"color:#1F2328;--shiki-dark:#D4D4D4\"> ],\u003c/span>\u003c/span>\n\u003cspan class=\"line\">\u003cspan style=\"color:#116329;--shiki-dark:#9CDCFE\"> \"liveStatistics\"\u003c/span>\u003cspan style=\"color:#1F2328;--shiki-dark:#D4D4D4\">: {\u003c/span>\u003c/span>\n\u003cspan class=\"line\">\u003cspan style=\"color:#116329;--shiki-dark:#9CDCFE\"> \"averageRating\"\u003c/span>\u003cspan style=\"color:#1F2328;--shiki-dark:#D4D4D4\">: \u003c/span>\u003cspan style=\"color:#0550AE;--shiki-dark:#B5CEA8\">8\u003c/span>\u003cspan style=\"color:#1F2328;--shiki-dark:#D4D4D4\">,\u003c/span>\u003c/span>\n\u003cspan class=\"line\">\u003cspan style=\"color:#116329;--shiki-dark:#9CDCFE\"> \"minRating\"\u003c/span>\u003cspan style=\"color:#1F2328;--shiki-dark:#D4D4D4\">: \u003c/span>\u003cspan style=\"color:#0550AE;--shiki-dark:#B5CEA8\">8\u003c/span>\u003cspan style=\"color:#1F2328;--shiki-dark:#D4D4D4\">,\u003c/span>\u003c/span>\n\u003cspan class=\"line\">\u003cspan style=\"color:#116329;--shiki-dark:#9CDCFE\"> \"maxRating\"\u003c/span>\u003cspan style=\"color:#1F2328;--shiki-dark:#D4D4D4\">: \u003c/span>\u003cspan style=\"color:#0550AE;--shiki-dark:#B5CEA8\">8\u003c/span>\u003cspan style=\"color:#1F2328;--shiki-dark:#D4D4D4\">,\u003c/span>\u003c/span>\n\u003cspan class=\"line\">\u003cspan style=\"color:#116329;--shiki-dark:#9CDCFE\"> \"ratedCount\"\u003c/span>\u003cspan style=\"color:#1F2328;--shiki-dark:#D4D4D4\">: \u003c/span>\u003cspan style=\"color:#0550AE;--shiki-dark:#B5CEA8\">1\u003c/span>\u003cspan style=\"color:#1F2328;--shiki-dark:#D4D4D4\">,\u003c/span>\u003c/span>\n\u003cspan class=\"line\">\u003cspan style=\"color:#116329;--shiki-dark:#9CDCFE\"> \"skippedCount\"\u003c/span>\u003cspan style=\"color:#1F2328;--shiki-dark:#D4D4D4\">: \u003c/span>\u003cspan style=\"color:#0550AE;--shiki-dark:#B5CEA8\">0\u003c/span>\u003cspan style=\"color:#1F2328;--shiki-dark:#D4D4D4\">,\u003c/span>\u003c/span>\n\u003cspan class=\"line\">\u003cspan style=\"color:#116329;--shiki-dark:#9CDCFE\"> \"failedCount\"\u003c/span>\u003cspan style=\"color:#1F2328;--shiki-dark:#D4D4D4\">: \u003c/span>\u003cspan style=\"color:#0550AE;--shiki-dark:#B5CEA8\">0\u003c/span>\u003c/span>\n\u003cspan class=\"line\">\u003cspan style=\"color:#1F2328;--shiki-dark:#D4D4D4\"> }\u003c/span>\u003c/span>\n\u003cspan class=\"line\">\u003cspan style=\"color:#1F2328;--shiki-dark:#D4D4D4\">}\u003c/span>\u003c/span>\n\u003c/code>\u003c/pre>"},{"filename":"409 Terminal state","html":"\u003cpre class=\"shiki shiki-themes github-light-default dark-plus\" style=\"background-color:#ffffff;--shiki-dark-bg:#0B0C0E;color:#1f2328;--shiki-dark:#D4D4D4\" language=\"json\">\u003ccode language=\"json\" numberOfLines=\"3\">\u003cspan class=\"line\">\u003cspan style=\"color:#1F2328;--shiki-dark:#D4D4D4\">{\u003c/span>\u003c/span>\n\u003cspan class=\"line\">\u003cspan style=\"color:#116329;--shiki-dark:#9CDCFE\"> \"error\"\u003c/span>\u003cspan style=\"color:#1F2328;--shiki-dark:#D4D4D4\">: \u003c/span>\u003cspan style=\"color:#0A3069;--shiki-dark:#CE9178\">\"Run is already in a terminal state\"\u003c/span>\u003c/span>\n\u003cspan class=\"line\">\u003cspan style=\"color:#1F2328;--shiki-dark:#D4D4D4\">}\u003c/span>\u003c/span>\n\u003c/code>\u003c/pre>"}],"attributes":[]}},"endpoint":{"title":"Submit Results","path":"/api/v1/custom-agent-evaluations/runs/{runId}/results","servers":[{"url":"http://localhost:4700"}],"method":"post","request":{"security":[{"title":"API Key Auth","parameters":{"header":{"x-api-key":{"type":"apiKey"}},"query":{},"cookie":{}}}],"parameters":{"query":{},"header":{"x-api-key":{"schema":[{"type":"string","description":"Project API key.","required":true}]}},"cookie":{},"path":{"runId":{"schema":[{"type":"string","description":"Run ID returned by `POST /runs` .","required":true}]}}},"body":{"application/json":{"schemaArray":[{"type":"object","properties":{"batchId":[{"type":"string","description":"Identifier for this batch, stored on each result row.","required":true}],"results":[{"type":"array","items":[{"type":"object","properties":{"idempotencyKey":[{"type":"string","description":"Unique key for this result within the run. Prevents duplicate scoring on retry (the engine also enforces uniqueness on `(runId, idempotencyKey)` at insert, so a racing duplicate counts as a duplicate, never an error). Recommended format: `{runId}:{caseId}:run-{runNumber}` . A result without one is counted in `failedValidation` and dropped.","required":true}],"questionIndex":[{"type":"number","description":"0-based index into the dataset’s `questions` array. Selects the case’s `expectedResults` , `judgeGuideline` , and other per-case grading inputs."}],"runNumber":[{"type":"number","description":"Which repetition this is (1 through `numberOfRequests` )."}],"caseId":[{"type":"string","description":"Case identifier. Defaults to `case-{questionIndex}` ."}],"input":[{"type":"object","properties":{},"description":"Agent input: `{ \"query\": string }` ."}],"output":[{"type":"object","properties":{},"description":"Agent output: `{ \"text\": string }` . Required unless `error` is set - a result with neither is counted in `failedValidation` and dropped."}],"error":[{"type":"object","properties":{},"description":"Error that occurred: `{ \"type\": string, \"message\": string }` . When set, the result is stored with status `\"failed\"` and rating 0; no judge call is made."}],"traceId":[{"type":"string","description":"ID of a trace previously ingested via Submit Trace (the Python SDK’s `client.tracer.trace(..., sync=True)` returns it as `span.trace_id` ). Linking a trace unlocks trajectory-aware judging (the judge sees the actual tool-call path), the trajectory and context match scorers, tool definitions for `toolContext: \"detailed\"` configs, and the dashboard’s View Trace on the result."}],"retrievalContext":[{"type":"array","items":[{"type":"string"}],"description":"What the agent actually retrieved for this case - a string or a list of chunk strings. Feeds `{context}` -referencing judge prompts (the RAG metric pack) and the context match scorer. When absent, the engine falls back to the linked trace’s recorded retrieval spans, then the case’s pinned `retrievalContext` ."}],"timings":[{"type":"object","properties":{},"description":"Performance metrics: `{ \"latencyMs\": number, \"inputTokens\": number, \"outputTokens\": number }` ."}],"isSmokeTestVariant":[{"type":"boolean","description":"Marks this row as a paraphrase variant from the run’s `smokeTestVariants` . Variant rows are excluded from every rating aggregate - `liveStatistics` , the run’s `averageRating` and `scorerBreakdown` , case statistics, low-scoring cases, and CI-gate aggregates - but still count toward the run’s `resultCount` ."}],"smokeTestVariantText":[{"type":"string","description":"The variant phrasing that was actually asked, when `isSmokeTestVariant` is `true` ."}]}}],"description":"Array of 1-10 result objects. Each result has:","required":true}]}}],"examples":{"example":{"value":{"batchId":"\u003cstring>","results":[{"idempotencyKey":"\u003cstring>","questionIndex":123,"runNumber":123,"caseId":"\u003cstring>","input":{},"output":{},"error":{},"traceId":"\u003cstring>","retrievalContext":["\u003cstring>"],"timings":{},"isSmokeTestVariant":true,"smokeTestVariantText":"\u003cstring>"}]}}}}}},"response":{"200":{"application/json":{"schemaArray":[{"type":"object","properties":{"runId":[{"type":"string","description":"Echoed back."}],"batchId":[{"type":"string","description":"Echoed back."}],"accepted":[{"type":"number","description":"New results scored and stored in this call."}],"duplicates":[{"type":"number","description":"Results skipped because their `idempotencyKey` already existed on this run."}],"failedValidation":[{"type":"number","description":"Results dropped (missing `idempotencyKey` , or both `output.text` and `error` absent)."}],"status":[{"type":"string","description":"The run’s status, e.g. `\"in_progress\"` ."}],"scoredResults":[{"type":"array","items":[{"type":"object","properties":{}}],"description":"Per-result scores, in submission order (validation-dropped results are omitted): `[{ idempotencyKey, rating, justification, status, vectorSimilarity, jaccardSimilarity, bleuScore, rougeScore, codeScorerResults, judgeScorerResults, deduped? }]` . `judgeScorerResults` carries one `{ scorerId, name, rating, justification }` entry per additional judge scorer (or scorer-group judge member) on the run, `null` when there are none. `status` is `\"scored\"` , `\"skipped\"` (judge could not score - the reason is in `justification` ), or `\"failed\"` (the result carried an `error` ). Duplicates echo the stored first-submission payload with `deduped: true` ."}],"liveStatistics":[{"type":"object","properties":{},"description":"Rating aggregate recomputed after this batch: `{ averageRating, minRating, maxRating, ratedCount, skippedCount, failedCount }` . Available on every batch response, no Analyze Run required; the Python SDK exposes the same numbers as `run.average_rating` and friends."}]}}],"examples":{"example":{"value":{"runId":"\u003cstring>","batchId":"\u003cstring>","accepted":123,"duplicates":123,"failedValidation":123,"status":"\u003cstring>","scoredResults":[{}],"liveStatistics":{}}}}}}},"deprecated":false}},"jsLinks":[],"subdomain":"agentx-ffe8d995","siteName":"AgentX","title":"Submit Results - AgentX","navMaps":{"firstHrefInVersion":{},"firstHrefInLanguage":{}},"lastModified":"2026-09-14T02:27:30.272Z","originalFileLocation":"api-reference/custom-eval/submit-results.mdx"},"headHtml":"\u003cscript data-mint-head>(function(){var d=document.documentElement,p=null;if(!p){try{p=localStorage.getItem(\"isDarkMode\")}catch(e){}if(p==='true')p='dark';else if(p==='false')p='light';if(p!=='light'\u0026\u0026p!=='dark'\u0026\u0026p!=='system')p=\"system\"}var r=p==='system'?(matchMedia(\"(prefers-color-scheme: dark)\").matches?'dark':'light'):p;d.classList.remove(r==='dark'?'light':'dark');d.classList.add(r);d.setAttribute(\"data-theme-preference\",p)})();\u003c/script>\u003cmeta data-mint-head name=\"description\" content=\"Submit a batch of agent outputs for synchronous judge scoring\">\u003cmeta data-mint-head name=\"application-name\" content=\"AgentX\">\u003cmeta data-mint-head name=\"generator\" content=\"Mintlify\">\u003clink data-mint-head rel=\"icon\" type=\"image/png\" sizes=\"192x192\" href=\"/mintlify-assets/_mintlify/favicons/agentx-ffe8d995/sticQuUfAA1h_xwo/_generated/favicon/android-chrome-192x192.png\" media=\"(prefers-color-scheme: light)\">\u003clink data-mint-head rel=\"apple-touch-icon\" type=\"image/png\" sizes=\"180x180\" href=\"/mintlify-assets/_mintlify/favicons/agentx-ffe8d995/sticQuUfAA1h_xwo/_generated/favicon/apple-touch-icon.png\">\u003clink data-mint-head rel=\"icon\" type=\"image/png\" sizes=\"16x16\" href=\"/mintlify-assets/_mintlify/favicons/agentx-ffe8d995/sticQuUfAA1h_xwo/_generated/favicon/favicon-16x16.png\" media=\"(prefers-color-scheme: light)\">\u003clink data-mint-head rel=\"icon\" type=\"image/png\" sizes=\"32x32\" href=\"/mintlify-assets/_mintlify/favicons/agentx-ffe8d995/sticQuUfAA1h_xwo/_generated/favicon/favicon-32x32.png\" media=\"(prefers-color-scheme: light)\">\u003clink data-mint-head rel=\"shortcut icon\" type=\"image/x-icon\" href=\"/mintlify-assets/_mintlify/favicons/agentx-ffe8d995/sticQuUfAA1h_xwo/_generated/favicon/favicon.ico\" media=\"(prefers-color-scheme: light)\">\u003clink data-mint-head rel=\"icon\" type=\"image/png\" sizes=\"192x192\" href=\"/mintlify-assets/_mintlify/favicons/agentx-ffe8d995/sticQuUfAA1h_xwo/_generated/favicon-dark/android-chrome-192x192.png\" media=\"(prefers-color-scheme: dark)\">\u003clink data-mint-head rel=\"icon\" type=\"image/png\" sizes=\"16x16\" href=\"/mintlify-assets/_mintlify/favicons/agentx-ffe8d995/sticQuUfAA1h_xwo/_generated/favicon-dark/favicon-16x16.png\" media=\"(prefers-color-scheme: dark)\">\u003clink data-mint-head rel=\"icon\" type=\"image/png\" sizes=\"32x32\" href=\"/mintlify-assets/_mintlify/favicons/agentx-ffe8d995/sticQuUfAA1h_xwo/_generated/favicon-dark/favicon-32x32.png\" media=\"(prefers-color-scheme: dark)\">\u003clink data-mint-head rel=\"shortcut icon\" type=\"image/x-icon\" href=\"/mintlify-assets/_mintlify/favicons/agentx-ffe8d995/sticQuUfAA1h_xwo/_generated/favicon-dark/favicon.ico\" media=\"(prefers-color-scheme: dark)\">\u003cmeta data-mint-head property=\"og:title\" content=\"Submit Results - AgentX\">\u003cmeta data-mint-head property=\"og:description\" content=\"Submit a batch of agent outputs for synchronous judge scoring\">\u003cmeta data-mint-head property=\"og:site_name\" content=\"AgentX\">\u003cmeta data-mint-head property=\"og:url\" content=\"https://developers.agentx.so/api-reference/custom-eval/submit-results\">\u003cmeta data-mint-head property=\"og:type\" content=\"website\">\u003cmeta data-mint-head property=\"og:image\" content=\"https://agentx-ffe8d995.mintlify.site/_mintlify/api/og/v2?division=Custom+Evaluations\u0026amp;title=Submit+Results\u0026amp;description=Submit+a+batch+of+agent+outputs+for+synchronous+judge+scoring\u0026amp;theme=2ebdc38e645cd0f197f78a50\u0026amp;signature=0d8a7ec3c07a8d49104dd715dae00dc9d66a9389de6ac1d195e929897fe41ae3\">\u003cmeta data-mint-head property=\"og:image:width\" content=\"1200\">\u003cmeta data-mint-head property=\"og:image:height\" content=\"630\">\u003cmeta data-mint-head name=\"twitter:card\" content=\"summary_large_image\">\u003cmeta data-mint-head name=\"twitter:title\" content=\"Submit Results - AgentX\">\u003cmeta data-mint-head name=\"twitter:description\" content=\"Submit a batch of agent outputs for synchronous judge scoring\">\u003cmeta data-mint-head name=\"twitter:image\" content=\"https://agentx-ffe8d995.mintlify.site/_mintlify/api/og/v2?division=Custom+Evaluations\u0026amp;title=Submit+Results\u0026amp;description=Submit+a+batch+of+agent+outputs+for+synchronous+judge+scoring\u0026amp;theme=2ebdc38e645cd0f197f78a50\u0026amp;signature=0d8a7ec3c07a8d49104dd715dae00dc9d66a9389de6ac1d195e929897fe41ae3\">\u003cmeta data-mint-head name=\"twitter:image:width\" content=\"1200\">\u003cmeta data-mint-head name=\"twitter:image:height\" content=\"630\">\u003cmeta data-mint-head name=\"msapplication-config\" content=\"/mintlify-assets/_mintlify/favicons/agentx-ffe8d995/sticQuUfAA1h_xwo/_generated/favicon/browserconfig.xml\">\u003cmeta data-mint-head name=\"apple-mobile-web-app-title\" content=\"AgentX\">\u003cmeta data-mint-head name=\"msapplication-TileColor\" content=\"#2A5699\">\u003clink data-mint-head rel=\"canonical\" href=\"https://developers.agentx.so/api-reference/custom-eval/submit-results\">\u003clink data-mint-head rel=\"alternate\" type=\"application/xml\" href=\"/sitemap.xml\">\u003clink data-mint-head rel=\"alternate\" type=\"text/markdown\" href=\"/api-reference/custom-eval/submit-results.md\">\u003cscript data-mint-head type=\"application/ld+json\">{\"@context\":\"https://schema.org\",\"@graph\":[{\"@type\":\"Organization\",\"@id\":\"https://developers.agentx.so/#organization\",\"name\":\"AgentX\",\"url\":\"https://developers.agentx.so\",\"logo\":{\"@type\":\"ImageObject\",\"url\":\"https://mintcdn.com/agentx-ffe8d995/ZTzZ3HDQJ-RJxymw/logo/light.png?fit=max\u0026auto=format\u0026n=ZTzZ3HDQJ-RJxymw\u0026q=85\u0026s=c1e30c949dc1cc2aaa3a137c1ee5c77f\"}},{\"@type\":\"WebSite\",\"@id\":\"https://developers.agentx.so#website\",\"name\":\"AgentX\",\"url\":\"https://developers.agentx.so\",\"publisher\":{\"@id\":\"https://developers.agentx.so/#organization\"}},{\"@type\":\"WebPage\",\"@id\":\"https://developers.agentx.so/api-reference/custom-eval/submit-results#webpage\",\"url\":\"https://developers.agentx.so/api-reference/custom-eval/submit-results\",\"name\":\"Submit Results\",\"description\":\"Submit a batch of agent outputs for synchronous judge scoring\",\"dateModified\":\"2026-09-14T02:27:30.272Z\",\"isPartOf\":{\"@id\":\"https://developers.agentx.so#website\"},\"breadcrumb\":{\"@id\":\"https://developers.agentx.so/api-reference/custom-eval/submit-results#breadcrumb\"}},{\"@type\":\"BreadcrumbList\",\"@id\":\"https://developers.agentx.so/api-reference/custom-eval/submit-results#breadcrumb\",\"itemListElement\":[{\"@type\":\"ListItem\",\"position\":1,\"name\":\"Custom Evaluations\",\"item\":\"https://developers.agentx.so/api-reference/custom-eval/overview\"},{\"@type\":\"ListItem\",\"position\":2,\"name\":\"Submit Results\",\"item\":\"https://developers.agentx.so/api-reference/custom-eval/submit-results\"}]},{\"@type\":[\"TechArticle\",\"APIReference\"],\"@id\":\"https://developers.agentx.so/api-reference/custom-eval/submit-results#apireference\",\"headline\":\"Submit Results\",\"name\":\"Submit Results\",\"description\":\"Submit a batch of agent outputs for synchronous judge scoring\",\"url\":\"https://developers.agentx.so/api-reference/custom-eval/submit-results\",\"mainEntityOfPage\":{\"@id\":\"https://developers.agentx.so/api-reference/custom-eval/submit-results#webpage\"},\"dateModified\":\"2026-09-14T02:27:30.272Z\",\"publisher\":{\"@id\":\"https://developers.agentx.so/#organization\"},\"isPartOf\":{\"@id\":\"https://developers.agentx.so#website\"}}]}\u003c/script>\u003cscript data-mint-head type=\"application/ld+json\">{\"@context\":\"https://schema.org\",\"@type\":\"WebSite\",\"name\":\"AgentX\",\"publisher\":{\"@type\":\"Organization\",\"name\":\"AgentX\"},\"provider\":{\"@type\":\"Organization\",\"name\":\"Mintlify\",\"url\":\"https://mintlify.com\"}}\u003c/script>","agentPointerHref":"/api-reference/custom-eval/submit-results.md","siteName":"AgentX","nav":{"global":null,"tabs":[{"tab":"Documentation","pages":[{"title":"Documentation","href":"/introduction"}]},{"tab":"Trace","pages":[{"title":"Trace","href":"/sdk/tracing"}]},{"tab":"Scorers \u0026 Monitoring","pages":[{"title":"Scorers \u0026 Monitoring","href":"/monitor/scorers"}]},{"tab":"Offline Evaluation","pages":[{"title":"Offline Evaluation","href":"/sdk/evaluations/overview"}]},{"tab":"Improve","pages":[{"title":"Improve","href":"/improve/tool-schemas"}]},{"tab":"CI/CD","pages":[{"title":"CI/CD","href":"/integrations/self-host-ci"}]},{"tab":"API Reference","groups":[{"group":"Tracing","pages":[{"title":"Submit Trace","api":"POST /api/v1/ingest/traces","href":"/api-reference/tracing/submit-trace"},{"title":"List Traces","api":"GET /api/v1/ingest/traces","href":"/api-reference/tracing/list-traces"},{"title":"Evaluate Trace","api":"POST /api/v1/ingest/traces/{traceId}/evaluate","href":"/api-reference/tracing/evaluate-trace"}]},{"group":"Custom Evaluations","pages":[{"title":"Custom Evaluations Overview","href":"/api-reference/custom-eval/overview"},{"title":"Create Dataset","api":"POST /api/v1/custom-agent-evaluations/datasets","href":"/api-reference/custom-eval/create-dataset"},{"title":"List Datasets","api":"GET /api/v1/custom-agent-evaluations/datasets","href":"/api-reference/custom-eval/list-datasets"},{"title":"Get Dataset","api":"GET /api/v1/custom-agent-evaluations/datasets/{datasetId}","href":"/api-reference/custom-eval/get-dataset"},{"title":"Create Run","api":"POST /api/v1/custom-agent-evaluations/runs","description":"Start a new evaluation run against a dataset","href":"/api-reference/custom-eval/create-run"},{"title":"Submit Results","api":"POST /api/v1/custom-agent-evaluations/runs/{runId}/results","href":"/api-reference/custom-eval/submit-results"},{"title":"Get Missing Results","api":"GET /api/v1/custom-agent-evaluations/runs/{runId}/missing-results","description":"Resume support: the idempotency keys a run has already accepted","href":"/api-reference/custom-eval/missing-results"},{"title":"Finalize Run","api":"POST /api/v1/custom-agent-evaluations/runs/{runId}/finalize","href":"/api-reference/custom-eval/finalize-run"},{"title":"Analyze Run","api":"POST /api/v1/custom-agent-evaluations/runs/{runId}/analyze","href":"/api-reference/custom-eval/analyze-run"},{"title":"Get Report","api":"GET /api/v1/custom-agent-evaluations/runs/{runId}/report","href":"/api-reference/custom-eval/get-report"},{"title":"Get Run","api":"GET /api/v1/custom-agent-evaluations/runs/{runId}","href":"/api-reference/custom-eval/get-run"},{"title":"List Runs","api":"GET /api/v1/custom-agent-evaluations/runs","href":"/api-reference/custom-eval/list-runs"},{"title":"List Models","api":"GET /api/v1/agent-monitoring/portability/models","href":"/api-reference/custom-eval/list-models"}]},{"group":"CI Gate","pages":[{"title":"CI/CD Overview","href":"/api-reference/ci-cd/overview"},{"title":"Create CI Run","api":"POST /api/v1/custom-agent-evaluations/runs","href":"/api-reference/ci-cd/create-run"},{"title":"Get Test Cases","api":"GET /api/v1/custom-agent-evaluations/datasets/{datasetId}","href":"/api-reference/ci-cd/get-test-cases"},{"title":"Submit Result","api":"POST /api/v1/custom-agent-evaluations/runs/{runId}/results","href":"/api-reference/ci-cd/submit-result"},{"title":"Finalize CI Run","api":"POST /api/v1/custom-agent-evaluations/runs/{runId}/finalize","href":"/api-reference/ci-cd/finalize-run"},{"title":"Get CI Gate","api":"GET /api/v1/custom-agent-evaluations/runs/{runId}/gate","href":"/api-reference/ci-cd/get-gate"}]},{"group":"Agent Registry","pages":[{"title":"List Agents","api":"GET /api/v1/agents","href":"/api-reference/tracked-agents/list"},{"title":"Register Agent","api":"POST /api/v1/agents","href":"/api-reference/tracked-agents/add"},{"title":"Get Agent","api":"GET /api/v1/agents/{id}","href":"/api-reference/tracked-agents/get"}]}]}]},"basePath":"","title":"Submit Results - AgentX","deploymentId":"dpl_DyXGnKeCjZyA3sUdfMJ7FjkdBefg","buildId":"dpl_DyXGnKeCjZyA3sUdfMJ7FjkdBefg","dynamics":["c2aa737661","2b790eda92","b0f84f53ff"],"chunks":["OpenApiReferenceProvider-CiUHMMOm.js","snippetPresets-BgfXU7Bw.js","utils-DWvOwgy_.js","style-to-object-BY3JEAdk.js","utils-BN7RXSMJ.js","v4-DIgz8zy6.js","addUniqueKeysToSchema-CFw8bq89.js","conditional-requirements-CFM0id0Q.js","cloneDeep-CAwdfOU2.js","_Uint8Array-BJr8e-ZL.js","isObject-FP1Yky1u.js","ApiExamples-XX0UgMZK.js","usePlaygroundInputsStore-DXDFSa1V.js","shallow-CdX5aREi.js","isEqual-CoOaxexF.js","CodeSnippets-Cg2wpzWm.js","CodeSnippet-D_HwgQO_.js","getApiPlaygroundDisplay-CmCoerhe.js","Page-CVJ-f_k1.js","DynamicMarkdown-CQM9Bhbv.js","SectionHeading-DpbrP8nQ.js","icon-BPCgijHy.js","MethodPill-GQ8DuY0c.js","getPastedJson-Cb5mT31m.js","Check2Icon-a7bUI6e8.js","axios-CsQoIw0U.js"]}