report = client.evaluations.get_report("rK7dP2qWx9TzB4mV6nJcE")print(report.summary)for case in report.low_scoring_cases: print(case)
{ "summary": "The agent performs well on direct how-to questions but struggles with multi-step billing queries.", "consistencyScore": 8, "instructionAdherence": { "score": 9, "analysis": "The agent generally follows the support tone guidelines.", "deviations": ["Q3 run 2: agent gave a refund without checking eligibility"], "rating": "high" }, "responsePatterns": { "similarities": ["Consistent greeting and closing"], "differences": ["Detail level varies on billing answers"], "outliers": ["Q3 run 2"], "rating": "medium" }, "reasoningAnalysis": { "cotQuality": "Reasoning is visible and mostly sound.", "reasoningPatterns": ["Looks up KB before answering"], "reasoningGaps": ["Skips eligibility checks under time pressure"], "rating": "medium" }, "toolUsageAnalysis": { "effectiveness": "KB search used appropriately on most cases.", "patterns": ["search_kb before every answer"], "issues": ["No billing tool call on refund cases"], "rating": "medium" }, "recommendations": [ { "category": "instructions", "priority": "high", "recommendation": "Add a guardrail that requires an eligibility check before issuing refunds", "reasoning": "One run issued a refund without confirming the customer's eligibility window" } ], "instructionChanges": [], "overallAssessment": { "strengths": ["Consistent tone", "Accurate password reset guidance"], "weaknesses": ["Billing edge cases", "Multi-step reasoning"], "rating": "medium" }, "runId": "rK7dP2qWx9TzB4mV6nJcE", "datasetId": "dS4tG7hNb2VxZ8kQ5wMyA", "status": "completed", "statistics": { "numberOfRuns": 6, "averageRating": 7.8, "minRating": 5, "maxRating": 10, "ratingVariance": 1.96 }, "lowScoringCases": [ { "query": "Why was I charged twice?", "response": "You may have been charged twice due to a pending authorization.", "rating": 5, "justification": "Agent did not escalate to the billing team as required." } ]}
{ "error": "No analysis found for this run. POST /runs/:runId/analyze first."}
Custom Evaluations
Get Report
Get the stored analysis report with statistics and low-scoring cases
report = client.evaluations.get_report("rK7dP2qWx9TzB4mV6nJcE")print(report.summary)for case in report.low_scoring_cases: print(case)
{ "summary": "The agent performs well on direct how-to questions but struggles with multi-step billing queries.", "consistencyScore": 8, "instructionAdherence": { "score": 9, "analysis": "The agent generally follows the support tone guidelines.", "deviations": ["Q3 run 2: agent gave a refund without checking eligibility"], "rating": "high" }, "responsePatterns": { "similarities": ["Consistent greeting and closing"], "differences": ["Detail level varies on billing answers"], "outliers": ["Q3 run 2"], "rating": "medium" }, "reasoningAnalysis": { "cotQuality": "Reasoning is visible and mostly sound.", "reasoningPatterns": ["Looks up KB before answering"], "reasoningGaps": ["Skips eligibility checks under time pressure"], "rating": "medium" }, "toolUsageAnalysis": { "effectiveness": "KB search used appropriately on most cases.", "patterns": ["search_kb before every answer"], "issues": ["No billing tool call on refund cases"], "rating": "medium" }, "recommendations": [ { "category": "instructions", "priority": "high", "recommendation": "Add a guardrail that requires an eligibility check before issuing refunds", "reasoning": "One run issued a refund without confirming the customer's eligibility window" } ], "instructionChanges": [], "overallAssessment": { "strengths": ["Consistent tone", "Accurate password reset guidance"], "weaknesses": ["Billing edge cases", "Multi-step reasoning"], "rating": "medium" }, "runId": "rK7dP2qWx9TzB4mV6nJcE", "datasetId": "dS4tG7hNb2VxZ8kQ5wMyA", "status": "completed", "statistics": { "numberOfRuns": 6, "averageRating": 7.8, "minRating": 5, "maxRating": 10, "ratingVariance": 1.96 }, "lowScoringCases": [ { "query": "Why was I charged twice?", "response": "You may have been charged twice due to a pending authorization.", "rating": 5, "justification": "Agent did not escalate to the billing team as required." } ]}
{ "error": "No analysis found for this run. POST /runs/:runId/analyze first."}
Returns the report Analyze Run stored for this run:
the narrative analysis hoisted to the top level, plus the run’s identifiers, the statistics
computed when the analysis ran, and the run’s low-scoring cases. Requires an analysis to exist -
a run that was never analyzed returns 404, so an empty report is never mistaken for a run
that scored nothing.
Authentication
string
required
Project API key.
Path Parameters
string
required
Run ID.
Response
Narrative analysis fields (top level, from the analysis body; absent when the analysis
failed):
Field
Type
Description
summary
string
A few sentences on how the agent performed overall
consistencyScore
number
0-10: how consistent responses were across similar inputs
[{ category, priority, recommendation, reasoning }] - category is one of "instructions", "tools", "knowledge", "reasoning", "consistency", "other"; priority and every rating above are "high" | "medium" | "low"
instructionChanges
array
Always [] for external agents (the engine doesn’t own the agent’s instructions)
overallAssessment
object
{ strengths[], weaknesses[], rating }
Run fields (always present):
string
Run ID.
string
Dataset the run scored against.
string
The stored analysis status: "completed" or "failed".
object | null
Computed when the analysis ran:
{ numberOfRuns, averageRating, minRating, maxRating, ratingVariance }, computed over rated
rows with smoke-test variants excluded - the same population as
Get Run’s averageRating, so the two agree.
array
Results with a rating of 5 or below (smoke-test variants excluded), derived fresh from the
run’s own rows: [{ query, response, rating, justification }].
Similarity aggregates (cosine/Jaccard/BLEU/ROUGE) are per-result, not report-level - read
them from Get Run’s results rows. strengths,
weaknesses, and the overall rating live under overallAssessment, not at the top level.
Errors
Status
Body
Meaning
404
{ "error": "Run not found" }
No run with this id in the project
404
{ "error": "No analysis found for this run. POST /runs/:runId/analyze first." }
report = client.evaluations.get_report("rK7dP2qWx9TzB4mV6nJcE")print(report.summary)for case in report.low_scoring_cases: print(case)
{ "summary": "The agent performs well on direct how-to questions but struggles with multi-step billing queries.", "consistencyScore": 8, "instructionAdherence": { "score": 9, "analysis": "The agent generally follows the support tone guidelines.", "deviations": ["Q3 run 2: agent gave a refund without checking eligibility"], "rating": "high" }, "responsePatterns": { "similarities": ["Consistent greeting and closing"], "differences": ["Detail level varies on billing answers"], "outliers": ["Q3 run 2"], "rating": "medium" }, "reasoningAnalysis": { "cotQuality": "Reasoning is visible and mostly sound.", "reasoningPatterns": ["Looks up KB before answering"], "reasoningGaps": ["Skips eligibility checks under time pressure"], "rating": "medium" }, "toolUsageAnalysis": { "effectiveness": "KB search used appropriately on most cases.", "patterns": ["search_kb before every answer"], "issues": ["No billing tool call on refund cases"], "rating": "medium" }, "recommendations": [ { "category": "instructions", "priority": "high", "recommendation": "Add a guardrail that requires an eligibility check before issuing refunds", "reasoning": "One run issued a refund without confirming the customer's eligibility window" } ], "instructionChanges": [], "overallAssessment": { "strengths": ["Consistent tone", "Accurate password reset guidance"], "weaknesses": ["Billing edge cases", "Multi-step reasoning"], "rating": "medium" }, "runId": "rK7dP2qWx9TzB4mV6nJcE", "datasetId": "dS4tG7hNb2VxZ8kQ5wMyA", "status": "completed", "statistics": { "numberOfRuns": 6, "averageRating": 7.8, "minRating": 5, "maxRating": 10, "ratingVariance": 1.96 }, "lowScoringCases": [ { "query": "Why was I charged twice?", "response": "You may have been charged twice due to a pending authorization.", "rating": 5, "justification": "Agent did not escalate to the billing team as required." } ]}
{ "error": "No analysis found for this run. POST /runs/:runId/analyze first."}
{"docsConfig":{"theme":"mint","$schema":"https://mintlify.com/docs.json","name":"AgentX","colors":{"primary":"#2A5699","light":"#5C8DCC","dark":"#1A3F73"},"logo":{"light":"https://mintcdn.com/agentx-ffe8d995/ZTzZ3HDQJ-RJxymw/logo/light.png?fit=max\u0026auto=format\u0026n=ZTzZ3HDQJ-RJxymw\u0026q=85\u0026s=c1e30c949dc1cc2aaa3a137c1ee5c77f","dark":"https://mintcdn.com/agentx-ffe8d995/ZTzZ3HDQJ-RJxymw/logo/dark.png?fit=max\u0026auto=format\u0026n=ZTzZ3HDQJ-RJxymw\u0026q=85\u0026s=35a521e090f7875c15bef7ae392af788"},"favicon":"https://mintcdn.com/agentx-ffe8d995/FRrvmlkH0m2BUb9c/logo/favicon.png?fit=max\u0026auto=format\u0026n=FRrvmlkH0m2BUb9c\u0026q=85\u0026s=e87d636d38c8e6e187b4be187e72381d","api":{"mdx":{"auth":{"method":"key","name":"x-api-key"},"server":"http://localhost:4700"}},"navbar":{"primary":{"type":"github","href":"https://github.com/AgentX-ai/AgentX-trace-eval"}},"footer":{"socials":{"website":"https://www.agentx.so/mcp/ai-evaluation","github":"https://github.com/AgentX-ai/AgentX-trace-eval"}},"styling":{"latex":true},"navigation":{"pages":[]}},"deployment":{"deploymentMetadataValue":{"subdomain":"agentx-ffe8d995","hasAudioNarration":false,"actualSubdomain":"agentx-ffe8d995","customDomains":["developers.agentx.so"],"gitSource":{"type":"github","owner":"agentx-ai","repo":"agentx-mintlify","deployBranch":"main","contentDirectory":"","isPrivate":true,"mountPath":"","label":""},"entitlements":{"AI_CHAT":{"status":"DISABLED"}},"buildId":"6ac683381cb9dd011c327d04","clientVersion":"0.0.4270","searchFilterCounts":[{"language":"default","tag":"API Reference","count":25},{"language":"default","tag":"Offline Evaluation","count":24},{"language":"default","tag":"Trace","count":17},{"language":"default","tag":"Scorers \u0026 Monitoring","count":16},{"language":"default","tag":"Custom Evaluations","count":13},{"language":"default","tag":"Documentation","count":13},{"language":"default","tag":"Framework Integrations","count":12},{"language":"default","tag":"Framework Examples","count":10},{"language":"default","tag":"Tracing","count":8},{"language":"default","tag":"Live Traffic Scoring","count":7}]},"assistantConfig":{"enableAskAiButton":false,"enableStarterQuestions":false,"enableContextualStarterQuestions":false,"enableAskAiSearchBar":true},"basePath":"","hasChatPermissions":false,"hasWebMcpPermissions":false,"toggles":[{"name":"kill-isr","enabled":true,"variant":{"name":"disabled","enabled":false,"feature_enabled":true},"impressionData":false},{"name":"custom-rbac-roles","enabled":true,"variant":{"name":"disabled","enabled":false,"feature_enabled":true},"impressionData":false},{"name":"agent-mintlify-slack","enabled":true,"variant":{"name":"disabled","enabled":false,"feature_enabled":true},"impressionData":false},{"name":"agent-integrations-settings","enabled":true,"variant":{"name":"disabled","enabled":false,"feature_enabled":true},"impressionData":false},{"name":"rbac-expansion","enabled":true,"variant":{"name":"disabled","enabled":false,"feature_enabled":true},"impressionData":false},{"name":"automations-pr-grouping","enabled":true,"variant":{"name":"disabled","enabled":false,"feature_enabled":true},"impressionData":false},{"name":"custom-prefix","enabled":true,"variant":{"name":"disabled","enabled":false,"feature_enabled":true},"impressionData":false},{"name":"analytics-v7","enabled":true,"variant":{"name":"disabled","enabled":false,"feature_enabled":true},"impressionData":false},{"name":"user-flows-analytics","enabled":true,"variant":{"name":"disabled","enabled":false,"feature_enabled":true},"impressionData":false},{"name":"no-publish-flow","enabled":true,"variant":{"name":"disabled","enabled":false,"feature_enabled":true},"impressionData":false},{"name":"workos-onboarding","enabled":true,"variant":{"name":"disabled","enabled":false,"feature_enabled":true},"impressionData":false},{"name":"snapshot-serving-live","enabled":true,"variant":{"name":"disabled","enabled":false,"feature_enabled":true},"impressionData":false},{"name":"stripTemperatureMiddleware","enabled":true,"variant":{"name":"disabled","enabled":false,"feature_enabled":true},"impressionData":false},{"name":"support-platform-pylon-zendesk","enabled":true,"variant":{"name":"disabled","enabled":false,"feature_enabled":true},"impressionData":false},{"name":"snapshot-serving","enabled":true,"variant":{"name":"disabled","enabled":false,"feature_enabled":true},"impressionData":false},{"name":"instant-rollback","enabled":true,"variant":{"name":"disabled","enabled":false,"feature_enabled":true},"impressionData":false},{"name":"granular-notifications-server","enabled":true,"variant":{"name":"disabled","enabled":false,"feature_enabled":true},"impressionData":false},{"name":"granular-notifications","enabled":true,"variant":{"name":"disabled","enabled":false,"feature_enabled":true},"impressionData":false},{"name":"automations-fable","enabled":true,"variant":{"name":"disabled","enabled":false,"feature_enabled":true},"impressionData":false},{"name":"fixed-ai-action-pricing","enabled":true,"variant":{"name":"disabled","enabled":false,"feature_enabled":true},"impressionData":false},{"name":"site-roles","enabled":true,"variant":{"name":"disabled","enabled":false,"feature_enabled":true},"impressionData":false},{"name":"automation-resolve-merge-conflict","enabled":true,"variant":{"name":"disabled","enabled":false,"feature_enabled":true},"impressionData":false},{"name":"scoped-publishing","enabled":true,"variant":{"name":"disabled","enabled":false,"feature_enabled":true},"impressionData":false},{"name":"mintql","enabled":true,"variant":{"name":"disabled","enabled":false,"feature_enabled":true},"impressionData":false},{"name":"page-level-analytics","enabled":true,"variant":{"name":"disabled","enabled":false,"feature_enabled":true},"impressionData":false},{"name":"editor-version-history-v2","enabled":true,"variant":{"name":"disabled","enabled":false,"feature_enabled":true},"impressionData":false},{"name":"billing-page-v2","enabled":true,"variant":{"name":"disabled","enabled":false,"feature_enabled":true},"impressionData":false},{"name":"full-page-search","enabled":true,"variant":{"name":"disabled","enabled":false,"feature_enabled":true},"impressionData":false},{"name":"workspace-projects","enabled":true,"variant":{"name":"disabled","enabled":false,"feature_enabled":true},"impressionData":false},{"name":"assistant-v5","enabled":true,"variant":{"name":"disabled","enabled":false,"feature_enabled":true},"impressionData":false},{"name":"fork-pr-previews","enabled":true,"variant":{"name":"disabled","enabled":false,"feature_enabled":true},"impressionData":false},{"name":"revalidation-jwt","enabled":true,"variant":{"name":"disabled","enabled":false,"feature_enabled":true},"impressionData":false},{"name":"workos-sso-scim","enabled":true,"variant":{"name":"disabled","enabled":false,"feature_enabled":true},"impressionData":false},{"name":"hobby-abuse-scan","enabled":true,"variant":{"name":"disabled","enabled":false,"feature_enabled":true},"impressionData":false},{"name":"dashboard-inbox","enabled":true,"variant":{"name":"disabled","enabled":false,"feature_enabled":true},"impressionData":false},{"name":"workos-signup","enabled":true,"variant":{"name":"disabled","enabled":false,"feature_enabled":true},"impressionData":false},{"name":"turbopuffer-dual-write","enabled":true,"variant":{"name":"disabled","enabled":false,"feature_enabled":true},"impressionData":false},{"name":"workos-enabled","enabled":true,"variant":{"name":"disabled","enabled":false,"feature_enabled":true},"impressionData":false},{"name":"approve-review-flow","enabled":true,"variant":{"name":"disabled","enabled":false,"feature_enabled":true},"impressionData":false},{"name":"editor-metadata","enabled":true,"variant":{"name":"disabled","enabled":false,"feature_enabled":true},"impressionData":false},{"name":"sso-setup-deadline-banner","enabled":true,"variant":{"name":"disabled","enabled":false,"feature_enabled":true},"impressionData":false},{"name":"turbopuffer-serve","enabled":true,"variant":{"name":"disabled","enabled":false,"feature_enabled":true},"impressionData":false},{"name":"live-agents","enabled":true,"variant":{"name":"disabled","enabled":false,"feature_enabled":true},"impressionData":false},{"name":"starter-pricing","enabled":true,"variant":{"name":"treatment","enabled":true,"payload":{"type":"string","value":"treatment"},"feature_enabled":true},"impressionData":false},{"name":"llms-txt-v2","enabled":true,"variant":{"name":"disabled","enabled":false,"feature_enabled":true},"impressionData":false},{"name":"wildcard-email-auth","enabled":true,"variant":{"name":"disabled","enabled":false,"feature_enabled":true},"impressionData":false}],"googleFontsInlined":false},"customCss":[]}
{"page":{"pagePath":"api-reference/custom-eval/get-report","slug":"/api-reference/custom-eval/get-report","pageMetadata":{"title":"Get Report","api":"GET /api/v1/custom-agent-evaluations/runs/{runId}/report","description":"Get the stored analysis report with statistics and low-scoring cases","href":"/api-reference/custom-eval/get-report"},"description":{"compiledSource":"\"use strict\";\nconst {jsx: _jsx} = arguments[0];\nconst {useMDXComponents: _provideComponents} = arguments[0];\nfunction _createMdxContent(props) {\n const _components = {\n p: \"p\",\n ..._provideComponents(),\n ...props.components\n };\n return _jsx(_components.p, {\n children: \"Get the stored analysis report with statistics and low-scoring cases\"\n });\n}\nfunction MDXContent(props = {}) {\n const {wrapper: MDXLayout} = {\n ..._provideComponents(),\n ...props.components\n };\n return MDXLayout ? _jsx(MDXLayout, {\n ...props,\n children: _jsx(_createMdxContent, {\n ...props\n })\n }) : _createMdxContent(props);\n}\nreturn {\n default: MDXContent\n};\n","frontmatter":{},"scope":{}},"pageType":"default","mdx":{"mdxSource":{"compiledSource":"\"use strict\";const{Fragment:_Fragment,jsx:_jsx,jsxs:_jsxs}=arguments[0];const{useMDXComponents:_provideComponents}=arguments[0];function _createMdxContent(props){const _components={a:`a`,code:`code`,p:`p`,tbody:`tbody`,td:`td`,th:`th`,thead:`thead`,tr:`tr`,..._provideComponents(),...props.components},{Heading,Note,ParamField,ResponseField,Table}=_components;if(!Heading)_missingMdxReference(`Heading`,true);if(!Note)_missingMdxReference(`Note`,true);if(!ParamField)_missingMdxReference(`ParamField`,true);if(!ResponseField)_missingMdxReference(`ResponseField`,true);if(!Table)_missingMdxReference(`Table`,true);return _jsxs(_Fragment,{children:[_jsxs(_components.p,{children:[`Returns the report `,_jsx(_components.a,{href:`/api-reference/custom-eval/analyze-run`,children:`Analyze Run`}),` stored for this run:\nthe narrative analysis hoisted to the top level, plus the run’s identifiers, the statistics\ncomputed when the analysis ran, and the run’s low-scoring cases. Requires an analysis to exist -\na run that was never analyzed returns `,_jsx(_components.code,{children:`404`}),`, so an empty report is never mistaken for a run\nthat scored nothing.`]}),`\n`,_jsx(Heading,{level:`2`,id:`authentication`,children:`Authentication`}),`\n`,_jsx(ParamField,{header:`x-api-key`,type:`string`,required:true,id:`param-x-api-key`,children:_jsx(_components.p,{children:`Project API key.`})}),`\n`,_jsx(Heading,{level:`2`,id:`path-parameters`,children:`Path Parameters`}),`\n`,_jsx(ParamField,{path:`runId`,type:`string`,required:true,id:`param-run-id`,children:_jsx(_components.p,{children:`Run ID.`})}),`\n`,_jsx(Heading,{level:`2`,id:`response`,children:`Response`}),`\n`,_jsx(_components.p,{children:`Narrative analysis fields (top level, from the analysis body; absent when the analysis\nfailed):`}),`\n`,_jsxs(Table,{children:[_jsx(_components.thead,{children:_jsxs(_components.tr,{children:[_jsx(_components.th,{children:`Field`}),_jsx(_components.th,{children:`Type`}),_jsx(_components.th,{children:`Description`})]})}),_jsxs(_components.tbody,{children:[_jsxs(_components.tr,{children:[_jsx(_components.td,{children:_jsx(_components.code,{children:`summary`})}),_jsx(_components.td,{children:`string`}),_jsx(_components.td,{children:`A few sentences on how the agent performed overall`})]}),_jsxs(_components.tr,{children:[_jsx(_components.td,{children:_jsx(_components.code,{children:`consistencyScore`})}),_jsx(_components.td,{children:`number`}),_jsx(_components.td,{children:`0-10: how consistent responses were across similar inputs`})]}),_jsxs(_components.tr,{children:[_jsx(_components.td,{children:_jsx(_components.code,{children:`instructionAdherence`})}),_jsx(_components.td,{children:`object`}),_jsx(_components.td,{children:_jsx(_components.code,{children:`{ score (0-10), analysis, deviations[], rating }`})})]}),_jsxs(_components.tr,{children:[_jsx(_components.td,{children:_jsx(_components.code,{children:`responsePatterns`})}),_jsx(_components.td,{children:`object`}),_jsx(_components.td,{children:_jsx(_components.code,{children:`{ similarities[], differences[], outliers[], rating }`})})]}),_jsxs(_components.tr,{children:[_jsx(_components.td,{children:_jsx(_components.code,{children:`reasoningAnalysis`})}),_jsx(_components.td,{children:`object`}),_jsx(_components.td,{children:_jsx(_components.code,{children:`{ cotQuality, reasoningPatterns[], reasoningGaps[], rating }`})})]}),_jsxs(_components.tr,{children:[_jsx(_components.td,{children:_jsx(_components.code,{children:`toolUsageAnalysis`})}),_jsx(_components.td,{children:`object`}),_jsx(_components.td,{children:_jsx(_components.code,{children:`{ effectiveness, patterns[], issues[], rating }`})})]}),_jsxs(_components.tr,{children:[_jsx(_components.td,{children:_jsx(_components.code,{children:`recommendations`})}),_jsx(_components.td,{children:`array`}),_jsxs(_components.td,{children:[_jsx(_components.code,{children:`[{ category, priority, recommendation, reasoning }]`}),` - `,_jsx(_components.code,{children:`category`}),` is one of `,_jsx(_components.code,{children:`\"instructions\"`}),`, `,_jsx(_components.code,{children:`\"tools\"`}),`, `,_jsx(_components.code,{children:`\"knowledge\"`}),`, `,_jsx(_components.code,{children:`\"reasoning\"`}),`, `,_jsx(_components.code,{children:`\"consistency\"`}),`, `,_jsx(_components.code,{children:`\"other\"`}),`; `,_jsx(_components.code,{children:`priority`}),` and every `,_jsx(_components.code,{children:`rating`}),` above are `,_jsx(_components.code,{children:`\"high\"`}),` | `,_jsx(_components.code,{children:`\"medium\"`}),` | `,_jsx(_components.code,{children:`\"low\"`})]})]}),_jsxs(_components.tr,{children:[_jsx(_components.td,{children:_jsx(_components.code,{children:`instructionChanges`})}),_jsx(_components.td,{children:`array`}),_jsxs(_components.td,{children:[`Always `,_jsx(_components.code,{children:`[]`}),` for external agents (the engine doesn’t own the agent’s instructions)`]})]}),_jsxs(_components.tr,{children:[_jsx(_components.td,{children:_jsx(_components.code,{children:`overallAssessment`})}),_jsx(_components.td,{children:`object`}),_jsx(_components.td,{children:_jsx(_components.code,{children:`{ strengths[], weaknesses[], rating }`})})]})]})]}),`\n`,_jsx(_components.p,{children:`Run fields (always present):`}),`\n`,_jsx(ResponseField,{name:`runId`,type:`string`,id:`param-run-id-1`,children:_jsx(_components.p,{children:`Run ID.`})}),`\n`,_jsx(ResponseField,{name:`datasetId`,type:`string`,id:`param-dataset-id`,children:_jsx(_components.p,{children:`Dataset the run scored against.`})}),`\n`,_jsx(ResponseField,{name:`status`,type:`string`,id:`param-status`,children:_jsxs(_components.p,{children:[`The stored analysis status: `,_jsx(_components.code,{children:`\"completed\"`}),` or `,_jsx(_components.code,{children:`\"failed\"`}),`.`]})}),`\n`,_jsx(ResponseField,{name:`statistics`,type:`object | null`,id:`param-statistics`,children:_jsxs(_components.p,{children:[`Computed when the analysis ran:\n`,_jsx(_components.code,{children:`{ numberOfRuns, averageRating, minRating, maxRating, ratingVariance }`}),`, computed over rated\nrows with smoke-test variants excluded - the same population as\n`,_jsx(_components.a,{href:`/api-reference/custom-eval/get-run`,children:`Get Run`}),`’s `,_jsx(_components.code,{children:`averageRating`}),`, so the two agree.`]})}),`\n`,_jsx(ResponseField,{name:`lowScoringCases`,type:`array`,id:`param-low-scoring-cases`,children:_jsxs(_components.p,{children:[`Results with a rating of 5 or below (smoke-test variants excluded), derived fresh from the\nrun’s own rows: `,_jsx(_components.code,{children:`[{ query, response, rating, justification }]`}),`.`]})}),`\n`,_jsx(Note,{children:_jsxs(_components.p,{children:[`Similarity aggregates (cosine/Jaccard/BLEU/ROUGE) are per-result, not report-level - read\nthem from `,_jsx(_components.a,{href:`/api-reference/custom-eval/get-run`,children:`Get Run`}),`’s `,_jsx(_components.code,{children:`results`}),` rows. `,_jsx(_components.code,{children:`strengths`}),`,\n`,_jsx(_components.code,{children:`weaknesses`}),`, and the overall rating live under `,_jsx(_components.code,{children:`overallAssessment`}),`, not at the top level.`]})}),`\n`,_jsx(Heading,{level:`2`,id:`errors`,children:`Errors`}),`\n`,_jsxs(Table,{children:[_jsx(_components.thead,{children:_jsxs(_components.tr,{children:[_jsx(_components.th,{children:`Status`}),_jsx(_components.th,{children:`Body`}),_jsx(_components.th,{children:`Meaning`})]})}),_jsxs(_components.tbody,{children:[_jsxs(_components.tr,{children:[_jsx(_components.td,{\"data-numeric\":`true`,children:_jsx(_components.code,{children:`404`})}),_jsx(_components.td,{children:_jsx(_components.code,{children:`{ \"error\": \"Run not found\" }`})}),_jsx(_components.td,{children:`No run with this id in the project`})]}),_jsxs(_components.tr,{children:[_jsx(_components.td,{\"data-numeric\":`true`,children:_jsx(_components.code,{children:`404`})}),_jsx(_components.td,{children:_jsx(_components.code,{children:`{ \"error\": \"No analysis found for this run. POST /runs/:runId/analyze first.\" }`})}),_jsx(_components.td,{children:`The run exists but was never analyzed`})]})]})]})]})}function MDXContent(props={}){const{wrapper:MDXLayout}={..._provideComponents(),...props.components};return MDXLayout?_jsx(MDXLayout,{...props,children:_jsx(_createMdxContent,{...props})}):_createMdxContent(props)}return{default:MDXContent};function _missingMdxReference(id,component){throw new Error(`Expected `+(component?`component`:`object`)+\" `\"+id+\"` to be defined: you likely forgot to import, pass, or provide it.\")}","frontmatter":{},"scope":{"config":{},"pageMetadata":{"title":"Get Report","api":"GET /api/v1/custom-agent-evaluations/runs/{runId}/report","description":"Get the stored analysis report with statistics and low-scoring cases","href":"/api-reference/custom-eval/get-report"}}},"mdxSourceWithNoJs":{"compiledSource":"\"use strict\";const{Fragment:_Fragment,jsx:_jsx,jsxs:_jsxs}=arguments[0];const{useMDXComponents:_provideComponents}=arguments[0];function _createMdxContent(props){const _components={a:`a`,code:`code`,p:`p`,pre:`pre`,span:`span`,tbody:`tbody`,td:`td`,th:`th`,thead:`thead`,tr:`tr`,..._provideComponents(),...props.components},{CodeBlock,Heading,Note,ParamField,RequestExample,ResponseExample,ResponseField,Table}=_components;if(!CodeBlock)_missingMdxReference(`CodeBlock`,true);if(!Heading)_missingMdxReference(`Heading`,true);if(!Note)_missingMdxReference(`Note`,true);if(!ParamField)_missingMdxReference(`ParamField`,true);if(!RequestExample)_missingMdxReference(`RequestExample`,true);if(!ResponseExample)_missingMdxReference(`ResponseExample`,true);if(!ResponseField)_missingMdxReference(`ResponseField`,true);if(!Table)_missingMdxReference(`Table`,true);return _jsxs(_Fragment,{children:[_jsxs(_components.p,{children:[`Returns the report `,_jsx(_components.a,{href:`/api-reference/custom-eval/analyze-run`,children:`Analyze Run`}),` stored for this run:\nthe narrative analysis hoisted to the top level, plus the run’s identifiers, the statistics\ncomputed when the analysis ran, and the run’s low-scoring cases. Requires an analysis to exist -\na run that was never analyzed returns `,_jsx(_components.code,{children:`404`}),`, so an empty report is never mistaken for a run\nthat scored nothing.`]}),`\n`,_jsx(Heading,{level:`2`,id:`authentication`,children:`Authentication`}),`\n`,_jsx(ParamField,{header:`x-api-key`,type:`string`,required:true,id:`param-x-api-key`,children:_jsx(_components.p,{children:`Project API key.`})}),`\n`,_jsx(Heading,{level:`2`,id:`path-parameters`,children:`Path Parameters`}),`\n`,_jsx(ParamField,{path:`runId`,type:`string`,required:true,id:`param-run-id`,children:_jsx(_components.p,{children:`Run ID.`})}),`\n`,_jsx(Heading,{level:`2`,id:`response`,children:`Response`}),`\n`,_jsx(_components.p,{children:`Narrative analysis fields (top level, from the analysis body; absent when the analysis\nfailed):`}),`\n`,_jsxs(Table,{children:[_jsx(_components.thead,{children:_jsxs(_components.tr,{children:[_jsx(_components.th,{children:`Field`}),_jsx(_components.th,{children:`Type`}),_jsx(_components.th,{children:`Description`})]})}),_jsxs(_components.tbody,{children:[_jsxs(_components.tr,{children:[_jsx(_components.td,{children:_jsx(_components.code,{children:`summary`})}),_jsx(_components.td,{children:`string`}),_jsx(_components.td,{children:`A few sentences on how the agent performed overall`})]}),_jsxs(_components.tr,{children:[_jsx(_components.td,{children:_jsx(_components.code,{children:`consistencyScore`})}),_jsx(_components.td,{children:`number`}),_jsx(_components.td,{children:`0-10: how consistent responses were across similar inputs`})]}),_jsxs(_components.tr,{children:[_jsx(_components.td,{children:_jsx(_components.code,{children:`instructionAdherence`})}),_jsx(_components.td,{children:`object`}),_jsx(_components.td,{children:_jsx(_components.code,{children:`{ score (0-10), analysis, deviations[], rating }`})})]}),_jsxs(_components.tr,{children:[_jsx(_components.td,{children:_jsx(_components.code,{children:`responsePatterns`})}),_jsx(_components.td,{children:`object`}),_jsx(_components.td,{children:_jsx(_components.code,{children:`{ similarities[], differences[], outliers[], rating }`})})]}),_jsxs(_components.tr,{children:[_jsx(_components.td,{children:_jsx(_components.code,{children:`reasoningAnalysis`})}),_jsx(_components.td,{children:`object`}),_jsx(_components.td,{children:_jsx(_components.code,{children:`{ cotQuality, reasoningPatterns[], reasoningGaps[], rating }`})})]}),_jsxs(_components.tr,{children:[_jsx(_components.td,{children:_jsx(_components.code,{children:`toolUsageAnalysis`})}),_jsx(_components.td,{children:`object`}),_jsx(_components.td,{children:_jsx(_components.code,{children:`{ effectiveness, patterns[], issues[], rating }`})})]}),_jsxs(_components.tr,{children:[_jsx(_components.td,{children:_jsx(_components.code,{children:`recommendations`})}),_jsx(_components.td,{children:`array`}),_jsxs(_components.td,{children:[_jsx(_components.code,{children:`[{ category, priority, recommendation, reasoning }]`}),` - `,_jsx(_components.code,{children:`category`}),` is one of `,_jsx(_components.code,{children:`\"instructions\"`}),`, `,_jsx(_components.code,{children:`\"tools\"`}),`, `,_jsx(_components.code,{children:`\"knowledge\"`}),`, `,_jsx(_components.code,{children:`\"reasoning\"`}),`, `,_jsx(_components.code,{children:`\"consistency\"`}),`, `,_jsx(_components.code,{children:`\"other\"`}),`; `,_jsx(_components.code,{children:`priority`}),` and every `,_jsx(_components.code,{children:`rating`}),` above are `,_jsx(_components.code,{children:`\"high\"`}),` | `,_jsx(_components.code,{children:`\"medium\"`}),` | `,_jsx(_components.code,{children:`\"low\"`})]})]}),_jsxs(_components.tr,{children:[_jsx(_components.td,{children:_jsx(_components.code,{children:`instructionChanges`})}),_jsx(_components.td,{children:`array`}),_jsxs(_components.td,{children:[`Always `,_jsx(_components.code,{children:`[]`}),` for external agents (the engine doesn’t own the agent’s instructions)`]})]}),_jsxs(_components.tr,{children:[_jsx(_components.td,{children:_jsx(_components.code,{children:`overallAssessment`})}),_jsx(_components.td,{children:`object`}),_jsx(_components.td,{children:_jsx(_components.code,{children:`{ strengths[], weaknesses[], rating }`})})]})]})]}),`\n`,_jsx(_components.p,{children:`Run fields (always present):`}),`\n`,_jsx(ResponseField,{name:`runId`,type:`string`,id:`param-run-id-1`,children:_jsx(_components.p,{children:`Run ID.`})}),`\n`,_jsx(ResponseField,{name:`datasetId`,type:`string`,id:`param-dataset-id`,children:_jsx(_components.p,{children:`Dataset the run scored against.`})}),`\n`,_jsx(ResponseField,{name:`status`,type:`string`,id:`param-status`,children:_jsxs(_components.p,{children:[`The stored analysis status: `,_jsx(_components.code,{children:`\"completed\"`}),` or `,_jsx(_components.code,{children:`\"failed\"`}),`.`]})}),`\n`,_jsx(ResponseField,{name:`statistics`,type:`object | null`,id:`param-statistics`,children:_jsxs(_components.p,{children:[`Computed when the analysis ran:\n`,_jsx(_components.code,{children:`{ numberOfRuns, averageRating, minRating, maxRating, ratingVariance }`}),`, computed over rated\nrows with smoke-test variants excluded - the same population as\n`,_jsx(_components.a,{href:`/api-reference/custom-eval/get-run`,children:`Get Run`}),`’s `,_jsx(_components.code,{children:`averageRating`}),`, so the two agree.`]})}),`\n`,_jsx(ResponseField,{name:`lowScoringCases`,type:`array`,id:`param-low-scoring-cases`,children:_jsxs(_components.p,{children:[`Results with a rating of 5 or below (smoke-test variants excluded), derived fresh from the\nrun’s own rows: `,_jsx(_components.code,{children:`[{ query, response, rating, justification }]`}),`.`]})}),`\n`,_jsx(Note,{children:_jsxs(_components.p,{children:[`Similarity aggregates (cosine/Jaccard/BLEU/ROUGE) are per-result, not report-level - read\nthem from `,_jsx(_components.a,{href:`/api-reference/custom-eval/get-run`,children:`Get Run`}),`’s `,_jsx(_components.code,{children:`results`}),` rows. `,_jsx(_components.code,{children:`strengths`}),`,\n`,_jsx(_components.code,{children:`weaknesses`}),`, and the overall rating live under `,_jsx(_components.code,{children:`overallAssessment`}),`, not at the top level.`]})}),`\n`,_jsx(Heading,{level:`2`,id:`errors`,children:`Errors`}),`\n`,_jsxs(Table,{children:[_jsx(_components.thead,{children:_jsxs(_components.tr,{children:[_jsx(_components.th,{children:`Status`}),_jsx(_components.th,{children:`Body`}),_jsx(_components.th,{children:`Meaning`})]})}),_jsxs(_components.tbody,{children:[_jsxs(_components.tr,{children:[_jsx(_components.td,{\"data-numeric\":`true`,children:_jsx(_components.code,{children:`404`})}),_jsx(_components.td,{children:_jsx(_components.code,{children:`{ \"error\": \"Run not found\" }`})}),_jsx(_components.td,{children:`No run with this id in the project`})]}),_jsxs(_components.tr,{children:[_jsx(_components.td,{\"data-numeric\":`true`,children:_jsx(_components.code,{children:`404`})}),_jsx(_components.td,{children:_jsx(_components.code,{children:`{ \"error\": \"No analysis found for this run. POST /runs/:runId/analyze first.\" }`})}),_jsx(_components.td,{children:`The run exists but was never analyzed`})]})]})]}),`\n`,_jsxs(RequestExample,{children:[_jsx(CodeBlock,{filename:`cURL`,numberOfLines:`2`,language:`shellscript`,children:_jsx(_components.pre,{className:`shiki shiki-themes github-light-default dark-plus`,style:{backgroundColor:`#ffffff`,\"--shiki-dark-bg\":`#0B0C0E`,color:`#1f2328`,\"--shiki-dark\":`#D4D4D4`},language:`shellscript`,children:_jsxs(_components.code,{language:`shellscript`,numberOfLines:`2`,children:[_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#953800`,\"--shiki-dark\":`#DCDCAA`},children:`curl`}),_jsx(_components.span,{style:{color:`#0A3069`,\"--shiki-dark\":`#CE9178`},children:` \"http://localhost:4700/api/v1/custom-agent-evaluations/runs/rK7dP2qWx9TzB4mV6nJcE/report\"`}),_jsx(_components.span,{style:{color:`#CF222E`,\"--shiki-dark\":`#D7BA7D`},children:` \\\\`})]}),`\n`,_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#0550AE`,\"--shiki-dark\":`#569CD6`},children:` -H`}),_jsx(_components.span,{style:{color:`#0A3069`,\"--shiki-dark\":`#CE9178`},children:` \"x-api-key: agtx_local_0f3c9a17d2b84e6a5c01b9f4e7d8a2c6431b5f97a0e2d4c8\"`})]}),`\n`]})})}),_jsx(CodeBlock,{filename:`Python SDK`,numberOfLines:`4`,language:`python`,children:_jsx(_components.pre,{className:`shiki shiki-themes github-light-default dark-plus`,style:{backgroundColor:`#ffffff`,\"--shiki-dark-bg\":`#0B0C0E`,color:`#1f2328`,\"--shiki-dark\":`#D4D4D4`},language:`python`,children:_jsxs(_components.code,{language:`python`,numberOfLines:`4`,children:[_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:`report `}),_jsx(_components.span,{style:{color:`#CF222E`,\"--shiki-dark\":`#D4D4D4`},children:`=`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:` client.evaluations.get_report(`}),_jsx(_components.span,{style:{color:`#0A3069`,\"--shiki-dark\":`#CE9178`},children:`\"rK7dP2qWx9TzB4mV6nJcE\"`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:`)`})]}),`\n`,_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#0550AE`,\"--shiki-dark\":`#DCDCAA`},children:`print`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:`(report.summary)`})]}),`\n`,_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#CF222E`,\"--shiki-dark\":`#C586C0`},children:`for`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:` case `}),_jsx(_components.span,{style:{color:`#CF222E`,\"--shiki-dark\":`#C586C0`},children:`in`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:` report.low_scoring_cases:`})]}),`\n`,_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#0550AE`,\"--shiki-dark\":`#DCDCAA`},children:` print`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:`(case)`})]}),`\n`]})})})]}),`\n`,_jsxs(ResponseExample,{children:[_jsx(CodeBlock,{filename:`200 OK`,numberOfLines:`60`,language:`json`,children:_jsx(_components.pre,{className:`shiki shiki-themes github-light-default dark-plus`,style:{backgroundColor:`#ffffff`,\"--shiki-dark-bg\":`#0B0C0E`,color:`#1f2328`,\"--shiki-dark\":`#D4D4D4`},language:`json`,children:_jsxs(_components.code,{language:`json`,numberOfLines:`60`,children:[_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:`{`})}),`\n`,_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#116329`,\"--shiki-dark\":`#9CDCFE`},children:` \"summary\"`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:`: `}),_jsx(_components.span,{style:{color:`#0A3069`,\"--shiki-dark\":`#CE9178`},children:`\"The agent performs well on direct how-to questions but struggles with multi-step billing queries.\"`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:`,`})]}),`\n`,_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#116329`,\"--shiki-dark\":`#9CDCFE`},children:` \"consistencyScore\"`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:`: `}),_jsx(_components.span,{style:{color:`#0550AE`,\"--shiki-dark\":`#B5CEA8`},children:`8`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:`,`})]}),`\n`,_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#116329`,\"--shiki-dark\":`#9CDCFE`},children:` \"instructionAdherence\"`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:`: {`})]}),`\n`,_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#116329`,\"--shiki-dark\":`#9CDCFE`},children:` \"score\"`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:`: `}),_jsx(_components.span,{style:{color:`#0550AE`,\"--shiki-dark\":`#B5CEA8`},children:`9`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:`,`})]}),`\n`,_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#116329`,\"--shiki-dark\":`#9CDCFE`},children:` \"analysis\"`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:`: `}),_jsx(_components.span,{style:{color:`#0A3069`,\"--shiki-dark\":`#CE9178`},children:`\"The agent generally follows the support tone guidelines.\"`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:`,`})]}),`\n`,_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#116329`,\"--shiki-dark\":`#9CDCFE`},children:` \"deviations\"`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:`: [`}),_jsx(_components.span,{style:{color:`#0A3069`,\"--shiki-dark\":`#CE9178`},children:`\"Q3 run 2: agent gave a refund without checking eligibility\"`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:`],`})]}),`\n`,_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#116329`,\"--shiki-dark\":`#9CDCFE`},children:` \"rating\"`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:`: `}),_jsx(_components.span,{style:{color:`#0A3069`,\"--shiki-dark\":`#CE9178`},children:`\"high\"`})]}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:` },`})}),`\n`,_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#116329`,\"--shiki-dark\":`#9CDCFE`},children:` \"responsePatterns\"`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:`: {`})]}),`\n`,_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#116329`,\"--shiki-dark\":`#9CDCFE`},children:` \"similarities\"`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:`: [`}),_jsx(_components.span,{style:{color:`#0A3069`,\"--shiki-dark\":`#CE9178`},children:`\"Consistent greeting and closing\"`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:`],`})]}),`\n`,_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#116329`,\"--shiki-dark\":`#9CDCFE`},children:` \"differences\"`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:`: [`}),_jsx(_components.span,{style:{color:`#0A3069`,\"--shiki-dark\":`#CE9178`},children:`\"Detail level varies on billing answers\"`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:`],`})]}),`\n`,_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#116329`,\"--shiki-dark\":`#9CDCFE`},children:` \"outliers\"`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:`: [`}),_jsx(_components.span,{style:{color:`#0A3069`,\"--shiki-dark\":`#CE9178`},children:`\"Q3 run 2\"`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:`],`})]}),`\n`,_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#116329`,\"--shiki-dark\":`#9CDCFE`},children:` \"rating\"`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:`: `}),_jsx(_components.span,{style:{color:`#0A3069`,\"--shiki-dark\":`#CE9178`},children:`\"medium\"`})]}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:` },`})}),`\n`,_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#116329`,\"--shiki-dark\":`#9CDCFE`},children:` \"reasoningAnalysis\"`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:`: {`})]}),`\n`,_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#116329`,\"--shiki-dark\":`#9CDCFE`},children:` \"cotQuality\"`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:`: `}),_jsx(_components.span,{style:{color:`#0A3069`,\"--shiki-dark\":`#CE9178`},children:`\"Reasoning is visible and mostly sound.\"`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:`,`})]}),`\n`,_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#116329`,\"--shiki-dark\":`#9CDCFE`},children:` \"reasoningPatterns\"`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:`: [`}),_jsx(_components.span,{style:{color:`#0A3069`,\"--shiki-dark\":`#CE9178`},children:`\"Looks up KB before answering\"`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:`],`})]}),`\n`,_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#116329`,\"--shiki-dark\":`#9CDCFE`},children:` \"reasoningGaps\"`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:`: [`}),_jsx(_components.span,{style:{color:`#0A3069`,\"--shiki-dark\":`#CE9178`},children:`\"Skips eligibility checks under time pressure\"`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:`],`})]}),`\n`,_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#116329`,\"--shiki-dark\":`#9CDCFE`},children:` \"rating\"`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:`: `}),_jsx(_components.span,{style:{color:`#0A3069`,\"--shiki-dark\":`#CE9178`},children:`\"medium\"`})]}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:` },`})}),`\n`,_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#116329`,\"--shiki-dark\":`#9CDCFE`},children:` \"toolUsageAnalysis\"`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:`: {`})]}),`\n`,_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#116329`,\"--shiki-dark\":`#9CDCFE`},children:` \"effectiveness\"`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:`: `}),_jsx(_components.span,{style:{color:`#0A3069`,\"--shiki-dark\":`#CE9178`},children:`\"KB search used appropriately on most cases.\"`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:`,`})]}),`\n`,_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#116329`,\"--shiki-dark\":`#9CDCFE`},children:` \"patterns\"`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:`: [`}),_jsx(_components.span,{style:{color:`#0A3069`,\"--shiki-dark\":`#CE9178`},children:`\"search_kb before every answer\"`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:`],`})]}),`\n`,_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#116329`,\"--shiki-dark\":`#9CDCFE`},children:` \"issues\"`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:`: [`}),_jsx(_components.span,{style:{color:`#0A3069`,\"--shiki-dark\":`#CE9178`},children:`\"No billing tool call on refund cases\"`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:`],`})]}),`\n`,_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#116329`,\"--shiki-dark\":`#9CDCFE`},children:` \"rating\"`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:`: `}),_jsx(_components.span,{style:{color:`#0A3069`,\"--shiki-dark\":`#CE9178`},children:`\"medium\"`})]}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:` },`})}),`\n`,_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#116329`,\"--shiki-dark\":`#9CDCFE`},children:` \"recommendations\"`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:`: [`})]}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:` {`})}),`\n`,_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#116329`,\"--shiki-dark\":`#9CDCFE`},children:` \"category\"`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:`: `}),_jsx(_components.span,{style:{color:`#0A3069`,\"--shiki-dark\":`#CE9178`},children:`\"instructions\"`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:`,`})]}),`\n`,_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#116329`,\"--shiki-dark\":`#9CDCFE`},children:` \"priority\"`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:`: `}),_jsx(_components.span,{style:{color:`#0A3069`,\"--shiki-dark\":`#CE9178`},children:`\"high\"`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:`,`})]}),`\n`,_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#116329`,\"--shiki-dark\":`#9CDCFE`},children:` \"recommendation\"`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:`: `}),_jsx(_components.span,{style:{color:`#0A3069`,\"--shiki-dark\":`#CE9178`},children:`\"Add a guardrail that requires an eligibility check before issuing refunds\"`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:`,`})]}),`\n`,_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#116329`,\"--shiki-dark\":`#9CDCFE`},children:` \"reasoning\"`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:`: `}),_jsx(_components.span,{style:{color:`#0A3069`,\"--shiki-dark\":`#CE9178`},children:`\"One run issued a refund without confirming the customer's eligibility window\"`})]}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:` }`})}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:` ],`})}),`\n`,_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#116329`,\"--shiki-dark\":`#9CDCFE`},children:` \"instructionChanges\"`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:`: [],`})]}),`\n`,_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#116329`,\"--shiki-dark\":`#9CDCFE`},children:` \"overallAssessment\"`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:`: {`})]}),`\n`,_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#116329`,\"--shiki-dark\":`#9CDCFE`},children:` \"strengths\"`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:`: [`}),_jsx(_components.span,{style:{color:`#0A3069`,\"--shiki-dark\":`#CE9178`},children:`\"Consistent tone\"`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:`, `}),_jsx(_components.span,{style:{color:`#0A3069`,\"--shiki-dark\":`#CE9178`},children:`\"Accurate password reset guidance\"`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:`],`})]}),`\n`,_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#116329`,\"--shiki-dark\":`#9CDCFE`},children:` \"weaknesses\"`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:`: [`}),_jsx(_components.span,{style:{color:`#0A3069`,\"--shiki-dark\":`#CE9178`},children:`\"Billing edge cases\"`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:`, `}),_jsx(_components.span,{style:{color:`#0A3069`,\"--shiki-dark\":`#CE9178`},children:`\"Multi-step reasoning\"`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:`],`})]}),`\n`,_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#116329`,\"--shiki-dark\":`#9CDCFE`},children:` \"rating\"`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:`: `}),_jsx(_components.span,{style:{color:`#0A3069`,\"--shiki-dark\":`#CE9178`},children:`\"medium\"`})]}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:` },`})}),`\n`,_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#116329`,\"--shiki-dark\":`#9CDCFE`},children:` \"runId\"`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:`: `}),_jsx(_components.span,{style:{color:`#0A3069`,\"--shiki-dark\":`#CE9178`},children:`\"rK7dP2qWx9TzB4mV6nJcE\"`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:`,`})]}),`\n`,_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#116329`,\"--shiki-dark\":`#9CDCFE`},children:` \"datasetId\"`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:`: `}),_jsx(_components.span,{style:{color:`#0A3069`,\"--shiki-dark\":`#CE9178`},children:`\"dS4tG7hNb2VxZ8kQ5wMyA\"`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:`,`})]}),`\n`,_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#116329`,\"--shiki-dark\":`#9CDCFE`},children:` \"status\"`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:`: `}),_jsx(_components.span,{style:{color:`#0A3069`,\"--shiki-dark\":`#CE9178`},children:`\"completed\"`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:`,`})]}),`\n`,_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#116329`,\"--shiki-dark\":`#9CDCFE`},children:` \"statistics\"`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:`: {`})]}),`\n`,_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#116329`,\"--shiki-dark\":`#9CDCFE`},children:` \"numberOfRuns\"`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:`: `}),_jsx(_components.span,{style:{color:`#0550AE`,\"--shiki-dark\":`#B5CEA8`},children:`6`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:`,`})]}),`\n`,_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#116329`,\"--shiki-dark\":`#9CDCFE`},children:` \"averageRating\"`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:`: `}),_jsx(_components.span,{style:{color:`#0550AE`,\"--shiki-dark\":`#B5CEA8`},children:`7.8`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:`,`})]}),`\n`,_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#116329`,\"--shiki-dark\":`#9CDCFE`},children:` \"minRating\"`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:`: `}),_jsx(_components.span,{style:{color:`#0550AE`,\"--shiki-dark\":`#B5CEA8`},children:`5`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:`,`})]}),`\n`,_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#116329`,\"--shiki-dark\":`#9CDCFE`},children:` \"maxRating\"`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:`: `}),_jsx(_components.span,{style:{color:`#0550AE`,\"--shiki-dark\":`#B5CEA8`},children:`10`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:`,`})]}),`\n`,_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#116329`,\"--shiki-dark\":`#9CDCFE`},children:` \"ratingVariance\"`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:`: `}),_jsx(_components.span,{style:{color:`#0550AE`,\"--shiki-dark\":`#B5CEA8`},children:`1.96`})]}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:` },`})}),`\n`,_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#116329`,\"--shiki-dark\":`#9CDCFE`},children:` \"lowScoringCases\"`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:`: [`})]}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:` {`})}),`\n`,_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#116329`,\"--shiki-dark\":`#9CDCFE`},children:` \"query\"`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:`: `}),_jsx(_components.span,{style:{color:`#0A3069`,\"--shiki-dark\":`#CE9178`},children:`\"Why was I charged twice?\"`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:`,`})]}),`\n`,_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#116329`,\"--shiki-dark\":`#9CDCFE`},children:` \"response\"`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:`: `}),_jsx(_components.span,{style:{color:`#0A3069`,\"--shiki-dark\":`#CE9178`},children:`\"You may have been charged twice due to a pending authorization.\"`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:`,`})]}),`\n`,_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#116329`,\"--shiki-dark\":`#9CDCFE`},children:` \"rating\"`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:`: `}),_jsx(_components.span,{style:{color:`#0550AE`,\"--shiki-dark\":`#B5CEA8`},children:`5`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:`,`})]}),`\n`,_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#116329`,\"--shiki-dark\":`#9CDCFE`},children:` \"justification\"`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:`: `}),_jsx(_components.span,{style:{color:`#0A3069`,\"--shiki-dark\":`#CE9178`},children:`\"Agent did not escalate to the billing team as required.\"`})]}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:` }`})}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:` ]`})}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:`}`})}),`\n`]})})}),_jsx(CodeBlock,{filename:`404 No analysis yet`,numberOfLines:`3`,language:`json`,children:_jsx(_components.pre,{className:`shiki shiki-themes github-light-default dark-plus`,style:{backgroundColor:`#ffffff`,\"--shiki-dark-bg\":`#0B0C0E`,color:`#1f2328`,\"--shiki-dark\":`#D4D4D4`},language:`json`,children:_jsxs(_components.code,{language:`json`,numberOfLines:`3`,children:[_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:`{`})}),`\n`,_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#116329`,\"--shiki-dark\":`#9CDCFE`},children:` \"error\"`}),_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:`: `}),_jsx(_components.span,{style:{color:`#0A3069`,\"--shiki-dark\":`#CE9178`},children:`\"No analysis found for this run. POST /runs/:runId/analyze first.\"`})]}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{style:{color:`#1F2328`,\"--shiki-dark\":`#D4D4D4`},children:`}`})}),`\n`]})})})]})]})}function MDXContent(props={}){const{wrapper:MDXLayout}={..._provideComponents(),...props.components};return MDXLayout?_jsx(MDXLayout,{...props,children:_jsx(_createMdxContent,{...props})}):_createMdxContent(props)}return{default:MDXContent};function _missingMdxReference(id,component){throw new Error(`Expected `+(component?`component`:`object`)+\" `\"+id+\"` to be defined: you likely forgot to import, pass, or provide it.\")}","frontmatter":{},"scope":{"config":{},"pageMetadata":{"title":"Get Report","api":"GET /api/v1/custom-agent-evaluations/runs/{runId}/report","description":"Get the stored analysis report with statistics and low-scoring cases","href":"/api-reference/custom-eval/get-report"}}}},"mdxExtracts":{"tableOfContents":[{"title":"Authentication","slug":"authentication","depth":2,"children":[]},{"title":"Path Parameters","slug":"path-parameters","depth":2,"children":[]},{"title":"Response","slug":"response","depth":2,"children":[]},{"title":"Errors","slug":"errors","depth":2,"children":[]}],"codeExamples":{"request":{"type":"RequestExample","children":[{"filename":"cURL","html":"\u003cpre class=\"shiki shiki-themes github-light-default dark-plus\" style=\"background-color:#ffffff;--shiki-dark-bg:#0B0C0E;color:#1f2328;--shiki-dark:#D4D4D4\" language=\"shellscript\">\u003ccode language=\"shellscript\" numberOfLines=\"2\">\u003cspan class=\"line\">\u003cspan style=\"color:#953800;--shiki-dark:#DCDCAA\">curl\u003c/span>\u003cspan style=\"color:#0A3069;--shiki-dark:#CE9178\"> \"http://localhost:4700/api/v1/custom-agent-evaluations/runs/rK7dP2qWx9TzB4mV6nJcE/report\"\u003c/span>\u003cspan style=\"color:#CF222E;--shiki-dark:#D7BA7D\"> \\\u003c/span>\u003c/span>\n\u003cspan class=\"line\">\u003cspan style=\"color:#0550AE;--shiki-dark:#569CD6\"> -H\u003c/span>\u003cspan style=\"color:#0A3069;--shiki-dark:#CE9178\"> \"x-api-key: agtx_local_0f3c9a17d2b84e6a5c01b9f4e7d8a2c6431b5f97a0e2d4c8\"\u003c/span>\u003c/span>\n\u003c/code>\u003c/pre>"},{"filename":"Python SDK","html":"\u003cpre class=\"shiki shiki-themes github-light-default dark-plus\" style=\"background-color:#ffffff;--shiki-dark-bg:#0B0C0E;color:#1f2328;--shiki-dark:#D4D4D4\" language=\"python\">\u003ccode language=\"python\" numberOfLines=\"4\">\u003cspan class=\"line\">\u003cspan style=\"color:#1F2328;--shiki-dark:#D4D4D4\">report \u003c/span>\u003cspan style=\"color:#CF222E;--shiki-dark:#D4D4D4\">=\u003c/span>\u003cspan style=\"color:#1F2328;--shiki-dark:#D4D4D4\"> client.evaluations.get_report(\u003c/span>\u003cspan style=\"color:#0A3069;--shiki-dark:#CE9178\">\"rK7dP2qWx9TzB4mV6nJcE\"\u003c/span>\u003cspan style=\"color:#1F2328;--shiki-dark:#D4D4D4\">)\u003c/span>\u003c/span>\n\u003cspan class=\"line\">\u003cspan style=\"color:#0550AE;--shiki-dark:#DCDCAA\">print\u003c/span>\u003cspan style=\"color:#1F2328;--shiki-dark:#D4D4D4\">(report.summary)\u003c/span>\u003c/span>\n\u003cspan class=\"line\">\u003cspan style=\"color:#CF222E;--shiki-dark:#C586C0\">for\u003c/span>\u003cspan style=\"color:#1F2328;--shiki-dark:#D4D4D4\"> case \u003c/span>\u003cspan style=\"color:#CF222E;--shiki-dark:#C586C0\">in\u003c/span>\u003cspan style=\"color:#1F2328;--shiki-dark:#D4D4D4\"> report.low_scoring_cases:\u003c/span>\u003c/span>\n\u003cspan class=\"line\">\u003cspan style=\"color:#0550AE;--shiki-dark:#DCDCAA\"> print\u003c/span>\u003cspan style=\"color:#1F2328;--shiki-dark:#D4D4D4\">(case)\u003c/span>\u003c/span>\n\u003c/code>\u003c/pre>"}],"attributes":[]},"response":{"type":"ResponseExample","children":[{"filename":"200 OK","html":"\u003cpre class=\"shiki shiki-themes github-light-default dark-plus\" style=\"background-color:#ffffff;--shiki-dark-bg:#0B0C0E;color:#1f2328;--shiki-dark:#D4D4D4\" language=\"json\">\u003ccode language=\"json\" numberOfLines=\"60\">\u003cspan class=\"line\">\u003cspan style=\"color:#1F2328;--shiki-dark:#D4D4D4\">{\u003c/span>\u003c/span>\n\u003cspan class=\"line\">\u003cspan style=\"color:#116329;--shiki-dark:#9CDCFE\"> \"summary\"\u003c/span>\u003cspan style=\"color:#1F2328;--shiki-dark:#D4D4D4\">: \u003c/span>\u003cspan style=\"color:#0A3069;--shiki-dark:#CE9178\">\"The agent performs well on direct how-to questions but struggles with multi-step billing queries.\"\u003c/span>\u003cspan style=\"color:#1F2328;--shiki-dark:#D4D4D4\">,\u003c/span>\u003c/span>\n\u003cspan class=\"line\">\u003cspan style=\"color:#116329;--shiki-dark:#9CDCFE\"> \"consistencyScore\"\u003c/span>\u003cspan style=\"color:#1F2328;--shiki-dark:#D4D4D4\">: \u003c/span>\u003cspan style=\"color:#0550AE;--shiki-dark:#B5CEA8\">8\u003c/span>\u003cspan style=\"color:#1F2328;--shiki-dark:#D4D4D4\">,\u003c/span>\u003c/span>\n\u003cspan class=\"line\">\u003cspan style=\"color:#116329;--shiki-dark:#9CDCFE\"> \"instructionAdherence\"\u003c/span>\u003cspan style=\"color:#1F2328;--shiki-dark:#D4D4D4\">: {\u003c/span>\u003c/span>\n\u003cspan class=\"line\">\u003cspan style=\"color:#116329;--shiki-dark:#9CDCFE\"> \"score\"\u003c/span>\u003cspan style=\"color:#1F2328;--shiki-dark:#D4D4D4\">: \u003c/span>\u003cspan style=\"color:#0550AE;--shiki-dark:#B5CEA8\">9\u003c/span>\u003cspan style=\"color:#1F2328;--shiki-dark:#D4D4D4\">,\u003c/span>\u003c/span>\n\u003cspan class=\"line\">\u003cspan style=\"color:#116329;--shiki-dark:#9CDCFE\"> \"analysis\"\u003c/span>\u003cspan style=\"color:#1F2328;--shiki-dark:#D4D4D4\">: \u003c/span>\u003cspan style=\"color:#0A3069;--shiki-dark:#CE9178\">\"The agent generally follows the support tone guidelines.\"\u003c/span>\u003cspan style=\"color:#1F2328;--shiki-dark:#D4D4D4\">,\u003c/span>\u003c/span>\n\u003cspan class=\"line\">\u003cspan style=\"color:#116329;--shiki-dark:#9CDCFE\"> \"deviations\"\u003c/span>\u003cspan style=\"color:#1F2328;--shiki-dark:#D4D4D4\">: [\u003c/span>\u003cspan style=\"color:#0A3069;--shiki-dark:#CE9178\">\"Q3 run 2: agent gave a refund without checking eligibility\"\u003c/span>\u003cspan style=\"color:#1F2328;--shiki-dark:#D4D4D4\">],\u003c/span>\u003c/span>\n\u003cspan class=\"line\">\u003cspan style=\"color:#116329;--shiki-dark:#9CDCFE\"> \"rating\"\u003c/span>\u003cspan style=\"color:#1F2328;--shiki-dark:#D4D4D4\">: \u003c/span>\u003cspan style=\"color:#0A3069;--shiki-dark:#CE9178\">\"high\"\u003c/span>\u003c/span>\n\u003cspan class=\"line\">\u003cspan style=\"color:#1F2328;--shiki-dark:#D4D4D4\"> },\u003c/span>\u003c/span>\n\u003cspan class=\"line\">\u003cspan style=\"color:#116329;--shiki-dark:#9CDCFE\"> \"responsePatterns\"\u003c/span>\u003cspan style=\"color:#1F2328;--shiki-dark:#D4D4D4\">: {\u003c/span>\u003c/span>\n\u003cspan class=\"line\">\u003cspan style=\"color:#116329;--shiki-dark:#9CDCFE\"> \"similarities\"\u003c/span>\u003cspan style=\"color:#1F2328;--shiki-dark:#D4D4D4\">: [\u003c/span>\u003cspan style=\"color:#0A3069;--shiki-dark:#CE9178\">\"Consistent greeting and closing\"\u003c/span>\u003cspan style=\"color:#1F2328;--shiki-dark:#D4D4D4\">],\u003c/span>\u003c/span>\n\u003cspan class=\"line\">\u003cspan style=\"color:#116329;--shiki-dark:#9CDCFE\"> \"differences\"\u003c/span>\u003cspan style=\"color:#1F2328;--shiki-dark:#D4D4D4\">: [\u003c/span>\u003cspan style=\"color:#0A3069;--shiki-dark:#CE9178\">\"Detail level varies on billing answers\"\u003c/span>\u003cspan style=\"color:#1F2328;--shiki-dark:#D4D4D4\">],\u003c/span>\u003c/span>\n\u003cspan class=\"line\">\u003cspan style=\"color:#116329;--shiki-dark:#9CDCFE\"> \"outliers\"\u003c/span>\u003cspan style=\"color:#1F2328;--shiki-dark:#D4D4D4\">: [\u003c/span>\u003cspan style=\"color:#0A3069;--shiki-dark:#CE9178\">\"Q3 run 2\"\u003c/span>\u003cspan style=\"color:#1F2328;--shiki-dark:#D4D4D4\">],\u003c/span>\u003c/span>\n\u003cspan class=\"line\">\u003cspan style=\"color:#116329;--shiki-dark:#9CDCFE\"> \"rating\"\u003c/span>\u003cspan style=\"color:#1F2328;--shiki-dark:#D4D4D4\">: \u003c/span>\u003cspan style=\"color:#0A3069;--shiki-dark:#CE9178\">\"medium\"\u003c/span>\u003c/span>\n\u003cspan class=\"line\">\u003cspan style=\"color:#1F2328;--shiki-dark:#D4D4D4\"> },\u003c/span>\u003c/span>\n\u003cspan class=\"line\">\u003cspan style=\"color:#116329;--shiki-dark:#9CDCFE\"> \"reasoningAnalysis\"\u003c/span>\u003cspan style=\"color:#1F2328;--shiki-dark:#D4D4D4\">: {\u003c/span>\u003c/span>\n\u003cspan class=\"line\">\u003cspan style=\"color:#116329;--shiki-dark:#9CDCFE\"> \"cotQuality\"\u003c/span>\u003cspan style=\"color:#1F2328;--shiki-dark:#D4D4D4\">: \u003c/span>\u003cspan style=\"color:#0A3069;--shiki-dark:#CE9178\">\"Reasoning is visible and mostly sound.\"\u003c/span>\u003cspan style=\"color:#1F2328;--shiki-dark:#D4D4D4\">,\u003c/span>\u003c/span>\n\u003cspan class=\"line\">\u003cspan style=\"color:#116329;--shiki-dark:#9CDCFE\"> \"reasoningPatterns\"\u003c/span>\u003cspan style=\"color:#1F2328;--shiki-dark:#D4D4D4\">: [\u003c/span>\u003cspan style=\"color:#0A3069;--shiki-dark:#CE9178\">\"Looks up KB before answering\"\u003c/span>\u003cspan style=\"color:#1F2328;--shiki-dark:#D4D4D4\">],\u003c/span>\u003c/span>\n\u003cspan class=\"line\">\u003cspan style=\"color:#116329;--shiki-dark:#9CDCFE\"> \"reasoningGaps\"\u003c/span>\u003cspan style=\"color:#1F2328;--shiki-dark:#D4D4D4\">: [\u003c/span>\u003cspan style=\"color:#0A3069;--shiki-dark:#CE9178\">\"Skips eligibility checks under time pressure\"\u003c/span>\u003cspan style=\"color:#1F2328;--shiki-dark:#D4D4D4\">],\u003c/span>\u003c/span>\n\u003cspan class=\"line\">\u003cspan style=\"color:#116329;--shiki-dark:#9CDCFE\"> \"rating\"\u003c/span>\u003cspan style=\"color:#1F2328;--shiki-dark:#D4D4D4\">: \u003c/span>\u003cspan style=\"color:#0A3069;--shiki-dark:#CE9178\">\"medium\"\u003c/span>\u003c/span>\n\u003cspan class=\"line\">\u003cspan style=\"color:#1F2328;--shiki-dark:#D4D4D4\"> },\u003c/span>\u003c/span>\n\u003cspan class=\"line\">\u003cspan style=\"color:#116329;--shiki-dark:#9CDCFE\"> \"toolUsageAnalysis\"\u003c/span>\u003cspan style=\"color:#1F2328;--shiki-dark:#D4D4D4\">: {\u003c/span>\u003c/span>\n\u003cspan class=\"line\">\u003cspan style=\"color:#116329;--shiki-dark:#9CDCFE\"> \"effectiveness\"\u003c/span>\u003cspan style=\"color:#1F2328;--shiki-dark:#D4D4D4\">: \u003c/span>\u003cspan style=\"color:#0A3069;--shiki-dark:#CE9178\">\"KB search used appropriately on most cases.\"\u003c/span>\u003cspan style=\"color:#1F2328;--shiki-dark:#D4D4D4\">,\u003c/span>\u003c/span>\n\u003cspan class=\"line\">\u003cspan style=\"color:#116329;--shiki-dark:#9CDCFE\"> \"patterns\"\u003c/span>\u003cspan style=\"color:#1F2328;--shiki-dark:#D4D4D4\">: [\u003c/span>\u003cspan style=\"color:#0A3069;--shiki-dark:#CE9178\">\"search_kb before every answer\"\u003c/span>\u003cspan style=\"color:#1F2328;--shiki-dark:#D4D4D4\">],\u003c/span>\u003c/span>\n\u003cspan class=\"line\">\u003cspan style=\"color:#116329;--shiki-dark:#9CDCFE\"> \"issues\"\u003c/span>\u003cspan style=\"color:#1F2328;--shiki-dark:#D4D4D4\">: [\u003c/span>\u003cspan style=\"color:#0A3069;--shiki-dark:#CE9178\">\"No billing tool call on refund cases\"\u003c/span>\u003cspan style=\"color:#1F2328;--shiki-dark:#D4D4D4\">],\u003c/span>\u003c/span>\n\u003cspan class=\"line\">\u003cspan style=\"color:#116329;--shiki-dark:#9CDCFE\"> \"rating\"\u003c/span>\u003cspan style=\"color:#1F2328;--shiki-dark:#D4D4D4\">: \u003c/span>\u003cspan style=\"color:#0A3069;--shiki-dark:#CE9178\">\"medium\"\u003c/span>\u003c/span>\n\u003cspan class=\"line\">\u003cspan style=\"color:#1F2328;--shiki-dark:#D4D4D4\"> },\u003c/span>\u003c/span>\n\u003cspan class=\"line\">\u003cspan style=\"color:#116329;--shiki-dark:#9CDCFE\"> \"recommendations\"\u003c/span>\u003cspan style=\"color:#1F2328;--shiki-dark:#D4D4D4\">: [\u003c/span>\u003c/span>\n\u003cspan class=\"line\">\u003cspan style=\"color:#1F2328;--shiki-dark:#D4D4D4\"> {\u003c/span>\u003c/span>\n\u003cspan class=\"line\">\u003cspan style=\"color:#116329;--shiki-dark:#9CDCFE\"> \"category\"\u003c/span>\u003cspan style=\"color:#1F2328;--shiki-dark:#D4D4D4\">: \u003c/span>\u003cspan style=\"color:#0A3069;--shiki-dark:#CE9178\">\"instructions\"\u003c/span>\u003cspan style=\"color:#1F2328;--shiki-dark:#D4D4D4\">,\u003c/span>\u003c/span>\n\u003cspan class=\"line\">\u003cspan style=\"color:#116329;--shiki-dark:#9CDCFE\"> \"priority\"\u003c/span>\u003cspan style=\"color:#1F2328;--shiki-dark:#D4D4D4\">: \u003c/span>\u003cspan style=\"color:#0A3069;--shiki-dark:#CE9178\">\"high\"\u003c/span>\u003cspan style=\"color:#1F2328;--shiki-dark:#D4D4D4\">,\u003c/span>\u003c/span>\n\u003cspan class=\"line\">\u003cspan style=\"color:#116329;--shiki-dark:#9CDCFE\"> \"recommendation\"\u003c/span>\u003cspan style=\"color:#1F2328;--shiki-dark:#D4D4D4\">: \u003c/span>\u003cspan style=\"color:#0A3069;--shiki-dark:#CE9178\">\"Add a guardrail that requires an eligibility check before issuing refunds\"\u003c/span>\u003cspan style=\"color:#1F2328;--shiki-dark:#D4D4D4\">,\u003c/span>\u003c/span>\n\u003cspan class=\"line\">\u003cspan style=\"color:#116329;--shiki-dark:#9CDCFE\"> \"reasoning\"\u003c/span>\u003cspan style=\"color:#1F2328;--shiki-dark:#D4D4D4\">: \u003c/span>\u003cspan style=\"color:#0A3069;--shiki-dark:#CE9178\">\"One run issued a refund without confirming the customer's eligibility window\"\u003c/span>\u003c/span>\n\u003cspan class=\"line\">\u003cspan style=\"color:#1F2328;--shiki-dark:#D4D4D4\"> }\u003c/span>\u003c/span>\n\u003cspan class=\"line\">\u003cspan style=\"color:#1F2328;--shiki-dark:#D4D4D4\"> ],\u003c/span>\u003c/span>\n\u003cspan class=\"line\">\u003cspan style=\"color:#116329;--shiki-dark:#9CDCFE\"> \"instructionChanges\"\u003c/span>\u003cspan style=\"color:#1F2328;--shiki-dark:#D4D4D4\">: [],\u003c/span>\u003c/span>\n\u003cspan class=\"line\">\u003cspan style=\"color:#116329;--shiki-dark:#9CDCFE\"> \"overallAssessment\"\u003c/span>\u003cspan style=\"color:#1F2328;--shiki-dark:#D4D4D4\">: {\u003c/span>\u003c/span>\n\u003cspan class=\"line\">\u003cspan style=\"color:#116329;--shiki-dark:#9CDCFE\"> \"strengths\"\u003c/span>\u003cspan style=\"color:#1F2328;--shiki-dark:#D4D4D4\">: [\u003c/span>\u003cspan style=\"color:#0A3069;--shiki-dark:#CE9178\">\"Consistent tone\"\u003c/span>\u003cspan style=\"color:#1F2328;--shiki-dark:#D4D4D4\">, \u003c/span>\u003cspan style=\"color:#0A3069;--shiki-dark:#CE9178\">\"Accurate password reset guidance\"\u003c/span>\u003cspan style=\"color:#1F2328;--shiki-dark:#D4D4D4\">],\u003c/span>\u003c/span>\n\u003cspan class=\"line\">\u003cspan style=\"color:#116329;--shiki-dark:#9CDCFE\"> \"weaknesses\"\u003c/span>\u003cspan style=\"color:#1F2328;--shiki-dark:#D4D4D4\">: [\u003c/span>\u003cspan style=\"color:#0A3069;--shiki-dark:#CE9178\">\"Billing edge cases\"\u003c/span>\u003cspan style=\"color:#1F2328;--shiki-dark:#D4D4D4\">, \u003c/span>\u003cspan style=\"color:#0A3069;--shiki-dark:#CE9178\">\"Multi-step reasoning\"\u003c/span>\u003cspan style=\"color:#1F2328;--shiki-dark:#D4D4D4\">],\u003c/span>\u003c/span>\n\u003cspan class=\"line\">\u003cspan style=\"color:#116329;--shiki-dark:#9CDCFE\"> \"rating\"\u003c/span>\u003cspan style=\"color:#1F2328;--shiki-dark:#D4D4D4\">: \u003c/span>\u003cspan style=\"color:#0A3069;--shiki-dark:#CE9178\">\"medium\"\u003c/span>\u003c/span>\n\u003cspan class=\"line\">\u003cspan style=\"color:#1F2328;--shiki-dark:#D4D4D4\"> },\u003c/span>\u003c/span>\n\u003cspan class=\"line\">\u003cspan style=\"color:#116329;--shiki-dark:#9CDCFE\"> \"runId\"\u003c/span>\u003cspan style=\"color:#1F2328;--shiki-dark:#D4D4D4\">: \u003c/span>\u003cspan style=\"color:#0A3069;--shiki-dark:#CE9178\">\"rK7dP2qWx9TzB4mV6nJcE\"\u003c/span>\u003cspan style=\"color:#1F2328;--shiki-dark:#D4D4D4\">,\u003c/span>\u003c/span>\n\u003cspan class=\"line\">\u003cspan style=\"color:#116329;--shiki-dark:#9CDCFE\"> \"datasetId\"\u003c/span>\u003cspan style=\"color:#1F2328;--shiki-dark:#D4D4D4\">: \u003c/span>\u003cspan style=\"color:#0A3069;--shiki-dark:#CE9178\">\"dS4tG7hNb2VxZ8kQ5wMyA\"\u003c/span>\u003cspan style=\"color:#1F2328;--shiki-dark:#D4D4D4\">,\u003c/span>\u003c/span>\n\u003cspan class=\"line\">\u003cspan style=\"color:#116329;--shiki-dark:#9CDCFE\"> \"status\"\u003c/span>\u003cspan style=\"color:#1F2328;--shiki-dark:#D4D4D4\">: \u003c/span>\u003cspan style=\"color:#0A3069;--shiki-dark:#CE9178\">\"completed\"\u003c/span>\u003cspan style=\"color:#1F2328;--shiki-dark:#D4D4D4\">,\u003c/span>\u003c/span>\n\u003cspan class=\"line\">\u003cspan style=\"color:#116329;--shiki-dark:#9CDCFE\"> \"statistics\"\u003c/span>\u003cspan style=\"color:#1F2328;--shiki-dark:#D4D4D4\">: {\u003c/span>\u003c/span>\n\u003cspan class=\"line\">\u003cspan style=\"color:#116329;--shiki-dark:#9CDCFE\"> \"numberOfRuns\"\u003c/span>\u003cspan style=\"color:#1F2328;--shiki-dark:#D4D4D4\">: \u003c/span>\u003cspan style=\"color:#0550AE;--shiki-dark:#B5CEA8\">6\u003c/span>\u003cspan style=\"color:#1F2328;--shiki-dark:#D4D4D4\">,\u003c/span>\u003c/span>\n\u003cspan class=\"line\">\u003cspan style=\"color:#116329;--shiki-dark:#9CDCFE\"> \"averageRating\"\u003c/span>\u003cspan style=\"color:#1F2328;--shiki-dark:#D4D4D4\">: \u003c/span>\u003cspan style=\"color:#0550AE;--shiki-dark:#B5CEA8\">7.8\u003c/span>\u003cspan style=\"color:#1F2328;--shiki-dark:#D4D4D4\">,\u003c/span>\u003c/span>\n\u003cspan class=\"line\">\u003cspan style=\"color:#116329;--shiki-dark:#9CDCFE\"> \"minRating\"\u003c/span>\u003cspan style=\"color:#1F2328;--shiki-dark:#D4D4D4\">: \u003c/span>\u003cspan style=\"color:#0550AE;--shiki-dark:#B5CEA8\">5\u003c/span>\u003cspan style=\"color:#1F2328;--shiki-dark:#D4D4D4\">,\u003c/span>\u003c/span>\n\u003cspan class=\"line\">\u003cspan style=\"color:#116329;--shiki-dark:#9CDCFE\"> \"maxRating\"\u003c/span>\u003cspan style=\"color:#1F2328;--shiki-dark:#D4D4D4\">: \u003c/span>\u003cspan style=\"color:#0550AE;--shiki-dark:#B5CEA8\">10\u003c/span>\u003cspan style=\"color:#1F2328;--shiki-dark:#D4D4D4\">,\u003c/span>\u003c/span>\n\u003cspan class=\"line\">\u003cspan style=\"color:#116329;--shiki-dark:#9CDCFE\"> \"ratingVariance\"\u003c/span>\u003cspan style=\"color:#1F2328;--shiki-dark:#D4D4D4\">: \u003c/span>\u003cspan style=\"color:#0550AE;--shiki-dark:#B5CEA8\">1.96\u003c/span>\u003c/span>\n\u003cspan class=\"line\">\u003cspan style=\"color:#1F2328;--shiki-dark:#D4D4D4\"> },\u003c/span>\u003c/span>\n\u003cspan class=\"line\">\u003cspan style=\"color:#116329;--shiki-dark:#9CDCFE\"> \"lowScoringCases\"\u003c/span>\u003cspan style=\"color:#1F2328;--shiki-dark:#D4D4D4\">: [\u003c/span>\u003c/span>\n\u003cspan class=\"line\">\u003cspan style=\"color:#1F2328;--shiki-dark:#D4D4D4\"> {\u003c/span>\u003c/span>\n\u003cspan class=\"line\">\u003cspan style=\"color:#116329;--shiki-dark:#9CDCFE\"> \"query\"\u003c/span>\u003cspan style=\"color:#1F2328;--shiki-dark:#D4D4D4\">: \u003c/span>\u003cspan style=\"color:#0A3069;--shiki-dark:#CE9178\">\"Why was I charged twice?\"\u003c/span>\u003cspan style=\"color:#1F2328;--shiki-dark:#D4D4D4\">,\u003c/span>\u003c/span>\n\u003cspan class=\"line\">\u003cspan style=\"color:#116329;--shiki-dark:#9CDCFE\"> \"response\"\u003c/span>\u003cspan style=\"color:#1F2328;--shiki-dark:#D4D4D4\">: \u003c/span>\u003cspan style=\"color:#0A3069;--shiki-dark:#CE9178\">\"You may have been charged twice due to a pending authorization.\"\u003c/span>\u003cspan style=\"color:#1F2328;--shiki-dark:#D4D4D4\">,\u003c/span>\u003c/span>\n\u003cspan class=\"line\">\u003cspan style=\"color:#116329;--shiki-dark:#9CDCFE\"> \"rating\"\u003c/span>\u003cspan style=\"color:#1F2328;--shiki-dark:#D4D4D4\">: \u003c/span>\u003cspan style=\"color:#0550AE;--shiki-dark:#B5CEA8\">5\u003c/span>\u003cspan style=\"color:#1F2328;--shiki-dark:#D4D4D4\">,\u003c/span>\u003c/span>\n\u003cspan class=\"line\">\u003cspan style=\"color:#116329;--shiki-dark:#9CDCFE\"> \"justification\"\u003c/span>\u003cspan style=\"color:#1F2328;--shiki-dark:#D4D4D4\">: \u003c/span>\u003cspan style=\"color:#0A3069;--shiki-dark:#CE9178\">\"Agent did not escalate to the billing team as required.\"\u003c/span>\u003c/span>\n\u003cspan class=\"line\">\u003cspan style=\"color:#1F2328;--shiki-dark:#D4D4D4\"> }\u003c/span>\u003c/span>\n\u003cspan class=\"line\">\u003cspan style=\"color:#1F2328;--shiki-dark:#D4D4D4\"> ]\u003c/span>\u003c/span>\n\u003cspan class=\"line\">\u003cspan style=\"color:#1F2328;--shiki-dark:#D4D4D4\">}\u003c/span>\u003c/span>\n\u003c/code>\u003c/pre>"},{"filename":"404 No analysis yet","html":"\u003cpre class=\"shiki shiki-themes github-light-default dark-plus\" style=\"background-color:#ffffff;--shiki-dark-bg:#0B0C0E;color:#1f2328;--shiki-dark:#D4D4D4\" language=\"json\">\u003ccode language=\"json\" numberOfLines=\"3\">\u003cspan class=\"line\">\u003cspan style=\"color:#1F2328;--shiki-dark:#D4D4D4\">{\u003c/span>\u003c/span>\n\u003cspan class=\"line\">\u003cspan style=\"color:#116329;--shiki-dark:#9CDCFE\"> \"error\"\u003c/span>\u003cspan style=\"color:#1F2328;--shiki-dark:#D4D4D4\">: \u003c/span>\u003cspan style=\"color:#0A3069;--shiki-dark:#CE9178\">\"No analysis found for this run. POST /runs/:runId/analyze first.\"\u003c/span>\u003c/span>\n\u003cspan class=\"line\">\u003cspan style=\"color:#1F2328;--shiki-dark:#D4D4D4\">}\u003c/span>\u003c/span>\n\u003c/code>\u003c/pre>"}],"attributes":[]}},"endpoint":{"title":"Get Report","path":"/api/v1/custom-agent-evaluations/runs/{runId}/report","servers":[{"url":"http://localhost:4700"}],"method":"get","request":{"security":[{"title":"API Key Auth","parameters":{"header":{"x-api-key":{"type":"apiKey"}},"query":{},"cookie":{}}}],"parameters":{"query":{},"header":{"x-api-key":{"schema":[{"type":"string","description":"Project API key.","required":true}]}},"cookie":{},"path":{"runId":{"schema":[{"type":"string","description":"Run ID.","required":true}]}}},"body":{}},"response":{"200":{"application/json":{"schemaArray":[{"type":"object","properties":{"runId":[{"type":"string","description":"Run ID."}],"datasetId":[{"type":"string","description":"Dataset the run scored against."}],"status":[{"type":"string","description":"The stored analysis status: `\"completed\"` or `\"failed\"` ."}],"statistics":[{"type":"object","properties":{},"description":"Computed when the analysis ran: `{ numberOfRuns, averageRating, minRating, maxRating, ratingVariance }` , computed over rated rows with smoke-test variants excluded - the same population as Get Run ’s `averageRating` , so the two agree."}],"lowScoringCases":[{"type":"array","items":[{"type":"object","properties":{}}],"description":"Results with a rating of 5 or below (smoke-test variants excluded), derived fresh from the run’s own rows: `[{ query, response, rating, justification }]` ."}]}}],"examples":{"example":{"value":{"runId":"\u003cstring>","datasetId":"\u003cstring>","status":"\u003cstring>","statistics":{},"lowScoringCases":[{}]}}}}}},"deprecated":false}},"jsLinks":[],"subdomain":"agentx-ffe8d995","siteName":"AgentX","title":"Get Report - AgentX","navMaps":{"firstHrefInVersion":{},"firstHrefInLanguage":{}},"lastModified":"2026-09-16T19:20:37.575Z","originalFileLocation":"api-reference/custom-eval/get-report.mdx"},"headHtml":"\u003cscript data-mint-head>(function(){var d=document.documentElement,p=null;if(!p){try{p=localStorage.getItem(\"isDarkMode\")}catch(e){}if(p==='true')p='dark';else if(p==='false')p='light';if(p!=='light'\u0026\u0026p!=='dark'\u0026\u0026p!=='system')p=\"system\"}var r=p==='system'?(matchMedia(\"(prefers-color-scheme: dark)\").matches?'dark':'light'):p;d.classList.remove(r==='dark'?'light':'dark');d.classList.add(r);d.setAttribute(\"data-theme-preference\",p)})();\u003c/script>\u003cmeta data-mint-head name=\"description\" content=\"Get the stored analysis report with statistics and low-scoring cases\">\u003cmeta data-mint-head name=\"application-name\" content=\"AgentX\">\u003cmeta data-mint-head name=\"generator\" content=\"Mintlify\">\u003clink data-mint-head rel=\"icon\" type=\"image/png\" sizes=\"192x192\" href=\"/mintlify-assets/_mintlify/favicons/agentx-ffe8d995/sticQuUfAA1h_xwo/_generated/favicon/android-chrome-192x192.png\" media=\"(prefers-color-scheme: light)\">\u003clink data-mint-head rel=\"apple-touch-icon\" type=\"image/png\" sizes=\"180x180\" href=\"/mintlify-assets/_mintlify/favicons/agentx-ffe8d995/sticQuUfAA1h_xwo/_generated/favicon/apple-touch-icon.png\">\u003clink data-mint-head rel=\"icon\" type=\"image/png\" sizes=\"16x16\" href=\"/mintlify-assets/_mintlify/favicons/agentx-ffe8d995/sticQuUfAA1h_xwo/_generated/favicon/favicon-16x16.png\" media=\"(prefers-color-scheme: light)\">\u003clink data-mint-head rel=\"icon\" type=\"image/png\" sizes=\"32x32\" href=\"/mintlify-assets/_mintlify/favicons/agentx-ffe8d995/sticQuUfAA1h_xwo/_generated/favicon/favicon-32x32.png\" media=\"(prefers-color-scheme: light)\">\u003clink data-mint-head rel=\"shortcut icon\" type=\"image/x-icon\" href=\"/mintlify-assets/_mintlify/favicons/agentx-ffe8d995/sticQuUfAA1h_xwo/_generated/favicon/favicon.ico\" media=\"(prefers-color-scheme: light)\">\u003clink data-mint-head rel=\"icon\" type=\"image/png\" sizes=\"192x192\" href=\"/mintlify-assets/_mintlify/favicons/agentx-ffe8d995/sticQuUfAA1h_xwo/_generated/favicon-dark/android-chrome-192x192.png\" media=\"(prefers-color-scheme: dark)\">\u003clink data-mint-head rel=\"icon\" type=\"image/png\" sizes=\"16x16\" href=\"/mintlify-assets/_mintlify/favicons/agentx-ffe8d995/sticQuUfAA1h_xwo/_generated/favicon-dark/favicon-16x16.png\" media=\"(prefers-color-scheme: dark)\">\u003clink data-mint-head rel=\"icon\" type=\"image/png\" sizes=\"32x32\" href=\"/mintlify-assets/_mintlify/favicons/agentx-ffe8d995/sticQuUfAA1h_xwo/_generated/favicon-dark/favicon-32x32.png\" media=\"(prefers-color-scheme: dark)\">\u003clink data-mint-head rel=\"shortcut icon\" type=\"image/x-icon\" href=\"/mintlify-assets/_mintlify/favicons/agentx-ffe8d995/sticQuUfAA1h_xwo/_generated/favicon-dark/favicon.ico\" media=\"(prefers-color-scheme: dark)\">\u003cmeta data-mint-head property=\"og:title\" content=\"Get Report - AgentX\">\u003cmeta data-mint-head property=\"og:description\" content=\"Get the stored analysis report with statistics and low-scoring cases\">\u003cmeta data-mint-head property=\"og:site_name\" content=\"AgentX\">\u003cmeta data-mint-head property=\"og:url\" content=\"https://developers.agentx.so/api-reference/custom-eval/get-report\">\u003cmeta data-mint-head property=\"og:type\" content=\"website\">\u003cmeta data-mint-head property=\"og:image\" content=\"https://agentx-ffe8d995.mintlify.site/_mintlify/api/og/v2?division=Custom+Evaluations\u0026amp;title=Get+Report\u0026amp;description=Get+the+stored+analysis+report+with+statistics+and+low-scoring+cases\u0026amp;theme=2ebdc38e645cd0f197f78a50\u0026amp;signature=54a9a053085c83f73c77c19db8cacf55748e786274b3081fde5974fe3e288bb2\">\u003cmeta data-mint-head property=\"og:image:width\" content=\"1200\">\u003cmeta data-mint-head property=\"og:image:height\" content=\"630\">\u003cmeta data-mint-head name=\"twitter:card\" content=\"summary_large_image\">\u003cmeta data-mint-head name=\"twitter:title\" content=\"Get Report - AgentX\">\u003cmeta data-mint-head name=\"twitter:description\" content=\"Get the stored analysis report with statistics and low-scoring cases\">\u003cmeta data-mint-head name=\"twitter:image\" content=\"https://agentx-ffe8d995.mintlify.site/_mintlify/api/og/v2?division=Custom+Evaluations\u0026amp;title=Get+Report\u0026amp;description=Get+the+stored+analysis+report+with+statistics+and+low-scoring+cases\u0026amp;theme=2ebdc38e645cd0f197f78a50\u0026amp;signature=54a9a053085c83f73c77c19db8cacf55748e786274b3081fde5974fe3e288bb2\">\u003cmeta data-mint-head name=\"twitter:image:width\" content=\"1200\">\u003cmeta data-mint-head name=\"twitter:image:height\" content=\"630\">\u003cmeta data-mint-head name=\"msapplication-config\" content=\"/mintlify-assets/_mintlify/favicons/agentx-ffe8d995/sticQuUfAA1h_xwo/_generated/favicon/browserconfig.xml\">\u003cmeta data-mint-head name=\"apple-mobile-web-app-title\" content=\"AgentX\">\u003cmeta data-mint-head name=\"msapplication-TileColor\" content=\"#2A5699\">\u003clink data-mint-head rel=\"canonical\" href=\"https://developers.agentx.so/api-reference/custom-eval/get-report\">\u003clink data-mint-head rel=\"alternate\" type=\"application/xml\" href=\"/sitemap.xml\">\u003clink data-mint-head rel=\"alternate\" type=\"text/markdown\" href=\"/api-reference/custom-eval/get-report.md\">\u003cscript data-mint-head type=\"application/ld+json\">{\"@context\":\"https://schema.org\",\"@graph\":[{\"@type\":\"Organization\",\"@id\":\"https://developers.agentx.so/#organization\",\"name\":\"AgentX\",\"url\":\"https://developers.agentx.so\",\"logo\":{\"@type\":\"ImageObject\",\"url\":\"https://mintcdn.com/agentx-ffe8d995/ZTzZ3HDQJ-RJxymw/logo/light.png?fit=max\u0026auto=format\u0026n=ZTzZ3HDQJ-RJxymw\u0026q=85\u0026s=c1e30c949dc1cc2aaa3a137c1ee5c77f\"}},{\"@type\":\"WebSite\",\"@id\":\"https://developers.agentx.so#website\",\"name\":\"AgentX\",\"url\":\"https://developers.agentx.so\",\"publisher\":{\"@id\":\"https://developers.agentx.so/#organization\"}},{\"@type\":\"WebPage\",\"@id\":\"https://developers.agentx.so/api-reference/custom-eval/get-report#webpage\",\"url\":\"https://developers.agentx.so/api-reference/custom-eval/get-report\",\"name\":\"Get Report\",\"description\":\"Get the stored analysis report with statistics and low-scoring cases\",\"dateModified\":\"2026-09-16T19:20:37.575Z\",\"isPartOf\":{\"@id\":\"https://developers.agentx.so#website\"},\"breadcrumb\":{\"@id\":\"https://developers.agentx.so/api-reference/custom-eval/get-report#breadcrumb\"}},{\"@type\":\"BreadcrumbList\",\"@id\":\"https://developers.agentx.so/api-reference/custom-eval/get-report#breadcrumb\",\"itemListElement\":[{\"@type\":\"ListItem\",\"position\":1,\"name\":\"Custom Evaluations\",\"item\":\"https://developers.agentx.so/api-reference/custom-eval/overview\"},{\"@type\":\"ListItem\",\"position\":2,\"name\":\"Get Report\",\"item\":\"https://developers.agentx.so/api-reference/custom-eval/get-report\"}]},{\"@type\":[\"TechArticle\",\"APIReference\"],\"@id\":\"https://developers.agentx.so/api-reference/custom-eval/get-report#apireference\",\"headline\":\"Get Report\",\"name\":\"Get Report\",\"description\":\"Get the stored analysis report with statistics and low-scoring cases\",\"url\":\"https://developers.agentx.so/api-reference/custom-eval/get-report\",\"mainEntityOfPage\":{\"@id\":\"https://developers.agentx.so/api-reference/custom-eval/get-report#webpage\"},\"dateModified\":\"2026-09-16T19:20:37.575Z\",\"publisher\":{\"@id\":\"https://developers.agentx.so/#organization\"},\"isPartOf\":{\"@id\":\"https://developers.agentx.so#website\"}}]}\u003c/script>\u003cscript data-mint-head type=\"application/ld+json\">{\"@context\":\"https://schema.org\",\"@type\":\"WebSite\",\"name\":\"AgentX\",\"publisher\":{\"@type\":\"Organization\",\"name\":\"AgentX\"},\"provider\":{\"@type\":\"Organization\",\"name\":\"Mintlify\",\"url\":\"https://mintlify.com\"}}\u003c/script>","agentPointerHref":"/api-reference/custom-eval/get-report.md","siteName":"AgentX","nav":{"global":null,"tabs":[{"tab":"Documentation","pages":[{"title":"Documentation","href":"/introduction"}]},{"tab":"Trace","pages":[{"title":"Trace","href":"/sdk/tracing"}]},{"tab":"Scorers \u0026 Monitoring","pages":[{"title":"Scorers \u0026 Monitoring","href":"/monitor/scorers"}]},{"tab":"Offline Evaluation","pages":[{"title":"Offline Evaluation","href":"/sdk/evaluations/overview"}]},{"tab":"Improve","pages":[{"title":"Improve","href":"/improve/tool-schemas"}]},{"tab":"CI/CD","pages":[{"title":"CI/CD","href":"/integrations/self-host-ci"}]},{"tab":"API Reference","groups":[{"group":"Tracing","pages":[{"title":"Submit Trace","api":"POST /api/v1/ingest/traces","href":"/api-reference/tracing/submit-trace"},{"title":"List Traces","api":"GET /api/v1/ingest/traces","href":"/api-reference/tracing/list-traces"},{"title":"Evaluate Trace","api":"POST /api/v1/ingest/traces/{traceId}/evaluate","href":"/api-reference/tracing/evaluate-trace"}]},{"group":"Custom Evaluations","pages":[{"title":"Custom Evaluations Overview","href":"/api-reference/custom-eval/overview"},{"title":"Create Dataset","api":"POST /api/v1/custom-agent-evaluations/datasets","href":"/api-reference/custom-eval/create-dataset"},{"title":"List Datasets","api":"GET /api/v1/custom-agent-evaluations/datasets","href":"/api-reference/custom-eval/list-datasets"},{"title":"Get Dataset","api":"GET /api/v1/custom-agent-evaluations/datasets/{datasetId}","href":"/api-reference/custom-eval/get-dataset"},{"title":"Create Run","api":"POST /api/v1/custom-agent-evaluations/runs","href":"/api-reference/custom-eval/create-run"},{"title":"Submit Results","api":"POST /api/v1/custom-agent-evaluations/runs/{runId}/results","href":"/api-reference/custom-eval/submit-results"},{"title":"Get Missing Results","api":"GET /api/v1/custom-agent-evaluations/runs/{runId}/missing-results","href":"/api-reference/custom-eval/missing-results"},{"title":"Finalize Run","api":"POST /api/v1/custom-agent-evaluations/runs/{runId}/finalize","href":"/api-reference/custom-eval/finalize-run"},{"title":"Analyze Run","api":"POST /api/v1/custom-agent-evaluations/runs/{runId}/analyze","description":"Run AI analysis over a run's results and store the structured report","href":"/api-reference/custom-eval/analyze-run"},{"title":"Get Report","api":"GET /api/v1/custom-agent-evaluations/runs/{runId}/report","href":"/api-reference/custom-eval/get-report"},{"title":"Get Run","api":"GET /api/v1/custom-agent-evaluations/runs/{runId}","description":"Get a single evaluation run with statistics and per-result scores","href":"/api-reference/custom-eval/get-run"},{"title":"List Runs","api":"GET /api/v1/custom-agent-evaluations/runs","href":"/api-reference/custom-eval/list-runs"},{"title":"List Models","api":"GET /api/v1/agent-monitoring/portability/models","href":"/api-reference/custom-eval/list-models"}]},{"group":"CI Gate","pages":[{"title":"CI/CD Overview","href":"/api-reference/ci-cd/overview"},{"title":"Create CI Run","api":"POST /api/v1/custom-agent-evaluations/runs","href":"/api-reference/ci-cd/create-run"},{"title":"Get Test Cases","api":"GET /api/v1/custom-agent-evaluations/datasets/{datasetId}","href":"/api-reference/ci-cd/get-test-cases"},{"title":"Submit Result","api":"POST /api/v1/custom-agent-evaluations/runs/{runId}/results","href":"/api-reference/ci-cd/submit-result"},{"title":"Finalize CI Run","api":"POST /api/v1/custom-agent-evaluations/runs/{runId}/finalize","href":"/api-reference/ci-cd/finalize-run"},{"title":"Get CI Gate","api":"GET /api/v1/custom-agent-evaluations/runs/{runId}/gate","href":"/api-reference/ci-cd/get-gate"}]},{"group":"Agent Registry","pages":[{"title":"List Agents","api":"GET /api/v1/agents","href":"/api-reference/tracked-agents/list"},{"title":"Register Agent","api":"POST /api/v1/agents","href":"/api-reference/tracked-agents/add"},{"title":"Get Agent","api":"GET /api/v1/agents/{id}","href":"/api-reference/tracked-agents/get"}]}]}]},"basePath":"","title":"Get Report - AgentX","deploymentId":"dpl_DyXGnKeCjZyA3sUdfMJ7FjkdBefg","buildId":"dpl_DyXGnKeCjZyA3sUdfMJ7FjkdBefg","dynamics":["c2aa737661","2b790eda92","b0f84f53ff"],"chunks":["OpenApiReferenceProvider-CiUHMMOm.js","snippetPresets-BgfXU7Bw.js","utils-DWvOwgy_.js","style-to-object-BY3JEAdk.js","utils-BN7RXSMJ.js","v4-DIgz8zy6.js","addUniqueKeysToSchema-CFw8bq89.js","conditional-requirements-CFM0id0Q.js","cloneDeep-CAwdfOU2.js","_Uint8Array-BJr8e-ZL.js","isObject-FP1Yky1u.js","ApiExamples-XX0UgMZK.js","usePlaygroundInputsStore-DXDFSa1V.js","shallow-CdX5aREi.js","isEqual-CoOaxexF.js","CodeSnippets-Cg2wpzWm.js","CodeSnippet-D_HwgQO_.js","getApiPlaygroundDisplay-CmCoerhe.js","Page-CVJ-f_k1.js","DynamicMarkdown-CQM9Bhbv.js","SectionHeading-DpbrP8nQ.js","icon-BPCgijHy.js","MethodPill-GQ8DuY0c.js","getPastedJson-Cb5mT31m.js","Check2Icon-a7bUI6e8.js","axios-CsQoIw0U.js"]}