> ## Documentation Index
> Fetch the complete documentation index at: https://developers.agentx.so/llms.txt
> Use this file to discover all available pages before exploring further.

# AI Analysis Report

> What .analyze() returns, strengths, weaknesses, instruction adherence, and recommendations

`.analyze()` is the last step in the evaluation chain. It runs the same durable, multi-stage pipeline as the dashboard's "Analyze" button: each response is scored by 1-3 LLM judges, then reduced through question- and cluster-level summaries into one final qualitative report, returned as a `Report` object.

```python theme={null}
report = client.evaluations.run(...).execute(my_agent).finalize().analyze(
    mode="auto",                                    # "auto" (default) | "sync" | "batch"
    quality_mode="quality_first",                   # "quality_first" (default) | "balanced"
    judges=["gpt-5.5", "claude-opus-4-8"],           # 1-3 model ids; omit for a single gpt-5.5 judge
)

report.summary                 # str | None, overall narrative summary
report.consistency_score       # float | None, 0-10, run-to-run consistency
report.instruction_adherence   # ReportInstructionAdherence | None
report.response_patterns       # ReportResponsePatterns | None
report.reasoning_analysis      # ReportReasoningAnalysis | None
report.tool_usage_analysis     # ReportToolUsageAnalysis | None
report.strengths               # list[str]
report.weaknesses              # list[str]
report.overall_rating          # str | None, "high" | "medium" | "low"
report.recommendations         # list[ReportRecommendation]
```

<Note>
  Numeric scores (`average_rating`, similarity metrics) are ready right after `.finalize()`, so you don't need to wait for `.analyze()` just to see how the run went. Because `.analyze()` polls a job until it completes, it can take noticeably longer than a single LLM call for larger runs; progress is shown in the terminal while it waits.
</Note>

## Controlling the analysis

| Parameter       | Values                                  | Description                                                                                                                                                                        |
| --------------- | --------------------------------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| `mode`          | `"auto"` (default), `"sync"`, `"batch"` | How item scoring executes server-side; `"auto"` picks based on run size                                                                                                            |
| `quality_mode`  | `"quality_first"`, `"balanced"`         | `"quality_first"` runs a second judge on every item; `"balanced"` samples based on risk                                                                                            |
| `judges`        | 1-3 model ids                           | Which LLM(s) score each response. The first always runs; a second confirms, a third only breaks a tie between the first two. Defaults to a single judge, `["gpt-5.5"]`, if omitted |
| `poll_interval` | seconds, default `5.0`                  | How often to check job status while waiting                                                                                                                                        |
| `timeout`       | seconds, default `1800.0`               | Give up waiting after this long (the job keeps running server-side; call `get_report()` later to check on it)                                                                      |

Check on a long-running analysis without calling `.analyze()` again, even from a separate script execution:

```python theme={null}
status = client.evaluations.get_analysis_status(run_id)
print(status.status, status.progress.overall_percentage)
```

## Fields

| Field                   | Shape                                                                       | Description                                                                                    |
| ----------------------- | --------------------------------------------------------------------------- | ---------------------------------------------------------------------------------------------- |
| `summary`               | `str`                                                                       | Overall narrative summary of the run                                                           |
| `consistency_score`     | `float`                                                                     | 0–10, how consistent responses were across repeated runs of the same question                  |
| `instruction_adherence` | `{ score, analysis, deviations: [str], rating }`                            | How well responses followed [`subject.agentInstructions`](/sdk/evaluations/evaluation-subject) |
| `response_patterns`     | `{ similarities: [str], differences: [str], outliers: [str], rating }`      | Cross-run consistency patterns                                                                 |
| `reasoning_analysis`    | `{ cot_quality, reasoning_patterns: [str], reasoning_gaps: [str], rating }` | Quality of the agent's chain-of-thought, when traced                                           |
| `tool_usage_analysis`   | `{ effectiveness, patterns: [str], issues: [str], rating }`                 | How well the agent used its tools, when tool calls were traced                                 |
| `strengths`             | `list[str]`                                                                 | What the agent did well                                                                        |
| `weaknesses`            | `list[str]`                                                                 | Where the agent fell short                                                                     |
| `overall_rating`        | `"high" \| "medium" \| "low"`                                               | Coarse overall quality rating                                                                  |
| `recommendations`       | `[{ category, priority, recommendation, reasoning }]`                       | Actionable, prioritized fixes                                                                  |

Each `rating` sub-field is one of `"high"`, `"medium"`, `"low"`. `recommendations[].category` is one of `"instructions"`, `"tools"`, `"knowledge"`, `"reasoning"`, `"consistency"`, `"other"`; `priority` is `"high"`, `"medium"`, or `"low"`.

## Reading it in your terminal

```python theme={null}
report = client.evaluations.run(...).execute(my_agent).finalize().analyze()

print(f"Rating: {report.overall_rating}")
print(report.summary)

for rec in report.recommendations:
    print(f"[{rec.priority}] {rec.category}: {rec.recommendation}")
```

Or open `report.dashboard_url` to review the full report, per-question breakdowns, and instruction-change suggestions in the dashboard.

## Per-result rows without an analysis

`results()` returns the raw scored rows - no analysis job, no extra judge calls - for asserting
on individual results in scripts and CI:

```python theme={null}
run = client.evaluations.run(...).execute(my_agent).finalize()

for r in run.results():
    print(r["rating"], r["questionText"], r["justification"])
    for scorer in r.get("codeScorerResults") or []:       # code scorers, trajectory/context match
        print(" ", scorer["name"], scorer["score"])
```

Each row carries rating/justification, the linked `traceId`, latency and token counts,
similarity metrics, and every named scorer row. `client.evaluations.get_run(run_id)` fetches
the same payload later, from a different process.