> ## Documentation Index > Fetch the complete documentation index at: https://developers.agentx.so/llms.txt > Use this file to discover all available pages before exploring further. # Dataset Coverage > Does your eval dataset look anything like production? Insights joins classified traffic to your cases and shows which high-traffic topics have no test behind them Your datasets say what you test. Production says what actually happens. **Dataset coverage** (Insights → Dataset coverage) is the join: it groups classified production traffic into topics, measures how deeply your dataset cases cover each topic, and ranks the gaps by what closing them is worth. ## Prerequisites * **Topics classification on** (Settings → Monitoring Defaults) - coverage is measured against the topics Monitor derives from real traffic, so an install with nothing classified shows an empty state, not zeroes. * **`OPENAI_API_KEY`** for embeddings. Without one the page still works but runs in a labelled **Approximate** mode: coverage falls back to counting cases against a per-topic target instead of measuring depth, and off-map detection is disabled (lexical matching cannot prove a case matches nothing). ## The three headline numbers | Card | What it answers | | ----------------------------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | **Traffic-weighted coverage** | Of the classified traffic, how much has a test behind it. Weighted by topic traffic share, so covering a busy topic moves it more than covering a rare one. | | **Topics at target** | How many topics meet their case target (which scales with traffic and observed risk), with the covered / underrepresented / missing split. | | **Risk-weighted coverage** | Of the traffic that already fails (issues, negative sentiment), how much is tested. The gap between this and traffic-weighted coverage is the useful finding: "you test what is common, not what is dangerous." | A fourth card, **Cases off the map**, lists dataset cases matching no topic production asks - candidates to retire (each opens with its expected results and nearest topic, so retiring is a read decision, not a guess). ## Coverage is depth, not case count A topic's coverage is a facility-location value over the topic's real traces: each production trace is scored against its nearest case, so **near-duplicate cases add nothing** and the number cannot be inflated by generating copies. A topic whose traces carry no embeddings falls back to case-count-against-target for that topic alone, and says so ("cases against target - no depth available"). The "covered" threshold is the same similarity band `addCaseToDataset` uses for dedupe - covered literally means *the dataset would reject this query as a duplicate*. ## Working the queue The topics table ranks by **priority** = traffic exposed x (1 - coverage) x risk amplification - a covered topic is worth nothing to work on however busy, and a risky topic outranks a healthy one carrying several times its traffic. The **Next up** chips surface the top three. Selecting a topic opens its detail rail: * the four per-topic numbers (traffic with trace/session counts, cases against target, coverage with its basis, observed risk with its issue/negative split), * **Production traces in this topic** - the newest real requests with the classifier's verdict, so a gap is judged by reading what production actually asked. Each trace in the list carries an **Add test case** button, the manual path for turning that one request into a case, * **Suggested action** with one button: **Generate N cases from traces** creates candidate cases from the topic's own traces through the standard curation path (deduped, versioned, `expectedResults` deliberately left for you to write - a generated case is a scaffold to review, never a fabricated assertion). ## The probe: "is this already tested?" The probe card answers coverage for one specific query - or a pasted list (a launch spec, a support macro export) as a pre-launch gate. Verdicts distinguish a **real gap** (production asks this, nothing tests it) from **untested and unasked** (nothing tests it, but nobody asks - deliberately neutral, because writing that test is work nobody needs). ## The coverage map The **Map** tab draws production traffic and your dataset cases in **one picture**: a joint UMAP projection in **question space** - a trace's input-only embedding against a case's query-only embedding - so an identical question from both sources lands in the same spot. (The coverage table's topic matching runs in interaction space, `input+output` vs `query+expected`; that is the right space for "is this behavior tested" but the wrong one for this picture, since it separates identical questions whenever the actual answer differs from the expected one.) Blue dots are classified production traces; orange dots are dataset cases. A translucent **density cloud** glows around the points in each source's color - clouds emerge only where points actually cluster, blue and orange blend where dataset cases sit on production traffic, and a blue region with **no orange tint is the gap**, visible without reading a number. Reading aids: * **Source chips** (Both / Production / Datasets) filter the view; the clouds follow. * The **topic legend** shows `production · dataset` point counts per topic - `34 · 0` is an untested topic before you have read the chart at all - and hovering a topic spotlights its points and clouds while dimming everything else. * **Unmatched cases** (tests no production topic claims) get a dashed chip; they are the same cases the Off-map tab lists. * The map inherits the page's date range and dataset filter; cases or traces still warming show as an "N still indexing" chip rather than silently missing (traces classified before the question-space column existed backfill in bounded batches as the map is viewed). Two honest limitations: UMAP coordinates carry no absolute meaning (only relative distance does, and the layout changes between fetches), and the map requires embeddings on both sides - unlike the coverage table it has no lexical fallback, because a position IS a similarity claim. ## Scoping and export The date range (presets from 6 hours to 90 days, or a custom range - custom spans are capped at one year; the default is the past 30 days) and the dataset filter scope every number on the page, the map included. **Export** writes exactly the rows on screen as CSV; the filename is prefixed `provisional-` when the numbers are still a floor (case embeddings indexing) or `approximate-` when they are lexical. ## API * `GET /api/v1/insights/coverage?window=30d&datasetId=...` - the full coverage result. * `GET /api/v1/insights/coverage/map` - the coverage map (same scoping parameters; a separate route because the joint UMAP fit is heavier than the coverage sweep, so you only pay for it with the Map tab open). Needs at least 10 embedded points across both sources; below that it returns `insufficientData: true` with the reason. * `POST /api/v1/insights/topics/curate` with `{ topic, datasetId, window?, limit? }` - the "Generate N cases from traces" action (returns `{ ok, added, duplicates, considered }`). `limit` defaults to 6 and is capped at 12; anything larger is a bulk import, which has its own surface. * `POST /api/v1/insights/probe` and `/probe/batch` - the probe. Both GET routes accept either `window` (`24h` / `7d` / `30d`) or an explicit custom range as `from` and `to` (epoch milliseconds, `to > from`, capped at one year) - the same range override every monitoring dashboard route takes. `datasetId` and `datasetIds` are interchangeable, each accepted repeated or comma-separated.