> ## Documentation Index
> Fetch the complete documentation index at: https://developers.agentx.so/llms.txt
> Use this file to discover all available pages before exploring further.

# Code Scorers

> Grade eval results with exact JavaScript, alongside the judge and similarity metrics

Evaluate's judge scoring and similarity metrics (vector/Jaccard/BLEU/ROUGE) cover most grading needs, but sometimes what you want to check is exact code, not a model's opinion - did the output actually call this function, is this valid JSON, does the response stay under N words. Datasets and Evaluator configs both support attaching one or more **code scorers**: plain JavaScript functions, run in-process (`node:vm`, sandboxed - no network, no filesystem, a 3-second timeout) against each result's `input`/`output`/`expected` - plus `toolCalls` (the linked trace's recorded tool calls) when the result carries a `trace_id`, so a scorer can assert on tool behavior. They run right alongside the judge and similarity metrics, not instead of them.

From the dashboard: Governance → Evaluate → Datasets (or Evaluator) → create or edit → **Code Scorers** section → **Add scorer**. Each one gets a name and a JavaScript function body:

From the SDK (dataset builder, so the scorer is versioned with the dataset it guards):

```python theme={null}
dataset = client.evaluations.datasets.builder(
    name="Support gate",
    code_scorers=[{
        "name": "mentions no competitor",
        "code": "const bad = /rivalcorp/i.test(output); return { score: bad ? 0 : 1 };",
    }],
).add_case(query="...", expected_results="...").publish()
```

```js theme={null}
const wordCount = output.trim().split(/\s+/).length;
if (wordCount <= 60) {
  return { score: 1, reasoning: `Concise: ${wordCount} words.` };
}
return { score: 0.5, reasoning: `Wordy: ${wordCount} words (over the 60-word guideline).` };
```

Return a bare number (0-1) or `{ score, reasoning }`. A scorer that throws, times out, or returns something unexpected degrades to `{ score: null, error }` for that one result - it never blocks the judge rating or any other score from being computed.

From the SDK, attach scorers when building a grading config:

```python theme={null}
settings = client.evaluations.settings.builder(
    name="Strict grading",
    acceptance_criteria="...",
    code_scorers=[
        {
            "name": "conciseness",
            "enabled": True,
            "code": (
                "const wordCount = output.trim().split(/\\s+/).length;\n"
                "return wordCount <= 60"
                "  ? { score: 1, reasoning: `Concise: ${wordCount} words.` }"
                "  : { score: 0.5, reasoning: `Wordy: ${wordCount} words.` };"
            ),
        }
    ],
).publish()
```

Each result's scorer rows come back via [`run.results()`](/sdk/evaluations/analysis-report#per-result-rows-without-an-analysis) as `codeScorerResults`. The runnable version is `sample-scripts/selfhost_demo/03_evaluate_with_a_dataset.py`.

<Note>
  [Expected-trajectory matching](/sdk/evaluations/build-dataset#expected-trajectories) reports
  through the same per-result scorer rows: a case with `expected_tools` shows a
  "Trajectory match (mode)" row next to your own code scorers, pass/fail with the
  expected-vs-actual call lists in its reasoning.
</Note>