> ## Documentation Index
> Fetch the complete documentation index at: https://developers.agentx.so/llms.txt
> Use this file to discover all available pages before exploring further.

# Coming from DeepEval

> Metric-name mapping, what carries over, and what works differently

DeepEval is a metrics library; AgentX is a self-host evaluation platform. If you've built evals
on DeepEval, almost everything you wrote maps directly - and the platform half (live scoring,
review, calibration, dashboards) is what you gain. This page is the honest mapping, based on a
side-by-side run of identical test cases through both systems.

## Metric mapping

| DeepEval                        | AgentX                                                    | Notes                                                                                                                                                                                                                                           |
| ------------------------------- | --------------------------------------------------------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| `FaithfulnessMetric`            | **RAG: Faithfulness**                                     | AgentX is stricter by design: claims must be *supported by* the context, not merely non-contradicting. In our comparison, a fully unsupported answer scored 1.00 on DeepEval ("no contradictions") and 1/10 here.                               |
| `AnswerRelevancyMetric`         | **RAG: Answer Relevancy**                                 | Both judge direction only, never correctness (that's Faithfulness's job).                                                                                                                                                                       |
| `ContextualRelevancyMetric`     | **RAG: Context Relevancy**                                | AgentX judges holistically with per-chunk reasoning; DeepEval scores a statement ratio, which can cliff (a retrieval containing the exact answer scored 0.00 in our comparison when a sibling chunk diluted the ratio).                         |
| `ContextualPrecisionMetric`     | **RAG: Contextual Precision**                             | Same contract, including "no relevant chunks scores 0".                                                                                                                                                                                         |
| `ContextualRecallMetric`        | **RAG: Contextual Recall**                                | Both require a reference answer. DeepEval raises `MissingTestCaseParamsError` mid-run without one; AgentX marks the scorer **Needs a reference answer** - live scoring is refused up front and reference-less dataset cases skip with a reason. |
| `GEval(criteria=...)`           | **Any judge scorer**                                      | Your GEval criteria paste straight into a scorer's acceptance/rejection/evaluation criteria - and the same scorer then also scores live traffic.                                                                                                |
| `TaskCompletionMetric`          | **Agent: Task Completion**                                | Trajectory-anchored: the judge sees the tools called, in order, with failures.                                                                                                                                                                  |
| `ToolCorrectnessMetric`         | **Agent: Tool Correctness** + `expectedTools` on cases    | For exact matching, dataset cases' `expected_tools` is deterministic - zero LLM cost. The judge template covers the semantic cases exact matching can't.                                                                                        |
| `StepEfficiencyMetric`          | **Agent: Step Efficiency**                                | Loops, redundant calls, dead ends - judged from the trajectory.                                                                                                                                                                                 |
| `KnowledgeRetentionMetric`      | **Session: Knowledge Retention**                          | Enable live scoring with per-session scope; the judge grades the whole transcript.                                                                                                                                                              |
| `RoleAdherenceMetric`           | **Session: Role Adherence**                               | Same session mechanism.                                                                                                                                                                                                                         |
| `ToxicityMetric` / `BiasMetric` | **Safety: Harmful Content** / **Safety: Bias & Fairness** | Judge-based, meaning-level. The zero-cost template patterns (secrets, PII, profanity) catch literal matches without any LLM spend.                                                                                                              |
| `assert_test` (pytest)          | `agentx.testing.assert_evaluation`                        | Same ergonomic; the verdict is also recorded in the dashboard's CI gate history.                                                                                                                                                                |

## What works differently (on purpose)

* **One scorer, both surfaces.** A DeepEval metric is an offline construct; online evaluation
  lives in its Confident AI cloud. An AgentX judge scorer grades dataset runs *and* live traffic
  with one rubric - see [the two grading modes](/monitor/scorers#the-two-grading-modes).
* **0-10 with prose, not 0-1 ratios.** DeepEval scores are claim/statement ratios (auditable
  arithmetic, occasional cliffs). AgentX judges holistically and states its counts in the
  justification ("2 of 3 claims supported") - auditable prose, no cliffs.
* **Hallucinations fail hard.** DeepEval's ratio math gave a known contradiction 0.50 in our
  comparison; AgentX's exact-fact rule floors a wrong core fact to 0-3 regardless of polish.

## What we don't have (and where to go instead)

* **Academic benchmarks** (MMLU, HumanEval, GSM8K...): DeepEval ships 17. These benchmark
  foundation models, not your agents - if you need them, run DeepEval's benchmark suite; it
  coexists fine with AgentX in the same codebase.
* **Dataset synthesizer**: AgentX builds datasets [from production traces](/evaluation/datasets-from-production)
  and generates grounded cases from pasted source text (plain text or Markdown only); DeepEval's
  synthesizer adds document-file parsing and larger-scale generation.
* **Red-teaming**: use their `deepteam`; AgentX's safety scorers monitor production, they don't attack it.
* **Multimodal metrics** (image, audio, voice): DeepEval only.

## Migration in practice

```python theme={null}
# DeepEval
metric = GEval(name="Helpfulness", criteria="Answers using the retrieved policy...")
assert_test(LLMTestCase(input=q, actual_output=a), [metric])

# AgentX - same rubric, plus it now also scores live traffic
scorer = client.monitor.judge_scorers.create(
    name="Helpfulness",
    judge={"acceptanceCriteria": "Answers using the retrieved policy..."},
)
report = client.evaluations.run(
    dataset_id=ds,
    subject={"kind": "custom_agent", "displayName": "Helpfulness Agent", "framework": "other"},
    scorer_id=scorer.id,
).execute(agent).finalize()
assert_evaluation(report, min_rating=7.0, no_regression=True)
```