> ## Documentation Index
> Fetch the complete documentation index at: https://developers.agentx.so/llms.txt
> Use this file to discover all available pages before exploring further.

# Coming from DeepEval

> Metric-name mapping, what carries over, and what works differently

DeepEval is a metrics library; AgentX is a self-host evaluation platform. If you've built evals
on DeepEval, almost everything you wrote maps directly - and the platform half (live scoring,
review, calibration, dashboards) is what you gain. This page is the honest mapping, based on a
side-by-side run of identical test cases through both systems.

## Metric mapping

| DeepEval | AgentX | Notes |
| - | - | - |
| `FaithfulnessMetric` | **RAG: Faithfulness** | AgentX is stricter by design: claims must be *supported by* the context, not merely non-contradicting. In our comparison, a fully unsupported answer scored 1.00 on DeepEval ("no contradictions") and 1/10 here. |
| `AnswerRelevancyMetric` | **RAG: Answer Relevancy** | Both judge direction only, never correctness (that's Faithfulness's job). |
| `ContextualRelevancyMetric` | **RAG: Context Relevancy** | AgentX judges holistically with per-chunk reasoning; DeepEval scores a statement ratio, which can cliff (a retrieval containing the exact answer scored 0.00 in our comparison when a sibling chunk diluted the ratio). |
| `ContextualPrecisionMetric` | **RAG: Contextual Precision** | Same contract, including "no relevant chunks scores 0". |
| `ContextualRecallMetric` | **RAG: Contextual Recall** | Both require a reference answer. DeepEval raises `MissingTestCaseParamsError` mid-run without one; AgentX marks the scorer **Needs a reference answer** - live scoring is refused up front and reference-less dataset cases skip with a reason. |
| `GEval(criteria=...)` | **Any judge scorer** | Your GEval criteria paste straight into a scorer's acceptance/rejection/evaluation criteria - and the same scorer then also scores live traffic. |
| `TaskCompletionMetric` | **Agent: Task Completion** | Trajectory-anchored: the judge sees the tools called, in order, with failures. |
| `ToolCorrectnessMetric` | **Agent: Tool Correctness** + `expectedTools` on cases | For exact matching, dataset cases' `expected_tools` is deterministic - zero LLM cost. The judge template covers the semantic cases exact matching can't. |
| `StepEfficiencyMetric` | **Agent: Step Efficiency** | Loops, redundant calls, dead ends - judged from the trajectory. |
| `KnowledgeRetentionMetric` | **Session: Knowledge Retention** | Enable live scoring with per-session scope; the judge grades the whole transcript. |
| `RoleAdherenceMetric` | **Session: Role Adherence** | Same session mechanism. |
| `ToxicityMetric` / `BiasMetric` | **Safety: Harmful Content** / **Safety: Bias & Fairness** | Judge-based, meaning-level. The zero-cost template patterns (secrets, PII, profanity) catch literal matches without any LLM spend. |
| `assert_test` (pytest) | `agentx.testing.assert_evaluation` | Same ergonomic; the verdict is also recorded in the dashboard's CI gate history. |

## What works differently (on purpose)

* **One scorer, both surfaces.** A DeepEval metric is an offline construct; online evaluation
  lives in its Confident AI cloud. An AgentX judge scorer grades dataset runs *and* live traffic
  with one rubric - see [the two grading modes](/monitor/scorers#the-two-grading-modes).
* **0-10 with prose, not 0-1 ratios.** DeepEval scores are claim/statement ratios (auditable
  arithmetic, occasional cliffs). AgentX judges holistically and states its counts in the
  justification ("2 of 3 claims supported") - auditable prose, no cliffs.
* **Hallucinations fail hard.** DeepEval's ratio math gave a known contradiction 0.50 in our
  comparison; AgentX's exact-fact rule floors a wrong core fact to 0-3 regardless of polish.

## What we don't have (and where to go instead)

* **Academic benchmarks** (MMLU, HumanEval, GSM8K...): DeepEval ships 17. These benchmark
  foundation models, not your agents - if you need them, run DeepEval's benchmark suite; it
  coexists fine with AgentX in the same codebase.
* **Dataset synthesizer**: AgentX builds datasets [from production traces](/evaluation/datasets-from-production)
  and generates grounded cases from pasted source text (plain text or Markdown only); DeepEval's
  synthesizer adds document-file parsing and larger-scale generation.
* **Red-teaming**: use their `deepteam`; AgentX's safety scorers monitor production, they don't attack it.
* **Multimodal metrics** (image, audio, voice): DeepEval only.

## Migration in practice

```python theme={null}
# DeepEval
metric = GEval(name="Helpfulness", criteria="Answers using the retrieved policy...")
assert_test(LLMTestCase(input=q, actual_output=a), [metric])

# AgentX - same rubric, plus it now also scores live traffic
scorer = client.monitor.judge_scorers.create(
    name="Helpfulness",
    judge={"acceptanceCriteria": "Answers using the retrieved policy..."},
)
report = client.evaluations.run(
    dataset_id=ds,
    subject={"kind": "custom_agent", "displayName": "Helpfulness Agent", "framework": "other"},
    scorer_id=scorer.id,
).execute(agent).finalize()
assert_evaluation(report, min_rating=7.0, no_regression=True)
```


This documentation is built and hosted on [Mintlify](https://mintlify.com), a developer documentation platform.