User Feedback

Your app already renders the agent’s answers, and your users already know which ones were bad. A vote button next to each response, forwarded to AgentX, gives you human ground truth with zero LLM cost: it triages real complaints, powers the Downvote rate KPI, and calibrates every automated judge against actual human reactions. Feedback is deliberately not a scorer. Scorers (the Scorers tab - template patterns, LLM judges, custom endpoints) are judgments you opt into; a user’s vote is what actually happened. AgentX keeps three separate streams:

Stream	Source	Where it lands
Operational outcomes	The trace itself (errors, failed tools, empty responses)	KPI failure metrics, Top failing
Scorer detections	Scorers you enabled	Signals, KPI metrics
Human feedback	Your users’ votes	Signals, Downvote rate, Judge Calibration

Reporting a vote

From your app’s vote handler, with the trace_id you kept from the traced agent call:

client.feedback.report(
    trace_id=trace_id,
    rating="down",                            # "up" or "down"
    comment="It never answered my question",  # optional - the user's own words
    end_user_id=current_user.id,              # optional - your identifier, opaque to AgentX
)

One call does four things:

Attaches the vote to the trace - up/down chips (and the comment) at the top of the trace’s detail view in Observe.
Raises a signal on a downvote - a Negative user feedback signal lands in Monitor for triage, tagged with a User feedback chip. The user is the detector: no sampling, no judge call, nothing to configure or enable.
Feeds Judge Calibration - every vote (up and down) is recorded as an outcome report, so AgentX’s automated verdicts get measured against real human reactions. See Outcomes & Judge Calibration.
Moves the Downvote rate KPI - Overview’s card shows the share of votes in the window that were “down” (vote-denominated: of the users who reacted, how many were unhappy).

Reading votes back:

votes = client.feedback.list(trace_id)   # the same rows the trace dialog shows as chips
kpis = client.monitor.kpis(window="7d")  # includes downvoteRate and its delta vs the prior 7d

The wire equivalents are POST /feedback ({"traceId", "rating", "comment?", "endUserId?"}) and GET /feedback/trace/:traceId.

Feedback vs. outcomes

Both are ground truth; they differ in who reports. Feedback is a human vote with up/down semantics, forwarded live from your UI. An outcome is an after-the-fact system result (“ticket reopened”, reported by a workflow or webhook) with a free-form label. Both feed the same calibration math, and neither ever counts as “AgentX flagged it in advance” - they are the reports the judges get measured against, never predictions that inflate agreement.

Try it

sample-scripts/selfhost_demo/14_user_feedback.py traces three simulated agent replies, votes on them (two up, one down with a comment), and prints the resulting signal, the vote rows on the trace, and the moved downvote rate - no LLM key needed.

Live Traffic Scoring

Ground Truth & Calibration

Reporting a vote

Feedback vs. outcomes

Try it

​Reporting a vote

​Feedback vs. outcomes

​Try it

Reporting a vote

Feedback vs. outcomes

Try it