VOCABULARY / ENTRY
Trust Calibration
trust-calibration
When a system's signals about its own capability and limits match its actual reliability, the user's trust matches it too: they neither accept output blindly where the system tends to be wrong, nor doubt it needlessly where it tends to be right. Trust calibration is about the correspondence between confidence signals and real performance, not about making people trust the system more overall.
Recognizing it
- The system proactively states which situations or tasks it tends to get wrong, not only where it performs well
- Confidence or certainty cues roughly track the actual probability that an output is correct, rather than a fixed, flattering score
- A way to verify an output before acting on it is available, sized to the consequence
Concept boundary
The goal of trust calibration is to make trust match the system's actual capability, not to maximize trust; over-trust (accepting a wrong output that should have been questioned) and under-trust (ignoring a correct output that could have been used) are both calibration failures, and neither is worse than the other. It concerns the correspondence between confidence signals and real accuracy, distinct from "explainability" — which concerns why the system produced a given result and what reasoning it used, a separate entry not yet written; this entry only mentions it without drawing out the comparison.
When to consider it
- Output is adopted directly for decisions, and the system's accuracy is uneven across tasks, inputs, or groups
- Users lack first-hand experience with the system's capability and are prone to judging trustworthiness by tone, personality, or interface polish
- System capability shifts with task type, data freshness, or version updates, so a past impression may already be outdated
When to be cautious
- Treating "raising trust" itself as the design goal, without checking whether trust actually matches measured accuracy
- Using confident phrasing, personality, or a polished interface to mask the system's real error rate in a given situation
- Requiring the same level of verification for every low-stakes, easily corrected output, creating needless friction
Tradeoff
Clearly communicating weak spots and providing a way to verify reduces both over-acceptance and under-use, but it adds interface information and interaction steps; requiring the same verification for every output regardless of consequence undercuts the judgment cost that calibration is meant to save.
Conditional rules · Editorial advice
Related entries
Calibrating trust depends on making the system's current status and uncertainty clear; visibility is the precondition for calibration signals to be communicated at all.
Human-centered design requires understanding how people actually interpret and use system output, which reinforces the goal of calibrating trust and avoiding over- or under-reliance.
Offering a verification step before a costly adoption is the same kind of design move as error prevention's "check or confirm before submitting" — only here the thing being checked is the system's judgment rather than the user's own input.
Sources and what they support
Guideline 2 calls for helping users understand "how well the system can do what it can do," i.e. how often it may make mistakes; Guideline 10 calls for the system to disambiguate or gracefully degrade — scoping down the services it offers — when it cannot determine the user's goal.
Retrieved: 2026-09-15 · Archived 2026-09-16Automation is often problematic because people fail to rely on it appropriately; because people respond to technology socially, trust shapes reliance on automation, especially when complexity and unanticipated situations make full understanding of the system impractical, so trust ends up guiding reliance instead.
Retrieved: 2026-09-15 · Archived 2025-05-18Automation bias is the tendency to favor an automated system's suggestions and ignore other correct information, an "error of commission" where over-trust leads to accepting a suggestion that should have been questioned; there is also an "error of omission," where a user skips proper review and misses a problem the system failed to catch, corresponding to under-trust or disuse of the automation.
Retrieved: 2026-09-15 · Archived 2026-09-16Definitions reference these sources. Digital specimens, selection advice, relationships, and recipes are editorial work and have not been validated through user research.
Type extensions and implementation notes
This entry has no operable specimen yet; the illustration is an editorial sketch. Applying it to a product still requires verifying that confidence or certainty signals match measured accuracy, that the verification flow is actually workable in the real setting, and that differences in system capability across tasks or groups are honestly communicated.
{
"evidenceType": "A design interpretation of human-AI interaction guidelines and automation-trust research",
"limit": "This entry does not measure how much any specific calibration technique reduces over-acceptance or under-use, nor does it prescribe a universal \"right\" confidence threshold; trust calibration remains an active research area, and this entry cites only verifiable definitional claims, not experimental effect sizes it has not read."
}Open the Agent JSON file for this entryThis file repeats every field of this entry verbatim and adds its sources, archive status, and editorial scope.