R0043/2026-04-01/Q001/SRC05/E01¶
Formal terminological distinctions between overreliance, automation bias, sycophancy, and trust
URL: https://arxiv.org/html/2509.08010v1
Extract¶
The paper provides formal definitions that disambiguate related terms:
- Overreliance — "adopting a system output when that output is wrong, or delegating to a system when such delegation is undesirable." Critically: overreliance is a behavior, not an attitude or feeling.
- Automation bias — a cognitive tendency to favor automated systems. The paper distinguishes this from overreliance: automation bias is the mental tendency; overreliance is the behavioral manifestation.
- Sycophancy — a model characteristic: "the tendency of models to be overly agreeable or sycophantic can reinforce users' existing beliefs." Creates feedback loops discouraging critical evaluation.
- Trust — a psychological state distinct from the behavioral consequence (overreliance). Trust is an attitude; overreliance is an action.
- Underreliance / algorithm aversion — the opposite failure: refusing to use accurate AI recommendations.
JUDGMENT: This is the clearest formal taxonomy found in any source. It establishes that sycophancy (model property), automation bias (human cognition), overreliance (human behavior), and trust (human attitude) are related but categorically distinct concepts.
Relevance to Hypotheses¶
| Hypothesis | Relationship | Strength |
|---|---|---|
| H1 | Supports | Confirms cross-domain vocabulary exists and can be formally mapped |
| H2 | N/A | Does not address domain gaps |
| H3 | Strongly supports | Demonstrates the terms are NOT synonyms — they describe different aspects of the same system |
Context¶
This paper is the strongest single source for the vocabulary mapping because it explicitly disambiguates terms that other sources use interchangeably. The multi-institution authorship (Oxford, Cambridge, Princeton, Stanford, OpenAI) gives it cross-community credibility.
Notes¶
The paper's framing of overreliance as a behavior (not a cognition or model property) provides the bridge between AI safety terminology (sycophancy = model) and human factors terminology (automation bias = cognition). Overreliance is what happens in the middle.