Skip to content

R0044 — Expanded Vocabulary Research

Mode: Query · Status: Active · Tags: sycophancy, automation-bias, human-factors, regulated-industries

Input

  1. Using the expanded vocabulary (automation bias, automation complacency, overtrust, overreliance, acquiescence problem, calibrated trust, confirmation bias amplification, alert fatigue, commission error, inappropriate trust), search for enterprise or government requirements, deployment standards, or procurement specifications that constrain AI system behavior — not just human operator behavior — to prevent the system from reinforcing user assumptions or providing agreeable-but-incorrect output. Focus on defense, healthcare, aviation, and financial services.
  2. Using the same expanded vocabulary, search for research on the consequences of AI systems that agree with users rather than challenge them, specifically in high-stakes professional contexts (engineering, medicine, military operations, financial analysis). Look for case studies, incident reports, or empirical studies where agreeable AI output led to measurable harm or near-misses.
  3. Has anyone in the regulated industries (aviation, defense, healthcare, finance) published research or guidance that explicitly connects the human-factors concept of "automation bias" or "overtrust" to the AI safety concept of "sycophancy"? Is anyone bridging these two vocabularies?
  4. The DoD CaTE (Calibrated AI Trust and Expectations) center was identified as having the most sophisticated regulated-industry vocabulary for this problem. What has CaTE published about calibrating trust in AI systems, and does their work address the system-side behavior (AI adjusting output to match user expectations) or only the human-side behavior (users trusting AI too much)?

Runs

2026-03-29 — Initial run

Mode: Query · Queries: 4 · Prompt: Unified Research Methodology v1 · Model: Claude Opus 4.6

First investigation of expanded vocabulary across regulated industries.

2026-04-01 — Rerun

Mode: Query · Queries: 4 · Prompt: Unified Research Methodology v1 · Model: Claude Opus 4.6 (1M context)

Isolated rerun. System-side regulatory constraints remain absent; lab evidence strong but field incidents sparse; no formal vocabulary bridge exists; CaTE focuses on human-side trust calibration only.