R0044/2026-04-01¶
This run investigated whether the expanded vocabulary from human factors research (automation bias, overtrust, complacency, commission error, alert fatigue) surfaces regulatory requirements, empirical evidence, and cross-disciplinary bridges that the AI safety term "sycophancy" alone would miss.
Queries¶
Q001 — Regulatory constraints on AI system behavior — Medium confidence
Query: Using the expanded vocabulary, search for enterprise or government requirements that constrain AI system behavior — not just human operator behavior — to prevent the system from reinforcing user assumptions or providing agreeable-but-incorrect output.
Answer: Regulated industries have extensively addressed human-side behavior but have produced almost no requirements constraining AI system-side behavior. The EU AI Act Article 14 and NIST AI 600-1 come closest but focus on transparency and interface design rather than constraining output generation.
| Hypothesis | Status | Probability |
|---|---|---|
| H1: Enforceable system-side requirements exist | Eliminated | — |
| H2: Partial/emerging requirements exist | Supported | — |
| H3: No requirements at all | Eliminated | — |
Confidence: Medium · Sources: 6 · Searches: 3
Q002 — Consequences of agreeable AI in professional contexts — Medium confidence
Query: Search for research on the consequences of AI systems that agree with users rather than challenge them, specifically in high-stakes professional contexts.
Answer: Strong experimental evidence documents measurable harms (Sharma et al. 2026 in Science: 49% more affirmation than humans; Nature Comms: 12-22% severe clinical errors via false confirmation). However, field incident reports from professional domains attributing harm to AI agreement behavior specifically remain sparse.
| Hypothesis | Status | Probability |
|---|---|---|
| H1: Extensive field evidence exists | Eliminated | — |
| H2: Lab evidence strong, field evidence sparse | Supported | — |
| H3: No empirical evidence | Eliminated | — |
Confidence: Medium · Sources: 5 · Searches: 3
Q003 — Bridging automation bias and sycophancy vocabularies — Medium confidence
Query: Has anyone explicitly connected the human-factors concept of automation bias/overtrust to the AI safety concept of sycophancy?
Answer: No formal vocabulary mapping exists. Ibrahim et al. (2025) come closest, using both vocabulary sets in a unified overreliance framework. Most researchers remain in one tradition — Malmqvist's sycophancy survey makes zero reference to human factors research despite studying the same phenomenon's downstream effects.
| Hypothesis | Status | Probability |
|---|---|---|
| H1: Formal bridge exists | Eliminated | — |
| H2: Partial/functional bridge exists | Supported | — |
| H3: No bridging at all | Eliminated | — |
Confidence: Medium · Sources: 3 · Searches: 2
Q004 — DoD CaTE publications and scope — Medium confidence
Query: What has CaTE published, and does it address system-side behavior or only human-side behavior?
Answer: CaTE has published one TEVV guidebook for LAWS (April 2025). Its scope covers both system evaluation and operator trust measurement, but emphasizes the human side. CaTE does not address AI output behavioral constraints and does not use sycophancy vocabulary. Its "calibrated trust" answers "Does operator trust match system capability?" not "Is the system manipulating operator trust?"
| Hypothesis | Status | Probability |
|---|---|---|
| H1: CaTE addresses AI output behavior | Eliminated | — |
| H2: Both sides, human emphasis | Supported | — |
| H3: Human-side only | Eliminated | — |
Confidence: Medium · Sources: 3 · Searches: 2
Collection Analysis¶
Cross-Cutting Patterns¶
| Pattern | Queries Affected | Significance |
|---|---|---|
| System-side regulatory gap | Q001, Q004 | Every sector addresses automation bias through human-side requirements; system-side output behavioral constraints are absent |
| Vocabulary silo | Q003, Q004 | Human factors and AI safety communities study overlapping phenomena under different names with limited cross-referencing |
| Lab-to-field evidence gap | Q002 | Strong experimental evidence of harm from sycophantic AI exists but field incident documentation is absent |
| Healthcare as most advanced domain | Q001, Q002 | Healthcare has both the most specific regulations (FDA CDS) and the most relevant empirical evidence (false confirmation errors) |
Collection Statistics¶
| Metric | Value |
|---|---|
| Queries investigated | 4 |
| Queries answered with medium confidence | 4 (Q001, Q002, Q003, Q004) |
Source Independence Assessment¶
Sources are highly independent across queries: EU legislation (Article 14), US federal standards (NIST AI 600-1), sector-specific regulators (FDA, FINRA, FAA), academic research from multiple institutions (Stanford, Oxford, Cambridge, Georgetown, U. Penn), defense institutions (SEI/CMU, DoD), and independent news outlets. No single institution or funding source dominates the evidence base. The main dependence risk is within Q004, where all sources are from CaTE's institutional ecosystem.
Collection Gaps¶
| Gap | Impact | Mitigation |
|---|---|---|
| PDF content inaccessible (NIST AI 600-1, CSET brief, CaTE guidebook) | May miss specific system-side provisions in full text | Used secondary sources and metadata; key findings likely captured |
| Classified/restricted procurement specifications | May contain system-side requirements not publicly documented | Acknowledged as a limitation; no mitigation available |
| Engineering and finance domain evidence for Q002 | Two of four target domains have no specific evidence | Documented as gaps; mechanism evidence from other domains may be transferable |
| Conference proceedings and working papers | May contain vocabulary bridging not indexed in web search | Acknowledged; Ibrahim et al. preprint was found despite being recent |
Collection Self-Audit¶
| Domain | Rating | Notes |
|---|---|---|
| Eligibility criteria | Low risk | Consistent criteria across all 4 queries |
| Search comprehensiveness | Some concerns | PDF inaccessibility and procurement specification opacity |
| Evaluation consistency | Low risk | Same scoring framework applied to all 17 sources |
| Synthesis fairness | Low risk | Contradictory evidence surfaced when found; gaps documented |
Resources¶
Summary¶
| Metric | Value |
|---|---|
| Queries investigated | 4 |
| Files produced | 112 |
| Sources scored | 17 |
| Evidence extracts | 17 |
| Results dispositioned | 17 selected + 133 rejected = 150 total |
Tool Breakdown¶
| Tool | Uses | Purpose |
|---|---|---|
| WebSearch | 16 | Search queries across regulatory, academic, and news sources |
| WebFetch | 16 | Page content retrieval (10 successful, 6 failed due to 403/303/PDF errors) |
| Write | 112 | File creation for complete evidence archive |
| Read | 2 | Reading methodology and output format specifications |
| Edit | 0 | No edits required |
| Bash | 2 | Directory creation and file counting |
Token Distribution¶
| Category | Tokens |
|---|---|
| Input (context) | ~500,000 |
| Output (generation) | ~150,000 |
| Total | ~650,000 |