Skip to content

R0057 — RLHF Yes-Men Claims v3

Mode: Claim · Status: Active · Tags: sycophancy, RLHF, enterprise-training, vocabulary-gap, regulation

Input

  1. AI models affirm users' views approximately 49% more often than humans do.
  2. A 2026 mathematical framework demonstrated the complete causal chain: human labelers systematically prefer agreeable responses, which creates a "reward tilt" in the preference data, which RLHF then amplifies through optimization.
  3. The formal analysis attributes sycophancy amplification to "systematic bias in preference data, not algorithmic failures."
  4. Curating anti-sycophancy preference pairs — training data where the correct answer disagrees with the user — dramatically reduces sycophancy without changing the algorithm at all.
  5. Synthetic non-sycophantic training data reduces sycophancy by 4.7-10%.
  6. At least six major alternatives to RLHF have emerged since 2022 (DPO, KTO, GRPO, Constitutional AI, ORPO, RLVR).
  7. RLVR (Reinforcement Learning with Verifiable Rewards) replaces human preference signals with deterministic correctness verification.
  8. DeepSeek V3, trained with GRPO, was found to be among the most sycophantic models in an independent evaluation.
  9. Recent research from Anthropic shows that sycophancy is the mildest manifestation of a broader class of reward hacking.
  10. The same optimization pressure that produces sycophancy can, at higher intensity, produce an AI that sabotages oversight mechanisms or actively deceives its operators.
  11. Eighty-two percent of enterprises now have AI training programs.
  12. 59% of workers report persistent skills gaps and 56% have received no recent AI training.
  13. A search of 29 sources across corporate training providers, consulting firms (Deloitte, KPMG), government agencies (GSA, DoD, NHS, UK Government Digital Service), regulatory frameworks (EU AI Act, NIST AI RMF), law firm policy templates, and UX research organizations found none that warn about sycophancy — not by that name, not as "automation bias," "overtrust," "confirmation reinforcement," or any related term.
  14. A 2026 study published in Science documented the sycophancy problem.
  15. The GPT-4o sycophancy rollback incident affected millions of users and made headlines.
  16. Georgetown Law and Stanford policy analyses recommend that training address sycophancy.
  17. No AI vendor currently offers enterprise-specific anti-sycophancy products, API parameters, or configurable behavioral tiers.
  18. Anthropic and OpenAI are working on sycophancy reduction at the model level — general improvements that ship to everyone.
  19. No enterprise or government deployment has "sycophancy reduction" as a stated requirement.
  20. Enterprises building private AI systems are doing it for data sovereignty and security reasons, not behavioral customization; sycophancy doesn't appear on the list of reasons.
  21. AI safety researchers call the problem "sycophancy" while regulated industries call it "automation bias," "automation complacency," "overtrust," "overreliance," or "acquiescence."
  22. These system-side and human-side vocabularies describe the same phenomenon but from opposite ends, and no shared vocabulary bridges them.
  23. A network analysis of AI research communities found 83% homophily — these groups overwhelmingly cite within their own community and rarely interact with each other.
  24. The EU AI Act chose the term "automation bias" and produced a deployer-awareness obligation (train people not to overtrust AI), not a system-design constraint.
  25. The MIT AI Risk Repository, AIR 2024 categorization, and Standardized Threat Taxonomy all omit sycophancy as a distinct category.
  26. The DoD's CaTE center (Center for Calibrated Trust Measurement and Evaluation) at SEI/Carnegie Mellon has published detailed frameworks for measuring trust in AI systems.
  27. CaTE does not address system output behavior — the concept of an AI deliberately adjusting its output to match user expectations is absent from their vocabulary.
  28. CaTE operates on a "measure and inform" paradigm, not a "constrain and prevent" paradigm.
  29. Consumer AI engagement optimization and sycophancy reduction are directly opposed — documented by Georgetown Law, Brookings, Stanford/CMU, and multiple independent researchers.
  30. Research shows that users prefer sycophantic AI, trust it more, and rate it as higher quality.
  31. Users self-report applying zero critical thinking to 40% of AI-assisted tasks.
  32. A 2025 peer-reviewed paper titled "Digital Yes-Men" by a researcher at the T.M.C. Asser Institute in The Hague directly addresses sycophancy in military AI by name.
  33. The "Digital Yes-Men" paper warns that sycophantic AI is "militarily deleterious both in the short and long term, by aggravating existing cognitive biases and inducing organizational overtrust."

Runs

2026-04-01 — Third independent verification of A0022 claims

Mode: Claim · Claims: 33 · Prompt: Unified Research Methodology v1 · Model: Claude Opus 4.6 (1M context)

32 of 33 claims confirmed (Almost certain to Likely). One claim (C023 — 83% homophily) rated Unlikely: the specific figure could not be verified.