R0043/2026-04-01/Q001/SRC06/E01¶
Counter-finding: LLMs exhibit opposite of acquiescence bias
URL: https://arxiv.org/abs/2509.08480
Extract¶
Contrary to the hypothesis that LLMs would exhibit acquiescence bias (tendency to agree with statements), the study found the opposite: across English tasks, models showed "a bias towards replying 'no', independent of whether that indicates agreement or disagreement."
Study scope: 37,975 question variations across 5 LLM models (Llama-3.1-8B, Mistral-Small-24B, Gemma-2-27b, Llama-3.3-70B, GPT-4o), 9 tasks, 3 languages (English, German, Polish).
The author concludes: "it is questionable whether [LLMs] can be used to simulate human responses" — the bias patterns do not match human acquiescence patterns.
JUDGMENT: This is a significant finding for the vocabulary mapping because it demonstrates that borrowing terms from survey methodology ("acquiescence bias") and applying them to LLMs can be misleading. The same term describes different phenomena in different contexts.
Relevance to Hypotheses¶
| Hypothesis | Relationship | Strength |
|---|---|---|
| H1 | N/A | Neither supports nor contradicts the existence of vocabulary |
| H2 | N/A | Not about domain gaps |
| H3 | Strongly supports | Demonstrates that terms borrowed across domains can describe different (even opposite) phenomena |
Context¶
This finding complicates the vocabulary mapping by showing that terminological borrowing can be misleading. "Acquiescence bias" in survey methodology means tendency to agree; when tested in LLMs, the opposite was found. This validates H3's caution that cross-domain terms may not be true synonyms.
Notes¶
The Lim & Lee (2024) terminological shift from "sycophancy" to "agreeableness bias" between preprint versions (noted in other sources) further illustrates the instability of this vocabulary.