E02¶


Research	R0023 — Counterproductive advice and prompt lifecycle
Run	2026-03-25
Query	Q001
Source	SRC03
Evidence	SRC03-E02
Type	Statistical

Low-knowledge personas actively reduce accuracy in specific models.

URL: https://gail.wharton.upenn.edu/research-and-insights/playing-pretend-expert-personas/

Extract¶

On GPQA Diamond: - o4-mini: All three low-knowledge personas (Layperson, Young Child, Toddler) underperformed baseline - GPT-4o: "Toddler" persona produced statistically significant accuracy reduction

Gemini 2.5 Flash frequently declined answering when given out-of-domain expert personas, refusing an average of 10.56 out of 25 trials per question, typically citing lack of relevant expertise.

Relevance to Hypotheses¶

Hypothesis	Relationship	Strength
H1	Supports	Low-knowledge personas actively degrade performance — personas can harm in both directions
H2	Contradicts	Consistent degradation pattern across multiple conditions
H3	Supports	Effect varies by model, demonstrating context-dependence

Context¶

The refusal behavior of Gemini 2.5 Flash is a particularly noteworthy failure mode — the model takes the persona assignment so literally that it refuses to answer questions outside the persona's domain, making it less useful than the baseline with no persona.