R0023/2026-03-25/Q001/SRC04
EMNLP 2024 peer-reviewed study: personas in system prompts do not improve LLM performance
Source
| Field |
Value |
| Title |
When "A Helpful Assistant" Is Not Really Helpful: Personas in System Prompts Do Not Improve Performances of Large Language Models |
| Publisher |
Association for Computational Linguistics |
| Author(s) |
Mingqian Zheng, Jiaxin Pei, Lajanugen Logeswaran, Moontae Lee, David Jurgens |
| Date |
2024-11 |
| URL |
https://aclanthology.org/2024.findings-emnlp.888/ |
| Type |
Peer-reviewed research paper |
Summary
| Dimension |
Rating |
| Reliability |
High |
| Relevance |
High |
| Bias: Missing data |
Low risk |
| Bias: Measurement |
Low risk |
| Bias: Selective reporting |
Low risk |
| Bias: Randomization |
N/A — not an RCT |
| Bias: Protocol deviation |
N/A — not an RCT |
| Bias: COI/Funding |
Low risk |
Rationale
| Dimension |
Rationale |
| Reliability |
Peer-reviewed at EMNLP, a top-tier NLP conference. 162 roles, 4 LLM families, 2,410 factual questions. Rigorous methodology. |
| Relevance |
Independently confirms the Wharton persona findings. Provides additional evidence from a different research group with different methodology. |
| Bias flags |
Low risk. Published through competitive peer review. Multiple authors from academic institutions. |
| Evidence ID |
Summary |
| SRC04-E01 |
Personas have no or small negative effects; expert persona underperforms base model (68.0% vs. 71.6%) |