R0027/2026-03-26/Q001/SRC07/E02¶
English prompts outperform translated prompts across all languages tested
URL: https://arxiv.org/html/2504.17720v2
Extract¶
"Using English prompts consistently outperformed translated prompts (72.7% vs. 67.2% average)." This 5.5 percentage point advantage of English prompts held across all tested languages, indicating that even when task data is in a non-English language, English instructions yield better performance.
Relevance to Hypotheses¶
| Hypothesis | Relationship | Strength |
|---|---|---|
| H1 | Supports | Directly demonstrates prompt language affects performance |
| H2 | Contradicts | Clear, consistent advantage for English prompts |
| H3 | Supports | The advantage is consistent but varies in magnitude by language |
Context¶
This finding is consistent with SRC06 (Arabic prompting study) and suggests a general pattern: models have been optimized for English instruction-following regardless of task language.