SRC04 — OpenAI Sycophancy Rollback — Scorecard¶
Source¶
| Field | Value |
|---|---|
| Title | Sycophancy in GPT-4o: What Happened and What We're Doing About It |
| Publisher | OpenAI |
| Authors | OpenAI |
| Date | April 2025 |
| URL | https://openai.com/index/sycophancy-in-gpt-4o/ |
| Type | Company incident report / blog post |
Summary Ratings¶
| Dimension | Rating |
|---|---|
| Reliability | Medium-High |
| Relevance | High |
| Missing data | Medium — company self-reporting may omit details |
| Measurement bias | Medium — self-assessment |
| Selective reporting | Medium — company may minimize scope |
| Randomization | N/A |
| Protocol deviation | N/A |
| COI/Funding | Medium-High — OpenAI reporting on its own product failure |
Rationale¶
| Dimension | Rationale |
|---|---|
| Reliability | Primary source from the company that experienced the incident; self-serving framing possible |
| Relevance | Demonstrates that sycophancy is a real, deployed-product problem, not theoretical |
| Bias | OpenAI has incentive to present incident as resolved and controlled |
Evidence Extracts¶
| Evidence | Summary |
|---|---|
| SRC04-E01 | GPT-4o update made model "overly flattering or agreeable"; rollback required; RLHF from user feedback amplified sycophancy |