R0050/2026-03-31-02/Q001/SRC02/E01¶
PolitiFact's Truth-O-Meter is a structured claim-rating scale, not an evidence quality hierarchy
URL: https://www.politifact.com/article/2018/feb/12/principles-truth-o-meter-politifacts-methodology-i/
Extract¶
PolitiFact's Truth-O-Meter uses six ratings reflecting claim accuracy:
- TRUE: Accurate, nothing significant missing
- MOSTLY TRUE: Accurate but needs clarification
- HALF TRUE: Partially accurate, leaves out important details
- MOSTLY FALSE: Element of truth but ignores critical facts
- FALSE: Not accurate
- PANTS ON FIRE: Not accurate and makes a ridiculous claim
The editorial process involves: (1) reporter researches and proposes a rating, (2) assigning editor reviews, (3) three-editor panel discusses using standardized questions, (4) panel votes (two votes required to decide). Editors ask: Is the statement literally accurate? Does it allow multiple interpretations? Did the speaker provide supporting evidence?
Key finding: The Truth-O-Meter rates the truthfulness of claims, not the quality of evidence. There is no hierarchy of evidence types (e.g., primary source vs. expert opinion vs. statistical data). The burden of proof is on the speaker. There is no calibrated uncertainty language — the scale is categorical, not probabilistic. There is no structured bias assessment of sources.
Relevance to Hypotheses¶
| Hypothesis | Relationship | Strength |
|---|---|---|
| H1 | Contradicts | Scale rates claims not evidence; no calibrated uncertainty; no bias domains; no source tiers |
| H2 | Supports | The six-point scale is a structured assessment tool, even if it measures claim truthfulness rather than evidence quality |
| H3 | Contradicts | The Truth-O-Meter is clearly a formal, structured rating system with defined categories |
Context¶
PolitiFact launched in 2007 and won the Pulitzer Prize for National Reporting in 2009. The Truth-O-Meter is perhaps the most widely recognized structured rating system in journalism. Its existence directly contradicts H3 (no structured tools), but its design — rating claims rather than evaluating evidence quality — means it does not constitute an evidence hierarchy.