R0050/2026-03-31-02/Q001 — Assessment¶
BLUF¶
No journalistic fact-checking framework includes all four structural features (hierarchical evidence quality scale, calibrated uncertainty language, structured bias assessment domains, formal source reliability tiering). NewsGuard comes closest with a quantified, nine-criteria source reliability scoring system, but it evaluates outlets rather than individual evidence. PolitiFact has a structured claim-rating scale, but it assesses claim truthfulness rather than evidence quality. The absence of calibrated uncertainty language and structured bias assessment domains is universal across all examined frameworks.
Probability¶
Rating: Very likely (80-95%) that no journalistic fact-checking framework includes all four features
Confidence in assessment: High
Confidence rationale: Eight primary sources examined across seven named frameworks plus academic literature. All sources are authoritative (primary methodology documents from the organizations themselves). High source agreement on the absence of the four features. Academic research independently confirms the finding.
Reasoning Chain¶
-
IFCN's Code of Principles requires fact-checkers to use "the same high standards of evidence" and to prioritize primary sources, but defines no structured evidence quality scale or calibrated language. [SRC01-E01, High reliability, High relevance]
-
PolitiFact's Truth-O-Meter provides a six-point structured rating scale (True through Pants on Fire), but this rates claim truthfulness, not evidence quality. There is no formal evidence hierarchy underlying the ratings. [SRC02-E01, High reliability, High relevance]
-
NewsGuard's nine-criteria, 100-point scoring system with five rating tiers is the closest analog to formal source reliability tiering in journalism. It separates credibility (62.5 pts) from transparency (37.5 pts) domains. However, it evaluates news outlets, not individual evidence or sources within stories. [SRC03-E01, SRC03-E02, High reliability, High relevance]
-
BBC Editorial Guidelines require evidence to be "well sourced" and "sound" but provide no structured framework for evaluating evidence quality. [SRC04-E01, High reliability, Medium relevance]
-
Bellingcat requires minimum 3-source corroboration and uses structured investigation phases (collect-analyze-integrate), but explicitly avoids source-level reliability blanket ratings. [SRC05-E01, High reliability, Medium relevance]
-
SPJ Code of Ethics provides principle-level verification guidance without any structured tools. [SRC06-E01, High reliability, Medium relevance]
-
The Verification Handbook explicitly rejects standardized one-size-fits-all methodology in favor of adaptive strategies. [SRC07-E01, High reliability, Medium relevance]
-
Academic research (2025) independently identifies three epistemological challenges in fact-checking and empirically demonstrates that different fact-checkers produce different results on the same claims — a consequence of operating without standardized evidence evaluation frameworks. [SRC08-E01, Medium-High reliability, High relevance]
-
JUDGMENT: The consistent absence of all four structural features across seven frameworks, confirmed by independent academic analysis, establishes with high confidence that journalism as a discipline has not developed the kind of formalized evidence evaluation infrastructure found in intelligence analysis (ICD 203) or scientific research (GRADE, Cochrane). [JUDGMENT]
-
JUDGMENT: The Verification Handbook's explicit rejection of standardized methodology suggests this is partly a deliberate philosophical choice — journalism values contextual judgment over formalized scoring. This is important context: the absence is not purely an oversight but reflects professional values. [JUDGMENT]
Evidence Base Summary¶
| Source | Description | Reliability | Relevance | Key Finding |
|---|---|---|---|---|
| SRC01 | IFCN Code of Principles | High | High | Procedural commitments without structural evidence tools |
| SRC02 | PolitiFact Methodology | High | High | Claim rating scale, not evidence quality hierarchy |
| SRC03 | NewsGuard Criteria | High | High | Closest to source reliability tiering (outlet-level) |
| SRC04 | BBC Editorial Guidelines | High | Medium | Principle-based verification, no structured tools |
| SRC05 | Bellingcat Methodology | High | Medium | Structured phases, 3-source minimum, no scoring |
| SRC06 | SPJ Code of Ethics | High | Medium | Pure principle-based approach |
| SRC07 | Verification Handbook | High | Medium | Explicitly rejects standardized methodology |
| SRC08 | Fact-checking Epistemology (2025) | Medium-High | High | Academic confirmation of methodology gaps |
Collection Synthesis¶
| Dimension | Assessment |
|---|---|
| Evidence quality | Robust — primary sources from all seven named frameworks plus academic literature |
| Source agreement | High — all sources converge on the absence of the four structural features |
| Source independence | High — each framework was developed independently by different organizations |
| Outliers | NewsGuard is the outlier with the most formalized scoring system; the Verification Handbook is an outlier in explicitly rejecting standardization |
Detail¶
The evidence tells a consistent story: journalism's verification methodologies are procedural and principle-based, relying on editorial judgment, multi-source corroboration, and transparency rather than formalized scoring systems. This is not accidental — the Verification Handbook's explicit rejection of standardized approaches and Bellingcat's deliberate avoidance of blanket source reliability ratings suggest professional values that prioritize contextual judgment over structural formalization.
The one significant partial exception is NewsGuard, which has created a quantified, multi-criteria source reliability scoring system. However, NewsGuard evaluates news outlets as institutions, not individual pieces of evidence within an investigation. It is an outlet credibility assessment tool, not an evidence quality evaluation framework.
Gaps¶
| Missing Evidence | Impact on Assessment |
|---|---|
| Washington Post Fact Checker methodology (4-Pinocchio scale) | Low — likely similar to PolitiFact (claim rating, not evidence hierarchy) |
| Full Fact (UK) detailed methodology | Low — IFCN signatory, likely follows IFCN standards |
| FactCheck.org detailed methodology | Low — no published evidence quality scale expected |
| Internal newsroom verification checklists (if they exist) | Medium — unpublished internal tools could contain structural features not visible in public standards |
Researcher Bias Check¶
Declared biases: No researcher profile was provided.
Influence assessment: The query's framing implies that the four features are desirable standards. This framing could bias toward characterizing their absence as a "gap" rather than a deliberate design choice. The assessment addresses this by noting the Verification Handbook's explicit rejection of standardization.
Cross-References¶
| Entity | ID | File |
|---|---|---|
| Hypotheses | H1, H2, H3 | hypotheses/ |
| Sources | SRC01, SRC02, SRC03, SRC04, SRC05, SRC06, SRC07, SRC08 | sources/ |
| ACH Matrix | — | ach-matrix.md |
| Self-Audit | — | self-audit.md |