Q003 — Self-Audit¶


Research	R0050 — Journalism Disciplines
Run	2026-03-31-02
Query	Q003

ROBIS 4-Domain Audit¶

Domain 1: Eligibility Criteria¶

Rating: Low risk

Criterion	Assessment
Were evidence types defined before searching?	Yes — structured classification procedures (decision trees, checklists, scoring rubrics)
Did criteria shift after seeing results?	No — consistent throughout
Were exclusion criteria applied consistently?	Yes

Notes: The query clearly defines "procedural implementation" vs. "conceptual framework" — this provided unambiguous eligibility criteria.

Domain 2: Search Comprehensiveness¶

Rating: Low risk

Criterion	Assessment
Multiple search strategies used	Yes — three search rounds targeting implementation, platform policy, and newsroom training
Searches designed to test each hypothesis	Yes — specifically searched for operationalized, procedural, decision tree implementations
All results dispositioned	Yes — 50 results returned, 6 selected, 44 rejected
Source diversity achieved	Yes — original report, creator's organization, EU policy, practitioner handbook, secondary review

Notes: Searches specifically used terms like "operationalized," "decision tree," and "classification tool" to find evidence that would support H1.

Domain 3: Evaluation Consistency¶

Rating: Low risk

Criterion	Assessment
All sources scored using same framework	Yes
Evidence typed consistently	Yes
ACH matrix applied	Yes
Diagnosticity analysis performed	Yes

Notes: Consistent application across five sources.

Domain 4: Synthesis Fairness¶

Rating: Low risk

Criterion	Assessment
All hypotheses given fair hearing	Yes — searched specifically for H1 evidence (operationalized implementations)
Contradictory evidence surfaced	Yes — noted that internal platform procedures may exist unpublished
Confidence calibrated to evidence	Yes
Gaps acknowledged	Yes — three gaps identified

Notes: The most important fairness consideration: the assessment acknowledges that platforms may have internal classification procedures that are not publicly documented. This is identified as a gap rather than evidence for H1.

Domain 5: Source-Back Verification¶

Rating: Low risk

Source	Claim in Assessment	Source Actually Says	Match?
SRC01	Report titled "Toward an Interdisciplinary Framework"	Title confirmed	Yes
SRC02	Crosscheck project involved 37 French newsrooms	Source confirms 37 newsrooms	Yes
SRC03	EU Code signed by Google, Meta, Microsoft, TikTok	Source confirms signatories	Yes
SRC04	Seven content types presented as descriptive categories	WebFetch confirmed: no decision tools present	Yes

Discrepancies found: 0

Corrections applied: None needed

Unresolved flags: None

Notes: All factual claims verified against source content.

Overall Assessment¶

Overall risk of bias: Low risk

The research specifically targeted evidence that would support H1 (taxonomy has been operationalized) through multiple search strategies. The finding that no operationalization was found — despite searching the creators' own organization, the most consequential policy adoption, and the leading practitioner handbook — provides high confidence in H2.

Researcher Bias Check¶

Confirmation bias risk: Low. The query's binary framing could bias toward the "conceptual" answer, but multiple searches specifically targeted operationalization evidence.
Negativity bias risk: Some concerns. The assessment finds an absence (no operationalization), which could reflect insufficient searching rather than genuine absence. Mitigated by examining the most likely venues for operationalization (First Draft, Verification Handbook).