Q002 — Self-Audit¶


Research	R0050 — Journalism Disciplines
Run	2026-03-31-02
Query	Q002

ROBIS 4-Domain Audit¶

Domain 1: Eligibility Criteria¶

Rating: Low risk

Criterion	Assessment
Were evidence types defined before searching?	Yes — formal truth-seeking methodologies with structured evidence evaluation
Did criteria shift after seeing results?	No — consistent across all eight disciplines
Were exclusion criteria applied consistently?	Yes — same novelty assessment standard applied to all

Notes: The "novelty" criterion (contributes concepts not in the nine reference frameworks) was clearly defined before searching.

Domain 2: Search Comprehensiveness¶

Rating: Some concerns

Criterion	Assessment
Multiple search strategies used	Yes — five search rounds across all eight disciplines
Searches designed to test each hypothesis	Yes — searched for both formal methods and overlap with reference set
All results dispositioned	Yes — 90 results returned, 10 selected, 80 rejected
Source diversity achieved	Partial — relied on Wikipedia for several frameworks; primary standards fetched for law and auditing

Notes: The breadth of disciplines (eight) necessitated some reliance on secondary sources (Wikipedia). Primary sources were accessed for the two most novel disciplines (law, auditing). The three non-novel disciplines (Bradford Hill, OCEBM, CASP) were adequately characterized through secondary sources because the key finding (overlap with GRADE/Cochrane) is well-established in the literature.

Domain 3: Evaluation Consistency¶

Rating: Low risk

Criterion	Assessment
All sources scored using same framework	Yes — identical scorecard format
Evidence typed consistently	Yes
ACH matrix applied	Yes
Diagnosticity analysis performed	Yes

Notes: Consistent application across nine sources.

Domain 4: Synthesis Fairness¶

Rating: Low risk

Criterion	Assessment
All hypotheses given fair hearing	Yes
Contradictory evidence surfaced	Yes — three disciplines found to NOT contribute novelty
Confidence calibrated to evidence	Yes
Gaps acknowledged	Yes — three potential disciplines identified as gaps

Notes: The finding that Bradford Hill, OCEBM, and CASP do not contribute novel concepts demonstrates that the assessment is not biased toward finding novelty in every discipline.

Domain 5: Source-Back Verification¶

Rating: Low risk

Source	Claim in Assessment	Source Actually Says	Match?
SRC01	Six-tier hierarchy with probabilities 42%-90%	Duke study reports these exact figures	Yes
SRC02	Dual-axis sufficiency-appropriateness framework	AS 1105 explicitly defines both axes	Yes
SRC04	RPN = P x S x D	Standard FMEA formula	Yes
SRC06	Seven Scandinavian criteria	Wikipedia cites Olden-Jorgensen 1998 and Thuren 1997	Yes

Discrepancies found: 0

Corrections applied: None needed

Unresolved flags: None

Notes: Key factual claims verified against sources.

Overall Assessment¶

Overall risk of bias: Low risk

The research covered all eight named disciplines with consistent evaluation criteria. The novelty assessment against nine reference frameworks provides a clear standard. The finding of three non-novel disciplines demonstrates balanced evaluation.

Researcher Bias Check¶

Confirmation bias risk: Low. The query expects novelty; three disciplines were found non-novel, demonstrating resistance to confirmation bias.
Coverage bias risk: Some concerns. Eight disciplines is broad; some were examined via secondary sources rather than primary standards documents. Mitigated by accessing primary sources for the most novel disciplines.