Skip to content

R0002/2026-03-13/C005 — Assessment
Research R0002 — Research Standards for AI-Assisted Writing
Run 2026-03-13
Claim C005

BLUF

Factually confirmed with nuance on editorial characterization. RoB 2 has exactly five bias domains and COI/funding is absent. The "conspicuous" characterization is defensible given published debate, but one-sided — the tool developers have articulated deliberate reasons for the design choice. A more precise formulation would be "deliberately excluded, though this decision is debated."

Probability

Rating: Very likely (80-95%)

Confidence in assessment: High

Confidence rationale: The factual elements (five domains, COI absent) are confirmed by the official Cochrane documentation with no ambiguity. The debate is well-documented with published arguments on both sides. The only uncertainty is interpretive — whether "conspicuously absent" fairly characterizes a deliberate design choice.

Reasoning Chain

  1. The claim states RoB 2 has five domains and COI/funding is "conspicuously absent." [Claim text]
  2. Cochrane's official documentation confirms exactly five domains. [SRC01-E01, High reliability, High relevance]
  3. The original RoB tool's "other bias" domain (where COI was assessed 5.8% and funding 15.6% of the time) was removed in RoB 2. [SRC02-E01, High reliability, High relevance]
  4. External researchers recommend adding funding source to future revisions. [SRC02-E02, peer-reviewed recommendation]
  5. However, a co-developer of RoB 2 (Sterne) published a formal argument that funding should NOT be included, and that the TACIT tool is the appropriate separate instrument. [SRC03-E01, High reliability but high COI]
  6. Inference: The absence is genuinely debated in the literature, which supports "conspicuous." However, the developers' deliberate defense complicates the implication that the absence is a deficiency.
  7. Conclusion: Rating of "Very likely" reflects factual accuracy with a minor reservation about the editorial characterization.

Evidence Base Summary

Source Description Reliability Relevance Key Finding
SRC01 Cochrane RoB 2 High High Five domains confirmed, no COI mechanism
SRC02 Nejadghaderi 2024 High High "Other bias" removed; recommends adding funding
SRC03 Sterne 2024 High (COI noted) High Defends absence as deliberate; TACIT under development

Collection Synthesis

Dimension Assessment
Evidence quality High — official documentation, peer-reviewed studies, developer commentary
Source agreement Full agreement on facts; normative interpretation is contested
Source independence Moderate — the debate is between tool developers and external critics
Outliers Sterne's defense is a notable counterpoint to the "deficiency" framing

Detail

All sources agree on the factual elements: RoB 2 has five domains, COI/funding is not among them, and the "other bias" domain from RoB 1 was removed. The tension is entirely normative — whether the absence is a deficiency (critics) or a principled design choice (developers). The fact that a co-developer published a formal defense, and that a separate tool (TACIT) is being developed, supports the "conspicuous" characterization while simultaneously complicating the "deficiency" implication. The debate is real, published, and ongoing.

Gaps

# Missing Evidence Impact on Assessment
1 TACIT tool documentation Would clarify whether the separate tool adequately addresses the gap
2 AMSTAR 2 comparison AMSTAR 2 includes COI assessment, suggesting broader community value

The gaps do not materially affect the factual assessment. The TACIT tool's development status would inform the temporal relevance of the "conspicuous" characterization — if TACIT is adopted, the gap may be closed.

Researcher Bias Check

Declared biases: Author may apply anti-corporate framing to the absence of COI assessment.

Influence assessment: The bias risk was mitigated by the search design — the targeted search for the COI debate (S02) surfaced both critics and defenders. Sterne's counterargument was included and given full weight. The synthesis explicitly notes that "conspicuous" is "defensible but one-sided" rather than simply accepting the anti-corporate framing.

Cross-References

Entity ID File
Hypotheses H1, H2, H3 hypotheses/
Sources SRC01, SRC02, SRC03 sources/
ACH Matrix C005 ach-matrix.md
Self-Audit C005 self-audit.md