C014 — Claim Definition¶


Research	R0052 — Methodology Claims
Run	2026-03-31
Claim	C014

Claim as Received¶

The process self-audit (ROBIS) catches process errors but not interpretation errors — an agent can follow every step correctly and still mischaracterize what a source says.

Claim as Clarified¶

ROBIS (Risk of Bias in Systematic Reviews) assesses process compliance through four domains but has a blind spot: it cannot detect cases where a reviewer follows all methodological steps correctly yet misrepresents or mischaracterizes the content of included sources. The claim implies ROBIS needs supplementation (such as the source-back verification step in the methodology prompt).

BLUF¶

The claim is substantially correct. ROBIS focuses on process domains (eligibility, search, evaluation, synthesis) and does assess interpretation in Phase 3. However, it relies on reviewer self-report and cannot independently verify whether source characterizations are accurate. An agent could follow every ROBIS-auditable step correctly and still mischaracterize sources.

Scope¶

Domain: Systematic review methodology / bias assessment
Timeframe: 2015-present (ROBIS publication)
Testability: Verifiable against ROBIS tool documentation

Assessment Summary¶

Probability: Very likely (80-95%)

Confidence: Medium-High

Hypothesis outcome: H2 best supported — ROBIS does address interpretation in Phase 3 but cannot independently verify source characterizations, so the practical claim holds.

[Full assessment in assessment.md.]

Status¶

Field	Value
Date created	2026-03-31
Date completed	2026-03-31
Researcher profile	Phil Moore
Prompt version	ai-research-methodology v1 research.md
Revisit by	2027-03-31
Revisit trigger	ROBIS is updated to include independent source-back verification