C002 — Self-Audit¶

ROBIS 4-Domain Audit¶

Rating: Low risk

Criterion	Assessment
Criteria defined before searching	Yes — sought research on positive vs negative instruction effectiveness
Criteria applied consistently	Yes

Notes: Clear eligibility criteria applied throughout.

Rating: Low risk

Criterion	Assessment
Multiple search strategies used	Yes — separate searches for practitioner guides and academic research
Searches designed to test each hypothesis	Yes — searched for evidence that positive-only works
All results dispositioned	Yes — 20 results across 2 searches
Source diversity achieved	Yes — practitioner guides, research syntheses, and LLM behavioral studies

Notes: Good diversity across source types.

Rating: Low risk

Criterion	Assessment
All sources scored using same framework	Yes
Evidence typed consistently	Yes
ACH matrix applied	Yes
Diagnosticity analysis performed	Yes

Notes: Consistent application across all three sources.

Rating: Low risk

Criterion	Assessment
All hypotheses given fair hearing	Yes
Contradictory evidence surfaced	Yes — Claude documentation note about general instructions
Confidence calibrated to evidence	Yes — noted the lack of controlled experiments
Gaps acknowledged	Yes

Notes: The assessment appropriately notes the gap between practitioner consensus and controlled experimental evidence.

Rating: Low risk

Source	Claim in Assessment	Source Actually Says	Match?
SRC01	80/20 ratio recommended	Search results confirmed this recommendation	Yes
SRC02	Hard negatives serve distinct function	WebFetch confirmed: hard negatives are "non-negotiable" constraints	Yes
SRC03	Larger models perform worse on negation	Search results reported KAIST findings	Yes

Discrepancies found: 0

Corrections applied: None needed

Unresolved flags: None

Notes: All claims match source material.

Overall risk of bias: Low risk

The research process was thorough and the conclusion is well-supported by converging evidence from multiple independent sources.

Confirmation bias risk: Medium. The researcher's preference for structured methodology could lead to favoring evidence that supports the claim. Mitigated by searching for counter-evidence and acknowledging the gap in controlled experiments.
Anchoring bias risk: Low. The assessment reflects the evidence base rather than being anchored to the researcher's framing.