Q003 — Self-Audit¶


Research	R0023 — Counterproductive advice and prompt lifecycle
Run	2026-03-25
Query	Q003

ROBIS 4-Domain Audit¶

Domain 1: Eligibility Criteria¶

Rating: Pass

Criterion	Assessment
Evidence criteria defined before searching	Yes — sought empirical studies with measurable cross-version comparisons
Criteria applied consistently	Yes

Domain 2: Search Comprehensiveness¶

Rating: Some concerns

Criterion	Assessment
Multiple search strategies used	Yes — 2 queries
All results dispositioned	Yes — 20 results
Source diversity achieved	Limited — one peer-reviewed study, one tangential study, one vendor article

Notes: The evidence base is genuinely thin. This is a finding, not a search failure — there are very few rigorous published studies on cross-version prompt degradation.

Domain 3: Evaluation Consistency¶

Rating: Pass

Criterion	Assessment
All sources scored using same framework	Yes
ACH matrix applied	Yes

Domain 4: Synthesis Fairness¶

Rating: Pass

Criterion	Assessment
All hypotheses given fair hearing	Yes
Gaps acknowledged	Yes — thin evidence base prominently noted

Overall Assessment¶

Overall risk of bias: Low risk

The main limitation is the thin evidence base, which is a property of the field, not a research process failure.

Researcher Bias Check¶

Confirmation bias risk: The query frames the phenomenon as "degradation" which is inherently negative. The research reported mixed effects (some improvements) to compensate.