R9990/2026-03-31/C001 — Self-Audit¶
ROBIS 4-Domain Audit¶
Domain 1: Eligibility Criteria¶
Rating: Low risk
| Criterion | Assessment |
|---|---|
| Evidence criteria defined before searching | Yes — defined as: evidence addressing STAR or behavioral interview formats, neurodivergent conditions (ADHD, dyslexia, autism), cognitive mechanisms (working memory, executive function), and interview performance data |
| Criteria applied consistently | Yes — same criteria used across all five searches |
| Criteria shifted after seeing results | No — the inclusion of SRC08 (counterpoint) was planned from search design (S01 specifically sought both supporting and contradicting evidence) |
Notes: Eligibility criteria were stable throughout. The decision to include autism-related sources (not explicitly named in the claim) was made at Step 1 (clarification) because neurodivergent individuals include autism spectrum, and the claim's examples ("such as") are non-exhaustive.
Domain 2: Search Comprehensiveness¶
Rating: Low risk
| Criterion | Assessment |
|---|---|
| Multiple search strategies used | Yes — 5 searches targeting different angles: direct claim (S01), ADHD mechanism (S02), dyslexia specifics (S03), STAR criticism (S04), peer-reviewed validity (S05) |
| Searches designed to test each hypothesis | Yes — S01 and S05 sought contradicting evidence (counterpoints and validity studies); S02/S03 sought mechanism evidence; S04 sought structural criticism |
| All results dispositioned | Yes — 50 results returned, all dispositioned: 9 selected + 41 rejected = 50 total |
| Source diversity achieved | Yes — peer-reviewed studies (3), practitioner accounts (2), advocacy organizations (2), industry commentary (2) |
Notes: The dyslexia-specific evidence base (S03) was thinner than ADHD or autism — only one source selected from 10 results, and it was testimonial rather than peer-reviewed. This is a genuine limitation but reflects the state of the evidence landscape, not a search failure.
Domain 3: Evaluation Consistency¶
Rating: Low risk
| Criterion | Assessment |
|---|---|
| All sources scored using same framework | Yes — all 9 sources scored on reliability, relevance, and 6 bias domains |
| Evidence typed consistently | Yes — Factual, Reported, Analytical, Statistical, Testimonial applied per source content |
| ACH matrix applied | Yes — all 9 evidence extracts mapped against all 3 hypotheses |
| Diagnosticity analysis performed | Yes — most and least diagnostic evidence identified with reasoning |
Notes: The peer-reviewed sources (SRC03, SRC06, SRC07) received higher reliability ratings than practitioner/advocacy sources, which is appropriate given the evidence hierarchy.
Domain 4: Synthesis Fairness¶
Rating: Low risk
| Criterion | Assessment |
|---|---|
| All hypotheses given fair hearing | Yes — H3 (STAR is beneficial) was actively searched for and the counterpoint (SRC08) was included and analyzed |
| Contradictory evidence surfaced | Yes — SRC08 directly contradicts the claim and was included as evidence |
| Confidence calibrated to evidence | Yes — "Very likely" (not "Almost certain") reflects the nuance introduced by SRC08 and the lack of direct STAR-specific peer-reviewed studies |
| Gaps acknowledged | Yes — dyslexia evidence thinness, pediatric-only WM study, paywalled studies all flagged |
Notes: The synthesis deliberately avoided confirming the claim as stated. The conclusion that the claim is "partially correct" rather than "fully correct" reflects genuine nuance in the evidence, not hedging.
Domain 5: Source-Back Verification¶
Rating: Low risk
For each source cited in the assessment, re-read the source and verify that the assessment accurately represents what the source says.
| Source | Claim in Assessment | Source Actually Says | Match? |
|---|---|---|---|
| SRC01 | STAR assumes sequential thinking, problematic for ADHD | "The STAR model assumes sequential thinking, which doesn't work for everyone" | Yes |
| SRC02 | Retrieval failure — "brain can't grab" examples | "my brain just can't grab one" | Yes |
| SRC03 | ADHD impacts Change Order, Mentally Recreate, Encourage Concentration | Source identifies these three as most impacted CI components | Yes |
| SRC06 | Central executive WM deficits d=1.62-2.03 | Source reports d = 1.62-2.03 for domain-general central executive | Yes |
| SRC07 | Autistic candidates scored 3.41 vs 3.91 | Source reports M=3.41 vs M=3.91 | Yes |
| SRC07 | Adaptations helped both groups, autistic more | Source confirms both groups improved, autistic showed greater improvement | Yes |
| SRC08 | STAR provides "much-needed anchor" for neurodivergent candidates | Source uses exact phrase "much-needed anchor" | Yes |
| SRC09 | Dyslexia symptoms worsen under interview stress | "symptoms of dyslexia will worsen for many under interview stress" | Yes |
Discrepancies found: 0
Corrections applied: None needed
Unresolved flags: None
Notes: All source attributions verified against original content. No interpretation drift detected. The characterization of SRC03 as a "forensic interview study" (not employment interview) is accurately noted in the evidence context sections.
Overall Assessment¶
Overall risk of bias: Low risk
The research process was systematic, comprehensive, and fair. The most significant limitation is the reliance on indirect evidence for dyslexia (no peer-reviewed studies directly examining dyslexia and STAR/behavioral interviews) and the use of a forensic interview study (SRC03) and a pediatric WM study (SRC06) as proxies for adult employment STAR interviews. These are reasonable analogies but not direct evidence. The counterpoint evidence (SRC08) was actively sought, included, and influenced the final assessment toward H2 rather than H1.
Researcher Bias Check¶
- Confirmation bias risk: The claim is sympathetic and socially desirable to confirm (supporting neurodivergent individuals). Extra scrutiny was applied to sources supporting the claim. The inclusion of SRC08 (counterpoint) and the conclusion of "partially correct" rather than "fully correct" mitigate this risk.
- Availability bias risk: The evidence base is weighted toward English-language, Western (US/UK/Australia) sources. Interview practices and neurodivergent support may differ in other cultural contexts.
- Anchoring bias risk: The claim itself frames STAR as "problematic," which could anchor the search toward confirming problems. The S05 search (behavioral interview validity) was specifically designed to seek evidence that standard interviews work well for neurodivergent candidates, counteracting this anchor.