SRC01-E01 — Elicit: Structured But Not Analytically Rigorous¶
Description¶
Elicit implements the most structured research workflow among AI tools but lacks the five queried analytical rigor dimensions.
URL¶
Extract¶
Elicit's Systematic Review workflow delivers human-level accuracy in evidence synthesis. Features: structured table-based data extraction (custom columns: "Intervention," "Outcome measures," "Sample size"); semantic search across 138M+ papers; 99.4% extraction accuracy (1,502/1,511 data points correct). Limitation: 39.5% average sensitivity in search — cannot serve as sole search tool. Cannot extract from figures.
Relevance to Hypotheses¶
| Hypothesis | Relevance | Strength |
|---|---|---|
| H1 — Comprehensive frameworks exist | Contradicts (structured but not analytically rigorous) | Strong |
| H2 — No structured features | Contradicts (highly structured extraction) | Strong |
| H3 — Partial features exist | Supports | Strong |
Feature checklist against queried dimensions:
| Dimension | Present? | Notes |
|---|---|---|
| Calibrated probability language | No | No probability calibration |
| Formal bias assessment | No | No risk-of-bias scoring |
| Competing hypotheses | No | No ACH or equivalent |
| Search transparency logging | Partial | Shows papers found but not complete search log |
| Self-audit mechanisms | No | No methodological self-check |
Context¶
FACT: Elicit is the most structured AI research tool evaluated. JUDGMENT: Its strength is in data organization and extraction, not in analytical reasoning or evidence quality assessment.