SRC04-E01 — Scientific Evaluation of IC Standards¶
Description¶
Marcoci et al. use controlled experiments to evaluate ICD 203 reliability, applying scientific methodology to assess IC standards but not integrating the two into a unified framework.
URL¶
https://www.tandfonline.com/doi/full/10.1080/02684527.2018.1546265
Extract¶
Three assumptions behind ICD 203: (1) tradecraft standards can be employed consistently; (2) tradecraft standards sufficiently capture the key elements of good reasoning; (3) good reasoning leads to more accurate judgments. Individual assessors do not reliably rate analytic product quality using the ICD 203 rubric, unless ratings of three or more assessors are averaged. The results suggest that optimism about ICD 203 may be compromised when evaluations are undertaken by single assessors.
Relevance to Hypotheses¶
| Hypothesis | Relevance | Strength |
|---|---|---|
| H1 — Unified frameworks exist | N/A | — |
| H2 — Domains remain siloed | Supports (uses scientific method to evaluate IC, but as external assessment not integration) | Moderate |
| H3 — Partial bridges exist | Supports (scientific evaluation of IC standards = partial bridge) | Moderate |
Context¶
FACT: The paper applies social science experimental methods to evaluate IC standards. JUDGMENT: This demonstrates that scientific methodology can be applied to IC analytical standards, but the paper's goal is evaluation rather than unification.