R0007/2026-03-19/C004 — Assessment

Research	R0007 — AI Made Everyone Faster
Run	2026-03-19
Claim	C004

BLUF

Large individual variation in software engineering productivity is one of the most consistently replicated findings in the field. The 2023 study confirms a 2.44x ratio between top and bottom halves. The claim is substantially correct.

Probability

Rating: Very likely (80-95%)

Confidence in assessment: Medium-High

Confidence rationale: The pattern of large variation is beyond dispute. The specific 2.4x ratio and log-normal characterization are supported by the 2023 study but represent one study's findings.

Reasoning Chain

1. Sackman, Erikson, and Grant (1968) found ratios of approximately 20:1 in coding time, 25:1 in debugging time, 5:1 in program size, and 10:1 in execution speed among professional programmers [SRC01-E01, High, High].
2. Even after correcting for methodological flaws in Sackman's study, a 10:1 difference remained [SRC01-E01, High, High].
3. The general finding of "order-of-magnitude differences among programmers" has been confirmed by subsequent studies spanning decades [SRC01-E01, High, High].
4. A 2023 study in Information and Software Technology found a coefficient of variance of 0.55 with the top 50% having 2.44x higher average productivity than the bottom 50% [SRC01-E02, High, High].
5. The 2023 study examined distribution characteristics, consistent with the claim about log-normal distributions [SRC01-E02, Medium, High].
6. JUDGMENT: The claim accurately characterizes a robust and well-replicated finding.

Evidence Base Summary

Source	Description	Reliability	Relevance	Key Finding
SRC01	Multiple studies on programmer variation	High	High	Large variation confirmed; 2.44x ratio confirmed

Collection Synthesis

Dimension	Assessment
Evidence quality	Robust — decades of replicated findings
Source agreement	High — all studies confirm large variation
Source independence	High — independent studies across decades
Outliers	None significant

Detail

The research literature spanning 55+ years consistently finds large individual differences in programmer productivity. The specific magnitude varies by study and measurement approach, but the existence of substantial variation is not in dispute. The 2023 study's 2.44x ratio is more conservative than earlier estimates (which found 10:1 or even 20:1 ratios), likely reflecting improved methodology.

Gaps

Missing Evidence	Impact on Assessment
Full text of Oliveira/Duarte de Oliveira 2023 paper	Could not verify distribution type claims directly
Comprehensive meta-analysis of all studies 1968-2023	Would strengthen "every major study" claim

Researcher Bias Check

Declared biases: None applicable.

Influence assessment: The "10x programmer" narrative has cultural significance in software engineering. The 2.44x figure from the 2023 study is notably more conservative than popular claims, suggesting the research is not inflating the finding.

Cross-References

Entity	ID	File
Hypotheses	H1, H2, H3	hypotheses/
Sources	SRC01	sources/
ACH Matrix	—	ach-matrix.md
Self-Audit	—	self-audit.md