C001 — CI Adoption Rate in Open Source Projects — Assessment¶
Contents¶
The 40% figure traces to Decan et al.'s finding of 43.9% GitHub Actions adoption in 68K repositories by January 2022. This measures only one CI platform among active repos with 300+ stars and commits. Total CI adoption across all platforms is likely higher, and may exceed 50% for active projects in well-established ecosystems (npm repos showed >50% CI adoption by May 2021). The claim's directional value remains: a substantial portion of open source projects lack CI. However, the specific 40% figure should be cited with its source and limitations.
Evidence Synthesis¶
Evidence quality: Medium — Two academic papers provide empirical CI adoption figures, but they measure different populations (68K general GitHub repos vs. 91K npm repos vs. 5K most popular repos) and different tools (GitHub Actions only vs. all CI). No study directly measures 'all CI in all open source projects' as the claim implies. The studies are peer-reviewed and methodologically sound within their scopes, but the extrapolation to the claim's broader framing is imprecise.
Source agreement: Medium — Sources agree directionally that CI adoption is in the 30-50% range for GitHub-hosted open source projects, but disagree on the precise figure: Decan et al. report 43.9% GHA adoption in 68K repos, Kinsman et al. report 30% GHA adoption in the 5K most popular repos, and Golzadeh et al. report >50% CI adoption in npm repos. The variation depends on population definition and which CI tools are counted.
Independence: Partially independent. Decan et al. and Kinsman et al. conducted separate studies with different samples. However, both mine GitHub metadata using similar methods, and Decan et al. cite Golzadeh et al., indicating awareness of each other's work. The findings converge through independent analysis but draw from the same underlying data source (GitHub).
Outliers¶
- https://link.springer.com/article/10.1007/s10664-023-10369-w: Reports 30% GHA adoption among the 5,000 most popular repos, lower than the 43.9% from Decan et al. — Kinsman et al. measured only GitHub Actions (not all CI tools) and only among the most popular repos. The population and tool scope differences explain the divergence. Interestingly, the most popular repos may use CI tools other than GHA (e.g., Jenkins, self-hosted), which would not be detected by looking only at GHA workflows.
Probability Assessment¶
- C001-H1: Likely (55-70%)
- The 43.9% figure from Decan et al. for GHA adoption closely matches the claimed ~40%. However, GHA is not the only CI system — projects using Travis CI, CircleCI, Jenkins, etc. would push total CI adoption higher. The claim says 'CI at all,' which should include all systems. For GHA specifically, 40% is approximately correct; for all CI combined, the figure is likely higher (possibly 50-60% for active projects).
- C001-H2: Roughly even chance (45-55%)
- Golzadeh et al. found >50% CI adoption in npm repos by May 2021, and GHA adoption has continued to grow since. For active, maintained projects in well-established ecosystems, CI adoption may be significantly above 40%. But this depends heavily on population definition — including all GitHub repos (many inactive or trivial) would lower the figure.
- C001-H3: Likely (60-75%)
- The strongest nuance: the 43.9% figure comes from repos with 300+ stars and 300+ commits — already a filtered population. For all GitHub repositories (including inactive, personal, and trivial projects), CI adoption would be lower. For active, maintained projects, it would be higher. The claim's framing of 'open source projects' without specifying population makes this ambiguity inherent.
- C001-H4: Likely (60-75%)
- Strong evidence that GHA became the dominant CI solution within 18 months and that npm repos exceeded 50% CI adoption by May 2021. The 40% figure (from Jan 2022 data for GHA alone) is likely already outdated as of 2026. GHA adoption has continued to grow, and the overall CI adoption rate across active projects is likely above 40% now. Verdict: The claim is likely approximately correct (55-70%) for the specific measurement it derives from (GitHub Actions adoption in active repos circa 2022), but is likely becoming outdated. Total CI adoption across all CI systems for active projects is probably above 40% and may exceed 50%. The claim's value is directional — it correctly identifies that a substantial portion of open source projects lack CI.
Evidence Gaps¶
Expected but not found: - No study measuring total CI adoption across all CI platforms (GHA + Travis + CircleCI + Jenkins + others) for a representative sample of open source projects - No study measuring CI adoption trends over time (longitudinal) to confirm whether 40% is current or outdated - No study stratifying CI adoption by project activity level to quantify the active vs. inactive project effect
Unanswered questions: - What is the total CI adoption rate when counting all CI platforms, not just GitHub Actions? - How has CI adoption changed between January 2022 (the Decan et al. measurement) and 2026? - What is the CI adoption rate specifically among projects that accept external contributions (the population most relevant to the article's argument)?
Impact on confidence: The gaps moderately reduce confidence. The 43.9% GHA-only figure is likely a lower bound for total CI adoption, meaning the claim's 40% figure may significantly understate actual adoption. However, the directional point (substantial fraction of projects lack CI) remains robust.