Q001 — Comprehensive CI Gate Adoption Rates in OSS — Assessment¶
Contents¶
The fraction of OSS projects enforcing comprehensive CI gate tooling is unmeasured but almost certainly very small (likely single-digit percentages). Dependency update tools show the highest adoption (Dependabot 69.2%, Renovate 21.0% in GHA workflows), but these primarily update Action versions, not scan for vulnerabilities. SAST adoption is poorly quantified and appears low. Container scanning, type checking enforcement, and coverage threshold enforcement are essentially unquantified for the OSS ecosystem. No study measures co-adoption of all five tool categories simultaneously.
Evidence Synthesis¶
Evidence quality: Limited — No study measuring comprehensive CI gate adoption (all tool categories simultaneously) was found. Individual tool adoption data exists for Dependabot (69.2%) and Renovate (21.0%) from the Mons/Radboud study. SAST adoption is referenced only indirectly through a small-sample study (Zampetti et al., 20 Java projects). No data on container scanning, type checking enforcement, or coverage threshold enforcement as CI checks was found.
Source agreement: Low — Sources address different dimensions of the question with no overlap: the Mons/Radboud paper covers dependency update tools in GHA, the Zampetti reference covers SAST in Travis CI, and the OpenSSF Scorecard includes SAST as a check but does not report adoption rates. No two sources address the same sub-question with comparable methodology.
Independence: The available sources are independent but cover different dimensions of the question, making cross-validation impossible.
Probability Assessment¶
Confidence: Low
Evidence Gaps¶
Expected but not found: - No study measuring co-adoption of multiple CI security tool categories was found. - No GitHub Octoverse or GitHub data reporting CodeQL adoption rates was found. - No data on container image scanning adoption in CI was found. - No data on type checking enforcement (mypy --strict, TypeScript strict) as CI gates was found. - No data on code coverage threshold enforcement as a required CI check was found. - No data stratifying CI tool adoption by governance model (foundation vs. community) was found.
Unanswered questions: - What fraction of GitHub projects have CodeQL or any SAST tool as a required check (not just present in workflow)? - What fraction of projects use container scanning in CI? - How does comprehensive CI gate adoption differ between CNCF/Apache projects and community-maintained projects? - Does strict type checking enforcement correlate with project maturity?
Impact on confidence: The extensive gaps are the primary reason for Low confidence. Almost every sub-question lacks direct empirical data. The answer is largely based on inference from partial evidence and the meaningful absence of comprehensive adoption studies. This absence itself supports the conclusion that comprehensive CI gate suites are not standard practice.