Q001 — Security Tool Adoption Rates in OSS CI Pipelines — Assessment¶
Contents¶
Security tool adoption in open source CI is highly uneven: SCA (Dependabot) at ~69%, SAST (CodeQL) at 10-30%. Container scanning, type checking, and coverage threshold data are unmeasured. The primary adoption driver is platform integration/defaults, not security awareness. Comprehensive gate adoption (all categories simultaneously) is extremely rare and unquantified.
Evidence Synthesis¶
Evidence quality: Medium — Strong data on two categories (SAST at 10-30% via CodeQL longitudinal study, SCA at 69.2% Dependabot adoption via workflow mining). No data found on container scanning, type checking, or code coverage threshold enforcement rates. The evidence is concentrated in SAST and SCA categories, leaving significant blind spots.
Source agreement: Medium — Sources agree that SAST adoption is low (10-30%) and SCA/Dependabot adoption is high (69.2%). They agree that false positive rates are high for SAST (68-75%) and detection rates are low (11-38% for real-world vulnerabilities). No disagreement was found, but the agreement is limited to the two categories where data exists.
Independence: Moderately independent. The CodeQL adoption study (arxiv) and the Semgrep/CodeQL comparison (konvu.com) use different methodologies — longitudinal GitHub mining vs. tool benchmarking. The Dependabot adoption figure comes from workflow mining. These represent three different research approaches converging on consistent findings.
Probability Assessment¶
Confidence: Medium
Evidence Gaps¶
Expected but not found: - Container scanning tool adoption rates (Trivy, Grype, Snyk Container) in open source CI - Type checking enforcement (mypy, TypeScript strict) as CI gate adoption rates - Code coverage threshold enforcement rates in open source CI - Comprehensive gate adoption rate — projects using SAST + SCA + container scanning + coverage simultaneously - Stratification of tool adoption by project size, language, or governance model
Unanswered questions: - What fraction of projects enforce Dependabot as a merge gate vs. just receiving passive alerts? - How does security tool adoption vary by programming language ecosystem? - What is the adoption rate of comprehensive security gate suites (all tool categories together)? - Does foundation backing (CNCF, Apache, Linux Foundation) measurably increase security tool adoption?
Impact on confidence: The gaps significantly limit the answer's comprehensiveness. We can characterize SAST and SCA adoption well, but three of the five tool categories asked about (container scanning, type checking, coverage thresholds) have no available data. The answer is Medium confidence for the categories covered but Low confidence for the overall picture.