Run Overview¶
Contents¶
- Axioms
- Claims
- C001 — CI Adoption Rate in Open Source Projects — The claim is likely approximately correct (55-70%) for the specific measurement it derives from (GitHub Actions adoption in active repos circa 2022), but is likely becoming outdated.
- C002 — OpenSSF Scorecard Average for Critical Projects — The claim is roughly even chance to likely correct in its specific form (40-55%), but directionally correct with higher confidence (75-85%). The 5.4 figure comes from Chainguard's Wolfi analysis, not OpenSSF's 1 million critical projects.
- C003 — OSSRA Vulnerability Prevalence and Growth Rate — The claim is almost certainly correct as quoted (95-99%) but very likely misleading without context (80-90%).
- C004 — Fix Availability for Vulnerable Open Source Components — The claim is almost certainly correct (95-99%) as stated, but very likely misleading (80-90%) because fix availability does not equal fix practicality.
- Queries
- Collection Analysis
Axioms¶
A001¶
This research is conducted in the context of understanding open source CI/CD practices to evaluate whether a comprehensive, zero-configuration auditing stack (enforcing SAST, SCA, container scanning, strict type checking, SBOM generation, and 100% code coverage as CI gates) represents meaningful differentiation in the open source ecosystem.
Claims¶
C001 — CI Adoption Rate in Open Source Projects — The claim is likely approximately correct (55-70%) for the specific measurement it derives from (GitHub Actions adoption in active repos circa 2022), but is likely becoming outdated.¶
Claim: Only approximately 40% of open source projects use continuous integration at all.
Verdict: The 40% figure traces to Decan et al.'s finding of 43.9% GitHub Actions adoption in 68K repositories by January 2022. This measures only one CI platform among active repos with 300+ stars and commits. Total CI adoption across all platforms is likely higher, and may exceed 50% for active projects in well-established ecosystems (npm repos showed >50% CI adoption by May 2021). The claim's directional value remains: a substantial portion of open source projects lack CI. However, the specific 40% figure should be cited with its source and limitations.
| Hypothesis | Status |
|---|---|
| H1: ** | — |
| H2: ** | — |
| H3: ** | — |
| H4: ** | — |
Sources: 13 · Searches: 5
C002 — OpenSSF Scorecard Average for Critical Projects — The claim is roughly even chance to likely correct in its specific form (40-55%), but directionally correct with higher confidence (75-85%). The 5.4 figure comes from Chainguard's Wolfi analysis, not OpenSSF's 1 million critical projects.¶
Claim: The average OpenSSF Scorecard score across the top one million critical open source projects is 5.4 out of 10.
Verdict: The 5.4 average Scorecard score is confirmed from Chainguard's analysis of 1,500 Wolfi upstream repos, not from OpenSSF's full 1 million critical projects. The 1 million project scanning program exists and publishes results to BigQuery. The 0-10 scoring methodology works as described. The specific 5.4 figure for the full population is unverified, though Chainguard notes that scores in the 4-6 range are 'typical' for open source projects. The popularity-score correlation suggests the critical projects set (selected for importance/popularity) might score higher than 5.4.
| Hypothesis | Status |
|---|---|
| H1: ** | — |
| H2: ** | — |
| H3: ** | — |
Sources: 13 · Searches: 4
C003 — OSSRA Vulnerability Prevalence and Growth Rate — The claim is almost certainly correct as quoted (95-99%) but very likely misleading without context (80-90%).¶
Claim: 87% of audited codebases contain at least one known open source vulnerability, with an average of 581 vulnerabilities per codebase representing a 107% year-over-year increase.
Verdict: All three figures (87%, 581, 107%) are confirmed verbatim from the 2026 OSSRA report. However, the figures require significant caveats: the sample is 947 commercial codebases from M&A due diligence (non-representative), the median is only 78 vulnerabilities (vs. 581 mean, showing extreme right-skew), an expert characterizes the 581 as 'largely transitive dependency sprawl,' and the 107% increase is partly attributable to CVE database growth (Linux Kernel CNA adding 5,000+ CVEs in 2024). The researcher should cite these figures accurately but present the caveats to avoid misleading readers.
| Hypothesis | Status |
|---|---|
| H1: ** | — |
| H2: ** | — |
| H3: ** | — |
| H4: ** | — |
Sources: 13 · Searches: 4
C004 — Fix Availability for Vulnerable Open Source Components — The claim is almost certainly correct (95-99%) as stated, but very likely misleading (80-90%) because fix availability does not equal fix practicality.¶
Claim: 95% of vulnerable open source components consumed by downstream projects have a known fix available that has not been applied.
Verdict: Sonatype's report confirms that 94.9% of vulnerable components have a newer, non-vulnerable version available. However, Moderne's independent analysis of 1,307 Java dependencies shows that only 30% of fixes are simple patch bumps, 50% require minor version upgrades, and 10% require major version upgrades. Combined with the finding that 80% of enterprise dependencies remain unmanaged and 13% of Log4j downloads are still vulnerable 3+ years after a non-breaking fix, the 95% figure significantly overstates how actionable the available fixes are.
| Hypothesis | Status |
|---|---|
| H1: ** | — |
| H2: ** | — |
| H3: ** | — |
Sources: 13 · Searches: 4
Queries¶
Q001 — Security Tool Adoption Rates in OSS CI Pipelines — Medium¶
Query: What fraction of open source projects on GitHub enforce comprehensive CI gate tooling — specifically static analysis (SAST), dependency/vulnerability scanning (SCA), container image scanning, strict type checking, and code coverage thresholds — as required checks on every commit or pull request? How does adoption vary by project size, programming language ecosystem, and whether the project is backed by a foundation or company versus community-maintained?
Answer: Security tool adoption in open source CI is highly uneven: SCA (Dependabot) at ~69%, SAST (CodeQL) at 10-30%. Container scanning, type checking, and coverage threshold data are unmeasured. The primary adoption driver is platform integration/defaults, not security awareness. Comprehensive gate adoption (all categories simultaneously) is extremely rare and unquantified.
Confidence: Medium · Sources: 13 · Searches: 5
Q002 — Barriers to Security Tooling Adoption in OSS — Medium¶
Query: What are the primary barriers to adoption of comprehensive security scanning pipelines in open source projects, and what evidence exists for 'security fatigue' — the phenomenon where initial post-incident surges in security tooling adoption (e.g., after Log4Shell) reverse over time? What role do setup complexity, false-positive rates, and maintenance burden play in driving abandonment?
Answer: Primary barriers are: lack of awareness, supply chain mistrust, high false positive rates (68-75% for SAST), setup complexity, and the perception that security features are unnecessary. Direct evidence for 'security fatigue' (spike-and-decline) is absent. The pattern is better characterized as persistent non-adoption. The Log4j case shows 13% still vulnerable 3+ years later despite a non-breaking fix, suggesting inertia rather than reversal.
Confidence: Medium · Sources: 13 · Searches: 5
Q003 — Large OSS Projects and Bespoke CI Tooling — Low¶
Query: Among well-resourced open source projects (Linux kernel, PostgreSQL, Node.js, Kubernetes, Python CPython), what CI and quality enforcement tooling do they actually run, and why do they build bespoke tooling rather than adopting standard commercial or open-source SAST/SCA scanners like CodeQL, Semgrep, and Trivy? Is there evidence that these tools are considered inadequate, too noisy, or inapplicable to large-scale projects?
Answer: Direct evidence about the CI tooling decisions of the five named projects was not found through web search. Standard SAST tools have documented fundamental limitations (cannot detect business logic vulnerabilities, authorization bypass, race conditions; 68-75% false positive rates; combined detection of only 38.8%). These limitations provide a plausible rationale for bespoke tooling but the specific evaluation-and-rejection decisions are undocumented in available sources. The 'active rejection' framing is unsupported — the reality is likely passive non-adoption, historical path dependency, or complementary use.
Confidence: Low · Sources: 13 · Searches: 5
Collection Analysis¶
Collection Statistics¶
| Metric | Value |
|---|---|
| Items investigated | 8 |
| Sources scored | 13 |
| Searches executed | 32 |