Q003 — Large OSS Projects and Bespoke CI Tooling — Assessment¶
Contents¶
Direct evidence about the CI tooling decisions of the five named projects was not found through web search. Standard SAST tools have documented fundamental limitations (cannot detect business logic vulnerabilities, authorization bypass, race conditions; 68-75% false positive rates; combined detection of only 38.8%). These limitations provide a plausible rationale for bespoke tooling but the specific evaluation-and-rejection decisions are undocumented in available sources. The 'active rejection' framing is unsupported — the reality is likely passive non-adoption, historical path dependency, or complementary use.
Evidence Synthesis¶
Evidence quality: Limited — Only 2 validated packets remain for this query after verbatim validation (1 dropped). No direct evidence about the specific CI tooling decisions of the five named projects (Linux kernel, PostgreSQL, Node.js, Kubernetes, CPython) was found in fetchable form. The evidence addresses SAST tool limitations in general but not how specific large projects evaluate or reject standard tools. Search results mentioned KernelCI and 0-day bot but fetched content did not yield quotable passages about tool evaluation decisions.
Source agreement: Medium — The two remaining sources agree that standard SAST tools have fundamental limitations (cannot detect business logic vulnerabilities, authorization bypass, race conditions) and that rule coverage is the primary detection lever. But the evidence base is too thin for strong agreement assessment.
Independence: Limited assessment possible. The two sources (arxiv workflow mining study and konvu.com tool comparison) use different methodologies but neither directly addresses the question of large project CI tooling choices.
Probability Assessment¶
Confidence: Low
Evidence Gaps¶
Expected but not found: - Direct documentation of SAST/SCA tool evaluation decisions by Linux kernel, PostgreSQL, Node.js, Kubernetes, or CPython maintainers - Mailing list discussions or meeting notes where standard tools were evaluated and rejected - Comparative analysis of bespoke vs. standard tool effectiveness for large codebases - Interview data from project maintainers about their CI tooling decisions
Unanswered questions: - Do these five projects use any standard SAST/SCA tools alongside their bespoke tooling? - Have any of these projects formally evaluated and rejected standard SAST/SCA tools? - Is the bespoke tooling primarily for testing/quality or specifically for security scanning? - Would a zero-configuration tool (as described in the axiom) address the barriers that lead to bespoke tooling?
Impact on confidence: The gaps severely limit confidence. This query requires primary source research (examining project repositories, mailing lists, and meeting notes) that was not achievable through web search alone. The answer is speculative rather than evidence-based for the specific projects named. The general SAST limitation findings are better supported.