SRC012 — https://arxiv.org/html/2605.07900v1¶
Contents¶
Metadata¶
| Field | Value |
|---|---|
| URL | https://arxiv.org/html/2605.07900v1 |
| Authors | Academic researchers (arXiv preprint, May 2026) |
| Date | May 2026 |
Content Summary¶
Largest academic study of CodeQL analyzing 3,993 CVEs from 1,622 repositories across 114 CodeQL versions (~20 billion LOC analyzed). Finds CodeQL detected 171 CVEs total, with 83 detectable before the fix was applied. GitHub reports storing CodeQL databases for 200,000+ repositories. 21 CVEs were lost between versions, showing detection instability.
Reliability: High¶
Large-scale academic study with transparent methodology and reproducible analysis.
Relevance: High¶
Provides CodeQL adoption data (200K repos) and effectiveness analysis — relevant to Q001 and Q003.
Bias Assessment¶
| Domain | Rating | Rationale |
|---|---|---|
| Missing Data | Low risk | Comprehensive longitudinal analysis with clear inclusion/exclusion criteria. |
| Measurement | Low risk | Automated analysis with reproducible methodology across 114 tool versions. |
| Selective Reporting | Low risk | Reports both positive (detections) and negative (instability, lost detections) findings. |
| Randomization | N/A | Observational study. |
| Protocol Deviation | N/A | Not an RCT. |
| Conflict Of Interest | Low risk | Academic researchers with no disclosed commercial interest. |