C002 — OpenSSF Scorecard 5.4 Average: Wrong Population — Unlikely as stated (20-45%)
Contents
Summary
Claim: The average OpenSSF Scorecard score across the top one million critical open source projects is 5.4 out of 10.
Bottom Line: The claim misattributes the 5.4 average to 'the top one million critical open source projects' when it actually comes from a Chainguard analysis of approximately 1,511 Wolfi upstream repositories. The 5.4 figure may be a reasonable rough estimate of typical Scorecard scores across open source — Chainguard notes scores in the 4-6 range are 'typical' based on past research — but the specific population attribution is incorrect. The researcher should cite the Chainguard analysis directly or verify against the OpenSSF BigQuery dataset.
Results
| Artifact |
Description |
| Input |
Original text, clarification, scope, vocabulary |
| Assessment |
Evidence synthesis, probability assessment, gaps |
| Self-Audit |
Process audit across 4 ROBIS domains |
| Reading List |
Prioritized source list |
Hypotheses
| ID |
Label |
Status |
| H1 |
|
— |
| H2 |
|
— |
| H3 |
|
— |
Searches
| ID |
Target |
Returned |
Selected |
| S01 |
H1 |
? |
? |
| S02 |
H1 |
? |
? |
| S03 |
H2 |
? |
? |
| S04 |
H3 |
? |
? |
Sources
| ID |
Title |
Reliability |
Relevance |
| SRC001 |
https://mir.cs.illinois.edu/marinov/publications/HiltonETAL1 |
High |
High |
| SRC002 |
https://decan.lexpage.net/files/SANER-2022a.pdf |
High |
High |
| SRC003 |
https://www.sciencedirect.com/science/article/abs/pii/S01641 |
High |
High |
| SRC004 |
https://blog.jetbrains.com/teamcity/2026/03/best-ci-tools/ |
Medium |
Medium |
| SRC005 |
https://www.chainguard.dev/unchained/wolfis-upstream-securit |
Medium |
High |
| SRC006 |
https://www.scworld.com/news/open-source-vulnerabilities-per |
Medium |
High |
| SRC007 |
https://www.sonatype.com/state-of-the-software-supply-chain/ |
Medium |
High |
| SRC008 |
https://arxiv.org/html/2409.07669v2 |
High |
High |
| SRC009 |
https://www.pixee.ai/blog/sast-false-positives-reduction |
Medium |
High |
| SRC010 |
https://embeddedbits.org/how-is-the-linux-kernel-tested-embe |
Medium |
High |
| SRC011 |
https://www.kernel.org/doc/html/v6.5/dev-tools/testing-overv |
High |
High |
| SRC012 |
https://arxiv.org/html/2605.07900v1 |
High |
High |
| SRC013 |
https://www.nist.gov/news-events/news/2026/04/nist-updates-n |
High |
High |
| SRC014 |
https://www.moderne.ai/blog/security-dependency-updates-unma |
Medium |
High |
| SRC015 |
https://contribute.cncf.io/resources/services/hosted-tools/ |
High |
Medium |
| SRC016 |
https://www.linuxfoundation.org/research/maintainer-perspect |
High |
High |
Evidence Snapshot
| Dimension |
Rating |
| Evidence quality |
Medium |
| Source agreement |
Medium |
Revisit Triggers
- [data_update] OpenSSF publishes official aggregate statistics (mean, median, distribution) for the Scorecard scores across the 1M critical projects scan.
- [study] An academic study analyzes the OpenSSF BigQuery public dataset and reports aggregate Scorecard statistics.
- [data_update] OpenSSF Scorecard v5.0.0 scoring methodology is fully rolled out and new aggregate statistics are published.
- [study] Chainguard or another organization repeats the analysis on a larger, more representative population.
← Back to run overview