C001 — Self-Audit¶


Research	R0058 — Candidate evidence test
Run	2026-04-03
Claim	C001

ROBIS 4-Domain Audit¶

Domain 1: Eligibility Criteria¶

Rating: Low risk

Criterion	Assessment
Criteria defined before searching	Yes — the claim specifies two testable numerical assertions (83% homophily, 1% bridging) against which evidence was evaluated
Criteria consistent throughout	Yes — no shifting of what counts as relevant evidence after seeing search results
Scope appropriate to claim	Yes — bibliometric and network analysis sources are the appropriate evidence type for claims about research community structure

Notes: The eligibility criteria were straightforward given the numerical specificity of the claim. The primary challenge was distinguishing between evidence that addresses the specific safety-ethics divide versus broader AI ethics community structure.

Domain 2: Search Comprehensiveness¶

Rating: Some concerns

Criterion	Assessment
Multiple search strategies used	Yes — 5 searches: candidate evidence fetch, direct search, contradicting evidence search, independent bibliometrics, bridge workshops
Searches designed to test each hypothesis	Yes — S03 specifically sought contradicting evidence; S04 sought independent corroboration; S05 sought qualitative context
All results dispositioned	Yes — 61 results returned across all searches; all dispositioned as selected or rejected with rationale
Source diversity achieved	Some concerns — 3 of 4 scored sources are academic papers/preprints; the community is small and Roytburg & Miller dominates search results for this specific topic

Notes: The primary concern is that this is a very specific claim sourced from a single study. No replication or contradicting quantitative study was found. The search space is inherently narrow because bibliometric analysis of the AI safety-ethics divide is a niche topic with limited literature.

Domain 3: Evaluation Consistency¶

Rating: Low risk

Criterion	Assessment
All sources scored using same framework	Yes — all 4 sources scored on reliability, relevance, and 6 bias domains
Evidence typed consistently	Yes — all evidence extracts classified as Statistical, Analytical, or Reported
ACH matrix applied	Yes — 6 evidence extracts evaluated against 3 hypotheses
Diagnosticity analysis performed	Yes — most and least diagnostic evidence identified with reasoning

Notes: The candidate evidence (SRC01) was scored on equal terms with search-discovered sources. Its reliability was rated Medium-High (not High) due to preprint status, despite being the researcher-provided source.

Domain 4: Synthesis Fairness¶

Rating: Low risk

Criterion	Assessment
All hypotheses given fair hearing	Yes — H3 (materially wrong) was pursued through dedicated contradicting-evidence searches
Contradictory evidence surfaced	Yes — SRC02-E01 (94% connected component) was identified as potentially contradicting the divide narrative, though it measures a different network
Confidence calibrated to evidence	Yes — assessment is "Likely" (not "Very likely" or higher) due to single-study dependence and the partial-correctness finding
Gaps acknowledged	Yes — absence of replication studies and venue limitations documented

Notes: The researcher's conflict of interest (author of blog where this claim will appear) was considered. The finding that the claim is partially incorrect demonstrates the assessment was not biased toward confirmation.

Domain 5: Source-Back Verification¶

Rating: Low risk

For each source cited in the assessment, re-read the source and verify that the assessment accurately represents what the source says.

Source	Claim in Assessment	Source Actually Says	Match?
SRC01	83.1% global homophily reported in Figure 1 caption	Figure 1 caption: "densely insular (83.1% homophily)"	Yes
SRC01	Top 1% of authors by degree control 58.0% of cross-disciplinary paths	Paper states: "Top 1% of authors by degree control 58.0% of cross-disciplinary paths"	Yes
SRC01	Mixed papers represent 9.5% of filtered corpus	Paper states: "Mixed papers represent only 9.5% of filtered corpus"	Yes
SRC01	6,442 papers analyzed	Paper states: "Total papers analyzed: 6,442"	Yes
SRC02	94% of institutions in largest connected component	Paper states: "94% of the nodes belong to the largest connected component"	Yes
SRC03	Safety underrepresented in AIES/FAccT	Paper states: "system safety does not appear as a prominent measure"	Yes
SRC04	Workshop identifies "epistemic fractures"	Workshop page uses the phrase "epistemic fractures"	Yes

Discrepancies found: 0

Corrections applied: None needed

Unresolved flags: None

Notes: All numerical claims and characterizations in the assessment match the source material. The critical finding — that the "1% bridging" is a mischaracterization — was verified by re-reading the source's actual statement about network degree concentration versus the claim's paraphrase about bridging authorship rates.

Overall Assessment¶

Overall risk of bias: Low risk

The research process was conducted systematically with appropriate scope. The main limitation is the narrow evidence base — this is a niche topic with a single primary quantitative study. The finding that the claim is partially incorrect (rather than fully confirmed) demonstrates that the assessment resisted confirmation bias from the researcher-provided candidate evidence.

Researcher Bias Check¶

Confirmation bias risk (from candidate evidence): The researcher provided the primary source as candidate evidence, which could anchor the analysis toward confirmation. MITIGATED by: the finding that the "1% bridging" component is a mischaracterization demonstrates the analysis was not simply confirming what the researcher expected.
Pro-infrastructure bias: The researcher's declared bias toward structural/systematic explanations could make them predisposed to accept claims about structural community divides. MITIGATED by: the analysis focused on verifying specific numbers rather than the narrative framing.
Conflict of interest: The researcher will publish this claim in a blog article, creating an incentive for the claim to be confirmed. MITIGATED by: the partial-correctness finding provides the researcher with a correction to make, which is more valuable than false confirmation.