R0043/2026-04-01/Q002/SRC04/E01¶
FDA requirements for AI medical device accuracy and human factors evaluation
URL: https://www.fda.gov/media/184856/download
Extract¶
FDA requirements for AI-enabled devices include:
- Performance tracking: manufacturers must "track accuracy metrics, identify performance drift, and detect unexpected behavior"
- Human factors: importance of evaluating "human in the loop" performance — "how a human interprets the AI and ultimately makes clinical decisions"
- Bias testing: representative data requirements across patient populations, though "the Agency does not provide any guidance on specific methods or thresholds for bias testing"
- Transparency: "appropriate level of transparency (clarity) of the output and the algorithm aimed at users"
JUDGMENT: The FDA addresses output accuracy and human-AI interaction quality but does not identify sycophancy or agreement-seeking as a specific risk category. The "human in the loop" evaluation could theoretically detect sycophancy effects (if clinicians over-accept AI recommendations), but the FDA does not require testing for this specific failure mode. The healthcare concept of "acquiescence" — AI confirming clinician hypotheses instead of challenging them — is not addressed.
Relevance to Hypotheses¶
| Hypothesis | Relationship | Strength |
|---|---|---|
| H1 | Partially supports | Accuracy and human factors requirements exist |
| H2 | Supports | No sycophancy-specific requirements |
| H3 | Supports | Indirect coverage through accuracy and human factors |
Context¶
The FDA's approach reflects a focus on clinical outcomes (accuracy, bias) rather than model behaviors (sycophancy). This means a sycophantic AI medical device could potentially pass FDA review if its outputs were accurate on average, even if it reinforced incorrect clinician hypotheses in specific cases.