Latest news

31 articles

RSS ↗
Audit reports lower rare-disease AI benchmark scores after stricter data isolation

Audit reports lower rare-disease AI benchmark scores after stricter data isolation

A preprint on AI systems for rare-disease diagnosis reports that mean Recall@1 fell from 54.3% to 40.3% when strict separation between datasets was enforced. Its authors also report lower accuracy for supervised classifiers evaluated on external cohorts. The findings raise questions about how some benchmarks are measured, but do not by themselves establish these systems’ clinical usefulness or safety.