The ARISE network, backed by Stanford and Harvard, has published its first annual synthesis of clinical AI: clear progress in reasoning, but evaluation and integration into care remain the weak points.
ARISE, a research network associating Stanford and Harvard, publishes the first annual edition of its State of Clinical AI Report. The document is intended to synthesize the advances, available evidence and persistent difficulties of artificial intelligence applied to care, covering both the performance of models, their evaluation, their insertion into hospital workflows and tools intended directly for patients.
The central finding is that of a "saw-toothed" ability: models have made significant progress in prediction and clinical reasoning, but remain poor at recognizing their own areas of uncertainty—precisely the skill that matters most in a medical decision-making context.
Evaluating differently
The authors — Peter Brodeur, Ethan Goh, Adam Rodman, and Jonathan Chen — call for moving away from question-and-answer benchmarks to evaluate systems on real, unstructured, multi-round data. They also emphasize an often-overlooked point: the design of the interaction between clinician and machine weighs as much as the raw capabilities of the model. For patient-facing tools, the report recommends making prospective randomized trials the standard of evaluation, not the exception.
Read the full article on the original source
Go to the source →