Clinical AI Hits 65% Accuracy. Clinicians Still Obeyed Wrong Answers. That Is the Problem.
A clinical-and-blood-model AI tool improved lung cancer disease-control prediction from 57% to 65% across 2,396 patients. Clinicians in the study also accepted incorrect AI suggestions, which is the finding nobody is bold-facing. Separately, ARPA-H announced ADVOCATE, a four-year $62.7 million program for FDA-authorized agentic AI in cardiovascular care involving Atman Health, Tempus AI, and Updoc.
This demonstrates 'automation bias in clinical decision support.' When an AI system is mostly right, humans begin deferring to it even when it is wrong. The 57% to 65% improvement is real. The dangerous finding is that physicians accepted incorrect suggestions from a system they had learned to generally trust. Accuracy gains do not eliminate this risk. They amplify it.
The clinical study involved 2,396 patients with a lung cancer prediction tool moving accuracy from 57% to 65%. ARPA-H announced the $62.7 million ADVOCATE program on September 9 with clinical agents from Atman Health, Tempus AI, and Updoc plus Stanford University.
- Open any AI assistant and ask it for medical information about a condition you actually know well, perhaps something you or a family member have experienced.
- Read the response carefully and identify at least one detail you can confirm from your own knowledge and at least one you cannot verify.
- Ask a follow-up question probing the unverifiable detail. Watch whether the AI doubles down or corrects itself. The lesson: your job is to be the skeptical clinician, not the one who accepts incorrect suggestions.