Using artificial intelligence for some tasks may actually be contributing to heavier workloads for some physicians, according to new research from Dartmouth.
AI tools can sometimes worsen doctors’ workloads by introducing errors and adding unnecessary details into their written messages to patients, according to Dartmouth’s study, which was presented on Tuesday at the Annual Meeting of the Association for Computational Linguistics in San Diego, California. The report also suggests that physicians may actually spend more time editing AI-generated responses than it would take to write them without using AI tools.
“We find that AI can sound like a doctor but not think like one,” Dartmouth Assistant Professor Of Computer Science Sarah Preum said in a statement.
Preum is the study’s co-corresponding author alongside Parker Seegmiller, a Guarini School of Graduate and Advanced Studies PhD candidate in Preum’s PersistLab, according to Dartmouth’s news release.
The researchers also evaluated physician responses created by several AI platforms, including Claude, Google Gemini, ChatGPT, Llama, Aloe, and Qwen. The research suggests that these AI-generated answers sometimes fail to align with how doctors would actually write information.
Common AI-generated errors included responses that were overly lengthy, left out essential follow-up questions and highlighted medical details that were irrelevant, inaccurate or inconsistent with clinical practice, according to the report.
“There are smaller studies that say, ‘Oh, AI is amazing,’ but we realized there is a gap in the existing literature of a large-scale evaluation of this technology,” Preum said. “We didn’t just want to measure a platform’s accuracy, but whether it actually helps with the workload, which in this case is measured by how much editing the physician is doing.”
The study’s release comes as AI has started becoming more widely adopted by U.S. medical professionals over the years. An American Medical Association (AMA) survey released on March 12 shows that 81% of physicians reported using AI tools in professional settings.
A separate AMA survey released in February 2025 found that 61% of U.S. physicians felt worried that health plans’ AI usage is “increasing prior authorization denials, exacerbating avoidable patient harms and escalating unnecessary waste now and into the future.”
Recent polling also indicates that many Americans are turning to AI for health-related recommendations. A KFF survey released in March found that 32% of adults rely on AI for medical information and advice.
(DCNF)
