Dartmouth Study Reveals AI in Healthcare Can Increase Physician Workload With Flawed Responses

Study shows how flawed AI responses increase physician workloads

Artificial intelligence is spreading rapidly in health care, with the goal of streamlining critical but onerous clerical tasks such as note-taking and charting so that physicians and nurses can devote more time to patients. But even when AI can free up doctors to correspond with patients, it may fall short in helping them do it by introducing errors and extraneous details into their messages, according to a new Dartmouth study presented at the 2026 Annual Meeting of the Association for Computational Linguistics and published in the conference proceedings. The result is that physicians may spend more time editing responses than it would've taken to write them, the researchers report. "We find that AI can sound like a doctor but not think like one," says Sarah Preum, an assistant professor of computer science and the study's co-corresponding author with Parker Seegmiller, a graduate researcher in Preum's PersistLab at Dartmouth. The researchers conducted the first large-scale study of an online patient portal that uses AI to draft responses from physicians to patients. The team developed a tool that compares AI-generated replies to a dataset of real responses they developed with health care professionals from Dartmouth Health. They then analyzed 146,000 conversations between 10,105 patients and their primary care physicians at the large rural health system. The study was approved by the Dartmouth Health Institutional Review Board, and the team used the required methods to protect patient privacy, including anonymizing data as needed. The researchers also used their tool to evaluate physician responses drafted by Claude, Gemini, and ChatGPT, as well as the three smaller commercial platforms, Llama, Aloe, and Qwen. We find that AI can sound like a doctor but not think like one." Sarah Preum, corresponding author and assistant professor of computer science The team reports that AI-generated answers frequently misalign with what clinicians would actually write. This includes automated responses that are too long, don't ask follow-up questions, and use irrelevant or inaccurate medical details. "There are smaller studies that say, 'Oh, AI is amazing,' but we realized there is a gap in the existing literature of a large-scale evaluation of this technology," Preum says. "We didn't just want to measure a platform's accuracy, but whether it actually helps with the workload, which in this case is measured by how much editing the physician is doing." For example, the portal's AI suggested telling a 32-year-old woman who is taking an acid reflux drug and was concerned about constant nausea that the medication might take some adjustment in diet. A physician replaced that by asking if there's any chance she was pregnant. Even little changes can add up over hundreds or thousands of messages, Preum says. "You don't want to integrate large language models into the workflow and just shift the bottleneck so that doctors are devoting their cognitive energy to playing AI janitor and fixing mistakes," Preum says. "But if we're not careful, that's a likely outcome." The researchers show, however, that adapting AI to how individual physicians communicate can improve accuracy by 33% and reduce editing by 26%. "If message generation is really efficient and high quality, if it asks the right things, then it really has potential to improve efficiency," says co-author Tim Burdick, an associate professor of community and family medicine in Dartmouth's Geisel School of Medicine and a family medicine physician at Dartmouth Health. "I don't foresee a time when the portal can respond to a patient without a clinician editing it first. But as we make the models better, we'll be able to address portal messages much more quickly and with less mental energy," Burdick says. The study shows that there are such things as "good" AI responses and provides a framework for implementing them into patient-physician portals, Preum says. These platforms are increasingly common among large health care systems and often customized, she says. "That took us a long time to figure out, but if you're trying to measure how effective this technology is, you need to define what a good response is," she says. "We can only improve what we can measure and objectively evaluate." The researchers created a technique called TADPOLE-or Thematic Agentic Direct Preference Optimization for Learning Enhancement-that trains AI platforms using the hybrid model they constructed from physician- and AI-generated responses. They plugged TADPOLE into the six commercial LLMs and found that drafted responses better matched physicians' standards for precision and information quality. "That could save a busy clinician an hour or two of work a day," Burdick says. Doctors and nurses today are inundated with messages from patients and caregivers who can write them online anytime, he says. An ongoing project between Burdick and the Preum Lab called PortalPal aims to streamline patient portals using AI, including by automating some steps in following up with patients to get more information. We're still nowhere near the point of having clinicians removed from the workflow." Tim Burdick, co-author and associate professor of community and family medicine Physicians who Burdick works with say that AI-generated drafts save about 25% of their time on shorter messages. "It's easier to make small edits to an LLM-generated message than it is to write it from scratch," he says. But longer drafts can include information that is not correct or accurate. "If you have to edit 75% of the message, you may be spending more time and energy on making changes than if you were to just write it from scratch," Burdick says. "I would guess we need to get to where the physician is editing less than 30% of the content before it has substantial benefit." One advantage of AI's verbosity is that it tends to be more empathetic and thorough than physicians crunched for time, the researchers find. For example, AI is more likely to tell a patient experiencing an upset stomach that it's sorry to hear they're feeling nauseated. This means AI could be used to help "nudge" doctors to show more understanding and care for the patient's situation, or answer patient's questions more effectively so that patients feel more heard, Preum says. The team produced sample responses such as showing empathy by praising patients for following a treatment plan ("You've been doing a great job with your tapering.") or planning for changes in symptoms ("If you're feeling dizzy, please call triage."). The researchers also find that 65% of all the portal messages they studied came from people over 55, with patients over 65 generating 24% of all messages. These figures suggest that patient portals in general should be designed to accommodate older people, Preum says. Future work will study how much actual time doctors spend editing automated drafts. The team also plans to evaluate their training model TADPOLE from the user perspective, studying if and how it lightens a physician's workload, and how doctors and patients rate its performance. "This is one of the first studies that uses real patient portal messages to establish a generative AI model. In that regard, it's innovative and shows us that this is not a simple task," Burdick says. "We're still nowhere near the point of having clinicians removed from the workflow." Burdick, Preum, and Seegmiller worked with co-authors Joseph Gatto, who received his PhD from Dartmouth this year; Sarah Greer, a former physician at Dartmouth Health; and 2026 Dartmouth graduates Ganza Belise Isingizwe and Rohan Ray. Source: Dartmouth College Journal references: Seegmiller, P., et al. (2026). How Much Would a Clinician Edit This Draft? Evaluating LLM Alignment for Patient Message Response Drafting. ACL Anthology. DOI: 10.18653/v1/2026.acl-long.1505. https://aclanthology.org/2026.acl-long.1505

Editing AI mistakes can cost doctors time when writing to patients

Errors and irrelevant details mean physicians may spend more time editing AI-drafted responses than it would take to write them, a large study of an online patient portal shows. Artificial intelligence is spreading rapidly in health care, with the goal of streamlining critical but onerous clerical tasks such as note-taking and charting so physicians and nurses can devote more time to patients. But even when AI can free up doctors to correspond with patients, it may fall short by introducing errors and extraneous details into their messages, according to a new Dartmouth study presented July 7 at the 2026 Annual Meeting of the Association for Computational Linguistics and published in the conference proceedings. The result is that physicians may spend more time editing responses than it would have taken to write them, the researchers report. "We find that AI can sound like a doctor but not think like one," says Sarah Preum, an assistant professor of computer science and the study's co-corresponding author with Parker Seegmiller, a graduate researcher in Preum's PersistLab at Dartmouth. The researchers conducted the first large-scale study of an online patient portal that uses AI to draft responses from physicians to patients. The team developed a tool that compares AI-generated replies with a dataset of real responses they developed with health care professionals from Dartmouth Health. They then analyzed 146,000 conversations between 10,105 patients and their primary care physicians in the large rural health system. The study was approved by the Dartmouth Health Institutional Review Board, and the team used the required methods to protect patient privacy, including anonymizing data as needed. The researchers also used their tool to evaluate physician responses drafted by Claude, Gemini and ChatGPT, as well as the three smaller commercial platforms Llama, Aloe and Qwen. The team reports that AI-generated answers frequently misalign with what clinicians would actually write. This includes automated responses that are too long, don't ask follow-up questions and use irrelevant or inaccurate medical details. "There are smaller studies that say, 'Oh, AI is amazing,' but we realized there is a gap in the existing literature in a large-scale evaluation of this technology," Preum says. "We didn't just want to measure a platform's accuracy, but whether it actually helps with the workload, which in this case is measured by how much editing the physician is doing." For example, the portal's AI suggested telling a 32-year-old woman who is taking an acid reflux drug and was concerned about constant nausea that the medication might require some adjustment in her diet. A physician replaced that by asking if there was any chance she was pregnant. Even little changes can add up over hundreds or thousands of messages, Preum says. "You don't want to integrate large language models into the workflow and just shift the bottleneck so that doctors are devoting their cognitive energy to playing AI janitor and fixing mistakes," she says. "But if we're not careful, that's a likely outcome." The researchers show, however, that adapting AI to how individual physicians communicate can improve accuracy by 33% and reduce editing by 26%. "If message generation is really efficient and high quality, if it asks the right things, then it really has the potential to improve efficiency," says co-author Tim Burdick, an associate professor of community and family medicine in Dartmouth's Geisel School of Medicine and a family medicine physician at Dartmouth Health. "I don't foresee a time when the portal can respond to a patient without a clinician editing it first. But as we make the models better, we'll be able to address portal messages much more quickly and with less mental energy," Burdick says. The study shows that there are such things as "good" AI responses and provides a framework for implementing them into patient-physician portals, Preum says. These platforms are increasingly common among large health care systems and often customized, she says. "That took us a long time to figure out, but if you're trying to measure how effective this technology is, you need to define what a good response is," she says. "We can only improve what we can measure and objectively evaluate." The researchers created a technique called TADPOLE -- Thematic Agentic Direct Preference Optimization for Learning Enhancement -- that trains AI platforms using the hybrid model they constructed from physician- and AI-generated responses. They plugged TADPOLE into the six commercial LLMs and found that drafted responses better matched physicians' standards for precision and information quality. "That could save a busy clinician an hour or two of work a day," Burdick says. Doctors and nurses today are inundated with messages from patients and caregivers who can write them online anytime, he says. An ongoing project between Burdick and the Preum Lab called PortalPal aims to streamline patient portals using AI, including by automating some steps in following up with patients to get more information. Physicians Burdick works with say that AI-generated drafts save about 25% of their time on shorter messages. "It's easier to make small edits to an LLM-generated message than it is to write it from scratch," he says. But longer drafts can include information that is not correct or accurate. "If you have to edit 75% of the message, you may be spending more time and energy making changes than if you were to just write it from scratch," Burdick says. "I would guess we need to get to where the physician is editing less than 30% of the content before it has substantial benefit." One advantage of AI's verbosity is that it tends to be more empathetic and thorough than physicians who are crunched for time, the researchers find. For example, AI is more likely to tell a patient experiencing an upset stomach that it's sorry to hear they're feeling nauseated. This means AI could be used to help "nudge" doctors to show more understanding and care for the patient's situation, or answer patients' questions more effectively so patients feel more heard, Preum says. The team produced sample responses such as showing empathy by praising patients for following a treatment plan ("You've been doing a great job with your tapering.") or planning for changes in symptoms ("If you're feeling dizzy, please call triage."). The researchers also found that 65% of all the portal messages they studied came from people over 55, with patients over 65 generating 24% of all messages. These figures suggest that patient portals in general should be designed to accommodate older people, Preum says. Future work will study how much actual time doctors spend editing automated drafts. The team also plans to evaluate their training model TADPOLE from the user perspective, studying whether and how it lightens a physician's workload, and how doctors and patients rate its performance. "This is one of the first studies that uses real patient portal messages to establish a generative AI model. In that regard, it's innovative and shows us that this is not a simple task," Burdick says. "We're still nowhere near the point of having clinicians removed from the workflow."