AI Scribes in NHS Make Critical Errors in Drug Names and Diagnoses, Healthwatch England Warns

AI scribes are getting drug names and diagnoses wrong in NHS records

A patient in England was told they had demyelination, the nerve damage that underlies conditions including multiple sclerosis. The test result had actually read "null demyelination", and an AI scribe had dropped the word that reversed the meaning. That case appears in a warning from Healthwatch England, the statutory patient watchdog, reported by the Guardian today. Its finding is that the tools now transcribing consultations for GPs and hospital doctors are getting drug names and diagnoses wrong, and that patients are often the ones who notice. Twenty-seven different AI scribes are in use across the health service in England. They sit in the consulting room, listen, and produce the note that goes into the record and the letter that goes to the patient. The errors collected by the watchdog are mundane in form and serious in effect. One scribe swapped a prescribed drug for a different one with a similar name, the kind of confusion that pharmacology spends considerable effort trying to design out. Another summary omitted a consultant's instruction that the patient should seek a repeat prescription for migraine medication. A third recorded a doctor telling a patient to continue their Prozac, when that doctor had neither prescribed it nor discussed it. The common thread is that nothing looked broken. A fluent, plausible note is exactly what these systems produce, which is why an error survives the glance a busy clinician gives it before signing off. "These inaccuracies may persist in their records if the patient doesn't catch them," the watchdog warned. That places the last line of defence on the person least equipped to know what the note should have said. Rachel Power, chief executive of the Patients Association, is among those raising concerns, alongside clinicians including the London GP Shier Ziser Dawood and Charlotte Blease of Uppsala University in Sweden. The objection is not to the technology so much as to its arrival without a safety net. Because there is no England-wide oversight of these tools. The Medicines and Healthcare products Regulatory Agency has not classified AI scribes as medical devices, which leaves them outside the regime that would test them for safety and effectiveness before deployment. The regulator did publish guidance in August clarifying where the line sits. A system that only transcribes what was said is not a device, while one that suggests a diagnosis or a treatment may well be, which puts a great deal of weight on how each product is described by its vendor. The incentive that follows is obvious enough. A scribe marketed as a passive transcriber avoids a regulatory process that a scribe marketed as a clinical assistant would have to complete. Transcription error is also a different failure from the one most AI safety work anticipates. Nobody here was misled by a hallucinated fact; a real sentence was rendered slightly wrong, and slightly wrong is sufficient when the sentence names a drug. None of which addresses why these tools spread so quickly. Clinical documentation is the administrative burden doctors complain about most, and a system that reliably removes an hour of typing a day will be adopted whether or not anyone has assessed it. The problem is what happens in between those two facts. A tool adopted for its speed, unassessed because of how it is categorised, producing a document that becomes the permanent clinical record, is a chain in which no single link is obviously anyone's responsibility. The remedy the watchdog points to is unglamorous and probably right. Patients should be told when a scribe is being used and given their notes to check, which turns an accidental safety mechanism into a deliberate one. Twenty-seven products, no device classification, and a check performed by whoever happens to read their letter carefully: that is the current arrangement in the NHS in England.

Doctors' AI scribes get names of drugs and diagnoses wrong, NHS watchdog warns

Exclusive: Patients identify errors in consultation transcripts that are missed by GPs, Healthwatch England finds AI technology that listens to and transcribes patients' consultations with doctors can put them at risk by getting the names of drugs and illnesses wrong, an NHS watchdog has warned. In one case a woman was left badly shaken when the AI scribe's summary of her conversation wrongly said she had demyelination - serious nerve damage that can lead to multiple sclerosis. It was only when the patient, an NHS health professional, queried the AI tool's record of the result of her MRI scan that the hospital corrected it to what it should have been - "null demyelination". "This was eventually corrected but was a very traumatising experience to be given an incorrect diagnosis because of AI and then be told it's a typo," said the woman, who asked not to be named. Healthwatch England has highlighted the case and other mistakes by AI scribes to show that the technology, which the NHS is rolling out rapidly, is a potential threat to patient safety. Unless detected, errors could end up in patients' medical records and affect their care, it said. In another case, an AI scribe confused the drug the GP had prescribed with a different one of a similar name - a blunder which again the patient, rather than the doctor, identified. On another occasion an AI-generated summary letter did not say that the hospital consultant had told the patient to seek a repeat prescription from their GP for their migraine, which could have left them unable to get their medication. Healthwatch, the statutory NHS patient champion, has heard "multiple stories from patients who have noticed these errors when a health professional hasn't", it said. "These inaccuracies may persist in their records if the patient doesn't catch them." The government's 10-year health plan for the NHS in England expects AI scribes to "liberate staff from their current burden of bureaucracy and administration, freeing up time to care and to focus on the patient". It is central to the planned "big shift" for the NHS from being an analogue to a digital-based service. A Healthwatch spokesperson said: "Healthcare has never been error-free. But our findings show the urgent need for clarity over how patients can report and get corrected any mistakes made by AI scribing tools or the professionals that use them." Rachel Power, chief executive of the Patients Association, said: "Trust and confidence in this technology depend on good communication and genuine partnership with patients and right now both are missing." Healthwatch added that it was "worrying" that the Medicines and Healthcare products Regulatory Agency has decided not to classify AI scribes as medical devices, which means there will be no England-wide oversight to ensure they are safe to use and effective. GPs and hospital doctors in England are already using 27 different AI scribes. Ministers have been warned that the NHS and medics could be sued over mistakes made by AI. AI's accuracy was in the spotlight recently when patients in Rotherham complained to their local Healthwatch that an AI receptionist used by some local GP practices did not understand their strong Yorkshire accents. Dr Shier Ziser Dawood, a GP in London, last year warned in a leading medical journal that AI scribes may prove "a double-edged sword" for family doctors. She recounted that an AI scribe said she had told her patient to "continue their Prozac" even though she had not prescribed or discussed that drug with them. That is an example of what are known as "hallucinations", where AI scribes refer to something that was not raised during the consultation. She warned that the need for doctors to review all transcripts in order to check for errors meant that AI tools are not yet proving time-saving. Given the belief of NHS bosses that scribes will mean GPs no longer have to take notes during an appointment, family doctors may be expected to see two more patients every day, even though NHS GP consultation times are already some of the shortest in the world, Dawood added, writing in the British Journal of General Practice. Dr Charlotte Blease, an expert in AI use in healthcare at Uppsala university in Sweden, said: "AI can and does make mistakes." Her research found that GPs who use ambient voice technology believe that errors are more likely to creep in when the consultation is with more than one person, with patients with a complex medical history, and with those whose first language is not English. But, she added: "The fact is, doctors can and do make mistakes without AI. And it is certainly possible the error rate is worse." More than half the 1,003 UK GPs in her survey last year believed that their ambient AI records were more accurate than those produced themselves, Blease said.