AI Chatbot Helps Doctors in 75% of Cases With Just 1 Hallucination in Clinical Trial

AI Chatbot AMIE Shows Promise in Real-World Urgent Care Setting

An eight-month clinical trial at Beth Israel Deaconess Medical Center has demonstrated that AI in healthcare can safely assist primary care physicians during urgent care visits. The study, published in The Lancet and representing Google's first-ever publication in the journal's main edition, involved 98 patients who interacted with AMIE (Articulate Medical Intelligence Explorer), Google's AI chatbot, before meeting their doctors.1

1

2

2

The patient-facing conversational AI gathered medical histories, explored symptoms, and generated summaries for physician review. Primary care providers reported that AI summaries for clinicians helped them prepare for visits in 75% of cases and potentially influenced their clinical approach in 57% of encounters.3

3

Doctors compared the technology's performance to "a high-performing medical student or resident," according to Peter Brodeur, a resident physician at Beth Israel Deaconess Medical Center.1

1

Safety Monitoring Reveals Minimal AI Hallucination Risk

The clinical trial was designed primarily to test safety, with board-certified physicians monitoring every conversation in real time. Across all 98 completed patient interactions, supervising physicians never needed to halt a conversation based on predefined safety criteria. The AI diagnostic chatbot experienced only one hallucination instance—misrepresenting a date as future when it was past.1

1

3

3

Physicians provided additional clinical clarification in just five cases. In one instance, when the AI application in healthcare listed cancer among potential diagnoses, a physician stepped in to reassure the patient that the bot was considering all possibilities.1

1

Adam Rodman, Director of AI Programs at Beth Israel and medical historian who helped conduct the trial, explained that the system's focus on clinical reasoning from relatively short conversations makes hallucination less likely compared to systems processing entire medical records.1

1

Diagnostic Accuracy and Workflow Benefits Show Strong Results

AMIE's differential diagnoses matched physicians' final diagnoses 90% of the time, demonstrating strong alignment between AI could aid urgent care assessments and human clinical judgment.2

2

The technology, powered by Google's flagship Gemini AI model, uses a more powerful version than what drives the company's search engine, specifically trained for clinical reasoning and medical diagnosis.1

1

For the 44 cases where providers reviewed AI-generated transcripts or summaries before appointments, the workflow benefits were clear. Jacob Koshy, a hospitalist at Beth Israel Deaconess Medical Center, noted that prospectively observing these interactions helped researchers understand both opportunities and limitations of the technology in routine patient care.3

3

Patient Trust Emerges as Critical Factor for AI Adoption

Patients rated Google's AI chatbot favorably across most quality measures, including its ability to listen, explain information, and help them feel at ease. Patient attitudes toward AI improved after interacting with the chatbot and remained elevated following their appointments with clinicians.3

3

However, patient trust remained a concern in two specific areas: confidentiality of shared information and the chatbot's honesty and trustworthiness. Rodman emphasized that patients only trusted the system because it operated alongside trained physicians. "None of our patients trusted it until they realized that it was handing-off their doctor," he said.1

1

Research suggests that patient trust in medical AI systems flows from relationships with human providers, underscoring the importance of maintaining strong patient-physician relationships.3

3

Future Implications for Strained Primary Care Systems

The study arrives as primary care systems worldwide face mounting pressure from an aging population, workforce changes, and increasingly complex medical needs. Robert Califf, a cardiologist at Duke and former FDA commissioner not involved in the trial, called this "the technology of the future" while cautioning that assumptions about reducing physician workload require further validation.1

1

Google has launched a nationwide trial comparing patient visits with and without chatbots, aiming to assess the technology at scale.1

1

Future research will explore which interaction characteristics build patient trust and how AI interactions can enhance rather than replace the patient-physician relationship. The findings suggest AI could improve patient-physician relationships by extending healthcare workers' capabilities, though experts warn these systems cannot yet replace trained doctors.2

2