AI Tools Simplify Medical Reports But Still Require Human Oversight to Prevent Clinical Errors

AI Language Models Tackle Medical Document Comprehension Crisis

Researchers from the Universidad Carlos III de Madrid have completed a comprehensive evaluation of AI tools designed to simplify medical reports for non-expert audiences, addressing a significant healthcare communication gap. The study, published in Scientific Reports1

1

, reveals that 46% of European citizens struggle to understand their medical documents, directly impacting their ability to manage health conditions and adhere to prescribed treatments.

"We developed a set of tools to help improve doctor-patient communication," explains study co-author Paloma Martínez, Full Professor in UC3M's Department of Computer Science. The AI language models can generate dual versions of documents—one for medical professionals and another tailored for patients, ensuring comprehension of abbreviations and complex phrasing found in reports2

2

.

Cognitive Accessibility Regulations Drive Research Urgency

The research arrives ahead of Royal Decree 707/2026, which takes effect on January 2, 2027. This regulation establishes Spain's first basic cognitive accessibility conditions, requiring public administrations to make documents, procedures, and services more understandable across justice, healthcare, employment, digital services, and transport sectors1

1

.

Co-author Lourdes Moreno emphasizes that plain language represents more than simple word substitution. "It involves structuring content coherently, adapting specialized vocabulary, and explicitly guiding users according to their informational needs," following the international plain language standard ISO 24495-1:20232

2

.

Rigorous Testing Against Expert-Prepared Standards

The research team evaluated 50 medical reports from clinical cases provided by the Andalusian Society of Family and Community Medicine. Each report received three independent adaptations from cognitive accessibility experts, yielding 150 adapted texts that served as benchmarks for model performance1

1

.

Four language model families underwent assessment: Llama 3.2 1B and 3B, Salamandra 2B and 7B, RigoChat-v2 7B, and Medical-mT5 3B. Researchers measured simplification, semantic similarity, readability and factual consistency, alongside environmental impact considerations2

2

.

No Perfect Model Emerges, But Clear Leaders Identified

Results demonstrate that no single model excels across all metrics. Llama 3.2-3B exhibited the most balanced profile for efficiency, robustness, and overall performance. Meanwhile, RigoChat-v2-7B distinguished itself when prioritizing output quality and readability1

1

.

This differentiation suggests healthcare organizations must select models based on specific priorities—whether optimizing for computational efficiency or maximizing patient comprehension quality.

Critical Errors Demand Human Oversight in Healthcare Applications

Analysis revealed three persistent error types across all four model families that underscore why human oversight remains essential. Oversimplification removes key clinical details patients need for informed decision-making. Clinical hallucinations involve inventing or altering data from original reports, creating dangerous misinformation. Incoherence manifests as fragmented sentences or abrupt transitions that hinder readability2

2

.

"It is precisely because of these errors that human oversight remains essential for these models, particularly in a context as sensitive as healthcare," Martínez emphasizes1

1

.

Open-Source Approach Enables Reproducible Research

The team has released all research materials to the scientific community, advancing reproducible research in medical AI applications. "Everything we developed—including the dataset, fine-tuned models, and prompts—is accessible in public repositories such as GitHub and Zenodo so others can build upon our work," Martínez confirms2

2

.

This open-source commitment allows global researchers to refine these AI tools and develop safeguards against identified error patterns. The work was conducted under HumanAI (Human-Centered AI: Towards Accessible and Personalized Language Models), funded by Spain's State Research Agency through project PID2023-148577OB-C211

1

.