The Outpost is a comprehensive collection of curated artificial intelligence software tools that cater to the needs of small business owners, bloggers, artists, musicians, entrepreneurs, marketers, writers, and researchers.
© 2025 TheOutpost.AI All rights reserved
Curated by THEOUTPOST
On Tue, 8 Oct, 4:06 PM UTC
5 Sources
[1]
Study finds AI not ready to run emergency rooms
UPI - AI isn't ready to run a hospital's emergency room just yet, a new study concluded. ChatGPT likely would ask for unnecessary x-rays and antibiotics for some patients, and admit others who don't really need hospital treatment, researchers reported in the journal Nature Communications. "This is a valuable message to clinicians not to blindly trust these models," said lead researcher Chris Williams, a postdoctoral scholar with the University of California, San Francisco. "ChatGPT can answer medical exam questions and help draft clinical notes, but it's not currently designed for situations that call for multiple considerations, like the situations in an emergency department," Williams added in a UCSF news release. For the new study, researchers challenged the ChatGPT AI model to provide the sort of recommendations an ER doctor would make after initially examining a patient. The team ran data from 1,000 prior ER visits past the AI, drawn from an archive of more than 251,000 visits. The AI had to answer "yes" or "no" as to whether each patient should be admitted, sent for X-rays or prescribed antibiotics. Overall, ChatGPT tended to recommend more services than were actually needed, results showed. The ChatGPT-4 model was eight per cent less accurate than human doctors, and ChatGPT-3.5 was 24 per cent less accurate. This tendency to overprescribe might be explained by the fact that the AI models are trained on the Internet, Williams said. Legitimate medical advice sites aren't designed to answer emergency medical questions, but to forward patients to a doctor who can. "These models are almost fine-tuned to say, 'seek medical advice,' which is quite right from a general public safety perspective," Williams said. "But erring on the side of caution isn't always appropriate in the ED setting, where unnecessary interventions could cause patients harm, strain resources and lead to higher costs for patients." To be more useful in the ER, AI models will need better frameworks built by designers who can thread the needle between catching serious illnesses while not asking for unnecessary exams and treatments, Williams said. "There's no perfect solution," he said, "But knowing that models like ChatGPT have these tendencies, we're charged with thinking through how we want them to perform in clinical practice."
[2]
Study finds AI is not ready to run emergency rooms
AI isn't ready to run a hospital's emergency room just yet, a new study concludes. ChatGPT likely would ask for unnecessary x-rays and antibiotics for some patients, and admit others who don't really need hospital treatment, researchers reported Tuesday in the journal Nature Communications. "This is a valuable message to clinicians not to blindly trust these models," said lead researcher Chris Williams, a postdoctoral scholar with the University of California, San Francisco. "ChatGPT can answer medical exam questions and help draft clinical notes, but it's not currently designed for situations that call for multiple considerations, like the situations in an emergency department," Williams added in a UCSF news release. For the new study, researchers challenged the ChatGPT AI model to provide the sort of recommendations an ER doctor would make after initially examining a patient. The team ran data from 1,000 prior ER visits past the AI, drawn from an archive of more than 251,000 visits. The AI had to answer "yes" or "no" as to whether each patient should be admitted, sent for X-rays or prescribed antibiotics. Overall, ChatGPT tended to recommend more services than were actually needed, results showed. The ChatGPT-4 model was 8% less accurate than human doctors, and ChatGPT-3.5 was 24% less accurate. This tendency to overprescribe might be explained by the fact that the AI models are trained on the internet, Williams said. Legitimate medical advice sites aren't designed to answer emergency medical questions, but to forward patients to a doctor who can. "These models are almost fine-tuned to say, 'seek medical advice,' which is quite right from a general public safety perspective," Williams said. "But erring on the side of caution isn't always appropriate in the ED setting, where unnecessary interventions could cause patients harm, strain resources and lead to higher costs for patients." To be more useful in the ER, AI models will need better frameworks built by designers who can thread the needle between catching serious illnesses while not asking for unnecessary exams and treatments, Williams said. "There's no perfect solution," he said, "But knowing that models like ChatGPT have these tendencies, we're charged with thinking through how we want them to perform in clinical practice."
[3]
AI May Not Be Ready to Run Emergency Rooms
TUESDAY, Oct. 8, 2024 (HealthDay News) -- AI isn't ready to run a hospital's emergency room just yet, a new study concludes. ChatGPT likely would ask for unnecessary x-rays and antibiotics for some patients, and admit others who don't really need hospital treatment, researchers reported Oct. 8 in the journal Nature Communications. "This is a valuable message to clinicians not to blindly trust these models," said lead researcher Chris Williams, a postdoctoral scholar with the University of California, San Francisco. "ChatGPT can answer medical exam questions and help draft clinical notes, but it's not currently designed for situations that call for multiple considerations, like the situations in an emergency department," Williams added in a UCSF news release. For the new study, researchers challenged the ChatGPT AI model to provide the sort of recommendations an ER doctor would make after initially examining a patient. The team ran data from 1,000 prior ER visits past the AI, drawn from an archive of more than 251,000 visits. The AI had to answer "yes" or "no" as to whether each patient should be admitted, sent for X-rays or prescribed antibiotics. Overall, ChatGPT tended to recommend more services than were actually needed, results showed. The ChatGPT-4 model was 8% less accurate than human doctors, and ChatGPT-3.5 was 24% less accurate. This tendency to overprescribe might be explained by the fact that the AI models are trained on the internet, Williams said. Legitimate medical advice sites aren't designed to answer emergency medical questions, but to forward patients to a doctor who can. "These models are almost fine-tuned to say, 'seek medical advice,' which is quite right from a general public safety perspective," Williams said. "But erring on the side of caution isn't always appropriate in the ED setting, where unnecessary interventions could cause patients harm, strain resources and lead to higher costs for patients." To be more useful in the ER, AI models will need better frameworks built by designers who can thread the needle between catching serious illnesses while not asking for unnecessary exams and treatments, Williams said. "There's no perfect solution," he said, "But knowing that models like ChatGPT have these tendencies, we're charged with thinking through how we want them to perform in clinical practice." SOURCE: University of California, San Francisco, news release, Oct. 8, 2024
[4]
Study finds that when it comes to emergency care, ChatGPT overprescribes
If ChatGPT were cut loose in the Emergency Department, it might suggest unneeded X-rays and antibiotics for some patients and admit others who didn't require hospital treatment, a new study from UC San Francisco has found. The researchers said that, while the model could be prompted in ways that make its responses more accurate, it's still no match for the clinical judgment of a human doctor. "This is a valuable message to clinicians not to blindly trust these models," said postdoctoral scholar Chris Williams, MB BChir, lead author of the study, which appears Oct. 8 in Nature Communications. "ChatGPT can answer medical exam questions and help draft clinical notes, but it's not currently designed for situations that call for multiple considerations, like the situations in an emergency department." Recently, Williams showed that ChatGPT, a large language model (LLM) that can be used for researching clinical applications of AI, was slightly better than humans at determining which of two emergency patients was most acutely unwell, a straightforward choice between patient A and patient B. With the current study, Williams challenged the AI model to perform a more complex task: providing the recommendations a physician makes after initially examining a patient in the ED. This includes deciding whether to admit the patient, get X-rays or other scans, or prescribe antibiotics. AI model is less accurate than a resident For each of the three decisions, the team compiled a set of 1,000 ED visits to analyze from an archive of more than 251,000 visits. The sets had the same ratio of "yes" to "no" responses for decisions on admission, radiology and antibiotics that are seen across UCSF Health's Emergency Department. Using UCSF's secure generative AI platform, which has broad privacy protections, the researchers entered doctors' notes on each patient's symptoms and examination findings into ChatGPT-3.5 and ChatGPT-4. Then, they tested the accuracy of each set with a series of increasingly detailed prompts. Overall, the AI models tended to recommend services more often than was needed. ChatGPT-4 was 8% less accurate than resident physicians, and ChatGPT-3.5 was 24% less accurate. Williams said the AI's tendency to overprescribe could be because the models are trained on the internet, where legitimate medical advice sites aren't designed to answer emergency medical questions but rather to send readers to a doctor who can. "These models are almost fine-tuned to say, 'seek medical advice," which is quite right from a general public safety perspective," he said. "But erring on the side of caution isn't always appropriate in the ED setting, where unnecessary interventions could cause patients harm, strain resources and lead to higher costs for patients." He said models like ChatGPT will need better frameworks for evaluating clinical information before they are ready for the ED. The people who design those frameworks will need to strike a balance between making sure the AI doesn't miss something serious, while keeping it from triggering unneeded exams and expenses. This means researchers developing medical applications of AI, along with the wider clinical community and the public, need to consider where to draw those lines and how much to err on the side of caution. "There's no perfect solution," he said, "But knowing that models like ChatGPT have these tendencies, we're charged with thinking through how we want them to perform in clinical practice."
[5]
Study reveals limitations of ChatGPT in emergency medicine
University of California - San FranciscoOct 8 2024 If ChatGPT were cut loose in the Emergency Department, it might suggest unneeded x-rays and antibiotics for some patients and admit others who didn't require hospital treatment, a new study from UC San Francisco has found. The researchers said that, while the model could be prompted in ways that make its responses more accurate, it's still no match for the clinical judgment of a human doctor. "This is a valuable message to clinicians not to blindly trust these models," said postdoctoral scholar Chris Williams, MB BChir, lead author of the study, which appears Oct. 8 in Nature Communications. "ChatGPT can answer medical exam questions and help draft clinical notes, but it's not currently designed for situations that call for multiple considerations, like the situations in an emergency department." Recently, Williams showed that ChatGPT, a large language model (LLM) that can be used for researching clinical applications of AI, was slightly better than humans at determining which of two emergency patients was most acutely unwell, a straightforward choice between patient A and patient B. With the current study, Williams challenged the AI model to perform a more complex task: providing the recommendations a physician makes after initially examining a patient in the ED. This includes deciding whether to admit the patient, get x-rays or other scans, or prescribe antibiotics. AI model is less accurate than a resident For each of the three decisions, the team compiled a set of 1,000 ED visits to analyze from an archive of more than 251,000 visits. The sets had the same ratio of "yes" to "no" responses for decisions on admission, radiology and antibiotics that are seen across UCSF Health's Emergency Department. Using UCSF's secure generative AI platform, which has broad privacy protections, the researchers entered doctors' notes on each patient's symptoms and examination findings into ChatGPT-3.5 and ChatGPT-4. Then, they tested the accuracy of each set with a series of increasingly detailed prompts. Overall, the AI models tended to recommend services more often than was needed. ChatGPT-4 was 8% less accurate than resident physicians, and ChatGPT-3.5 was 24% less accurate. Williams said the AI's tendency to overprescribe could be because the models are trained on the internet, where legitimate medical advice sites aren't designed to answer emergency medical questions but rather to send readers to a doctor who can. These models are almost fine-tuned to say, 'seek medical advice,' which is quite right from a general public safety perspective. But erring on the side of caution isn't always appropriate in the ED setting, where unnecessary interventions could cause patients harm, strain resources and lead to higher costs for patients." Chris Williams, MB BChir, lead author of the study He said models like ChatGPT will need better frameworks for evaluating clinical information before they are ready for the ED. The people who design those frameworks will need to strike a balance between making sure the AI doesn't miss something serious, while keeping it from triggering unneeded exams and expenses. This means researchers developing medical applications of AI, along with the wider clinical community and the public, need to consider where to draw those lines and how much to err on the side of caution. "There's no perfect solution," he said, "But knowing that models like ChatGPT have these tendencies, we're charged with thinking through how we want them to perform in clinical practice." University of California - San Francisco
Share
Share
Copy Link
A new study from UC San Francisco shows that AI models like ChatGPT are not yet ready to make critical decisions in emergency rooms, tending to overprescribe treatments and admissions compared to human doctors.
A recent study conducted by researchers at the University of California, San Francisco (UCSF) has revealed significant limitations in the ability of AI models like ChatGPT to make critical decisions in emergency room settings. The research, published in Nature Communications on October 8, 2024, highlights the potential risks of relying too heavily on AI for complex medical decision-making 1.
Led by postdoctoral scholar Chris Williams, the research team challenged ChatGPT to perform tasks typically handled by emergency room physicians. The AI was tasked with deciding whether to admit patients, order X-rays, or prescribe antibiotics based on initial examinations 2.
The study analyzed 1,000 emergency department visits, drawn from an archive of over 251,000 cases. The results showed that:
The study's findings raise important questions about the readiness of AI for critical healthcare applications. Williams emphasized that while ChatGPT can handle certain medical tasks, it's not designed for the complex, multi-faceted decision-making required in emergency departments 3.
The AI's tendency to overprescribe is attributed to its training on internet data, where medical advice often errs on the side of caution. While this approach may be appropriate for general public safety, it can lead to unnecessary interventions, potential harm to patients, and increased healthcare costs in an emergency room setting 4.
To improve AI's performance in emergency settings, researchers suggest:
Williams stressed the importance of not blindly trusting these models and the need for continued research to refine AI's capabilities in healthcare settings 5.
As AI continues to evolve, the challenge lies in harnessing its potential while ensuring patient safety and maintaining the irreplaceable value of human clinical judgment in complex medical scenarios.
Reference
[1]
[2]
[3]
[4]
Medical Xpress - Medical and Health News
|Study finds that when it comes to emergency care, ChatGPT overprescribes[5]
A new study reveals that while AI models perform well on standardized medical tests, they face significant challenges in simulating real-world doctor-patient conversations, raising concerns about their readiness for clinical deployment.
3 Sources
3 Sources
A recent study reveals that ChatGPT, when used alone, significantly outperformed both human doctors and doctors using AI assistance in diagnosing medical conditions, raising questions about the future of AI in healthcare.
6 Sources
6 Sources
A collaborative research study explores the effectiveness of GPT-4 in assisting physicians with patient diagnosis, highlighting both the potential and limitations of AI in healthcare.
3 Sources
3 Sources
A recent study reveals that ChatGPT, an AI language model, demonstrates superior performance compared to trainee doctors in assessing complex respiratory diseases. This breakthrough highlights the potential of AI in medical diagnostics and its implications for healthcare education and practice.
3 Sources
3 Sources
A new study reveals that AI-powered chatbots can improve physicians' clinical management reasoning, outperforming doctors using conventional resources and matching the performance of standalone AI in complex medical decision-making scenarios.
3 Sources
3 Sources
