AI Models in Healthcare Show Potential Bias in Treatment Recommendations, Study Reveals

Health Rounds: AI can have medical care biases too, a study reveals

April 9 (Reuters) - (To receive the full newsletter in your inbox for free sign up here) Artificial intelligence models may recommend different treatments for the same medical condition based solely on a patient's socioeconomic and demographic characteristics, researchers warn. The researchers invented nearly three dozen different patients and asked nine healthcare large language AI models how each one should be managed, in a thousand different emergency room situations. Despite identical clinical details, the AI models occasionally altered decisions based on patients' personal characteristics, affecting priority for care, diagnostic testing, treatment approach, and mental health evaluation, the researchers reported in Nature Medicine, opens new tab. For example, advanced diagnostic tests such as CT scans or MRI were more often recommended for high-income patients, while low-income patients were more frequently advised to undergo no further testing, somewhat mimicking real-world healthcare inequities. The problems were seen in both proprietary and open-source AI models, the researchers found. "AI has the power to revolutionize healthcare, but only if it's developed and used responsibly," study co-leader Dr. Girish Nadkarni of the Icahn School of Medicine at Mount Sinai in New York said in a statement. "By identifying where these models may introduce bias, we can work to refine their design, strengthen oversight, and build systems that ensure patients remain at the heart of safe, effective care," added coauthor Dr. Eyal Klang, also of the Icahn School. Potential fixes for Sjogren's saliva and tears symptoms Researchers are closer to being able to fix the life-altering dryness of the mouth and eyes that afflicts patients with Sjogren's syndrome, based on success of two approaches tested in mice. The symptoms of the autoimmune disorder can make it hard to speak, eat and sleep. But exactly how the disease shuts down the body's production of tears and saliva has been a mystery until now, researchers reported in the International Journal of Oral Science, opens new tab. Their new study found that early in the progression of Sjogren's syndrome, a protein called tricellulin, which clasps together the cells of the glands that produce tears and saliva, is destroyed. Loss of the tight cellular junctions results in inadequate saliva secretion, the researchers found. Two possible interventions - an investigational drug (AT1001) and an experimental molecule - each restored saliva secretion in the mice, one by repairing the cell junctions and the other by stopping the breakdown of the junctions before it began. Both restored normal gland function, offering a potential blueprint for human treatment, the researchers said. "This changes how we think about treating Sjogren's syndrome," study leader Dr. Xin Cong of Peking University said in a statement. "We're moving beyond simply calming inflammation. Now we can fix the actual structural damage in the glands," Xin said. "What's even more encouraging is that both approaches worked, which gives us real confidence in developing patient-ready therapies." Experimental drug shows promise for one type of MS An experimental drug originally developed to treat lymphomas is the first-ever to show an effect against a form of multiple sclerosis for which no approved treatments are available, researchers reported at the American Academy of Neurology meeting in San Diego. Sanofi's (SASY.PA), opens new tab tolebrutinib, an investigational oral Bruton's tyrosine kinase inhibitor, demonstrated a 31% delay in the onset of six-month confirmed disability progression in patients with non-active, non-relapsing secondary progressive multiple sclerosis, in a clinical trial. "This is the first clinical trial showing a positive effect in delaying disability progression in non-relapsing SPMS, a later form of the disease where neurological function gradually worsens over time and disability increases relentlessly," study leader Dr. Robert Fox of the Cleveland Clinic said in a statement. With 1,131 patients enrolled in the trial, the rate of confirmed disability progression at six months was 22.6% in the tolebrutinib group versus 30.7% in the placebo group, according to a report of the study published in The New England Journal of Medicine, opens new tab. More patients receiving tolebrutinib achieved improvement in disability, with a six-month confirmed disability improvement rate of 8.6% versus 4.5% with placebo, the researchers also reported. Markers of disease activity, including inflammation and tissue damage, also were reduced with tolebrutinib compared with placebo. Serious adverse events, particularly liver complications, were more frequent with tolebrutinib, which is currently under review for potential U.S. approval. "It appears that about one in 200 patients will have severe elevation of liver enzymes during the first three months of use, so careful monitoring is important, and the drug should be stopped immediately in those with liver enzyme elevations," Fox said. Separately, in two studies of patients with relapsing multiple sclerosis, tolebrutinib was not superior to Sanofi's Aubagio (teriflunomide) in decreasing annualized relapse rates, according to a second report, opens new tab in the same journal. Reporting by Nancy Lapid; editing by Bill Berkrot Our Standards: The Thomson Reuters Trust Principles., opens new tab Suggested Topics:Healthcare & Pharmaceuticals Nancy Lapid Thomson Reuters Nancy has been a health news reporter and editor at Reuters for more than a decade, covering important medical research advances. She is the author of our twice-a-week Reuters Health Rounds newsletter.

AI can have medical care biases too, a study reveals

Despite identical clinical details, the AI models occasionally altered decisions based on patients' personal characteristics, affecting priority for care, diagnostic testing, treatment approach, and mental health evaluation, the researchers reported in Nature Medicine.Artificial intelligence models may recommend different treatments for the same medical condition based solely on a patient's socioeconomic and demographic characteristics, researchers warn. The researchers invented nearly three dozen different patients and asked nine healthcare large language AI models how each one should be managed, in a thousand different emergency room situations. Despite identical clinical details, the AI models occasionally altered decisions based on patients' personal characteristics, affecting priority for care, diagnostic testing, treatment approach, and mental health evaluation, the researchers reported in Nature Medicine. For example, advanced diagnostic tests such as CT scans or MRI were more often recommended for high-income patients, while low-income patients were more frequently advised to undergo no further testing, somewhat mimicking real-world healthcare inequities. The problems were seen in both proprietary and open-source AI models, the researchers found. "AI has the power to revolutionize healthcare, but only if it's developed and used responsibly," study co-leader Dr. Girish Nadkarni of the Icahn School of Medicine at Mount Sinai in New York said in a statement. "By identifying where these models may introduce bias, we can work to refine their design, strengthen oversight, and build systems that ensure patients remain at the heart of safe, effective care," added coauthor Dr. Eyal Klang, also of the Icahn School. Potential fixes for Sjogren's saliva and tears symptoms Researchers are closer to being able to fix the life-altering dryness of the mouth and eyes that afflicts patients with Sjogren's syndrome, based on success of two approaches tested in mice. The symptoms of the autoimmune disorder can make it hard to speak, eat and sleep. But exactly how the disease shuts down the body's production of tears and saliva has been a mystery until now, researchers reported in the International Journal of Oral Science. Their new study found that early in the progression of Sjogren's syndrome, a protein called tricellulin, which clasps together the cells of the glands that produce tears and saliva, is destroyed. Loss of the tight cellular junctions results in inadequate saliva secretion, the researchers found. Two possible interventions - an investigational drug (AT1001) and an experimental molecule - each restored saliva secretion in the mice, one by repairing the cell junctions and the other by stopping the breakdown of the junctions before it began. Both restored normal gland function, offering a potential blueprint for human treatment, the researchers said. "This changes how we think about treating Sjogren's syndrome," study leader Dr. Xin Cong of Peking University said in a statement. "We're moving beyond simply calming inflammation. Now we can fix the actual structural damage in the glands," Xin said. "What's even more encouraging is that both approaches worked, which gives us real confidence in developing patient-ready therapies." Experimental drug shows promise for one type of MS An experimental drug originally developed to treat lymphomas is the first-ever to show an effect against a form of multiple sclerosis for which no approved treatments are available, researchers reported at the American Academy of Neurology meeting in San Diego. Sanofi's tolebrutinib, an investigational oral Bruton's tyrosine kinase inhibitor, demonstrated a 31% delay in the onset of six-month confirmed disability progression in patients with non-active, non-relapsing secondary progressive multiple sclerosis, in a clinical trial. "This is the first clinical trial showing a positive effect in delaying disability progression in non-relapsing SPMS, a later form of the disease where neurological function gradually worsens over time and disability increases relentlessly," study leader Dr. Robert Fox of the Cleveland Clinic said in a statement. With 1,131 patients enrolled in the trial, the rate of confirmed disability progression at six months was 22.6% in the tolebrutinib group versus 30.7% in the placebo group, according to a report of the study published in The New England Journal of Medicine. More patients receiving tolebrutinib achieved improvement in disability, with a six-month confirmed disability improvement rate of 8.6% versus 4.5% with placebo, the researchers also reported. Markers of disease activity, including inflammation and tissue damage, also were reduced with tolebrutinib compared with placebo. Serious adverse events, particularly liver complications, were more frequent with tolebrutinib, which is currently under review for potential U.S. approval. "It appears that about one in 200 patients will have severe elevation of liver enzymes during the first three months of use, so careful monitoring is important, and the drug should be stopped immediately in those with liver enzyme elevations," Fox said. Separately, in two studies of patients with relapsing multiple sclerosis, tolebrutinib was not superior to Sanofi's Aubagio (teriflunomide) in decreasing annualised relapse rates, according to a second report in the same journal.

Using AI for medical diagnosis? What you should know about its safety

Image credit: Getty Images Artificial intelligence models may recommend different treatments for the same medical condition based solely on a patient's socioeconomic and demographic characteristics, researchers warn. The researchers invented nearly three dozen different patients and asked nine healthcare large language AI models how each one should be managed, in a thousand different emergency room situations. How does AI alter decisions? Despite identical clinical details, the AI models occasionally altered decisions based on patients' personal characteristics, affecting priority for care, diagnostic testing, treatment approach, and mental health evaluation, the researchers reported in Nature Medicine. Read-UAE ranks in top 10 nations with most AI firms per capita For example, advanced diagnostic tests such as CT scans or MRI were more often recommended for high-income patients, while low-income patients were more frequently advised to undergo no further testing, somewhat mimicking real-world healthcare inequities. The problems were seen in both proprietary and open-source AI models, the researchers found. AI in healthcare: Responsibility that comes with its usage "AI has the power to revolutionise healthcare, but only if it's developed and used responsibly," study co-leader Dr Girish Nadkarni of the Icahn School of Medicine at Mount Sinai in New York said in a statement. "By identifying where these models may introduce bias, we can work to refine their design, strengthen oversight, and build systems that ensure patients remain at the heart of safe, effective care," added coauthor Dr Eyal Klang, also of the Icahn School. Potential fixes for Sjogren's saliva and tears symptoms Researchers are closer to being able to fix the life-altering dryness of the mouth and eyes that afflicts patients with Sjogren's syndrome, based on success of two approaches tested in mice. The symptoms of the autoimmune disorder can make it hard to speak, eat and sleep. But exactly how the disease shuts down the body's production of tears and saliva has been a mystery until now, researchers reported in the International Journal of Oral Science. Their new study found that early in the progression of Sjogren's syndrome, a protein called tricellulin, which clasps together the cells of the glands that produce tears and saliva, is destroyed. Loss of the tight cellular junctions results in inadequate saliva secretion, the researchers found. Two possible interventions - an investigational drug (AT1001) and an experimental molecule - each restored saliva secretion in the mice, one by repairing the cell junctions and the other by stopping the breakdown of the junctions before it began. Both restored normal gland function, offering a potential blueprint for human treatment, the researchers said. "This changes how we think about treating Sjogren's syndrome," study leader Dr Xin Cong of Peking University said in a statement. "We're moving beyond simply calming inflammation. Now we can fix the actual structural damage in the glands," Xin said. "What's even more encouraging is that both approaches worked, which gives us real confidence in developing patient-ready therapies." Experimental drug shows promise for one type of MS An experimental drug originally developed to treat lymphomas is the first-ever to show an effect against a form of multiple sclerosis for which no approved treatments are available, researchers reported at the American Academy of Neurology meeting in San Diego. Sanofi's tolebrutinib, an investigational oral Bruton's tyrosine kinase inhibitor, demonstrated a 31 per cent delay in the onset of six-month confirmed disability progression in patients with non-active, non-relapsing secondary progressive multiple sclerosis, in a clinical trial. "This is the first clinical trial showing a positive effect in delaying disability progression in non-relapsing SPMS, a later form of the disease where neurological function gradually worsens over time and disability increases relentlessly," study leader Dr Robert Fox of the Cleveland Clinic said in a statement. With 1,131 patients enrolled in the trial, the rate of confirmed disability progression at six months was 22.6 per cent in the tolebrutinib group versus 30.7 per cent in the placebo group, according to a report of the study published in The New England Journal of Medicine. More patients receiving tolebrutinib achieved improvement in disability, with a six-month confirmed disability improvement rate of 8.6 per cent versus 4.5 per cent with placebo, the researchers also reported. Markers of disease activity, including inflammation and tissue damage, also were reduced with tolebrutinib compared with placebo. Serious adverse events, particularly liver complications, were more frequent with tolebrutinib, which is currently under review for potential US approval. "It appears that about one in 200 patients will have severe elevation of liver enzymes during the first three months of use, so careful monitoring is important, and the drug should be stopped immediately in those with liver enzyme elevations," Fox said. Separately, in two studies of patients with relapsing multiple sclerosis, tolebrutinib was not superior to Sanofi's Aubagio (teriflunomide) in decreasing annualised relapse rates, according to a second report in the same journal.