Harvard's AI Tool PICTURE Revolutionizes Brain Cancer Diagnosis During Surgery

AI tool accurately distinguishes between glioblastoma and primary central nervous system lymphoma

Harvard Medical SchoolSep 29 2025 A Harvard Medical School-led research team has developed an AI tool that can reliably tell apart two look-alike cancers found in the brain but with different origins, behaviors, and treatments. The tool, called PICTURE (Pathology Image Characterization Tool with Uncertainty-aware Rapid Evaluations), distinguished with near-perfect accuracy between glioblastoma - the most common and aggressive brain tumor - and primary central nervous system lymphoma (PCNSL), a rarer cancer often mistaken for glioblastoma. While both can appear in the brain, glioblastoma arises from brain cells, whereas PCNSL develops from immune cells. Their similarities under the microscope often lead to misdiagnosis, with serious consequences for treatment. The work, supported in part by the National Institutes of Health, is described Sept. 29 in Nature Communications. The AI model is publicly available for other scientists to use and build upon, the team said. Correctly identifying look-alike tumors in the brain during surgery is one of the toughest diagnostic challenges in neuro-oncology, the researchers said. An accurate diagnosis while the patient is still in the operating room can help expedite critical treatment choices, such as whether to operate and remove the cancerous tissue - as should be done with glioblastoma - or leave it behind and opt for radiation and chemotherapy instead, the preferred therapy for PCNSL. Inaccurate or delayed diagnosis of cancers in the brain can lead to unnecessary surgery and delays in proper treatment. What makes the tool especially valuable is its ability to be deployed during surgery, providing critical insights in real time to surgeons and pathologists. Our model can minimize errors in diagnosis by distinguishing between tumors with overlapping features and help clinicians determine the best course of treatment based on a tumor's true identity." Kun-Hsing Yu, study senior author, associate professor of biomedical informatics in the Blavatnik Institute at HMS and HMS assistant professor of pathology at Brigham and Women's Hospital During brain tumor surgery, surgeons typically remove tumor tissue for rapid evaluation under a microscope. The evaluation is done by freezing the sample in liquid nitrogen, which can distort the cellular features somewhat but provides a quick, real-time assessment. The process takes 15 minutes or so. Based on the results of this first-glance evaluation, surgeons determine whether to remove the tumor or leave it behind and opt for radiation and chemotherapy. Then, over the next few days, pathologists conduct a more detailed and more reliable evaluation of the tumor sample. In about 1 in 20 cases, the initial diagnosis of a tumor changes on second read, Yu said. This is precisely where the new AI system could play a valuable role - removing uncertainty and reducing the risk for error during operation when critical decisions are made. "Our model shows reliable performance on frozen sections during brain surgery and in scenarios with significant diagnostic disagreement among human experts," he said. The tool was tested in five hospitals and outperformed both human pathologists and other AI models. A unique aspect of the new model is an "uncertainty detector," which allows it to not only distinguish between cancer types with high accuracy but also to signal when it's unsure in its judgment - an important feature for high-stakes medical scenarios. The new study builds on earlier work led by Yu to develop an AI system that could reliably decode the molecular features of different types of gliomas. How PICTURE spots brain-cancer doppelgangers Each year, more than 300,000 people worldwide are diagnosed with tumors in the brain or central nervous system, and more than 200,000 deaths occur as a result. The World Health Organization recognizes about 109 different types of brain and spinal cord tumors, each with its own unique features under the microscope or at the genetic level. Accurately distinguishing PCNSL from glioblastoma during surgery could allow surgeons to spare brain tissue instead of removing it. Patients with PCNSL are then referred for radiation and chemotherapy, the preferred treatments for this type of tumor. By contrast, glioblastoma requires surgical removal of as much of the cancerous brain tissue as possible. A near PICTURE-perfect performance The model - which Yu developed with study co-first authors Junhan Zhao and Shih-Yen Lin - was evaluated on 2,141 brain pathology slides collected worldwide, including rare cases across both frozen sections and formalin-fixed samples. It was designed to spot critical cancer features including tumor cell density, cell shape, and presence of necrosis. The scientists tested PICTURE's performance across five international hospitals in four countries. In every case, the AI model outperformed existing AI tools and traditional frozen-section assessment, the standard of care for real-time tumor typing. In tests, the PICTURE model correctly distinguished glioblastoma from PCNSL more than 98 percent of the time - a level of accuracy that held up when tested in five independent international patient groups. In addition, PICTURE identified samples belonging to 67 CNS cancers that were neither gliomas nor lymphomas. The model could spot tumors it had not seen during its training and, when it did, it raised a red flag for human review. In other words, the tool knew when it didn't know, Yu said, and this prevented the system from pigeonholing unclear cases into known categories. This feature renders the model unique among other AI systems, the researchers said. In comparison, other AI tools can differentiate in a binary, either-or fashion - disease A versus disease B. This is especially problematic for brain pathology, Yu noted, because there are more than 100 different subtypes of brain cancers, and many of them are relatively rare. PICTURE outperformed human pathologists in hard-to-distinguish tumors in the brain. In tests, human specialists showed significant disagreement on difficult diagnoses, with some tumor types misdiagnosed 38 percent of the time. PICTURE correctly identified all these cases, offering support when expert opinion varies. Launching PICTURE into the real world Deploying the tool could be a great opportunity for human-AI collaboration, the researchers said. They envision implementing the system across operating rooms and pathology departments as an initial filter to differentiate glioblastoma from PCNSL and inform in-the-OR treatment calls. Using the model could also democratize access to neuropathology, a highly specialized area of expertise with a dearth of specialists and uneven distribution of experts across the country and world. In addition, the tool can also be used as an educational tool for training the next generation of pathologists to recognize look-alike lesions in the brain where critical differences are obscured under similar appearance. The researchers noted that most tumor samples were obtained from white patients, so more research is needed to confirm the model's accuracy across diverse populations. And while the tool focused on glioblastoma and PCNSL, future work could expand it to other cancer types and combine it with genetic and molecular data for deeper insights. Source: Harvard Medical School Journal reference: Zhao, J., et al. (2025). Uncertainty-aware ensemble of foundation models differentiates glioblastoma from its mimics. Nature Communications. doi.org/10.1038/s41467-025-64249-6

AI Distinguishes Glioblastoma From Look-Alike Cancers During Surgery | Newswise

Newswise -- A Harvard Medical School-led research team has developed an AI tool that can reliably tell apart two look-alike cancers found in the brain but with different origins, behaviors, and treatments. The tool, called PICTURE (Pathology Image Characterization Tool with Uncertainty-aware Rapid Evaluations), distinguished with near-perfect accuracy between glioblastoma -- the most common and aggressive brain tumor -- and primary central nervous system lymphoma (PCNSL), a rarer cancer often mistaken for glioblastoma. While both can appear in the brain, glioblastoma arises from brain cells, whereas PCNSL develops from immune cells. Their similarities under the microscope often lead to misdiagnosis, with serious consequences for treatment. The work, supported in part by the National Institutes of Health, is described Sept. 29 in Nature Communications. The AI model is publicly available for other scientists to use and build upon, the team said Correctly identifying look-alike tumors in the brain during surgery is one of the toughest diagnostic challenges in neuro-oncology, the researchers said. An accurate diagnosis while the patient is still in the operating room can help expedite critical treatment choices, such as whether to operate and remove the cancerous tissue -- as should be done with glioblastoma -- or leave it behind and opt for radiation and chemotherapy instead, the preferred therapy for PCNSL. Inaccurate or delayed diagnosis of cancers in the brain can lead to unnecessary surgery and delays in proper treatment. What makes the tool especially valuable is its ability to be deployed during surgery, providing critical insights in real time to surgeons and pathologists. "Our model can minimize errors in diagnosis by distinguishing between tumors with overlapping features and help clinicians determine the best course of treatment based on a tumor's true identity," said study senior author Kun-Hsing Yu, associate professor of biomedical informatics in the Blavatnik Institute at HMS and HMS assistant professor of pathology at Brigham and Women's Hospital. During brain tumor surgery, surgeons typically remove tumor tissue for rapid evaluation under a microscope. The evaluation is done by freezing the sample in liquid nitrogen, which can distort the cellular features somewhat but provides a quick, real-time assessment. The process takes 15 minutes or so. Based on the results of this first-glance evaluation, surgeons determine whether to remove the tumor or leave it behind and opt for radiation and chemotherapy. Then, over the next few days, pathologists conduct a more detailed and more reliable evaluation of the tumor sample. In about 1 in 20 cases, the initial diagnosis of a tumor changes on second read, Yu said. This is precisely where the new AI system could play a valuable role -- removing uncertainty and reducing the risk for error during operation when critical decisions are made. "Our model shows reliable performance on frozen sections during brain surgery and in scenarios with significant diagnostic disagreement among human experts," he said. The tool was tested in five hospitals and outperformed both human pathologists and other AI models. A unique aspect of the new model is an "uncertainty detector," which allows it to not only distinguish between cancer types with high accuracy but also to signal when it's unsure in its judgment -- an important feature for high-stakes medical scenarios. The new study builds on earlier work led by Yu to develop an AI system that could reliably decode the molecular features of different types of gliomas. Each year, more than 300,000 people worldwide are diagnosed with tumors in the brain or central nervous system, and more than 200,000 deaths occur as a result. The World Health Organization recognizes about 109 different types of brain and spinal cord tumors, each with its own unique features under the microscope or at the genetic level. Accurately distinguishing PCNSL from glioblastoma during surgery could allow surgeons to spare brain tissue instead of removing it. Patients with PCNSL are then referred for radiation and chemotherapy, the preferred treatments for this type of tumor. By contrast, glioblastoma requires surgical removal of as much of the cancerous brain tissue as possible. The model -- which Yu developed with study co-first authors Junhan Zhao and Shih-Yen Lin -- was evaluated on 2,141 brain pathology slides collected worldwide, including rare cases across both frozen sections and formalin-fixed samples. It was designed to spot critical cancer features including tumor cell density, cell shape, and presence of necrosis. The scientists tested PICTURE's performance across five international hospitals in four countries. In every case, the AI model outperformed existing AI tools and traditional frozen-section assessment, the standard of care for real-time tumor typing. In tests, the PICTURE model correctly distinguished glioblastoma from PCNSL more than 98 percent of the time -- a level of accuracy that held up when tested in five independent international patient groups. In addition, PICTURE identified samples belonging to 67 CNS cancers that were neither gliomas nor lymphomas. The model could spot tumors it had not seen during its training and, when it did, it raised a red flag for human review. In other words, the tool knew when it didn't know, Yu said, and this prevented the system from pigeonholing unclear cases into known categories. This feature renders the model unique among other AI systems, the researchers said. In comparison, other AI tools can differentiate in a binary, either-or fashion -- disease A versus disease B. This is especially problematic for brain pathology, Yu noted, because there are more than 100 different subtypes of brain cancers, and many of them are relatively rare. PICTURE outperformed human pathologists in hard-to-distinguish tumors in the brain. In tests, human specialists showed significant disagreement on difficult diagnoses, with some tumor types misdiagnosed 38 percent of the time. PICTURE correctly identified all these cases, offering support when expert opinion varies. Deploying the tool could be a great opportunity for human-AI collaboration, the researchers said. They envision implementing the system across operating rooms and pathology departments as an initial filter to differentiate glioblastoma from PCNSL and inform in-the-OR treatment calls. Using the model could also democratize access to neuropathology, a highly specialized area of expertise with a dearth of specialists and uneven distribution of experts across the country and world. In addition, the tool can also be used as an educational tool for training the next generation of pathologists to recognize look-alike lesions in the brain where critical differences are obscured under similar appearance. The researchers noted that most tumor samples were obtained from white patients, so more research is needed to confirm the model's accuracy across diverse populations. And while the tool focused on glioblastoma and PCNSL, future work could expand it to other cancer types and combine it with genetic and molecular data for deeper insights. Additional authors on the paper include Junhan Zhao, Shih-Yen Lin, Raphaël Attias, Liza Mathews, Christian Engel, Guillaume Larghero, Dmytro Vremenko, Ting-Wan Kao, Tsung-Hua Lee, Yu-Hsuan Wang, Cheng Che Tsai, Eliana Marostica, Ying-Chun Lo, David Meredith, Keith L. Ligon, Omar Arnaout, Thomas Roetzer-Pejrimovsky, Shih-Chieh Lin, Natalie NC Shih, Nipon Chaisuriya, David J. Cook, Jung-Hsien Chiang, Chia-Jen Liu, Adelheid Woehrer, Jeffrey A. Golden, and MacLean P. Nasrallah. The work was supported in part by the National Institute of General Medical Sciences grant R35GM142879, the Department of Defense Peer Reviewed Cancer Research Program Career Development Award HT9425-23-1-0523, the Research Scholar Grant RSG-24-1253761-01-ESED (grant DOI: https://doi.org/10.53354/ACS.RSG-24-1253761-01-ESED.pc.gr.193749) from the American Cancer Society, the Google Research Scholar Award, the HMS Dean's Innovation Award, and the Blavatnik Center for Computational Biomedicine Award.