AI Evaluates AI: PROLIFERATE_AI Assesses Cardiac Diagnostic Tool in Emergency Departments

Testing AI with AI: Ensuring effective AI implementation in clinical practice

"AI is becoming more common in healthcare, but it doesn't always fit in smoothly with the vital work of our doctors and nurses," says Flinders University's Dr Maria Alejandra Pinero de Plaza, who led the research. "We need to confirm these systems are trustworthy and work consistently for everyone, ensuring they are able to support medical teams rather than slowing them down." Developed by Dr Pinero de Plaza and her team, PROLIFERATE_AI is a human-centred evaluation tool that uses artificial intelligence alongside researcher analysis to assess how well AI tools work in hospitals. "In order to understand if the AI systems are viable, we look at how easy they are to use, how well doctors and nurses adopt them, and how they impact patient care," says Dr Pinero de Plaza, a research fellow in Flinders' Caring Futures Institute. "It's not just about making AI accurate; it's about making sure it's easy to understand, adaptable, and genuinely helpful for doctors and patients when it matters most." Published in the International Journal of Medical Informatics, the study used PROLIFERATE_AI to assess the RAPIDx AI tool; designed to help emergency doctors quickly and accurately diagnose cardiac conditions by rapidly analysing clinical and biochemical data. With chest pain one of the most common reasons for ED visits, the South Australian health system has been part of an NHMRC-funded trial being run across 12 hospitals in metropolitan and rural SA, which is currently analysing its 12-month patient outcomes. Before and during the trial, the PROLIFERATE researchers evaluated the tool, with medical and nursing staff at the participating hospitals being provided the opportunity to share their insights on interacting with the RAPIDx AI tool. The results showed that while experienced clinicians, such as ED consultants and registrars, demonstrated high comprehension and engagement with the RAPIDx AI tool, less experienced users, including residents and interns, faced usability challenges. Registered nurses also reported strong emotional engagement with the tool, recognising its potential to enhance patient safety by reducing diagnostic uncertainty. "What sets PROLIFERATE_AI apart is its ability to provide actionable insights," says Dr Pinero de Plaza. "Rather than focusing solely on technical performance, we evaluate AI tools based on real-world usability and clinician trust, ensuring that these technologies are not just innovative but also practical and accessible. "We want to set a new standard for AI implementation, fundamental care, and evaluation standards, starting with emergency medicine." While the research demonstrated consultants and registrars benefited most from using RAPIDx AI, the study also highlighted the need for targeted training and workflow-aligned interfaces to improve adoption rates among new users. It also demonstrated clinician desires in having further automated data integration into the user interface. The authors say the findings highlight that AI in healthcare should always be developed with the end-users in mind. "Our goal is to create AI solutions that empower doctors and nurses, not replace them," says Dr Pinero de Plaza. "Technology alone cannot solve the complexities of emergency care. We need AI systems that work seamlessly with clinicians, support decision-making under pressure, and integrate smoothly into existing workflows. "We are committed to enhancing our AI tools to ensure they are instinctively user-friendly and to develop customised training programs that meet the diverse needs of healthcare professionals and others." Last month, the team received a CSIRO award from the On PRIME innovation program ($5000), which will aid in the continuous refinement of the predictive modelling and implementation science tool.

Testing AI with AI: Ensuring effective AI implementation in clinical practice

Using a pioneering artificial intelligence platform, Flinders University researchers have assessed whether a cardiac AI tool recently trialed in South Australian hospitals actually has the potential to assist doctors and nurses to rapidly diagnose heart issues in emergency departments. "AI is becoming more common in health care, but it doesn't always fit in smoothly with the vital work of our doctors and nurses," says Flinders University's Dr. Maria Alejandra Pinero de Plaza, who led the research. "We need to confirm these systems are trustworthy and work consistently for everyone, ensuring they are able to support medical teams rather than slowing them down." Developed by Dr. Pinero de Plaza and her team, PROLIFERATE_AI is a human-centered evaluation tool that uses artificial intelligence alongside researcher analysis to assess how well AI tools work in hospitals. "In order to understand if the AI systems are viable, we look at how easy they are to use, how well doctors and nurses adopt them, and how they impact patient care," says Dr. Pinero de Plaza, a research fellow at Flinders' Caring Futures Institute. "It's not just about making AI accurate; it's about making sure it's easy to understand, adaptable, and genuinely helpful for doctors and patients when it matters most." Published in the International Journal of Medical Informatics, the study used PROLIFERATE_AI to assess the RAPIDx AI tool, designed to help emergency doctors quickly and accurately diagnose cardiac conditions by rapidly analyzing clinical and biochemical data. With chest pain one of the most common reasons for ED visits, the South Australian health system has been part of a trial being run across 12 hospitals in metropolitan and rural SA, which is currently analyzing its 12-month patient outcomes. Before and during the trial, the PROLIFERATE researchers evaluated the tool, with medical and nursing staff at the participating hospitals being provided the opportunity to share their insights on interacting with the RAPIDx AI tool. The results show that while experienced clinicians, such as ED consultants and registrars, demonstrate high comprehension and engagement with the RAPIDx AI tool, less experienced users, including residents and interns, face usability challenges. Registered nurses also reported strong emotional engagement with the tool, recognizing its potential to enhance patient safety by reducing diagnostic uncertainty. "What sets PROLIFERATE_AI apart is its ability to provide actionable insights," says Dr. Pinero de Plaza. "Rather than focusing solely on technical performance, we evaluate AI tools based on real-world usability and clinician trust, ensuring that these technologies are not just innovative but also practical and accessible. "We want to set a new standard for AI implementation, fundamental care, and evaluation standards, starting with emergency medicine." While the research demonstrated consultants and registrars benefited most from using RAPIDx AI, the study also highlighted the need for targeted training and workflow-aligned interfaces to improve adoption rates among new users. It also demonstrated clinician desires in having further automated data integration into the user interface. The authors say the findings highlight that AI in health care should always be developed with the end-users in mind. "Our goal is to create AI solutions that empower doctors and nurses, not replace them," says Dr. Pinero de Plaza. "Technology alone cannot solve the complexities of emergency care. We need AI systems that work seamlessly with clinicians, support decision-making under pressure, and integrate smoothly into existing workflows. "We are committed to enhancing our AI tools to ensure they are instinctively user-friendly and to develop customized training programs that meet the diverse needs of health care professionals and others."

Evaluating the usability of AI in emergency medicine to support clinicians

Flinders UniversityFeb 6 2025 Using a pioneering artificial intelligence platform, Flinders University researchers have assessed whether a cardiac AI tool recently trialed in South Australian hospitals actually has the potential to assist doctors and nurses to rapidly diagnose heart issues in emergency departments. "AI is becoming more common in healthcare, but it doesn't always fit in smoothly with the vital work of our doctors and nurses," says Flinders University's Dr Maria Alejandra Pinero de Plaza, who led the research. "We need to confirm these systems are trustworthy and work consistently for everyone, ensuring they are able to support medical teams rather than slowing them down." Developed by Dr Pinero de Plaza and her team, PROLIFERATE_AI is a human-centered evaluation tool that uses artificial intelligence alongside researcher analysis to assess how well AI tools work in hospitals. In order to understand if the AI systems are viable, we look at how easy they are to use, how well doctors and nurses adopt them, and how they impact patient care. It's not just about making AI accurate; it's about making sure it's easy to understand, adaptable, and genuinely helpful for doctors and patients when it matters most." Dr. Pinero de Plaza, research fellow, Flinders' Caring Futures Institute Published in the International Journal of Medical Informatics, the study used PROLIFERATE_AI to assess the RAPIDx AI tool; designed to help emergency doctors quickly and accurately diagnose cardiac conditions by rapidly analysing clinical and biochemical data. With chest pain one of the most common reasons for ED visits, the South Australian health system has been part of an NHMRC-funded trial being run across 12 hospitals in metropolitan and rural SA, which is currently analysing its 12-month patient outcomes. Before and during the trial, the PROLIFERATE researchers evaluated the tool, with medical and nursing staff at the participating hospitals being provided the opportunity to share their insights on interacting with the RAPIDx AI tool. The results showed that while experienced clinicians, such as ED consultants and registrars, demonstrated high comprehension and engagement with the RAPIDx AI tool, less experienced users, including residents and interns, faced usability challenges. Registered nurses also reported strong emotional engagement with the tool, recognising its potential to enhance patient safety by reducing diagnostic uncertainty. "What sets PROLIFERATE_AI apart is its ability to provide actionable insights," says Dr Pinero de Plaza. "Rather than focusing solely on technical performance, we evaluate AI tools based on real-world usability and clinician trust, ensuring that these technologies are not just innovative but also practical and accessible. "We want to set a new standard for AI implementation, fundamental care, and evaluation standards, starting with emergency medicine." While the research demonstrated consultants and registrars benefited most from using RAPIDx AI, the study also highlighted the need for targeted training and workflow-aligned interfaces to improve adoption rates among new users. It also demonstrated clinician desires in having further automated data integration into the user interface. The authors say the findings highlight that AI in healthcare should always be developed with the end-users in mind. "Our goal is to create AI solutions that empower doctors and nurses, not replace them," says Dr Pinero de Plaza. "Technology alone cannot solve the complexities of emergency care. We need AI systems that work seamlessly with clinicians, support decision-making under pressure, and integrate smoothly into existing workflows. "We are committed to enhancing our AI tools to ensure they are instinctively user-friendly and to develop customised training programs that meet the diverse needs of healthcare professionals and others." Last month, the team received a CSIRO award from the On PRIME innovation program ($5000), which will aid in the continuous refinement of the predictive modelling and implementation science tool. The paper 'Human-centred AI for emergency cardiac care: Evaluating RAPIDx AI with PROLIFERATE_AI' by Maria Alejandra Pinero de Plaza, Kristina Lambrakis, Fernando Marmolejo-Ramos, Alline Beleigoli, Mandy Archibald, Lalit Yadav, Penelope McMillan, Robyn Clark, Michael Lawless, Erin Morton, Jeroen Hendriks, Alison Kitson, Renuka Visvanathan, Derek P. Chew, Carlos Javier Barrera Causil is published in the International Journal of Medical Informatics. DOI: 10.1016/j.ijmedinf.2025.105810. The study was supported by funding from the National Health and Medical Research Council (NHMRC), grant number GNT1191914. Flinders University Journal reference: Pinero de Plaza, M. A., et al. (2025). Human-centred AI for emergency cardiac care: Evaluating RAPIDx AI with PROLIFERATE_AI. International Journal of Medical Informatics. doi.org/10.1016/j.ijmedinf.2025.105810.