New collaborative plans to rank radiology AI applications

Academic radiology departments and the American College of Radiology (ACR) will anchor a new Healthcare AI Challenge Collaborative hosted by Mass General Brigham AI.

Described as a response to the complexities involved with responsible development and use of AI in healthcare, especially in radiology, the effort aims to generate rankings to provide industry, healthcare stakeholders, and the public with a transparent analysis and performance of various AI uses across a range of healthcare data and clinical scenarios, according to Mass General Brigham.

“The velocity of AI innovations and breadth of their healthcare applications ... leaves clinicians struggling to determine the effectiveness of these innovations in safely delivering value to healthcare providers and our patients," said Keith Dreyer, DO, PhD, Mass General Brigham chief data science officer and leader of the healthcare system’s AI business, in a statement.

Healthcare professionals with relevant healthcare credentials will be verified and granted access to the Healthcare AI Challenge to assess AI tools for effectiveness in specific medical tasks in a simulated environment. Participants can then provide their feedback on performance and utility to generate publicly available insights and analytics, according to Mass General Brigham.

“We need healthcare delivery communities to provide real-world experience of the application of AI at the point of care. That is what the Healthcare AI Challenge is designed to do,” said Alistair Erskine, MD, chief information and digital officer at Emory Healthcare and Emory University, in the statement.

Mass General Brigham noted that the Healthcare AI Challenge will add new AI solutions, events, multimodal diverse data, domain experts, and specialties to its interactive environment, including pathology, genomics, waveform, as well as public and specialized text-based foundation models.