Chest x-ray reading accuracy declines significantly in fatigued radiology residents after overnight shifts, and AI offers little help, according to a recent study. 

A team from Aachen University Hospital in Germany measured speed and accuracy among 10 residents when reading x-rays before and after a night-shift, and with and without AI, with findings suggesting AI may not be the urgently needed solution to mitigate radiologist fatigue.  

"Night-shift-induced fatigue impaired diagnostic accuracy without slowing reporting speed, raising patient safety concerns. The utilized AI model did not restore pre-shift performance," the researchers noted. The study was published October 22 in European Radiology. 

Rising study volumes and persistent workforce shortages are intensifying radiologist workload, the authors wrote. Prior research has linked fatigue to decreased detection of clinically relevant findings and reduced diagnostic confidence, they added. Although AI tools have shown potential to improve accuracy and alleviate workload, evidence that they can offset fatigue-related performance loss remains limited, the group noted.

To explore the issue, the group recruited 10 radiology residents (mean experience, 35.5 months) who each interpreted three sets of 33 bedside chest x-rays in a prospective intra-individual reader study, completing two sessions before a first-call on-site night shift and one session immediately after, the latter with and without AI assistance. They used an AI model they previously developed at their institution and which had shown excellent agreement with expert radiologists.

Technical setup and customized DICOM viewer. a) Radiology resident reads chest x-rays on a single-screen workstation equipped with screen-based eye tracking (bar unit below the screen). b) The x-ray (right) was displayed alongside the structured reporting template (left). c) Without AI assistance, readers reported the presence and severity of findings using drop-down menus. d) With AI assistance, the template was pre-filled with the model’s suggestions, which could be accepted or modified.European Radiology

Screen-based eye tracking identified fatigued readers by measuring how efficiently their eyes moved across each x-ray (saccadic main sequence slope), while diagnostic accuracy was gauged by how closely each resident’s interpretations of five findings (cardiomegaly, pulmonary congestion, pleural effusion, pulmonary opacities, and atelectasis) matched those of a panel of expert radiologists. 

According to the analysis, six of 10 residents were classified as fatigued. Their diagnostic accuracy declined significantly post-shift, from a κ of 0.71 to 0.61 (p

"Assistance by the specific AI model used in this study did not lead to a significant improvement in diagnostic accuracy and failed to restore performance to pre-shift levels, suggesting a limited ability to offset fatigue-related declines in performance," the authors wrote. 

Ultimately, the findings raise questions about whether current duty-hour regulations and staffing models adequately address fatigue-related performance risk, the authors noted. Future work should evaluate whether more comprehensive AI tools, such as systems that incorporate confidence scores or broader finding detection, can better support fatigued readers, they suggested. 

"More comprehensive or differently integrated AI models may provide different levels of support in future studies," the authors concluded. 

The full study is available here.