EP546 - Reducing Discrepancies in Pelvic MRI Reports for Rectal Cancer Surveillance during Non-Operative Management: An AI-Based Quality Improvement Study

Purpose/Background: Pelvic MRI is an essential tool for surveillance in non-operative management of rectal cancer. However, discrepancies in MRI reports lead to clinical uncertainty for providers and confusion for patients. In collaboration with a multi-disciplinary tumor board (MDTB) and a Radiology quality committee, we performed a quality improvement (QI) needs assessment to 1) analyze the frequency/type of MRI report discrepancies, 2) distinguish radiologist uncertainty from problematic reporting, and 3) strategize for future QI interventions.

Methods/Interventions: We searched for all patients diagnosed with rectal cancer from January 2018 to September 2025 based on ICD10 coding from a university hospital research repository. Eligible patients had at least one post-treatment pelvic MRI that reported clinical complete response to treatment, designated strictly as “ycT0N0” and using the Mercury 2016 tumor response grading system. We computationally screened for character strings or patterns in the text that contradicted the ycT0N0 designation and classified discrepancies by type. We then hand-searched screen-positive discrepant reports to confirm or rebut computational findings and determine the type of misclassification.

Results/Outcomes: We identified 1899 post-treatment pelvic MRIs from 827 unique eligible patients with rectal cancer. Among these, 677 MRIs (35.6%) from 216 patients (26.1%) reported "ycT0N0” stage. Computational screening identified 88 reports with discrepancies among 65 unique patients (30.1%). Discrepancies were classified computationally as: multiple stages in a single report (n=51), language suggesting the ongoing presence of tumor (n=9), or indications of diffusion/enhancement on MRI (n=29).

Among the 88 screen-positive reports, hand-searching confirmed 74 reports with discrepancies among 57 unique patients (26.4%). Hand-searching revealed 3 duplicate reports and 11 non-discrepant reports in which there were multiple masses (n=4); no designation of ycT0N0 (n=2); inflammation/proctitis without evidence of tumor (n=3); or unclear reasons for the discrepancy classification (n=2).

Among discrepant reports, 4 were corrected within 1 month; 3 corrections commented that staging reflected the depth of fibrosis rather than residual tumor. 10 of the discrepant reports noted limitations of imaging (e.g., patient movement, prothesis artifact, or limited planes or sequences) and legitimate radiologist uncertainty (e.g., no pre-treatment comparison).

Conclusion/Discussion: We identified discrepancies in MRI reports from more than 25% of patients with designated ycT0N0 tumors. While legitimate radiologist uncertainty was noted, many discrepant reports had no notations of uncertainty and may represent problematic reporting or a need for change in templated language. We are now incorporating these data into our MDTB and existing QI processes and have publicly posted our computational codes to facilitate future work on the quality of MRI reporting. Beyond important patient-provider communication during non-operative management, our findings have major implications for use of AI in summarization of patient records, reliance on AI to flag or correct discrepancies, and training AI to read images.