EPD104 - Improving Prediction of Complete Response for Rectal Cancer using a Clinical-Endoscopic Artificial Intelligence Model

ePoster of Distinction Presenter(s)

• 

  Kevin A. Chen, MD

  Clinical Fellow
  Cleveland Clinic
  Cleveland Clinic
  Cleveland, OH, United States

  All disclosed relationships have been reviewed and managed prior to the start of this activity. Any relevant relationships will be listed below. This individual has no financial relationships to disclose

Author(s)

• 

  Metincan Erkaya, MD

  Clinical Research Fellow
  Cleveland Clinic Digestive Diseases & Surgery Institute
  Cleveland Clinic
  Cleveland, OH, United States

  All disclosed relationships have been reviewed and managed prior to the start of this activity. Any relevant relationships will be listed below. This individual has no financial relationships to disclose
• JS

  Josh Sommovilla, MD

  Assistant Professor of Surgery
  Cleveland Clinic
  Cleveland, Ohio, United States

  Disclosure information not submitted.
• 

  Michael A. Valente, DO

  Associate Professor of Surgery
  Cleveland Clinic
  cleveland, Ohio, United States

  Disclosure information not submitted.
• SK

  Smitha Krishnamurthi, MD

  Staff
  Cleveland Clinic Foundation-Main Campus
  Cleveland, Ohio, United States

  Disclosure information not submitted.
• JM

  Jacob A. Miller, MD

  Assistant Professor
  Radiation Oncology, Cleveland Clinic
  Cleveland, Ohio, United States

  Disclosure information not submitted.
• AP

  Andrei S. Purysko, MD

  Staff Physician
  Section of Abdominal Imaging and Nuclear Radiology Department, Imaging Institute, Cleveland Clinic
  Cleveland, Ohio, United States

  Disclosure information not submitted.
• 

  David Liska, MD

  Associate Professor, Chair
  Department of Colorectal Surgery, Cleveland Clinic
  Cleveland Clinic, Cleveland, Ohio
  Cleveland, OH, United States

  All disclosed relationships have been reviewed and managed prior to the start of this activity. Any relevant relationships will be listed below. This individual has no relevant financial relationships.
• 

  Scott R. Steele, MD, MBA (he/him/his)

  President, Cleveland Clinic, Main Campus Submarket and Rupert B. Turnbull M.D., Endowed Chair in Colorectal Surgery
  Cleveland Clinic
  Cleveland Clinic Lerner College of Medicine of Case Western Reserve University
  Cleveland, OH, United States

  All disclosed relationships have been reviewed and managed prior to the start of this activity. Any relevant relationships will be listed below. This individual has no financial relationships to disclose

Senior Author(s)

• 

  Emre Gorgun, MD, MBA. FACS, FASCRS. (he/him/his)

  Vice Chair, Department of Colorectal Surgery
  Cleveland Clinic
  Cleveland Cliinic
  Cleveland, Ohio, United States

  All disclosed relationships have been reviewed and managed prior to the start of this activity. Any relevant relationships will be listed below. This individual has no relevant financial relationships.

Purpose/Background: The use of total neoadjuvant therapy (TNT) for locally advanced rectal cancer (LARC) has introduced a new paradigm for management of the disease, especially nonoperative management (NOM) for patients with clinical complete response (cCR). Current methods for evaluating response to neoadjuvant therapy, including MRI and endoscopy, are accurate, but imperfect for predicting oncologic outcomes. About 10% of patients who appear to have incomplete response ultimately have pathologic complete response (pCR) after undergoing surgery, while 30-40% of patients with cCR may develop local regrowth. We hypothesize that we can harness artificial intelligence (AI) to develop a predictive model which uses clinical data and endoscopic images to more accurately predict complete response (CR) for LARC after TNT.

Methods/Interventions: We used a previously collected institutional database which includes patients with LARC who underwent TNT, between January 2015 and May 2025. For a subset of patients, restaging endoscopy images were also captured. The primary endpoint was CR, defined as either pCR after undergoing primary surgery or sustained cCR after 2 years of NOM. Patients missing data on restaging or the primary endpoint were excluded. Baseline accuracy of MRI tumor response grade (mrTRG) and restaging endoscopy were calculated. We then developed a combined clinical-endoscopic model (Figure 1). First, a clinical model which included risk factors from the initial staging, TNT sequence and interruptions, and restaging MRI and endoscopy was trained. Various linear and tree-based modeling approaches were compared. Next, we implemented a vision transformer (ViT) which had been pretrained on the Kvasir gastrointestinal endoscopy dataset and fine-tuned it on restaging endoscopy images. Finally, we combined the clinical and endoscopic models. Models were evaluated using 4-fold nested cross-validation and metrics including area under the receiver operating characteristic curve (AUROC), accuracy, sensitivity, and specificity. For the ViT, heatmaps describing the model's attention to specific parts of the endoscopic image were generated.

Results/Outcomes: The dataset included 382 patients, with endoscopic images available for 65 patients. This included 328 patients who underwent surgery after completion of TNT, of whom 74 (22.6%) had pCR and 54 patients who underwent NOM, of whom 44 (80.0%) had 2-year sustained cCCR. In total, the models included 82 variables. The highest performing clinical model showed an AUROC of 0.78 for predicting CR, compared with 0.70 for mrTRG and 0.74 for endoscopic response. The combined clinical-endoscopic model (random forest and ViT) showed an AUROC of 0.85, an accuracy of 84.8%, sensitivity of 63.3%, and specificity of 91.9% (Figure 2A). Sample heatmaps for the ViT model are shown in Figure 2B.

Conclusion/Discussion: This study developed and internally validated a clinical-endoscopic model, which uses machine learning and computer vision to predict CR for rectal cancer after TNT. We found that the model matches or exceeds the accuracy of existing modalities for evaluating response to TNT, including MRI and endoscopy, and we expect model performance to improve as restaging is performed more routinely. With further data collection, model optimization, and prospective validation, this model has the potential to guide treatment for near-complete response, personalize NOM protocols, and improve patient selection for NOM after TNT.

Figure 1. Modeling strategy for combined clinical-endoscopic model
The combined clinical-endoscopic model to predict sustained clinical complete response or pathologic complete response after total neoadjuvant therapy for rectal cancer incorporates restaging endoscopic imaging, large-scale pre-training on colonoscopy, and clinical data including patient/tumor characteristics and treatment details. TNT - total neoadjuvant therapy, cCR - clinical complete response, pCR - pathologic complete response
Figure 2A/B. Receiver operating characteristic curves and model attention maps
2A. Receiver operating characteristic curves describing the accuracy of the MRI tumor regression score (mrTRG), endoscopic response, clinical model, and clinical-endoscopic model. 2B. Model attention maps from a vision transformer trained on endoscopic images. Each pair shows the original image (left) and the corresponding heatmap (right). Warmer colors indicate regions most influential in the model's decision.