EP756 - Using Artificial Intelligence to Increase Readability of ASCRS Online Patient

Monday, May 11, 2026

8:38 AM - 8:43 AM ET

ePoster Presenter(s)

MU

Melissa Ulbrick, MD

Resident
Lenox Hill Hospital
New York, New York, United States

Disclosure information not submitted.

Author(s)

Katie Carsky, MD, MS (she/her/hers)

Fellow
Cleveland Clinic Florida
Pembroke Pines, FL, United States

All of the relevant financial relationships listed below have been mitigated.
This individual has no financial relationships with ineligible companies.
AF

Alexander Farrell, DO

Resident Physician
Lenox Hill Hospital, Northwell Health
New York, New York, United States

All of the relevant financial relationships listed below have been mitigated.
This individual has no financial relationships with ineligible companies.

Senior Author(s)

RW

Richard L. Whelan, MD, FACRS

Attending Surgeon
NORTHWELL HEALTH, LENOX HILL HOSPITAL
New York, New York, United States

Disclosure information not submitted.

All of the relevant financial relationships listed below have been mitigated.

Katie Carsky, MD, MS: This individual has no financial relationships with ineligible companies.

Alexander Farrell, DO: This individual has no financial relationships with ineligible companies.

Purpose/Background: With the increasing accessibility of health information, patients frequent the internet to learn more about their conditions. The American Society of Colorectal Surgeons (ASCRS) has patient education resources on numerous colorectal conditions that are currently written at the average of a college level. The American Medical Association (AMA) and National Institutes of Health (NIH) recommend patient resources to be at a 6th and 8th grade level respectively. Patient education materials often exceed this goal, compromising comprehension and shared decision making. This study aimed to evaluate if artificial intelligence (AI) can improve the readability of ASCRS resources.

Methods/Interventions: 51 of the ASCRS patient education materials were rewritten using GPT-5 with the following prompt: "rewrite this text so it is understandable at a 6th grade reading level while keeping it medically accurate and complete”. The GPT-5 response was analyzed using a validated readability calculator, the Flesch-Kincaid (FK) grade level. A lower FK grade level is easier to read. The readability of the GPT-5 output was compared to the original ASCRS patient education materials using paired t-test and Pearson correlations.

Results/Outcomes: The average FK grade level of the ASCRS resources was 11.30 which correlates to college level. The average FK grade level of the GPT-5 output was 8.06 (SD 1.00) which correlates to 10th-12th grade. Compared to the original ASCRS resources, the GPT-5 outputs decreased FK grade level by 3.43 (P <.001) correlating with improved readability. 31% (n=16) of the GPT-5 outputs were at an 8th to 9th grade level compared to none of the original ASCRS resources. The Pearson correlation for the improvements in FK readability scores showed strong positive correlation between FK grade level and the number of words per sentence r = 0.73, P <.001 and between the FK grade level and the numbers of syllables per word r =0.71, P <.001.

Conclusion/Discussion: GPT-5 significantly improved the readability of ASCRS patient education resources. These improvements were primarily driven by decreasing sentence length and words with fewer syllables. However, none of the resources were at the 6th grade level as recommended by the AMA and 16 (31%) were at the 8th to 9th grade close to the recommendation by the NIH. Since a majority of the resources exceeded the readability targets, further advances in artificial intelligence are required to meet the goals set by the AMA and NIH.