EP153 - Digital Health Literacy in Colorectal Surgery: EHR Messaging Readability Analysis in Hemorrhoidectomy

Purpose/Background: Digital health literacy is a known determinant of health that impacts patient engagement and education. EHR-integrated patient messaging platforms are increasingly serving as a communication medium for exchange of medical information and should be assessed for accessibility to patients. In this study, readability statistics of EHR messages between colorectal surgery teams and their patients were analyzed to determine the Flesch-Kincaid reading grade level (FKRGL).

Methods/Interventions: Based on an IRB approved protocol, all messages were obtained for hemorrhoidectomy patients at a single institution including in 2 weeks preoperative and postoperative during the years of 2020-2025. Using Python (PSF, Wilmington, DE), the text data was cleaned and analysis in Microsoft Word (Microsoft, Redmond, WA) provided readability statistics including the FKRGL. Benchmarks of appropriate reading level include the recommendations of the American Medical Association of a 6th grade reading level and the National Institute of Health recommendation of an 8th grade reading level. Patient messaging topics were coded based on themes. Themes were included if they occurred in two or more messages and redundant themes were combined resulting in final themes: scheduling, pain, prescriptions, urination, insurance, general update, bowel movements, wound, and bleeding.

Results/Outcomes: An initial 150 messages were reviewed. 105 messages were sent by patients, 30 by healthcare staff, and 15 were standard preoperative instructions routinely sent to all patients. Patient messages had a median of 6th grade reading level (FKRGL median 6.1, IQR 4.0-8.0) and staff messages matched at a median 6th grade reading level (FKRGL median 6.1, IQR 3.6-8.0). Notably, standardized preoperative messages were at an 8th grade reading level (FKRGL median 8.1, IQR 6.3-9.6). Additional review was undertaken of other institutional standardized colorectal patient facing materials, which ranged from 8th-9th grade reading levels. The top three topics patients messaged providers about were pain (21%), general updates (18%), and scheduling (13%).

Conclusion/Discussion: This study demonstrates that basic natural language processing techniques can be reliably applied in aiding assessment of colorectal surgery patient message readability. Overall there was a provider and patient messaging readability match at the 6th grade level. However, the standardized materials that provide more complex and complete medical information were at a reading level notably higher than the observed reading level of our patient population as well as the national recommended guidelines. Further work is needed to examine the nature of messaging in surgeries of varying complexity, including literacy level, cadence, and thematic content; leveraging large language models to create customized patient information.

Hemorrhoidectomy EHR Messaging Themes
Figure 1. EHR messaging themes were coded throughout data analysis and compiled with themes generating <2 occurrences removed and themes with redundancy combined. The most common themes were pain (21%), general updates (18%), and scheduling (13%).