EP797 - Unpacking the Black Box: How Patient Characteristics and Disease Features Influence Large Language Model Recommendations Regarding Watch-and-Wait

Purpose/Background: There is rising interest in using large language models (LLMs) to guide clinical decision making; however, studies have focused on clinical accuracy without evaluating the underlying reasoning. Understanding which factors LLMs prioritize is essential to evaluate potential biases and ensure safe integration of medical care. We aim to identify which variables are prioritized by LLMs when providing treatment recommendations for patients with advanced rectal cancer.

Methods/Interventions: We developed a clinical vignette describing a hypothetical patient with locally advanced rectal cancer who achieved clinical complete response following total neoadjuvant therapy (TNT) who was eligible for either total mesorectal excision (TME) or watch-and-wait (WW). We then modified the clinical vignette as follows: "I am a colorectal surgeon. I have a patient who is a [35, 55, 75, 95] years old [Black, Asian, White, Native American, Pacific Islander, Hispanic], [Male, Female] who had a [low, middle, high] rectal cancer and is now status post TNT with a clinical complete response. They [do, do not] have multiple comorbidities. I [am, am not] worried about their medical compliance. They [do, do not] need to travel a long distance to the hospital for treatment. They state their highest priority is [quality of life, duration of life, avoiding an ostomy]. I want to advise them on how to best manage their cancer. Should they undergo a total mesorectal excision or do watch and wait?" A prompt was generated for every unique combination of characteristics resulting in 3,456 hypothetical patients. Four LLMs (Gemini, Anthropic, GPT 4.1, GPT 4.1-mini) were queried to recommend TME or WW for each hypothetical patient resulting in 13,824 individual recommendations. We performed logistic regression to calculate odds ratios (OR) for the influence of each characteristic on treatment recommendation with a 95% confidence interval (CI).

Results/Outcomes: Across all models, WW was recommended more frequently than TME (Anthropic 99%, Gemini 75%, GPT 4.1-Mini 65%, GPT 4.1 53%). Clinical characteristics consistently influenced the recommendations: TME was recommended less frequently with increasing age and in patients with multiple comorbidities. Compared to high rectal tumors, only GPT 4.1-mini recommended TME less often for low (OR 0.55 [95% CI: 0.44-0.69]) and middle (0.61 [0.49-0.76]) rectal tumors. Stated patient preferences had a large impact on treatment recommendations. LLMs recommended TME more often if the hypothetical patient prioritized duration of life and less often if QoL was prioritized. This variable was often multiple orders of magnitude more influential than any other factor. (Figure 1) Anthropic and GPT 4.1-mini both recommended TME more frequently for male patients (2.69 [1.3-6.02] and 1.46 [1.18-1.82] respectively) compared to female patients. GPT 4.1-mini more often recommended TME for Black (1.69 [1.21-2.36]), Native American (1.44 [1.03-2.01]), and Pacific Islander (1.41 [1.01-1.98]) patients.

Conclusion/Discussion: There was cross-platform inconsistency, with different models providing conflicting recommendations for the same patient scenario. Overall, LLMs strongly valued patient preferences when making recommendations but did not always recognize the clinical importance of other variables. Of note, some models demonstrated racial and gender biases. This inconsistency underscores the need for caution when interpreting LLM-generated medical advice.

Odds Ratios and 95% Confidence Intervals for TME vs WW across models
Odds ratios and 95% confidence intervals for predictors of total mesorectal excision (TME) versus watch-and-wait (WW) treatment recommendations across large language models. 'x' indicates no available estimate