EP580 - At the Level of a Fellow?: A study of the updated Colon & Rectal Surgery Resident Assessment Letter of Recommendation

Purpose/Background: Traditional letters of recommendation can be problematic as they are subject to bias, often use vague and inflated language, are time consuming for letter writers and lack standardization. As colon and rectal surgery fellowship becomes increasingly competitive, the field has moved to the use of a standardized resident assessment form instead of narrative letters of recommendation. This form has undergone several iterations, and this past year added a standardized rubric of candidate performance across multiple domains ranging from technical skills to clinical knowledge to situational awareness. Authors were also required to identify three of these domains as an applicant area of strength and one as a weakness. We analyzed whether the new standardized resident assessment forms were being used correctly and how candidate rankings were distributed across domains.

Methods/Interventions: This was a retrospective analysis of applications to the colorectal fellowship at a major academic institution for the 2026-2027 cycle. We collected demographic information about applicants as well as data on their residency training programs. Standardized resident assessment forms were reviewed to collect the distribution of candidate rankings across domains as well as information about the form authors. Chi-squared analyses were used to compare categorical variables.

Results/Outcomes: 106 colorectal fellowship applications were received. 49.1% of applicants were men. 56.6% of applicants trained at a community residency program, 41.5% at an academic program and 1.9% at a military program. 34.3% of applicants came from a program with a colorectal fellowship.

393 total letters of recommendation were submitted. 63.7% of authors were colorectal surgeons and 10.6% were colorectal fellowship program directors. The correct standardized resident assessment form was used 92.1% of the time. 1.8% of authors submitted an additional narrative statement despite multiple written instructions not to do so.

The vast majority of candidates were ranked as performing at the level of a fellow or an above average resident in all 9 domains of clinical performance. The distribution is shown in figure 1. Women scored higher in all domains except for clinical knowledge when compared to men, however this only reached statistical significance in "interactions with patients" (χ²(3) = 9.98, p = 0.019). Selected areas of candidate strengths and weakness are shown in figure 2.

There were no statistical differences in letter writer evaluation of technical skills, clinical knowledge or clinical judgement when comparing residents from programs with a colorectal fellowship to those without. This was also true when comparing evaluations by colorectal surgeons to those of non-colorectal surgeons. Residents from academic programs were ranked higher in clinical judgement compared to non-academic programs (χ²(2) = 6.77, p=0.034).

Conclusion/Discussion: Although most candidates were ranked as above average in all domains, this mostly remained consistent when evaluating for resident training characteristics as well as those of the letter writer. Although these results only represent candidates who applied to one institution, which may be seen as a weakness, this likely represents greater than half of the applicant pool based on historical data. Future directions of investigation may include surveying colorectal fellowship directors on the utility of the new resident standardized assessment.

Figure 1
Figure 2