Most medical students know that the road to a successful residency match is long, hard, and unforgiving. In the most recent NRMP match, in line with the last 10 years of data, there was a gulf of nearly 15,000 positions between the number of match registrants and available PGY-1 positions. Medical school applications demand an inerrant path of nearly perfect grades and a strong performance on the MCAT and most students who matriculate successfully have never or infrequently personally experienced disappointment with standardized testing. For these medical students, failure to perform at the level they would like or expect on Step 1 comes as a bewildering experience.
However, in the wake of this challenging news, a student’s future success in residency applications is contingent on his/her ability to rationally perform a root cause analysis and figure out why they didn’t perform as hoped and how to remedy their future performances on exams. It is the exceptionally rare case that a student successfully applies to medical school and performs satisfactorily through the pre-clinical years, only to arrive at USMLE exam time with an inability to perform at a high level to do an intrinsic defect.
Program directors and department chairs consistently report that the single most important criterion in their ranking of applicants is their Step 1 score. However, for students with an unremarkable performance on Step 1 of the USMLE, Step 2 CK presents a crucial opportunity to demonstrate their ability to perform at a high level on high-stakes certification exams; program directors report that it is the second most important piece of data in choosing applicants. Although I could recount numerous personal anecdotes from students with Step 1 scores as low as the 220s matching into competitive specialties such as dermatology, urology, and plastic surgery, the most important lesson in their stories and their common denominator is that they all scored very highly on Step 2 CK.
Getting to the Root Cause
Before I came to medical school, I was an Apache helicopter pilot and US Army officer. One of the key aspects of my job was not only to fly aircraft, but to regulate the maintenance workflow on the helicopters that I was responsible for. One of the tools I’d routinely employ to solve maintenance problems is called the 5 Whys, an iterative process examination technique developed within the Toyota Motor Corporation. One of the keys of this process is to remember that people don’t fail, processes do. As an example:
Problem: An aircraft’s battery is found to have unacceptably low voltage on pre-flight inspection.
Why?
The aircraft’s alternator has failed.
Why?
One of the aircraft’s alternator drive shafts bearings has seized.
Why?
The drive shaft bearing were not lubricated according to manufacturer prescribed inspection and maintenance requirements.
Why?
The aircraft was flown 20 hours past a mandatory inspection and maintenance window.
Why?
Mission-hour requirements exceeded the capacity of the unit’s maintenance team for the preceding 2 months. (root cause)
The same sort of analysis can be conducted for a student who has performed poorly on a USMLE exam. The answer, invariably, isn’t that the student didn’t try hard enough or wasn’t smart enough. Additionally, the process doesn’t always require a full five layers of iteration in order to arrive at a crucial realization. As an example:
Problem: I scored poorly on USMLE Step 1
Why?
I felt very pressured for time on each question block and ended up guessing on 5 questions per 40 question block each time. I was plagued with anxiety about rushing through each section during each subsequent question block.
Why?
When I practiced for the exam, I spent the majority of my time doing UWorld questions in untimed, tutor mode, and rarely practiced the test in real-world conditions until the week prior on a UWorld self-assessment.
Why?
It was difficult to make it through all of the material that I hoped to study during my dedicated study period.
Why?
I entered my dedicated study period without a clearly defined schedule that incorporated time for learning, consolidation, self-assessment, and practicing key test-taking skills like time management. (root cause)
Oftentimes, this process of introspection can be challenging; the assistance of an external party can be both useful and reassuring by identifying a process problem rather than a defect inherent to the test taker himself/herself (e.g. poor motivation or lack of intelligence).
Working with students in the past, the most common root causes of underperformance are:
Feeling overwhelmed by the sheer volume of content on the USMLE exams and focusing on mastery of minutiae instead of on conceptual mastery of material. This problem is also linked to a failure to create a study schedule that focuses on the most important concepts, forces the examinee to move to new material each day, encourages spaced repetition, and incorporates other key elements such as routine scheduled self-assessments.
Poor fundamental test taking strategy. Test-taking is an acquired skill incorporating time management skills, understanding the intent and patterns of the test-writers, and decision making that must be practiced and can be dramatically improved with peer coaching.
Relying on external content creators for study material instead of “actively” studying by making personalized Anki (or manual) flashcard decks. Education theory research has repeatedly demonstrating that more “active” study methods produce far superior retention.
Inadequate real-exam conditions practice that not only practice key facets of exam taking skills, but also instill the confidence in the test taker necessary to approach the exam on game day with the minimum level of anxiety possible. In many ways, standardized tests are as much a measure of a examinee’s ability to manage the emotional stresses of test day as they are a measure of the test content itself.
Under-performing your potential on Step 1 doesn’t have to be a fatal flaw in your residency application. The keys to success in moving forward are threefold: 1) Correctly identifying and addressing the root cause of the poor performance 2) Remedying the root cause and succeeding at a high level on Step 2 CK and 3) Being prepared to explain this process of introspection during interview season.
Joshua Caldwell is a fourth-year student at Harvard Medical School. He is applying in the 2019 AUA match and hopes to pursue a career as a urologic oncologist. He tutors with USMLE Pro for Step 2 CK.