USMLE Step 1
The USMLE Step 1 is the first part of the United States Medical Licensing Examination. It aims to assess whether medical school students or graduates can apply important concepts of the foundational sciences fundamental to the practice of medicine. US medical students who wish to seek licensure to practice medicine in the US typically take Step 1 at the end of the second year of medical school. Graduates of international medical schools must also take Step 1 if they want to practice in the US. Graduates from international medical schools must apply through ECFMG, and the registration fee is $915. For 2020, the NBME registration fee for the test is $645, with additional charges for applicants who choose a testing region outside the United States or Canada.
During COVID-19 pandemic and George Floyd Protests, the medical community criticized the USMLE for their refusal to revert Step 1 scoring to pass/fail before 2022 and for the gross mishandling of Step examinations during the pandemic, creating "chaos," further "disadvantages," "harm," "bias" and "inequity;" and called for an emergency committee who could address these issues in a timely manner without financial or other conflict of interests.
Current recommendations under consideration include retroactive step 1 pass/fail reporting for the class of 2022, or having each student choose how their Step 1 score is reported on their ERAS transcript beginning with the class of 2021.
History
Prior to 1992, the NBME Part I examination served as the staple basic science examination for medical students at the end of their second year. Upon the launch of the three-part United States Medical Licensing Examination, NBME Part I exam was carried forward in its new format, the USMLE Step 1 examination, which has since evolved to become an increasingly clinically-applied examination of the foundational sciences. The exam became computer based several years later. In May 2015, the USMLE began emphasizing of concepts regarding patient safety and quality improvement across all parts of the USMLE exam series, including Step 1.While the USMLE Step 1 has traditionally been taken after students complete foundational sciences and before core clinical clerkships, in recent years a number of medical schools have reformed their medical school curriculum, moving to having students take the USMLE Step 1 after core clinical clerkships or preliminary clinical training. Top medical schools such as Harvard Medical School, Vanderbilt University School of Medicine, Penn State College of Medicine, UCSF School of Medicine, University of Wisconsin School of Medicine and Public Health, Baylor College of Medicine, Weill Cornell Medical College, Perelman School of Medicine at the University of Pennsylvania, Vagelos College of Physicians and Surgeons at Columbia University, Stanford School of Medicine, University of Michigan, Yale School of Medicine, Roy J. and Lucille A. Carver College of Medicine at the University of Iowa and New York University School of Medicine have instituted these changes, requiring completion of the exam in the third year of medical school.
Format
The exam is currently an eight-hour computer-based test taken in a single-day, composed of seven 40-question sections with a maximum 280 multiple-choice questions. One hour is provided for each section, allotting an average of a minute and thirty seconds to answer each question. Between test sections, the test taker is allotted a cumulative 45 minutes for personal breaks. If the taker finishes any section before the allotted one hour time limit, the unused time is added to the break time total. The test is administered at any of several Prometric computer testing sites. Starting in 2022, the USMLE Step I examination will be reported as pass/fail rather than a three-digit score.Subjects
Step 1 is designed to test the knowledge learned during the basic science years of medical school as applied in the form of clinical vignettes. This includes anatomy, behavioral sciences, biochemistry, microbiology, pathology, pharmacology, and physiology, as well as to interdisciplinary areas including genetics, aging, immunology, nutrition, and molecular and cell biology. Epidemiology, medical ethics and questions on empathy are also emphasized. Each exam is dynamically generated for each test taker; while the general proportion of questions derived from a particular subject is the same, some test takers report that certain subjects are either emphasized or deemphasized.Scoring
Currently, students receive a three-digit score following sitting for the Step 1 examination. In 1999, the USMLE phased out the use of a percentile-based system in favor of a three-digit and two-digit scaled scoring system. In October 2011, two-digit scaled scores were no longer reported to any parties besides the examinees. In April 2013, the two-digit score was eliminated completely from the score report.While the USMLE program does not disclose how the three-digit score is calculated, Step 1 scores theoretically range from 1 to 300, most examinees score in the range of 140 to 260, the passing score is 194 and the national mean and standard deviation are approximately 229 and 20, respectively. Rise in the national mean Step 1 score has been observed with time, as shown in the table below. According to the National Resident Matching Program, the mean score for U.S. allopathic seniors who matched to residency programs in 2016 was 233.2.
It was announced on February 12th, 2020 that beginning no earlier than January 2022, USMLE Step 1 would transition to a pass/fail scoring system. Mean and median USMLE scores have increased significantly from 2003 to today leading to the testing board having to increase the score it takes to pass; independent research cited that Practice-question usage was self-reported as a completion fraction of UWorld, USMLE-Rx, or Kaplan question banks or self-assessments.
As of June 2020, The USMLE has not yet decided how the STEP 1 will be reported on the ERAS transcripts for the Class of 2022 or later when applying to residency. The USMLE wrote on their website that if you received a score on the STEP before 2022, the "USMLE program has not yet determined how the policy change will impact score reports and transcripts."
USMLE mishandling during COVID-19
With the ongoing COVID-19 pandemic, the USMLE and NBME were met with tremendous criticism by the medical community--including their parent organizations AMA and AAMC as well as letters and petitions from physicians and students-- for lack of flexibility and adaptability during COVID and for mishandling testing and refusing to revert step 1 scoring pass/fail before 2022.A summary of current complaints surrounding the USMLE mishandlings can be found and include the USMLE's "failure to take responsibility," "lack of communication," "day-of cancellations," and unequal access for test takers that created "inequity," "bias," "chaos" and "harm" to students and "shortchanging their clinical experience."
By not reverting to remote P/F during COVID, the USMLE is "distracting students from their medical education," said a doctor at a Stanford. Because of overemphasize of Step 1, students study for months, often at the expense of classes and clinical training. These further delays will further exacerbate that problem. For the few that were able to test during COVID, many of those students reported taking a day or more off of clinical rotations to fly to their exam, only to find out their exam was cancelled day-of and had to scramble for a new date months later, meaning they would miss more months of their medical education to study for Step 1.
The medical community was upset that despite the chaos, harm, and damages to the integrity of the test created by USMLE, the USME's concern was their stakeholders, not students. Despite numerous letters, petitions on change.org and the medical community's public pleas, Michael Barone, VP of NBME Licensure Programs, said that the USMLE would not consider moving up the anticipated pass/fail Step 1 scoring before 2022 because it would affect their "stakeholders". The medical community was upset that the USMLE prioritized residency programs over students and doctors. The Step scores were not designed to be used as a residency admissions metric; the scores we intended for medical licensing purposes only and have no meaning beyond pass/fail designation. While it is ultimately the Residency programs fault for misusing the step scores, the USMLE/NBME are aware of and directly enabling the misuse. The overemphasis of Step would not occur if the exam was scored as originally intended.
The resulting "harm and chaos" has created inequity and bias and damaged the integrity of the Step exams. To have the most "unbiased and equitable" exam, the medical community has asked the AAMC, ERAS, and USMLE to allow medical students to choose how their score would be reported on their ERAS transcript during this transition or retroactively report scores as pass/fail beginning with the class of 2022. To avoid the potential financial conflict of interest of the NBME and USMLE, the Coalition for Physician Accountability launched workgroups to address the downstream implications of COVID-19.
Numerous memes have gone viral regarding these matters.
Changes to Step 1 scoring
misuse of Step 1 Scoring to grant residency interviews
The medical community criticized the USMLE and residency programs for using an exam that is originally Pass/Fail to be used as the main screening tool in selecting applicants a residency interview, creating a "barrier that was never meant to exist." Residency directors are “overweighting a screening test in a manner not supported by evidence and for which the test was not specifically designed. Step 1 was designed as a pass/fail exam, and beyond the pass/fail designation, the scores were not meant to have meaning.For state licensing purposes, the exam always has been pass/fail, but over the past few years, its three-digit numeric score has been used as the main determination in granting medical interviews. In recent years, an applicant's Step 1 score has been cited by residency program directors as their most important criterion in selecting graduating medical students for their residency program, despite lack of evidence or data to support that perversion.
Average USMLE Step 1 scores for various residencies are available in .
Studies on Step 1 performance found that "Step 1 is neither precise nor does it predict student performance as residents beyond a certain threshold. With a standard error of eight points, two applicants with scores as far as 15 points apart may not be meaningfully different and yet several programs use singular cutoff points as screening tools." This, as well as the negative impact on student learning, cost, gross misuse of student time, and attempt to decrease racial bias, are amongst the many reasons that the USMLE announced it would revert to pass/fail reporting of Step 1 beginning 2022.
unintended consequences
In a , students started studying for Step 1 during their preclinical curriculum and increased the intensity of their study time until it reached a maximum of 11 hours a day over the period of a month before the exam. Instead of relying on their medical school curriculum, the students instead focused heavily on third-party study materials. While this “parallel curriculum” did lead to an increased Step 1 score, it often did not help them learn the medical school curriculum, because students were skipping lectures and classes."Racial bias and lack of diversity in specialities
Since 2001, there has been a strong plea to remove the Step 1 score barrier that affects all students but especially those of specific racial and ethnic groups. "Using Step 1 scores to screen residency applications puts students who are underrepresented in medicine at a disadvantage."Step 1 scores create unfortunate racial disparities. For Step 1, black and Latino students receive markedly lower scores than white students.
A showed that "when Step 1 scores were used to screen applicants for interviews, a significantly greater proportion of Black students were refused interviews." A showed that between 2005 and 2014, Black and Latino applicants were accepted into residency programs at a significantly lower rate than white applicants.
Further showed lack of diversity within specialities and that that underrepresented students were more likely to go into specialities that have lower Step 1 cut offs like Primary Care.
Public support for USMLE changes
The USMLE parent organizations, including the AAFP, AMSA, and AAMC wrote to the USMLE recommended broad, systemic changes to the medical program including changing Step 1 to Pass/Fail. "The current overemphasis on USMLE Step 1 is having overwhelming negative impact on students. This should be addressed immediately. A pass/fail score will help provide a more meaningful learning environment, improved emotional climate, and better student-student interactions, which can lead to better academic performance that includes USMLE tests. Additionally it will decrease racial bias for programs that use USMLE Step 1 scores to grant interviews." Further, they supported the move to Step 1 pass/fail citing unintentional negative impact of a single standardized exam on career exploration and selection.The AAFP wrote that changing Step 1 to pass/fail creates a "more equitable student evaluation and residency selection process, as it will reduce the impact of racial and other biases on residency selection. Factors that impact student experience with standardized testing perpetuate inequities and disparities that impact test performance, but do not predict or capture competency or skills for future physicians." The AMSA recommended immediate pass/fail to "Reduce the adverse impact of the current overemphasis on USMLE performance in residency screening" and reduce racial bias." The ECFMG and AMA support this transition as well.As part of a public plea for systemic changes to the misuse of USMLE STEP exams, STAT wrote that the Harmful impact of an exam that was designed to be pass/fail has caused so much unnecessary anguish.... "This multimillion-dollar industry has exploited the opportunity to extract thousands of dollars from already overly indebted students.
Public objection to USMLE changes
Immediately following the announcement by USMLE that Step 1 would become pass/fail in 2022, student outcry from the IMG community began over Twitter as well as several different international USMLE Step 1 groups and forums. International students expressed frustration and anger with ECFMG's statement, "The ECFMG supports the informed decision making of the NBME and FSMB Boards on these policy changes. We are looking forward to continually advocating for international graduates as well as participating in the important future conversations on residency selection and transition" due to growing concerns that the change will further decrease IMG chances of matching into US residency programs.Historically, IMGs have been at a significant disadvantage when applying to residency programs in the US. Program directors have been known to place a high emphasis on letters of recommendation, clinical grades, and research. Since international medical students come from medical schools with different grading systems and do not usually have access to well-known faculty and American research opportunities, USMLE Step 1 is often seen as a major opportunity to boost IMG residency applications with a high score. Without a numerical score on the USMLE Step 1, there is speculation that IMG's will be further be pushed out of the race for residency program spots.
Response from Katsufrakis and Chaudhry
Peter Katsufrakis, CEO of the NBME, and Humayun Chaudhry, CEO of the FSMB, argued that numerical scores can help residency programs differentiate among thousands of applicants for a limited number of slots. In an early version of the essay, they noted that if students spent less time preparing for Step 1 and "more time to activities that make them less prepared to provide quality care, such as binge-watching the most recent Netflix series or compulsively updating their Instagram account, this could negatively impact residency performance and ultimately patient safety." Within a few days, Katsufrakis and Chaudhry issued an apology and removed the phrase, but the comment fueled a Twitter storm, bringing Step 1 scoring to the forefront.While the NBME, USMLE, and FSMB were originally against these changes, as of 2020, they have changed their stance in support of public opinion. Humayun Chaudhry, the President of the FSMB,, later said that “although the primary purpose of the exam is to assess the knowledge and skills essential to safe patient care, the new policies will “address concerns about Step 1 scores impacting student well-being and medical education.” Because students put so much emphasis on getting a high USMLE Step 1 score, they often skimmed the medical school curriculum that they deemed to be not as relevant in order to get the maximum score on the USMLE Step 1.
Formation of INCUS
The current use of Step 1 scoring as a major determinant for granting residency interviews has been met with tremendous criticism by the medical community. The public outcry led to the formation of a committee and investigation in early 2019 called InCUS. In February 2020, InCUS concluded that residencies were overemphasizing the Step 1 and not using it for its original purpose for which it was designed as a means of licensing. The Step 1 exam was designed to be one of a four tests for granting a medical license. It was never intended to be used as a predictor of medical knowledge for which cutoffs or barriers could be justified or to enable racial bias. In comparing the four USMLE licensing exams, the Step 1 is least clinically relevant. The Step 1 focuses on basic sciences and is not an accurate representation of clinical knowledge. The Step 1 is generally taken before beginning of clinical rotations, meaning that at the time of Step 1, most students have not "stepped foot in a hospital" or worked with patients in a clinical setting. As such, it cannot accurately measure medical knowledge and has little clinical relevance, yet disproportionately determines a student's specialty. Further that the misuse of Step 1 scores by residencies was negatively impacting students medical education including but not limited to taking away from clinical experience, unnecessary anxiety, gross misuse of student time, and an inaccurate measure of clinical knowledge. The decision to move to pass/fail was also an effort to decrease racial bias of STEP exams. A list of InCUS participants can be found atIncus outcomes
In response to public outcry, in February 2020, the USMLE program announced a plan to change Step 1 score reporting to a pass/fail system in an effort "to reduce overemphasis on Step 1 performance while allowing licensing authorities to use the exam for its primary purpose of medical licensure eligibility." However, because of pressure from its "stakeholders," the USMLE posted on its website and twitter that this transition will occur no earlier than January 1, 2022.As of June 2020, the USMLE has not yet decided how the STEP 1 will be reported on the ERAS transcripts for the Class of 2022 or later when applying to residency. The USMLE wrote on their website that if you received a score on the STEP before 2022, the "USMLE program has not yet determined how the policy change will impact score reports and transcripts."
The USMLE changes were met with mostly positive responses. "In becoming pass/fail, Step 1 is reverting to what it was meant to be from the get-go: a minimum competency assessment. There has never been a sound basis on which to conclude that a score of 240 means that a student will be a better clinician than one who scores 220."
However, the USMLE was met with heavy criticism for refusal to move Step 1 to pass/ fail until 2022; these changes are long overdue and any further delay, would not bring justice for current students. During the COVID-19 pandemic, the USMLE wrote on their COVID-FAQ page and on twitter that despite receiving many reasonable inquiries to move to immediate pass/fail, they would not consider any early changes to Step 1 reporting as this would affect their "stakeholders."
Aftermath
One of the main complaints of the USMLE's decision to wait until 2022 does not make an effort to address current students. This systemic change have been "long overdue and further delay only adds to the problem'" with a major complaint being that the intense pressure attached to one’s numerical Step 1 score has been causing unhealthy levels of anxiety and stress for medical students during their first and second years. This is particularly problematic, considering that much of the material covered in Step 1 may not be especially relevant to the practice of these future doctors." Despite pleas by the medical community to move to pass or fail immediately, the USMLE responded they would not do so before 2022 to accommodate their "stakeholders." The USMLE met heavy criticism for the suggestion that residency programs are the stakeholders, not medical students. Further, their refusal to change the score reporting Not only acknowledges that they recognize the overemphasis and misuse of their exam but that they are also actively enabling the misuse.The other concern was that this will put more emphasis on Step 2. However, critics agree that in a choice between Step 1 or 2, Step 2 would be a "more accurate predictor of medical knowledge as it is usually taken after third year when the student has had a year of clinical experience, having worked with patients in a hospital." In Feb 2020, the Harvard Crimson wrote, "the fact that the Step 2 is a more clinically relevant exam than Step 1, makes it a better proxy for clinical acumen."
Effect on residency matching
The USMLE score is one of many factors considered by residency programs in selecting applicants. The median USMLE Step 1 scores for graduates of U.S. Medical Schools for various residencies are charted in Chart 6 on page 9 of "Charting Outcomes in the Match" available through the .Many residency programs use a "cutoff" score for Step 1, below which applicants are unlikely to be considered, although in some cases individuals with significantly higher Step 2 CK scores may still receive further consideration. The NRMP Residency Program Director survey contains more information, both overall and by specialty, regarding "cutoff" scores.
Studies on Step 1 performance found that "Step 1 is neither precise nor does it predict student performance as residents beyond a certain threshold. With a standard error of eight points, two applicants with scores as far as 15 points apart may not be meaningfully different and yet several programs use singular cutoff points as screening tools."
This, as well as the negative impact on student learning, cost, gross misuse of student time, and attempt to decrease racial bias, are amongst the many reasons that the USMLE announced it would revert to pass/fail reporting of STEP 1 beginning 2022. However, the delay in changes until 2022 does not come soon enough and will not help the current classes who will be unaffected by this future change.
Controversies
USMLE repsonse to COVID
The USMLE was met with heavy criticism for their lack of adaptability during the COVID.Students and physicians were upset that the USMLE refused to move up the 2022 deadline of making Step 1 pass or fail, in light of the testing barriers created by COVID. The USMLE responded that the "eventual change to Pass/Fail score reporting for Step 1 will impact their stakeholders," implying their stakeholders were not students.
Racial Bias of Step 1 scoring
Research demonstrates some differences in USMLE scores attributable to race and ethnicity, with self-identified Black, Asian, and Hispanic examinees showing score differences when compared with self-identified White examinees. "The mean USMLE step 1 score was significantly greater among white applicants as compared to black and Hispanic applicants. Depending on the threshold score, an African American was 3-6x less likely to be offered an interview.""61% of minority applicants were accepted into an orthopaedic residency versus 73% of White applicants. White and Asian applicants and residents had higher USMLE Step 1. White applicants and matriculated candidates had odds of Alpha Omega Alpha membership compared with Black, Hispanic, and Other groups.
One of the recommendations for Step 1 changing to pass or fail was to address racial differences. "Among self-identified racial groups, research shows that white students perform higher on the USMLE than any other group."
The American Academy of Family Physicians and Association of American Medical Colleges supported changing Step 1 to pass or fail to reduce racial bias.
Katsufrakis & Chaudhry comments against Step 1 changes
In Dec 2018, NBME President Peter Katsufrakis and FSMB President Humayun Chaudhry wrote in opposition of USMLE Step 1 changes in from Improving Residency Selection Requires Close Study and Better Understanding of Stakeholder Needs:"If students reduce time and effort devoted to preparing for Step 1, they may indeed devote attention to other activities that will prepare them to be good physicians. This would arguably be an ideal outcome of such a change. However, if students were to devote more time to activities that make them less prepared to provide quality care, such as binge-watching the most recent Netflix series or compulsively updating their Instagram account, this could negatively impact residency performance and ultimately patient safety. We know that assessment drives learning, so another concern resulting from a shift to pass/fail scoring may be a less knowledgeable physician population."
NBME Executive Salary
The NBME executives received public criticism after their salaries were released after increasing costs for students, showing many executives receiving high said to seven figures."Former NBME President Dr. Melnick's compensation increased from $399,160 in 2001 to over $1.2 million in 2016, almost perfectly in parallel with the tripling of USMLE costs." "As of 2016, NBME President also gets free first-class airfare for himself and his travel partner, as well as a membership to a Philadelphia social club. According to the 2017 Form 990, Schedule J, two lower executives received total compensation over $700,000; another two over $600,000; another three receiving over $500,000; and another 6 receiving over $400,000....Yet, the total number of test-takers for the USMLE Step 1 has been essentially unchanged for the past 10-15 years."
Pricing
NBME and USMLE were met with criticism when they announced their raising cost for STEP standardized tests.Kevin MD wrote, "Safeguards are needed to ensure fees for mandatory testing such as the USMLE do not exceed reasonable operating costs, particularly for financially vulnerable medical students."
As part of a broader public plea for systemic changes to the improper use of USMLE STEP 1 exams, STAT wrote that this "multimillion-dollar industry has exploited the opportunity to extract thousands of dollars from already overly indebted students. Registering for STEP exams test cost $645-$1300, while Step preparation materials and courses run much higher."
As of 2020, the USMLE currently charges $645 for Step and $60 for each NBME Self Assesments.