Evidencias Medicas : Holistic Review — Shaping the Medical Profession One Applicant at a TimeMedicina General y Familiar

Robert A. Witzburg, M.D., and Henry M. Sondheimer, M.D.

April 10, 2013DOI: 10.1056/NEJMp1300411

Comments open through April 17, 2013

Modern medicine has been characterized by rapid and accelerating progress in biomedical sciences as the foundation for clinical practice. In 1910, the Flexner Report established these sciences as the core of medical education.1 Admissions committees at U.S. medical schools have, for the past century, focused their attention largely on predictors of success in the foundational science curriculum, relying heavily on academic performance in the biologic and physical sciences and scores on the Medical College Admission Test (MCAT) in selecting applicants for medical school.
Abundant data support the contention that performance in the medical school science curriculum and on the U.S. Medical Licensing Examination (USMLE) Step 1 are predicted by performance on the MCAT and in the undergraduate science coursework required of medical school applicants.2 Key aspects of behavior, character, and performance that are essential for the practice of medicine, however, cannot be predicted from these measures. In addition, performance on standardized tests and in undergraduate sciences is influenced by myriad social, demographic, and economic factors that limit the utility of these measures in large segments of the potential applicant pool for medical school.
Over the past decade, individual medical schools, supported by the Association of American Medical Colleges (AAMC), have been working to expand the frame of reference for evaluating applicants for medical school. These efforts have come together under the “holistic review” rubric endorsed by the U.S. Supreme Court in 2003: “highly individualized, holistic review of each applicant's file, giving serious consideration to all the ways an applicant might contribute to a diverse educational environment.” Under such an approach, a school “seriously considers each `applicant's promise of making a notable contribution to the class by way of a particular strength, attainment, or characteristic — e.g., an unusual intellectual achievement, employment experience, nonacademic performance, or personal background.'”3
The AAMC Holistic Review Project has defined holistic review in medical school admissions as “a flexible, individualized way of assessing an applicant's capabilities by which balanced consideration is given to experiences, attributes, and academic metrics . . . and, when considered in combination, how the individual might contribute value as a medical student and future physician.”4
A holistic review process therefore emphasizes attributes, including learning ability, that are associated with excellence in physicians. Applicants are evaluated according to criteria that are institution-specific, mission-driven, broad-based, and applied consistently across the entire applicant pool at a given school. Holistic review does not abandon the assessment of aptitude in science. Rather, it places such measures in the broader context of the applicant's life experiences, with a particular focus on adversities overcome, challenges faced, advantages and opportunities encountered, and the applicant's demonstrated resilience in the face of difficult circumstances. Each factor, be it the undergraduate grade-point average (GPA), the MCAT score, or the leadership roles assumed in volunteer service organizations, is evaluated in the context of the complete portfolio of information available about the applicant. That is, a given level of accomplishment for one applicant may look very different in the context of another applicant with a different life story. Medical schools have many more qualified applicants than they can realistically interview, so holistic principles must be applied from the initial screening through the entire admissions process for their desired effect to be realized.
The imperative for a diverse physician workforce in an increasingly diverse society is one important driver of the move to take a more expansive view of excellence in medical student selection. This more comprehensive approach to considering a multitude of factors in evaluating all applicants provides a context for the inclusion of race, ethnic background, language, culture, and heritage, among other factors, in a way that is educationally sound and legally viable. The AAMC's Experience–Attributes–Metrics Model includes consideration of many dimensions of applicants, broadening the context in which their development, accomplishments, and potential can be evaluated. The metrics include grade trends in addition to the usual GPA and MCAT scores; attributes range from fields of interest, intellectual curiosity, and maturity to languages spoken, gender identity, and family status; and experiences may include everything from education and research to general life experiences.
In 2003, the Boston University School of Medicine (BUSM) became one of a number of U.S. medical schools to launch a systematic transition from a traditional admissions model based largely on the review of academic metrics to a comprehensive, holistic review process. It was a slow and deliberative transition, but by 2008, changes in the BUSM admissions program were clear and substantial, and the effects were evident in the entering class of 2009.
The BUSM Committee on Admissions first developed a mission statement for itself that reflected the concepts in the institutional mission statement and then created a set of decision-support tools using performance metrics, characteristics, and behaviors that are identified in that mission and used in a clearly defined and universally applied manner. The table

Mapping Desirable Physician Traits to Applicant Data. shows one such tool: a list of desirable traits for physicians matched with the elements of applicant data that reveal or predict those traits. Direct measures of these traits are often unavailable, so proxies are used. Holistic review is an information-hungry process; electronic processing greatly facilitates both the application and the evaluation of the program. Experiences, attributes, and academic metrics are evaluated and scored in a systematic and consistent manner across the entire applicant pool, with due consideration to the demonstrated validity of various criteria in predicting success in both medical school and medical practice. The BUSM program uses structured interviewing, rigorous training of participating faculty and staff, and systematic evaluation of data elements, all of which minimize the influence of conscious and unconscious bias.
Since BUSM became engaged in holistic review, the profile of its entering class has changed dramatically. Other factors, including changes both at the school and within society at large, have certainly influenced the school's ability to select and recruit strong students, but holistic review is at the core of the process. Students are culturally, linguistically, racially, ethnically, and demographically more diverse than previous classes, and according to the standard measures of undergraduate GPA and MCAT score, they are at least as well prepared academically (the average GPA and average MCAT score were 3.66 and 33.62 for the entering class of 2012, as compared with 3.57 and 31.68 for the entering class of 2008). Students from groups underrepresented in medicine now make up approximately 20% of the entering class, as compared with 11 to 12% before the adoption of holistic review. We observe that students are more frequently engaged in our campus community and in outside activities, and graduating students pursue a broad range of specialties and career paths. The general sense of the faculty, particularly those who teach our small-group problem seminars, is that the students are more collegial, more supportive of one another, more engaged in the curriculum, and more open to new ideas and to perspectives different from their own. Some of these observations are subjective and difficult to quantify, but there is a striking, and uncoached, consensus among the experienced faculty members.
Successful adoption of holistic review in medical school admissions requires a strong commitment by the school, but the process can be bolstered by support from the AAMC's Holistic Review Project, which has nurtured a community of physicians focused on the development and implementation of strategies for ongoing monitoring and improvement.5 This effort is led by a committed group of admissions officers, diversity directors, deans, medical educators, students, and residents from around the country who continue to remind us that medical school admissions is not merely about selecting next year's first-year class, but also about selecting the physicians who will successfully lead a rapidly evolving 21st-century health care workforce.

Disclosure forms provided by the authors are available with the full text of this article at NEJM.org.
This article was published on April 10, 2013, at NEJM.org.