Are virtual interviews here to stay?: Best practices from SGO Fellowship Directors

In early 2020, the COVID 19 pandemic required recruitment efforts for medical training programs to quickly pivot to a virtual platform given the national, institutional, and individual concerns about travel and exposure to and infection with COVID. This was particularly acute for those of us in gynecologic oncology as the fellowship interview season had already begun. Through the Society of Gynecologic Oncology (SGO) Program Directors’ Network, we came together as a community and decided that pivoting to virtual interviews would be in everyone’s best interest. Within a few short months, gynecologic oncology fellow applicants and fellowship programs interviewed, ranked, and matched – all virtually. The subsequent 2 interview cycles have also been virtual with rare exceptions. While the virtual interview experience has drawbacks, the past 3 years have also demonstrated the benefits of virtual interviews. For one, virtual interviews put less financial stress on trainees applying for fellowship. The average cost of candidates interviewing for gynecologic oncology fellowship is over $10,000, which is not insignificant on a trainee’s salary.¹ Additional research demonstrates that female and non-White applicants are disproportionately impacted by these costs.² Thus, virtual interviews increase equity of opportunity for all applicants, regardless of background.

Additionally, virtual interviews put less strain on applicants and their residency programs with respect to missed days of work, using vacation days to interview, and arranging for coverage.³ These benefits may have been greatest for applicants coming from smaller programs with limited flexibility. For fellowship programs, there were benefits in terms of scheduling, allowing faculty the flexibility to interview from off-campus sites, as well as lower costs. Programs also observed fewer canceled interviews as the uncertainty of travel was largely absent from the interview process. In the summer of 2022, our most recent interview cycle, 45,000 flights were canceled during our peak interview season.⁴

Supporting these perceived benefits, survey data from recent applicants for Ob/Gyn fellowship positions revealed that 82% of fellowship applicants felt confident in making their rank lists after interviewing virtually.⁵ In this same survey, almost half (44%) of applicants actually preferred virtual interviews while another 33% did not have a preference of virtual or in person.⁵ Essentially all applicants saw cost savings as an advantage to virtual interviews. In a survey of fellowship program directors, a large majority felt comfortable assessing a candidate and making their rank list after virtual interviews. They also felt that it made it easier to comply with duty hours. A majority (73%) also felt just as confident preparing their rank-list after virtual interviews vs. in-person.⁶ Despite these overall positive responses from PDs, difficulty in observing social interaction and gauging applicant interest may be a big challenge moving forward in the absence of in person interviews or pre-interview events.

At the same time, virtual interviews have created new challenges and have the potential to exacerbate old ones.  The newest and most cited drawback to the virtual interview platform is the lack of personal interactions — between programs and applicants or amongst applicants themselves. Many practicing gynecologic oncologists and recent graduates cite their experiences on the interview trail as formative for their careers. In a survey of 75 Ob/Gyn residents who applied to fellowships, a majority felt that they may not have all the necessary information needed to choose a fellowship program (60%) and had less of a feel for the culture of the program (79%) than they would have if they had done in-person interviews. Many of the respondents advocated for a hybrid format in the future (47%).⁷  In an effort to address the concern of applicants not being able to get a feel for a program without visiting in person, the NRMP has proposed a Rank Order List (ROL) “lock” for programs. In this proposal, programs would lock their rank list 2 weeks prior to the ROL deadline during which time applicants could visit programs that they were interested in. Applicants could use this visit to help put together their lists but programs would have already locked their lists and therefore applicants who were unable to visit would not be penalized. The current proposal is for a voluntary “lock” where programs could opt in and their participation would be known to applicants. ⁸

An old problem that is exacerbated by the virtual platform is interview hoarding, a phenomenon of over-application where the bulk of interview invitations are received by a small percentage of top applicants. This problem has already been demonstrated across specialties. ^9,10 Evidence of this phenomenon is hinted at in the survey by Ding et al in which 65% of respondents interviewed at more programs than they would have if interviews had been in person.⁷   A 2020 survey by Bernard et al sought to compare Ob/Gyn fellowship applicants who interviewed in person to those who interviewed virtually. While there was no difference in the number of initial applications per applicant, the applicants that interviewed virtually did more interviews and ranked more programs again suggesting a trend towards interview hoarding.³  If left unchecked, hoarding could undermine the gains realized in equity of opportunity.

Despite these concerns, we have successfully completed 3 interview cycles and the first virtual class is in their second year of training with no signs of widespread concerns. Interview hoarding is a problem that needs our attention. Fortunately, we have not observed any programs going unfilled (a consequence/concern of interviewing a limited number of applicants) and the proportion of unmatched applicants is decreasing. According to NRMP data from 2022, Ob/Gyn subspecialties (Gynecologic Oncology, Reproductive Endocrinology, and Maternal-Fetal Medicine) make up 3 of the 5 programs with the highest percentage of filled positions.¹¹ The problems of virtual interviews are real, but we must acknowledge that the old way of doing things was not perfect either.

Considering the benefits and being cognizant of changing expectations of applicants as well as the increased comfort with virtual platforms, the SGO Program Directors’ Network Interview Taskforce developed the following recommendations regarding the 2023 gynecologic oncology fellowship cycle.

Interviews for the 2023 gynecologic oncology fellowship match cycle (for candidates starting in Summer 2024) should be virtual. If programs wish to include in-person events, we recommend that it is clear to applicants that these opportunities are optional and take place after rank order lists have been submitted. In addition, we recommend that all programs continue to conduct their interviews with the principles of equity, cost, and fairness in mind. We also want to remind and encourage all programs to participate in the National Residency Match Program sponsored by SGO.  If a program decides to take a fellow applicant outside the match, we ask that they withdraw from the match process as early as possible.

Program directors are committed to improving the interview process for everyone involved. While incorporating virtual platforms was the most significant change in our interview format, going forward we note that other aspects of the interview season need attention.  Examples of areas for additional study and change include: finding ways for fellow applicants to get a better feel for programs in some form of in person event(s) possibly at national or regional meetings; better advising fellow applicants to apply to and interview at a more feasible number of programs to reduce both interview hoarding and maximize chances of matching; the best use of the standardized letter of evaluation; helping fellowship programs interview the right number of candidates and holistically screen applicants to maximize matching the right candidate for their program. These changes may be difficult but are necessary if we are to broaden the opportunity to train in gynecologic oncology to a new generation.

 Authors:
A. Shafer^a*, B. Davidson^b*, B. Rungruang^c, J.S. Ferriss^d

^a Division of Surgery, Department of Gynecologic Oncology and Reproductive Medicine, MD Anderson Cancer Center, Houston, TX 77030

^b Division of Gynecologic Oncology, Department of Obstetrics & Gynecology, Duke University, Durham, NC 27710

^c Division of Gynecologic Oncology, Department of Obstetrics & Gynecology, Medical College of Georgia at Augusta University, Augusta, GA 30912

^d Department of Gynecology and Obstetrics, Johns Hopkins University School of Medicine, Baltimore, MD 21287

*These authors contributed equally to this work.

Corresponding Author

Aaron Shafer, MD

Department of Gynecologic Oncology and Reproductive Medicine

MD Anderson Cancer Center

1155 Pressler St.

Unit 1362

Houston, TX 77030

Tel. (832) 750-1795

Fax (713) 792-7586

shafer.aaron@gmail.com

Author Contributions

Conceptualization: all authors

Methodology: all authors

Formal Analysis: all authors

Data Curation: all authors

Writing- Original Draft: all authors

Writing- Review and Editing: all authors

Declaration of Competing Interest

Dr. Davidson reports other from GSK and Onc Live, outside the submitted work.

All other authors have nothing to disclose.

References

1. Ferriss JS, Rose S, Rungruang B, et al. Society of Gynecologic Oncology recommendations for fellowship education during the COVID-19 pandemic and beyond: Innovating programs to optimize trainee success. Gynecol Oncol. 2021;160(1):271-278. doi:10.1016/j.ygyno.2020.10.009
2. Heisler CA, Botros-Brey S, Wang H, et al. Money Matters: Anticipated Expense of In-Person Obstetrics and Gynecology Fellowship Interviews Has Greater Impact for Underrepresented in Medicine and Women Applicants. Womens Health Rep. 2022;3(1):686-691. doi:10.1089/whr.2021.0114
3. Virtual Interviews for Obstetrics and Gynecology Fellowships: Adapting to the Pandemic or The New Normal? doi:10.21203/rs.3.rs-1564544/v1
4. Wichter Z. 45,000+ flights were canceled this summer. Here’s what flyers can expect this winter. USA TODAY. Accessed January 3, 2023. https://www.usatoday.com/story/travel/airline-news/2022/11/13/flight-cancellations-summer-2022-airline-improvements/10668152002/
5. Do Tran A, Heisler CA, Botros-Brey S, et al. Virtual Interviews Improve Equity and Wellbeing: Results of a Survey of Applicants to Obstetrics and Gynecology Subspecialty Fellowships. BMC Med Educ. 2022;22:620. doi:10.1186/s12909-022-03679-y
6. Ponterio JM, Keslar M, Lakhi NA. The virtual interview format for fellowship recruitment in obstetrics and gynecology: a nationwide survey of program directors. Med Educ Online. 2022;27(1):2054304. doi:10.1080/10872981.2022.2054304
7. Ding JJ, Has P, Hampton BS, Burrell D. Obstetrics and gynecology resident perception of virtual fellowship interviews. BMC Med Educ. 2022;22(1):58. doi:10.1186/s12909-022-03113-3
8. NRMP® Call for Public Comment – Consideration of Voluntary Locking Functionality for Program Rank Order Lists. NRMP. Published March 8, 2023. Accessed March 12, 2023. https://www.nrmp.org/about/news/2023/03/nrmp-call-for-public-comment-consideration-of-voluntary-locking-functionality-for-program-rank-order-lists/
9. Boyd CJ, Ananthasekar S, Vernon R, King TW, Saadeh PB. Interview Hoarding. Ann Plast Surg. 2021;87(1):1-2. doi:10.1097/SAP.0000000000002840
10. Moran SK, Nguyen JK, Grimm LJ, et al. Should Radiology Residency Interviews Remain Virtual? Results of a Multi-institutional Survey Inform the Debate. Acad Radiol. 2022;29(10):1595-1607. doi:10.1016/j.acra.2021.10.017
11. Fellowship Data & Reports | NRMP. Accessed October 10, 2022. https://www.nrmp.org/match-data-analytics/fellowship-data-reports/