ais

Hot Keywords
Intelligence

Top
Art Int Surg 2022;2:195-206. 10.20517/ais.2022.24 © The Author(s) 2022.
Open Access Original Article

Leveraging artificial intelligence for resident recruitment: can the dream of holistic review be realized?

Department of Surgery, University of Maryland Medical Center, Baltimore, MD 21201, USA.

Correspondence to: Dr. Stephen M. Kavic, Department of Surgery, University of Maryland Medical Center, 29 South Greene Street, GS 631, Baltimore, MD 21201, USA. E-mail: skavic@som.umaryland.edu

    Views:54 | Downloads:8 | Cited:0 | Comments:0 | :0
    Academic Editor: Andrew A. Gumbs | Copy Editor: Ke-Cui Yang | Production Editor: Ke-Cui Yang

    © The Author(s) 2022. Open Access This article is licensed under a Creative Commons Attribution 4.0 International License (https://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, sharing, adaptation, distribution and reproduction in any medium or format, for any purpose, even commercially, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made.

    Abstract

    Aim: The purpose of this study was to investigate if principles of Artificial Intelligence (AI), specifically Natural Language Processing (NLP), could be applied to the personal statements of general surgery residency applicants in order to gain valuable insight into the candidates and facilitate a more comprehensive assessment.

    Methods: The personal statements from individuals applying for a general surgery residency position during the 2021/22 application cycle (n = 1792) were analyzed using AI technology. Comparison groups were drawn from a database of documents from the general population and the personal statements of current general surgery residents (n = 64) at a single academic center. The study was conducted in collaboration with a leading language psychology and natural language processing organization.

    Results: Applicants exhibited a language-based personality that was highly self-assured (P < 0.0001) and trusting (P < 0.0001), and less stress-prone (P < 0.0001) and impulsive (P < 0.0001) than that of the general population. Compared to the general applicant pool, current residents were significantly more emotionally aware (P < 0.001) and organized (P < 0.001) and less self-assured (P < 0.001) and less driven by power (P < 0.001).

    Conclusion: Natural language processing technology can be utilized to assess the unique characteristics of general surgery resident applicants based on the content of their personal statements. In addition, candidates who successfully gain admission to a single academic program display different language-based personalities and drives compared to the general applicant pool. Incorporating these principles of artificial intelligence into the residency selection process could facilitate a more holistic evaluation of candidates.

    INTRODUCTION

    The field of natural language processing (NLP) is one of the fundamental components of Artificial Intelligence (AI) that equips computer algorithms with the capability of “understanding” human language[1]. In the NLP framework, theories of linguistics are combined with advanced AI techniques (artificial neural networks and deep machine learning) into a computer program that analyzes and interprets text and speech. Integrating behavioral and cognitive psychology with current NLP methodologies has enabled developers of AI systems to process language data in a way that can reliably provide insight into the individuals who produced the content [Figure 1][2].

    Figure 1. An illustration of the methodology of generating personality characteristics from personal statements based on Natural Language Processing (NLP) technology.

    There may be a role for this emerging technology within the Graduate Medical Education (GME) field. For instance, the application process for medical students seeking postgraduate training has become increasingly complex and uncertain. One of the significant challenges is the demand for a more holistic screening process when selecting candidates for residency interviews, calling for broadened consideration of characteristics, experience, and motivation, in addition to their academic performance[3]. While considering each student as a unique, whole individual rather than as a set of numerical test results, promoting diversity, and recruiting based on skills crucial to patient care and work ethic, is important, the practical application is ambiguous at best. Significant problems with the call for holistic assessment include that it is time-consuming, complicated, poorly understood, and non-standardized, making it susceptible to bias and error, which will lead many program leaders to forgo applying this to their residency recruitment[4]. Therefore, we must identify methods to address these shortcomings to achieve the benefits that holistic recruitment is intended to bring.

    These challenges can be addressed by incorporating theories of artificial intelligence, which involves the development of computer systems that can perform tasks typically requiring human perception and reasoning. In its most practical form, artificial intelligence software functions as an assistant, handling some of the more laborious aspects of a task so that experts can focus their attention on making decisions of greater importance. Accordingly, in the present study, the personal statements of general surgery applicants were analyzed using a commercially available NLP platform that incorporates psycholinguistics, statistical language modeling, and contemporary AI principles to gain insight into the characteristics of prospective surgeons. This work was intended to provide a basis for incorporating AI technology into the residency selection process to make further advancements in pursuing holistic review.

    METHODS

    Setting and participants

    This study examined the personal statements of medical students applying for general surgery residency positions at a single academic institution during the 2021/22 application cycle. All applicants’ personal statements were included to maximize generalizability and validity. Applicant personal statements were compared to a corpus of data from our partnering NLP group that represents national averages. Personal statements submitted via the Electronic Residency Application Service (ERAS) were prospectively collected. We prepared the personal statements for analysis by de-identifying, numbering, and collating them in an Excel spreadsheet that the NLP group could use for processing. The personal statements were not altered and were submitted verbatim as the applicants had written them. Furthermore, the personal statements of the current residents of the same academic institution (University of Maryland) were analyzed for comparison to the general applicant pool. The institutional review board deemed this study exempt from review. Figure 2 represents a flow diagram illustrating the study’s design.

    Figure 2. Flow diagram illustrating the study design and methods.

    Data collection and NLP measures

    The project was carried out in collaboration with an independent NLP and language psychology firm (Receptiviti®, Toronto, Canada)[5]. The partnering NLP organization was founded by leaders in linguistics, psychology, and computer science and developed the gold standard algorithm in language psychology, the Linguistic Inquiry and Word Count (LIWC). The LIWC model forms the basis of the proprietary NLP software used in this study. This software has been extensively studied for over 30 years to uncover and understand the emotions, psychology, personality, mindsets, and motivations embedded in all language data. LIWC has been thoroughly validated (used in over 19,000 academic research citations) and applied broadly, including in mental health, organizational psychology, medicine, recruitment optimization, leadership and succession planning, market research, customer intelligence, insider threat detection, finance, fraud detection, etc. Receptiviti owns the global commercial rights to LIWC.

    The Language-based Personality Framework is based on the Ocean Model of Personality[6]. This model has five main categories, each containing six subcategories, totaling 35 different personality measures. The Drives framework includes five metrics that provide information about the applicant’s motivations. Table 1 summarizes the characteristics extracted by NLP from the personal statements of the study participants.

    Table 1

    The NLP characteristics extracted from the personal statements of general surgery resident applicants

    NLP FrameworksCategoryFacetsCategory Summary
    PersonalityOpennessArtisticThis measure and its facets examine the degree to which a person is open to new ideas or new experiences
    Adventurous
    Intellectual
    Liberal
    Imaginative
    Emotionally aware
    ConscientiousnessSelf-assuredThis measure and its facets examine the degree to which a person is reliable, organized, disciplined, and deliberate
    Disciplined
    Ambitious
    Dutiful
    Cautious
    Organized
    ExtraversionSociableThis measure and its facets examine the degree to which a person feels energized or uplifted when interacting with others
    Friendly
    Assertive
    Active
    Energetic
    Cheerful
    AgreeablenessGenerousThis measure and its facets examine the degree to which a person is inclined to please others
    Trusting
    Cooperative
    Empathetic
    Genuine
    Humble
    NeuroticismImpulsiveThis measure and its facets examine the degree to which a person expresses signs of anxiety, unhappiness, pessimism, or depression
    Stress prone
    Anxiety prone
    Aggressive
    Melancholy
    Self conscious
    DrivesAffiliationThe degree to which a person is driven by their own internal need for affiliation with other individuals or groups
    AchievementThe degree to which a person is driven by an internal need for achievement
    Risk seekingThe degree to which a person is focused on engaging in risky behaviors or activities
    Risk aversionThe degree to which a person is focused on avoiding risk
    PowerThe degree to which a person is driven by an internal need for power or domination
    RewardThe degree to which a person is driven by an internal need for reward

    The partnering NLP group utilizes a proprietary data set consisting of over 350 samples of language from a diverse set of perspectives to measure the Personality and Drive frameworks. In our analysis, this proprietary dataset represents the “general population”. The scores generated for these personal attributes range from 0 to 100 (i.e., a score of 80 indicates that 80% of the samples in the benchmark dataset have a lower score than the language sample being evaluated). A detailed explanation of data extraction and score generation can be found on the partnering NLP documentation home page (https://docs.receptiviti.com/the-receptiviti-api).

    Statistical analysis

    SAS 9.4 was used for all data management and statistical analysis. The student’s t-test was used as our primary means of statistical inference testing, evaluating two independent groups with continuous dependent variables. Statistical significance was defined as α < 0.05, with all P-values reported as two-tailed.

    RESULTS

    Exploratory analysis and demographics

    There was a total of 1792 personal statements included in this study; 966 (54%) were self-identified male, and 826 (46%) were self-identified female. The bulk of the applicants reported being native English speakers (n = 1523, 85%), followed by advanced English speakers (n = 267, 15%) (2 participants did not report English proficiency). Most applicants were born in the United States (U.S.; n = 1139, 64%) and went to U.S. medical schools (n = 1105, 62%). The average word count for the personal statements was 671 (SD = 144), with the majority being between 500-1000 words (n = 1649, 92%).

    Surgical residency applicant measured characteristics

    Measures of personality

    Figure 2 presents a summary of the big-5 language-based personality of general surgery applicants (in blue) compared to the general population (in orange). Figure 3 shows the 30 subcategories of the big-5 personality measures for applicants compared to the general population. In their personal statements, general surgery applicants exhibited a language-based personality that was highly self-assured (mean = 71.2, SD = 9.87; P < 0.0001), trusting (mean = 67.40, SD = 9.11; P < 0.0001), cheerful (mean = 62.04, SD = 14.48; P < 0.0001), and cooperative (mean = 60.31, SD = 6.15; P < 0.0001). In addition, applicants presented language-based personalities that were considerably less stress-prone (mean = 25.24, SD = 9.54; P < 0.0001), melancholy (mean = 32.04, 11.97; P < 0.0001), emotionally aware (mean = 35.20, SD = 11.27; P < 0.0001), and impulsive (mean = 38.99, SD = 9.18; P < 0.0001).

    Figure 3. Comparison of the Big-5 personality measures derived from the personal statements of general surgery applicants (blue) to the general population (orange) using Natural Language Processing technology. Statistical analysis was performed using student’s t-test(***P < 0.0001).

    Measures of drive

    Figure 4 presents a summary of the language-based drive (i.e., psychological motivation) of general surgery applicants (in blue) in comparison to the general population (in orange). In their personal statements, general surgery applicants demonstrated that they were highly driven by achievement (mean of 90.61, SD of 9.08; P < 0.0001) and power (mean = 69.27, SD of 24.14; P < 0.0001). Moreover, applicants were found to be far less motivated by risk aversion (mean = 43.70, SD = 22.49; P < 0.0001).

    Figure 4. Comparison of the 30 components of the Big-5 personality measures derived from the personal statements of general surgery applicants (blue) to the general population (orange) using Natural Language Processing technology.

    General surgery residents compared to applicants

    There was a total of 64 personal statements for current general surgery residents at our institution that were analyzed in this study. Residents were significantly more self-assured (mean 67.7; P < 0.001), trusting (mean = 66;P < 0.001) and cheerful (mean = 64.1; P < 0.001) as compared to the general population. Residents were also significantly less stress-prone (mean = 27.7; P < 0.001), melancholy (mean = 35; P < 0.001), and cautious (mean = 36.6; P < 0.001).

    Figures 5 and 6 include the mean difference (MD) between the residents and applicants for the personality and drive measures, respectively. Comparing current residents to the general applicant pool, residents were significantly more emotionally aware (MD 5.39; P < 0.001) and organized (MD = 5.02; P < 0.001) and were significantly less disciplined (MD = -3.48; P < 0.001) and self-assured (MD = -3.48; P < 0.001). Residents were also significantly less driven by reward (MD = -11.16; P < 0.001) and power (MD = -10.47; P < 0.001).

    Figure 5. Comparison of drive measures derived from the personal statements of general surgery applicants (blue) to the general population (orange) using Natural Language Processing technology.

    Figure 6. The mean difference in the personality measures derived from the personal statements of current general surgery residents at a single academic institution and applicants who did not receive employment at that institution. Results were computed by subtracting the mean of the applicant’s personality measure from the mean of the current resident’s personality measure.

    DISCUSSION

    This study analyzed personal statements from general surgery applicants and current residents using a commercially available computational linguistics and psychoanalytic AI interface. We discovered that applicants to general surgery include unique characteristics in their personal statements that can be quantified using NLP. Candidates showed a language-based personality that was self-assured, trusting, intellectual, and not impulsive or prone to stress. All of these traits have face validity as desirable among surgeons. Applicants also exhibited a remarkable drive for achievement and power. Variations were notable between the applicant pool and current residents of a single academic institution when characteristics were compared. The residents were more emotionally aware, organized, humble, and less assertive and cautious. Furthermore, the residents were less motivated by power and achievement and more interested in avoiding risk.

    This is the first study to evaluate the use of AI in the evaluation of personal statements of applicants for general surgery. Medicine and artificial intelligence have a long and interconnected history. AI has been integral in developing many medical technologies, from diagnostic tools to treatments and surgical procedures[7]. In turn, medicine has been vital in advancing AI, providing data and insights that have helped to improve and refine these technologies[8]. As AI continues to evolve, it is poised to transform medicine in various ways. AI-based diagnostic tools could, for example, improve patient care by identifying diseases in coordination with expert clinicians[9]. AI-powered treatments could be personalized to each patient, making them more effective[10]. Furthermore, AI-assisted surgeries could help minimize complications and speed up recovery times[11]. Ultimately, the relationship between medicine and AI is poised to result in major advances in healthcare for patients around the world.

    AI plays an increasingly important role in graduate medical education[12]. In recent years, AI-based systems have been developed to provide personalized feedback to medical students, residents, and fellows[13]. These systems use data from various sources, including clinical encounters, social media, and electronic health records. By analyzing this data, AI-based systems can identify patterns and recommend interventions to improve a trainee’s performance. AI-based techniques can also be used to create virtual patients that can be used for educational purposes[14]. In the future, AI-based systems will likely play an even more significant role in graduate medical education, providing individualized feedback and creating custom learning experiences for each trainee[15].

    The selection of residents is among the many areas in which AI can be effectively incorporated into graduate medical education. In a landmark decision aiming to promote more holistic evaluations, the Federation of State Medical Boards (FSMB) and the National Board of Medical Examiners (NBME), sponsors of USMLE Step 1, announced that Step 1 score reporting will transition to pass/fail beginning in January 2022[16]. This has left both candidates and residency programs uncertain about the future of resident selection. In addition, critics have argued that the new guidelines will prove ineffective, with USMLE Step 2 becoming the de facto Step 1[17]. As a response to these concerns, the Association of Program Directors (APDS) has continued to recommend enhancing transparency (encouraging applicants to comprehend their chances of admission) in addition to integrating holistic review (considering both individual characteristics and academic performance) as a central component of the application process going forward[18].

    Currently, it remains unclear what part of the residency application package can be utilized to evaluate personal characteristics and how these qualities can be measured accurately. The personal statement is an integral component of the application materials that may provide valuable insights into the qualities that are part of a holistic review. Through the control of their narrative, applicants can demonstrate experiences, motivations, and personal characteristics. However, these attributes can be challenging to assess and quantify individually, which renders the use of a personal statement primarily subjective and unreliable. For instance, there is poor inter-rating reliability, programs report uncertainty regarding the role of the personal statement, and this document is given variable importance in residency selection[19-21]. The mission should be to find methods of making personal statements more useful, standardized, and informative so that they may be incorporated into the residency selection process with enhanced consistency and reproducibility. We can look to modern technology and artificial intelligence for assistance in achieving this goal.

    It is necessary to consider the ethical implications of AI involved in procedures that affect vulnerable populations, such as those seeking training positions to become independent practicing doctors. Ideally, machine learning models and the use of AI and NLP are objective and neutral. However, our perspectives, opinions, stereotypes, and other personal assumptions may impact the development and implementation of these technologies[22]. The article published in 2021 by Dirk Hovy et al. on ethics in artificial intelligence provides a detailed summary of the five primary sources of bias in natural language processing technology[23]. In machine learning, the language data used to develop NLP systems can lead to bias if produced predominantly by a small, homogeneous sample. It is necessary to incorporate diverse sources of information and challenge the stereotypes inherent in historical documents resulting from traditional worldviews and prejudices. Having perfected their API over the past 30 years with tens of thousands of validating studies, we partnered with a leading NLP group that aimed to reduce bias with the help of linguistics, social scientists, and psychologists. However, considerable research and development are needed to practically apply NLP to the residency application process. It will be necessary to standardize and validate the NLP platform to documents consistent with the values of medical recruitment so that it can reliably and effectively measure the important characteristics for selecting future physicians and surgeons that can meet the needs of a diverse patient population.

    An additional concern associated with applying NLP to resident recruitment is the possibility of manipulating the evaluation to obtain unfair advantages. The Residency MATCH is intended to be an ethical process, with minimal manipulation by applicants and programs[24]. However, it is possible that the applicants will misrepresent their statements to gain an unfair competitive edge in the match. The NLP group we work with has conducted extensive research to develop their evaluation of written text based on how it is written instead of the specific content words. The filler words, propositions, articles, and pronouns determine the correlation with the various emotional and drive scores generated, thus making it extremely difficult to manipulate this aspect of writing. The personal statement would likely become unreadable if this were attempted. It is also possible to assess them for multiple writing styles in such long documents to determine if another is writing their statements or if their style changes at different points in the document. However, further investigation will be required to ensure that no stakeholder is gaming the system to gain an unfair advantage.

    In the admissions process, “fit” plays a vital role as it identifies candidates who can be better supported and more likely to succeed at a given program[25,26]. In most cases, the fit is evaluated at the interview stage[27]. Despite this, it may be challenging to determine compatibility during interviews, making this practice not universally successful, a problem amplified by the advent of virtual interviews[28]. The advancements in NLP, a primary component of AI, have enabled the ability to “understand” language through artificial neural networks and deep learning techniques[29]. The latest innovation has resulted in AI’s ability to derive psychoanalytic measures from language data[30,31]. The findings of our study provide a foundation for incorporating AI, specifically NLP, into the residency selection process, aiming to gain a deeper and more complete understanding of the candidates and identify those that will be a good match for the program. Moreover, our results may serve as a basis for assisting programs in meeting their admissions objectives and increasing transparency while decreasing the reliance on quantitative measures such as USMLE Step exams.

    This study has several limitations that must be acknowledged, including its limited generalizability as a study of one institution. However, we incorporated a robust sample size and included personal statements from applicants outside the institution. Further, despite the limitations in our capacity to establish causal inferences, we believe that the prospective nature of this study enhances our ability to draw meaningful conclusions. Personal statements are highly edited documents that may include contributions from many individuals, skewing the writer’s psychological composition and impairing the validity of personality and drive measures. Our partnering NLP platform generates a Genuine variable that assesses the entire document for consistency and clarity as a control measure. The “genuineness” of the personal statements was no different from that of the general population. The variables were developed using general documents, which may mask meaningful variation as they may be more similar to each other than the documents used to generate the scores. Addressing these validity concerns is an important future direction of our research.

    Future research aims to improve the NLP characteristics extracted from residency application personal statements. In collaboration with our partnering NLP group, we are working to standardize future measures extracted from these documents to personal statements and expert opinions. Furthermore, we are expanding the evaluation of applicants’ personal statements to include more established and objective characteristic measures (such as leadership, organizational skills, professionalism, and intelligence).

    CONCLUSION

    Artificial intelligence is rapidly advancing and increasingly integrated into everyday life. Modern innovations have enabled a deeper and more trusted integration of AI in the medical field and medical education. Presently, the residency admissions process is in transition, with policymakers urging a holistic review, USMLE Step 1 becoming pass/fail, and programs uncertain of their future candidate selection practices. Considering this, we used NLP, an integral aspect of AI, to evaluate the personal statements submitted by candidates seeking general surgery training. We discovered that NLP could identify unique personalities and drive measures of the residency applicants. Additionally, we identified characteristic features of current residents relevant to an academic program that can be used to facilitate a more thorough match. The proposal is to integrate this emerging technology into the selection process to facilitate a more holistic review based upon the principle of fit, which will better assist applicants in finding the programs that can better promote their training and career development.

    DECLARATIONS

    Authors’ contributions

    Made substantial contributions to conception and design of the study and performed data analysis and interpretation and writing of the manuscript: John AS

    Made substantial contributions to conception and design of the study, performed data acquisition, manuscript writing, as well as provided administrative, technical, and material support as the principal investigator (PI): Kavic SM

    Availability of data and materials

    Not applicable.

    Financial support and sponsorship

    None.

    Conflicts of interest

    Both authors declared that there are no conflicts of interest.

    Ethical approval and consent to participate

    Not applicable.

    Consent for publication

    Not applicable.

    Copyright

    © The Author(s) 2022.

    References

    • 1. Chowdhary KR. Natural language processing. In: fundamentals of artificial intelligence. New Delhi: Springer India, 2020. pp. 603-49.

      DOI
    • 2. Boyd RL, Schwartz HA. Natural language analysis and the psychology of verbal behavior: the past, present, and future states of the field. J Lang Soc Psychol 2021;40:21-41.

      DOIPubMed PMC
    • 3. Nehemiah A, Roberts SE, Song Y, et al. Looking beyond the numbers: increasing diversity and inclusion through holistic review in general surgery recruitment. J Surg Educ 2021;78:763-9.

      DOIPubMed
    • 4. Sungar WG, Angerhofer C, Mccormick T, et al. Implementation of holistic review into emergency medicine residency application screening to improve recruitment of underrepresented in medicine applicants. AEM Educ Train 2021;5:S10-8.

      DOIPubMed PMC
    • 5. Receptiviti. Available from: https://www.receptiviti.com/ [Last accessed on 21 Dec 2022].

    • 6. Cobb-clark DA, Schurer S. The stability of big-five personality traits. Econ Lett 2012;115:11-5.

      DOI
    • 7. He J, Baxter SL, Xu J, Xu J, Zhou X, Zhang K. The practical implementation of artificial intelligence technologies in medicine. Nat Med 2019;25:30-6.

      DOIPubMed PMC
    • 8. Kaul V, Enslin S, Gross SA. History of artificial intelligence in medicine. Gastrointest Endosc 2020;92:807-12.

      DOIPubMed
    • 9. Kumar Y, Koul A, Singla R, Ijaz MF. Artificial intelligence in disease diagnosis: a systematic literature review, synthesizing framework and future research agenda. J Ambient Intell Human Comput ;2022:1-28.

      DOIPubMed PMC
    • 10. Johnson KB, Wei WQ, Weeraratne D, et al. Precision medicine, AI, and the future of personalized health care. Clin Transl Sci 2021;14:86-93.

      DOIPubMed PMC
    • 11. Tokuyasu T, Iwashita Y, Matsunobu Y, et al. Development of an artificial intelligence system using deep learning to indicate anatomical landmarks during laparoscopic cholecystectomy. Surg Endosc 2021;35:1651-8.

      DOIPubMed PMC
    • 12. Paranjape K, Schinkel M, Nannan Panday R, Car J, Nanayakkara P. Introducing artificial intelligence training in medical education. JMIR Med Educ 2019;5:e16048.

      DOIPubMed PMC
    • 13. Chan KS, Zary N. Applications and Challenges of Implementing Artificial Intelligence in Medical Education: Integrative Review. JMIR Med Educ 2019;5:e13930.

      DOIPubMed PMC
    • 14. Kron FW, Guetterman TC, Fetters MD. Response to “emerging roles of virtual patients in the age of AI”. AMA J Ethics 2019;21:E920-5.

      DOIPubMed PMC
    • 15. Chen X, Xie H, Zou D, Hwang G. Application and theory gaps during the rise of artificial intelligence in education. Comput Educ Artif Intell 2020;1:100002.

      DOI
    • 16. Humphrey HJ, Woodruff JN. The pass/fail decision for USMLE step 1-next steps. JAMA 2020;323:2022-3.

      DOIPubMed
    • 17. Pontell ME, Makhoul AT, Ganesh Kumar N, Drolet BC. The change of USMLE step 1 to pass/fail: perspectives of the surgery program director. J Surg Educ 2021;78:91-8.

      DOIPubMed PMC
    • 18. Association of program directors in surgery. APDS position statements. Available from: https://apds.org/about/apds-position-statements/ [Last accessed on 21 Dec 2022].

    • 19. Crane JT, Ferraro CM. Selection criteria for emergency medicine residency applicants. Acad Emerg Med 2000;7:54-60.

      DOIPubMed
    • 20. Demzik A, Filippou P, Chew C, et al. Gender-based differences in urology residency applicant personal statements. Urology 2021;150:2-8.

      DOIPubMed PMC
    • 21. Flanigan KL, Mears CT, Morrell DS. Value of personal statements to dermatology programs: a survey-based critical review. Dermatology Online J 2020;26:5.

      PubMed
    • 22. Bird E, Fox-Skelly J, Jenner N, Larbey R, Weitkamp E, Winfield A. The ethics of artificial intelligence: issues and initiatives: study panel for the future of science and technology. Brussels : European Union, 2020.

      DOI
    • 23. Hovy D, Prabhumoye S. Five sources of bias in natural language processing. Lang Linguist Compass 2021;15:e12432.

      DOIPubMed PMC
    • 24. List JM. Honesty and fairness in the residency match. Virtual Mentor 2012;14:1011-5.

      DOIPubMed
    • 25. Bowe SN. “Fit” Has a broader meaning: recognizing the utility of person-environment fit theory in residency recruitment and selection. J Grad Med Educ 2020;12:637-8.

      DOIPubMed PMC
    • 26. Caretta-Weyer HA. An outcomes-oriented approach to residency selection: implementing novel processes to align residency programs and applicants. Acad Med 2022;97:626-30.

      DOIPubMed
    • 27. Stephenson-Famy A, Houmard BS, Oberoi S, Manyak A, Chiang S, Kim S. Use of the interview in resident candidate selection: a review of the literature. J Grad Med Educ 2015;7:539-48.

      DOIPubMed PMC
    • 28. Labiner HE, Anderson CE, Maloney Patel N. Virtual recruitment in surgical residency programs. Curr Surg Rep 2021;9:25.

      DOIPubMed PMC
    • 29. Jayaratne M, Jayatilleke B. Predicting personality using answers to open-ended interview questions. IEEE Access 2020;8:115345-5.

      DOI
    • 30. Ahmad N, Siddique J. Personality assessment using twitter tweets. Procedia Comput Sci 2017;112:1964-73.

      DOI
    • 31. Rathi S, Verma JP, Jain R, Nayyar A, Thakur N. Psychometric profiling of individuals using Twitter profiles: a psychological natural language processing based approach. Concurr Comput 2022;34:e7029.

      DOI

    Cite This Article

    OAE Style

    John AS, Kavic SM. Leveraging artificial intelligence for resident recruitment: can the dream of holistic review be realized?. Art Int Surg 2022;2:195-206. http://dx.doi.org/10.20517/ais.2022.24

    AMA Style

    John AS, Kavic SM. Leveraging artificial intelligence for resident recruitment: can the dream of holistic review be realized?. Artificial Intelligence Surgery. 2022; 2(4):195-206. http://dx.doi.org/10.20517/ais.2022.24

    Chicago/Turabian Style

    John, Ace St, Stephen M. Kavic. 2022. "Leveraging artificial intelligence for resident recruitment: can the dream of holistic review be realized?" Artificial Intelligence Surgery. 2, no.4: 195-206. http://dx.doi.org/10.20517/ais.2022.24

    ACS Style

    John, AS.; Kavic SM. Leveraging artificial intelligence for resident recruitment: can the dream of holistic review be realized?. Art. Int. Surg. 20222, 195-206. http://dx.doi.org/10.20517/ais.2022.24

    Views
    54
    Downloads
    8
    Citations
     0
    Comments
    0

    0

    Comments

    Comments must be written in English. Spam, offensive content, impersonation, and private information will not be permitted. If any comment is reported and identified as inappropriate content by OAE staff, the comment will be removed without notice. If you have any queries or need any help, please contact us at support@oaepublish.com.

    © 2016-2022 OAE Publishing Inc., except certain content provided by third parties