Principal Investigator: Roman Nowygrod, Professor of Surgery (VP&S)
Summary: At the heart of medical education is the exchange of feedback, a critical tool in shaping a student's clinical and professional growth. Our project, "FeedForward," proposes the innovative use of artificial intelligence (AI) to improve the feedback process for medical students during their surgical clerkship at Columbia University's Vagelos College of Physicians and Surgeons. Currently, providing personalized and high-quality feedback is a time-consuming task that challenges educators, particularly in the context of formative (ongoing) and summative (conclusive) assessments. This project targets the inefficiencies in creating summative feedback and individualized learning plans (ILPs) for medical students, addressing the specific needs for scalability and quality enhancement in feedback mechanisms. The project introduces a custom-developed AI chatbot, leveraging Columbia's ChatGPT Enterprise, to automate the initial creation of summative feedback narratives and ILPs for medical students at the conclusion of their general surgery clerkship. This tool is designed to analyze compiled student performance evaluations (SPEs) and generate comprehensive, detailed, and actionable feedback that aligns closely with medical education performance objectives (MEPOs). Our project specifically aims to evaluate if AI-generated feedback, compared to human-generated can 1) match or exceed quality in terms of accuracy, relevance, and detail 2) match or exceed utility for various stakeholders (students, clerkship directors, program directors, and medical student performance evaluation (MSPE) writers), and 3) provide a novel tool to increase efficiency for those tasked with providing summative feedback. To assess these, we plan to use a combination of qualitative and quantitative methods to analyze the AI and human-generated feedback as well as direct feedback from a variety of stakeholders including students, clerkship directors, program directors, and MSPE writers on the clarity, relevance, and helpfulness of the AI-generated feedback. "FeedForward" aims to revolutionize the feedback process in surgical education by integrating AI, thereby improving both the quality of education and the operational efficiency for educators. With this project, we anticipate setting a benchmark for the future integration of AI across various educational settings, demonstrating substantial benefits for both learners and educational institutions.