Nudges in public health education: Tailored nudges to increase engagement in quantitative analysis coursework
Principal Investigator: Christine Mauro, Assistant Professor of Biostatistics, Mailman School of Public Health (MSPH)
Award Date: Fall 2024 - Spring 2025
Status:
Summary
To expand and enrich research activities of Phase 3 of this iterative project with increase participation and, in turn, statistical power. We will assess, in a randomized controlled trial, the impact of tailored educational nudges on student engagement and learning outcomes in an interdisciplinary biostatistics and epidemiology Core course. We hypothesize that tailored nudges, compared to generic nudges, will result in significantly higher 1) engagement with course materials and 2) learning outcomes. We further hypothesize that the effects of tailored nudges may differ by student characteristics, including baseline quantitative skills.
Collaborative Art & Play as Practice: Learning Inclusive Leadership Skills with AI-generated Art
Summary
Teaching inclusive leadership in business schools remains challenging in part because doing so requires developing students’ social-emotional skills—like getting curious about the self and others, and approaching, rather than avoiding, intergroup differences (Luckman, 2023). These skills cannot be learned through passive channels alone, that are often found in higher education. In the current project I utilize and test the efficacy of text-to-image generative-AI as a novel tool for business students and professionals to practice inclusive team leadership behaviors through collaboration and imaginative play exercises.
Art generative-AI is a promising tool for practicing inclusive behaviors as it 1) quickly makes what is abstract and theoretical, real and concrete; 2) elicits experiences of awe and amazement which can serve as a precursor to enacting prosocial behaviors (Piff et al., 2015); and 3) integrates arts-based (versus analytical) activities in the classroom which have been shown to enhance social-emotional skills (Yang et al., 2023)—all while allowing participants to develop their technical skills (i.e., prompt generaJon) as AI tools proliferate in day-to-day business functioning.
During the 2023-2024 academic year and with the support of SOLER, I have designed three team exercises utilizing text-to-image generative-AI for business students and professionals to engage in imaginative group play and practice inclusive leadership behaviors. The first team exercise created, a Design Challenge, was piloted in the MBA classroom with ~400 students. In addition, I have created and refined two additional exercises—Art Matching Task and Imaginative Story Illustration—with ~300 professionals. Preliminary data (both quantitative and qualitative) suggest these activities enhance interpersonal communication within teams, invite team members to get curious about their differences and unique backgrounds, and allow diverse teams to form stronger bonds early in their development and work trajectory.
With these pilots, I have also refined numerous exercise logistics, including staffing and technology requirements, appropriate assessment questionnaires and generative-AI software applications (i.e., testing Midjourney, Adobe Firefly, and ChatGPT4’s DALL-E). Importantly, I have also identified a comparison activity (the Marshmallow Design Challenge, a group exercise that does not utilize generative-AI) with which to compare the use of generative-AI in the development of inclusive team leadership behaviors. Finally, I have received IRB approval to conduct this comparison through an experimental design in the Lead classroom as well as in executive workshops with both student and professional populations. Next steps involve data collection and analysis across ten workshops (~500 participants) scheduled for the 2024-2025 academic year.
Principal Investigator: Dr. Ashli B. Carter, Lecturer, Management
Award Date: 2024 - 2025
Status:
Testing the Efficacy of Dialogic Feedback in Enhancing Public Speaking Skills
Principal Investigator: Zhongqi Shi, Senior Lecturer, East Asian Languages and Cultures
Award Date: Spring 2024 - Summer 2024
Status:
Summary
Effective public speaking skills are essential for college students to express ideas clearly, persuade others, and present themselves confidently. However, traditional feedback methods often leave students feeling confused and disengaged. This study proposes using a novel "dialogic feedback" approach, defined as interactive exchanges where interpretations are shared, meanings negotiated and expectations clarified, to potentially enhance feedback effectiveness. We aim to assess the efficacy of dialogic feedback in improving students' presentation skills and perceptions compared to traditional didactic feedback. Dialogic feedback offers students opportunities for enhanced comprehension and engagement through an interactive process. Employing a longitudinal research design, this study seeks to empirically assess the cognitive, emotional, and motivational benefits of dialogic feedback versus didactic feedback. Participants will be recruited from Business Chinese classes that emphasize developing communication and public speaking skills. They will be randomly assigned to either an experimental group receiving dialogic feedback sessions or a control group receiving didactic (one-way) feedback sessions. We hypothesize that students receiving dialogic feedback will outperform the control group across measured outcomes related to feedback retention, perceptions, and motivational behaviors. Data will be collected through post-presentation student surveys and research assistant evaluations of performances. This study aims to contribute insights into effective feedback practices to improve language education. Successful findings will be shared to inspire broader adoption of dialogic feedback, ultimately enhancing student learning and preparing them for success.
StreamLine
Summary
The rise of digital media and technological advances has led to a significant use of instructional videos across educational platforms, particularly in flipped classrooms as well as public streaming services. An increase in these practices has been notably observed, for example, from 2010 to 2015 in K-12 education, and the trend continued through the COVID-19 pandemic, which necessitated a transition to digital instruction. Our proposal, StreamLine, seeks to optimize engagement by investigating the efficacy of “chunking” instructional content in videos used for bridge modules for organic chemistry I courses as a preamble to the content they will learn in their upcoming studies. Bridge modules are a series of recorded lectures that bridge concepts between courses from different semesters. The hypothesis poses that shorter, concept-driven videos will enhance student understanding and retention of information compared to traditional longer videos. The existing literature suggests that segmenting content into manageable portions can significantly reduce cognitive load; therefore, this is expected to improve attention and information retention. Previous methodologies, which largely relied on self-reporting and lacked interactive elements, which yielded inadequate results regarding true engagement accurately. StreamLine will utilize eye-tracking technology alongside traditional survey methods to include a comprehensive analysis of student engagement with supplemental video content. We will evaluate “chunked” versus full-length video content by using empirical methods to assess attention metrics, with the goal of refining video instructional design for enhanced educational outcomes.
Principal Investigator: Talha Siddiqui, Lecturer, Chemistry
Award Date: Spring 2024 - Fall 2024
Status:
“FeedForward”: An Artificial Intelligence (AI) Tool for Creating Summative Feedback and Individualized Learning Plans (ILPs) for Surgery Clerkship Medical Students
Principal Investigator: Roman Nowygrod (MD), Department of Vascular Surgery, Columbia University Vagelos College of Physicians and Surgeons
Award Date: Summer 2024 - Summer 2025
Status:
Summary
At the heart of medical education is the exchange of feedback, a critical tool in shaping a student's clinical and professional growth. Our project, "FeedForward," proposes the innovative use of artificial intelligence (AI) to improve the feedback process for medical students during their surgical clerkship at Columbia University's Vagelos College of Physicians and Surgeons. Currently, providing personalized and high-quality feedback is a time-consuming task that challenges educators, particularly in the context of formative (ongoing) and summative (conclusive) assessments. This project targets the inefficiencies in creating summative feedback and individualized learning plans (ILPs) for medical students, addressing the specific needs for scalability and quality enhancement in feedback mechanisms. The project introduces a custom-developed AI chatbot, leveraging Columbia's ChatGPT Enterprise, to automate the initial creation of summative feedback narratives and ILPs for medical students at the conclusion of their general surgery clerkship. This tool is designed to analyze compiled student performance evaluations (SPEs) and generate comprehensive, detailed, and actionable feedback that aligns closely with medical education performance objectives (MEPOs). Our project specifically aims to evaluate if AI-generated feedback, compared to human-generated can 1) match or exceed quality in terms of accuracy, relevance, and detail 2) match or exceed utility for various stakeholders (students, clerkship directors, program directors, and medical student performance evaluation (MSPE) writers), and 3) provide a novel tool to increase efficiency for those tasked with providing summative feedback. To assess these, we plan to use a combination of qualitative and quantitative methods to analyze the AI and human-generated feedback as well as direct feedback from a variety of stakeholders including students, clerkship directors, program directors, and MSPE writers on the clarity, relevance, and helpfulness of the AI-generated feedback. "FeedForward" aims to revolutionize the feedback process in surgical education by integrating AI, thereby improving both the quality of education and the operational efficiency for educators. With this project, we anticipate setting a benchmark for the future integration of AI across various educational settings, demonstrating substantial benefits for both learners and educational institutions.
Preparing for college through COVID: impacts of disrupted and online high school learning on general chemistry readiness
Summary
Online and interrupted learning caused by the COVID-19 pandemic has led to decreases in high school students’ performance on standardized tests across all subjects and to decreases in their self-reported academic self-efficacy. Many students who experienced learning disruptions in high school are now in college, and future classes of college students will have experienced similar disruptions in earlier years. Higher education instructors, including those in chemistry, have felt that students struggle in ways that they did not use to and attribute this perceived change to COVID-related learning disruptions. However, no quantitative study has identified this effect in introductory chemistry, likelydue to the statistical challenge of isolating COVID-related discrepancies from other year-to-year changes in exam scores. This study proposes to surmount this statistical challenge by using Columbia’s large population of nontraditional students as an internal control. Specifically, the change in traditional vs. nontraditional exam performance will be compared for pre-2020 and post-2020 iterations of General Chemistry. Past years’ questions will be coded based on content, math or reading requirement, and complexity. Specific skills that have been strongly impacted by COVID-related learning disruption will then be identified and could be the targets of future curricular interventions.
Principal Investigator: Christopher Eckdahl, Lecturer, Chemistry
Award Date: Summer 2024 - Summer 2025
Status:
Practices and Implications around Concealment of Social Class Identities among Graduate Students in Engineering
Principal Investigator: Paul Ingram, Kravis Professor of Business, Columbia Business
School
Award Date: Summer 2024 - Summer 2025
Status:
Summary
Processes of identity management are key to understanding the experiences of disadvantaged individuals in education institutions, and therefore, inequality. We plan to use a novel “whole identity” approach, combined with natural language processing, to examine concealment of identity elements associated with lower social class standing among graduate students in engineering at Columbia University. Evidence elsewhere suggests that students in elite professional programs are motivated to conceal lower social class identity elements to avoid penalties in the job market, out of social embarrassment, and to appear as more attractive partners for building professional networks. These arguments, however, have been developed among law and business school students, contexts where the importance of social/cultural status may be higher relative to technical skills than among engineering students. It matters whether engineering students do conceal lower social class identities, as identity concealment is associated with lower wellbeing, lower performance, and lower commitment to the context in which concealment occurs. We will also examine the intersection of identities around social class, race and gender to test, for example, whether social class concealment is more or less common among female students. If the data collection opportunity allows, we also intend to conduct experiments aimed at reducing the costs of concealment and helping the students achieve better outcomes in their programs and beyond.
Evaluating the Impact of MetricsMentor - an interactive visual graphical platform for econometrics courses
Summary
This project evaluates the hypothesis that a new innovative course design enhances students’ learning outcomes in econometrics. Currently, the course relies on traditional lecture-style teaching, which may not fully engage students or help them grasp complex concepts like endogeneity problems in regression analysis. To address this gap, we are developing MetricsMentor, an interactive online platform that provides a hands-on learning experience. This platform allows students to explore different scenarios and visually understand the effects of specific endogeneity problems and their solutions. We will conduct controlled experiments to compare the effectiveness of the platform with traditional lecturing methods. Students will be randomly assigned to either use MetricsMentor or receive traditional instruction, and their performance will be assessed through quizzes before and after each simulation. Additionally, we will measure students’ attitudes toward econometrics before and after using MetricsMentor. By using both quantitative and qualitative methods, we aim to provide a comprehensive assessment of the platform’s impact on student learning outcomes and attitudes. We expect MetricsMentor to improve both the immediate comprehension of contents and overall attitudes toward econometrics among students.
Principal Investigator: Seyhan Erden, Lecturer, Economics
Award Date: Summer 2024 - Summer 2025
Status: