Projects

The projects highlighted below received Provost's SOLER Seed Grants (PSSGs).

Title: Evaluating augmented reality for embodied learning in introductory biochemistry

Principal Investigator: Brent Stockwell, Professor, Biological Sciences

Award Date: Summer 2023

Status: Completed, manuscript in prep.

Summary:  In response to the COVID-19 pandemic, which prompted a shift to online learning, interest in virtual reality (VR) as a tool for teaching biochemistry emerged. Through SOLER-supported and CUIT-supported studies conducted from 2020-2022, the potential of VR in enhancing student engagement and understanding of disciplinary concepts was explored, revealing positive student perceptions despite technical challenges. Now, with instruction returning to classrooms, attention turns to augmented reality (AR) as a complement to in-person learning, leveraging advanced hardware and software like the Quest Pro headsets and Nanome program. Collaborating with the CTL, this iteration aims to integrate embodied learning principles and explore AR's impact on student outcomes, motivation, and interest in further study or professional opportunities. By incorporating qualitative research methods and innovative pedagogical approaches, the study seeks to elucidate AR's potential to create immersive learning environments that foster meaningful connections and deeper understanding of complex biochemistry concepts.

*********************************************************************************************************************

Title: Flexible neuroscience- and technology-driven frameworks for the study of classroom engagement

Principal Investigator: Alfredo Spagna, Lecturer, Psychology

Award Date: Summer 2023

Status: Ongoing, manuscript submitted

Summary: Classroom engagement has a crucial impact on academic success. However, the role of affective, cognitive, and behavioral components of engagement in classroom learning remains elusive. We conducted a study within a real-world undergraduate classroom using a non-invasive, research-based approach to clarify the role of these components. We employed portable EEG headsets to measure cognitive engagement via theta/beta ratio, in-class quizzes to assess content retention, and post-class subjective questionnaires to index affective engagement by measuring feelings of learning and attention. As expected, content retention was positively related to affective engagement, while cognitive engagement positively related to content retention. Interestingly, however, cognitive engagement was negatively related to subsequent affective engagement, while holding content retention constant. Crucially, accounting for both of these sources of variance in the classroom provided a much more robust model of affective engagement, underscoring the need for real-world, multimodal learning analytics. We discuss how these approaches enable real-time studies of classroom engagement and can be integrated to develop neurofeedback interventions.  

*********************************************************************************************************************

Title: Promoting engagement in quantitative data analysis coursework via tailored educational nudges

Principal Investigator: Samantha Garbers, Associate Professor, Population and Family Health 

Award Date: Summer 2023

Status: Ongoing, manuscript submitted

Summary: Nudge theory – the behavioral economics idea that subtle environmental cues influence decision making – shows promise in education in the form of reminders, social comparisons, gain/loss framing, or goal setting. However, prior research, including ours, suggests that nudges are effective only when they tap into motivations for learning and engagement. We propose to test the hypothesis that nudges tailored to students’ motivations will enhance engagement and learning outcomes in an interdisciplinary Public Health course. Mailman School of Public Health (MPH) students spend their first semester in a Core curriculum that provides essential interdisciplinary knowledge. Research Methods and Application is a Core course taught by Biostatistics and Epidemiology faculty in a hybrid format (“Core Quant”). It is the most challenging Core course: 3% of students in Fall 2022 did not successfully complete it despite the availability of tutoring. Furthermore, student engagement with supportive materials such as slide decks and answer keys is low. In two SOLER-supported studies (2021-2022), we investigated the use of educational nudges in Core Quant. In Study 1, a randomized controlled trial revealed that nudges linking behavioral engagement (e.g., accessing readings, watching lecture videos) with assessment grades did not significantly affect student actions. Our findings were published in March 2023. In Study 2, we implemented a mixed-methods study (manuscript in preparation) of students’ motivations and behaviors featuring interviews and forced-choice analyses comparing nudge messages that will inform the development of tailored nudges to be tested in our proposed third study. Persona creation: We will create three personas with narrative descriptions and visual avatars for students to select, aligned with motivation profiles identified in our qualitative analysis and corresponding to established features of nudges: learners (intrinsically motivated), doers (task-oriented), and performers (grade-motivated). Nudge finalization: Each persona will have visual, tailored nudges deployed throughout the semester; a set of text-only generic nudges (without appeal to motivation) will be the control condition. Experimental study design: Students will identify their learning persona at the start of the semester; within each persona, students will be randomized 1:1 to receive either a generic or persona-specific nudge. The generic nudge will be embedded in a weekly email to students that summarizes upcoming work across the Core courses. Tailored nudges will be deployed through Courseworks’ Group feature and Announcement function. For blinding, the groups will be named numerically rather than by persona label; group membership will not be visible to students. Students randomized to the generic group, or who opt out, will be assigned to a group but will not receive a nudge via the announcements. We will compare the effect of tailored vs. generic nudges within each persona group and across all groups on the following outcomes: engagement with course materials and learning supports, pre-post change in trust and confidence scales, post-course empirical knowledge score, and course grade. If tailored nudges enhance engagement learning outcomes in this study, they can be implemented in courses with similar challenges within and beyond MPH. 

*********************************************************************************************************************

Title: Assessing graduate student attitudes toward ChatGPT and its effectiveness as a teaching tool for real estate finance

Principal Investigator: Chris Munsell, Associate Professor of Professional Practice, GSAPP

Award Date: Summer 2023

Status: Completed, manuscript in prep.

Summary: In the master's level course Real Estate Finance II, one challenging learning objective requires students to apply the mechanics and theory of time value of money to the joint venture waterfall, an essential concept. Students struggle to achieve this learning because it requires an advanced knowledge of Excel and financial theory. We believe that this project will be innovative because it will allow students and ourselves to understand how effective or ineffective AI can be as an external teaching aid in real estate finance. We propose a mixed-methods study to test the effectiveness of large language model generative AI (such as ChatGPT) as an alternative instructional practice in this context, with students using the technology in one homework assignment. We will compare students who are randomly assigned to complete the homework with or without the use of AI. We predict that students in the AI group will exhibit enhanced performance on their homework assignment, but inferior performance on their final exam. The non AI group we predict that they will not perform as well as the AI group. We will also ask both student groups to respond to open-ended questions that address their learning process, academic integrity reflections, and ethical decision making in real estate finance.
  

*********************************************************************************************************************

Title: Collaborative art & play as practice: learning inclusive leadership skills with AI-generated art

Principal Investigator: Ashli Carter, Lecturer, GSB

Co-Sponsor: Innovative Course Module Design Grant (CTL)

Award Date: Summer 2023

Status: Ongoing

Summary: More than ever, emerging leaders must learn how to create inclusive environments wherein individuals with diverse talents, backgrounds, and perspectives can thrive, and a culture of learning, growth, and experimentation can flourish. However, the social-emotional skills needed for inclusive leadership are learned by doing and not just knowing. When it comes to practicing inclusivity, there often exists a gap between what individuals understand on an abstract level versus their actual behaviors. This may occur in part because practicing inclusive behaviors – particularly as a novice – involves interpersonal risk-taking, which individuals may especially avoid in diverse settings. Group-based, imaginative play has recently been explored as a means for diverse teams to foster stronger team bonds and more effective collaboration. This play enhances team functioning by reducing intergroup anxiety and fostering creativity, trust, and a sense of psychological safety. We propose to evaluate a new tool for facilitating group-based, imaginative play and teaching inclusive leadership behaviors – art generative-AI. Art generative-AI is a promising tool for practicing inclusive behaviors as it allows students to engage in key social-emotional skills of inclusive leadership (i.e., interpersonal risk-taking and novel relationship building) while having fun together. Furthermore, generating art with AI tools may also aid learning inclusive behaviors by inspiring moments of collective awe and facilitating rapid and repeated experiences of collaboratively turning abstract ideas into reality. In the following proposal, we outline a study testing the effectiveness of a group-based, imaginative play exercise facilitated through art generative-AI for enhancing inclusive behaviors within MBA student teams.
 

*********************************************************************************************************************

Title: Sense of scale embodied: investigating the impact of object-based learning on undergraduate STEM education

Principal Investigator: Bradley Pitcher, Lecturer, Earth & Environmental Sciences

Award Date: Summer 2023

Status: Completed, manuscript in prep.

Summary: This research proposal aims to evaluate the hypothesis that incorporating physical objects and movement, using an embodied learning approach, enhances student learning and cultivates a better comprehension of scale. This is a major learning objective of Frontiers of Science (FoS), a core course for Columbia College first-year students. The traditional approach of teaching sense of scale via a written activity that involves sorting a list of objects inadequately addresses the learning objective. The innovative approach proposed in this study, sense of scale embodied (SSE), requires students to manipulate M&M candies to visualize large numbers, areas, and volumes, and to physically move around the room to estimate relative sizes and distances of planetary objects. The impact of the SSE will be compared against a traditional approach as measured by student performance on quizzes administered immediately after the activity and at the end of the semester. FoS is particularly well suited for this experiment because over 500 students are enrolled each semester and instructors each teach two sections, allowing them to teach one SSE and one traditional section. The results will be shared with FoS instructors and directors, and if SSE proves to be more effective, it will be permanently included in the curriculum. This proposed research is innovative as it would provide a unique direct comparison of the impact of an object-based learning activity on the development of students’ sense of scale, thereby greatly contributing to the growing body of research in favor of embodied pedagogical techniques in STEM higher education.

*********************************************************************************************************************

Title: Evaluating the effectiveness of an upstander training across modalities

Principal Investigator: Jean Marie Alves-Bradford, Associate Professor, Psychiatry (VP&S)

Co-Sponsor: Innovative Course Module Design Grant (CTL)

Award Date: Summer 2023

Status: In progress

Summary: Medical students experience bias incidents at alarmingly high rates of over 80% in some student samples. Allyship and Upstander skills are part of the AAMC Diversity Equity and Inclusion competencies, yet many schools have not yet implemented bias-response training. Students, faculty, and clinical team members can be trained to be “Upstanders,” those who act and intervene on behalf of themselves and others. Bias-response training has been shown to increase medical students’ self-assessed confidence responding to bias incidents. With prior Provost support, we developed trainings to facilitate skill uptake. Investigating causal relationships between prior experience, topic, and modality on effectiveness of learning has direct connections to the science of learning and implications for all teaching. However, this support was not designed to draw causal conclusions. Additionally, while many prior studies comparing asynchronous online versus offline teaching modalities have demonstrated equal effectiveness, online modules intended to increase confidence responding to microaggressions showed lowest effectiveness when compared to other topics within cultural competence. Furthermore, there is emerging evidence that prior experience with microaggressions impacts effectiveness of bias training. Therefore, this project has three core goals - to strengthen existing correlational findings with causal conclusions, to evaluate the effectiveness of in-person versus asynchronous online modality in teaching upstander skills, and to explore the relationship between existing experience with microaggressions and uptake of Upstander skills. Given that this project addresses a required medical education competency, this grant will be instrumental in determining how to move forward with efficiency and efficacy.

*********************************************************************************************************************

Title: Exploring the use of Large Language Models to generate text versions of neuroscience lectures

Principal Investigator: Darcy Kelley, Professor, Biological Sciences

Award Date: Summer 2023

Status: In progress

Summary: We propose a feasibility exploration of using LLMs (Large Language Models) to create a text version of the lectures comprising Developmental and Systems Neuroscience. The project addresses the following problem: the rapid pace of discovery renders commercial neuroscience texts obsolete upon publication. Undergraduates rely on texts to support learning, but what if those texts are wrong (out of date). Our goal is to pilot – using an LMM – a living, written text based on audio recorded in Spring 2023 lectures delivered by the PI. Each 2023 lecture was accompanied by illustrations (figures from the primary literature, reviews and text figures) in PowerPoint format. A secondary goal is to replace the 2023 figures (heterogeneous in style and source) with original illustrations in a uniform style, an important element for visual learners. The project is innovative because it will provide a means to replace a static, outdated source of information with a dynamic, current, written and visual resource that can be updated in real time. Ascertaining impact on 2024 students relative to 2023 will be a major challenge for this project; CTL Lead Teaching Fellows and other personnel will support the development of assessment frameworks. The value of original illustrations will be equally challenging to assess, but we will work closely with Nicoletta Barolini (CU Art Director, Office of Communications & Public Affairs) to develop and evaluate new illustrations. 

Title: Evaluating augmented reality (AR) using Microsoft Mesh and Hololens 2 for teaching deep expertise in introductory biochemistry

Principal Investigator: Brent Stockwell, Professor, Biological Sciences

Co-Sponsor: CUIT Emerging Technology Teaching and Learning Grant

Award Date: Summer 2022

Status: Deferred to 2023-2024

Summary: Learning biochemistry is challenging for students using chalkboards, PowerPoint or Zoom, due to 3D aspects of molecular conformations. AR/VR offer a solution—they can be more immersive and enhance attention due to the lack of distractions, and allow engagement with 3D objects, such as proteins, that cannot be directly experienced in the real world. We previously performed studies with support from the Provost’s Office and from SOLER using VR to teach biochemistry using Oculus Quest 1 headsets and Spatial.io, meeting with students to discuss 3D aspects of biochemistry, and Quest 2 headsets with the online platform Glue. We found that using VR was impactful in terms of engagement and learning improvements. We found that many students are highly engaged during the in- person experience as well as the VR experience, but that rendering detailed proteins and their motions is challenging with the limited processing power of the Quest 2 headsets. Most critically, we found that students need to be able to translate the insights from interacting with 3D models of proteins back to representing proteins in two dimensions in order to build on their knowledge with subsequent reading of papers, textbooks, and other 2D representations of 3D molecules. This translation from 3D back to 2D is difficult to do in VR because it is difficult to simultaneously view 3D and 2D representations of molecules with high fidelity. We hypothesize that Augmented Reality (AR) will provide the missing link in translating deep insights into the 3D structure-function relationship in protein molecules into 2D representations of these molecules that can be used outside of the AR/VR format.

*********************************************************************************************************************

Title: Neuro-DBER: A flexible neuroscience- and technology-driven framework for Discipline-Based Education Research

Principal Investigator: Alfredo Spagna, Lecturer, Psychology

Co-Sponsor: CUIT Emerging Technology Research Grant

Award Date: Summer 2022

Status: Ongoing, manuscript submitted

Summary: What predicts student success? The classroom is a dynamic setting where students and instructors of diverse backgrounds exchange ideas in the pursuit of learning. Many variables complicate this collaborative paradigm: composition changes due to student absence; content difficulty increases as the course deepens; student fatigue accumulates as the semester progresses. While the relative contribution of complicating variables has been studied in isolation, a comprehensive understanding of their synergistic effects remains elusive. What can be done to help instructors and students integrate solutions to these well-known challenges into a cohesive framework? This project aims to develop a flexible neuroscience and technology-driven framework for Discipline-Based Education Research (Neuro-DBER) to revolutionize teaching and learning in the classroom. Our system merges cognitive neuroscienceeducationand innovative technology to enable investigations of the relationships between subjective experience, body language, , real-time learning, and neural measures of in-class activities. Neuro-DBER is a flexible technological framework suitable to any field of study to address critical questions about classroom experiences, such as the gap between a student's subjective feelings versus actual learning measured. Featuring integration of advanced research technology, our framework facilitates thoughtful design of in-class activities that will thereby transform the way we teach and learn, yielding an inclusive environment conducive to effective teaching and addressing students’ distinctive needs.

*********************************************************************************************************************

Title: Increasing student engagement with course content and learning supports through mixed-methods mapping of motivation and learning behaviors

Principal Investigator: Samantha Garbers, Associate Professor, Population and Family Health 

Award Date: Summer 2022

Status: Ongoing, manuscript submitted

Summary: This project aims to conceptualize, quantify, and explore qualitatively student engagement and motivations in an interdisciplinary biostatistics and epidemiology Core course. At the Mailman School of Public Health, all incoming students spend their first semester participating in an interdisciplinary Core curriculum. The most challenging course in the curriculum, Research Methods & Application - Quantitative Data Analysis, is taught in a hybrid format with synchronous and asynchronous content. Student supports have been deployed in recent years, but some students continue to face challenges in successful completion of this course. As revealed by our prior SOLER-funded project – a randomized trial of behavioral nudges – student engagement with Core course content is low and varies substantially by week; most troublingly, engagement with learning supports for task management declined steadily over the course of the semester. Academic engagement is a critical construct known to improve course persistence, predict student success (including grades, degree completion), educational attainment, and occupational attainment. Interventions are needed to increase student engagement across the board, for all students, but particularly for students whose baseline assessments indicate the need for academic support. These interventions should aim to increase student engagement with both course content and task management learning supports. Intervention strategies should be tailored to students’ academic needs, baseline competencies, and informed by evidence on how students of varying baseline competency levels engage with these course elements. More nuanced data on how students of different competency levels engage with course elements – including qualitative data on student motivations for engagement – are needed to guide the development and deployment of interventions that can be tested using rigorous designs. In Phase 1, we will replicate student engagement analyses using learning management system learning analytics data for the 2021 course, stratifying by baseline competency level. In Phase 2, we will use rigorous qualitative methods as one-on-one interviews to explore student learning behaviors and motivations that underlie engagement. Our prior study did not demonstrate efficacy of behavioral nudges most likely because the nudges, though evidence-based, were not informed by data on what motivates students. In the final phase of the project, we will quantitatively explore student perspectives of motivating behavioral nudges using forced-choice paired comparison methods that assess preferences across a range of dimensions, including messaging, placement/delivery, timing, and design. This proposed work is an essential step before deploying a larger-scale trial testing nudge interventions with more complex features, such as tailored nudges based on baseline characteristics.

*********************************************************************************************************************

Title: An exploration of the anti-oppressive practice of 'un-grading' and its effects on student learning

Principal Investigator: Amy Werman, Lecturer, Social Work

Award Date: Summer 2022

Status: Completed

Summary: A significant challenge facing instructors at Columbia School of Social Work is the fair, equitable, and accurate assessment of students. Instructors use letter grades to evaluate student outcomes, but there is little agreement as to what grades indicate. There is, however, a shared implicit assumption that grades measure learning. The current study interrogates that assumption by exploring the implementation of an "ungrading" pedagogy in a Program Evaluation course. Ungrading is an innovative practice in which students receive feedback on assignments and revise their work based on that feedback; however, their work is not graded. Rather, the student is asked to reflect on what they learned and on the process of learning (their motivation, effort, curiosity), and then give themselves a grade on the assignment. Anecdotal evidence suggests that ungrading fosters learning because students focus on the process of learning instead of the outcome of a grade. This study aims to examine objective and subjective student experiences with ungrading relative to traditional alternatives.   

*********************************************************************************************************************

Title: How teacher-student connection improves abstract thinking

Principal Investigator: Tugce Bilgin, Lecturer, Ecology, Evolution, and Environmental Biology

Award Date: Summer 2022

Status: Completed, manuscript submitted

Summary: The ‘social breathing’ and ‘interactive brain’ hypotheses as well as models of learning dynamics, joint action, and teacher-student interactions posit that low-level perceiving and signaling systems in nonverbal behavior scaffold and support interpersonal coordination and performance in joint tasks, such as teaching and learning. This study attempts to quantify this relationship. Teacher-student dyads recruited from various sections of the Columbia College Core course 'Frontiers of Science' will use a 1-on-1 online portal to collaboratively complete simple and complex tasks. We will examine the relationships between nonverbal behavior, task complexity and performance, and dyad familiarity (i.e., whether or not the student and instructor were drawn from the same section.) 

*********************************************************************************************************************

Title: Interactive simulations to support inquiry-based statistics instruction in Frontiers of Science

Principal Investigator: Nicholas Bock, Lecturer, Earth & Environmental Sciences

Co-Sponsor: Innovative Course Design Grant (CTL)

Award Date: Summer 2022

Status: Completed

Summary: Frontiers of Science is a required course for all first-year Columbia College students. The goal of the course is to provide an introduction to current research across four different scientific disciplines while helping students develop a framework for approaching, analyzing, and interpreting quantitative information (hereafter referred to as the scientific "habits of mind"). Students come to Frontiers of Science with a wide range of prior science coursework and attitudes/interests/experiences in science. Therefore, a central challenge in planning individual seminar sessions is identifying learning activities that address course objectives while serving the learning needs of all students in the classroom. This challenge can be particularly acute when introducing statistical concepts. Frontiers of Science students are introduced to basic statistical concepts (e.g., measures of central tendency and dispersion) at the start of the semester. During the first three weeks of the semester, students are expected to develop an understanding of how  these concepts can be used to formulate statistically supported inferences (e.g., the use of standard error to evaluate confidence intervals and to support hypothesis tests). However, based on an introductory survey completed by students in seminar sections during the spring 2022 semester, only 50 percent of students reported having completed previous coursework in statistics. Therefore, there is a clear need within Frontiers of Science for instructional tools that are appropriate and challenging for students both with and without prior experience in statistics. We will develop a series of interactive simulations to support inquiry-based learning of basic statistical concepts in Frontiers of Science seminar sessions. We additionally propose to work with SOLER Initiative faculty to investigate the impact of the proposed instructional materials on learning outcomes in Frontiers of Science seminar sections. Because all incoming first-year students enroll in Frontiers of Science, student interactions with these simulations will impact learning outcomes across Columbia College. Moreover, because the statistical concepts to be introduced are foundational to data analysis across many different fields of study, these simulations will be of use to faculty across the  university; for example, the simulations may be adapted to introduce Earth Science students to statistical analysis of environmental data. The simulations will additionally expand upon interactive simulations previously developed by faculty in the School of International and Public Affairs (SIPA) as part of the Stats Interactives website.

Access the project proposal here.

Title: Evaluating Oculus Quest 2 and Glue as a Virtual Reality (VR) Platform for Learning Biochemistry

Principal Investigator: Brent Stockwell, Professor, Biological Sciences

Co-Sponsor: CUIT Emerging Technology Grant

Award Date: Summer 2021

Status: Data collection complete; analysis in progress

Summary: Learning biochemistry is challenging for students using chalkboards, Powerpoint and Zoom, due to 3D aspects of molecular conformations. In all environments, learning can be compromised by a lack of attention. VR offers a solution to both issues—it can be more immersive and enhance attention due to the lack of distractions, and it allows engagement with 3D objects, such as proteins, that cannot be directly experienced in the real world. We previously performed a pilot study using VR to teach biochemistry using Oculus Quest 1 headsets and Spatial.io, meeting with students to discuss 3D aspects of biochemistry. In Summer 2021, we improved upon that experience by upgrading the tools: Quest 2 headsets allowed up to 40 people to meet together in the online platform Glue, whereas Quest 1 headsets only allowed 15 people to meet. Glue offers more realistic animated avatars that provide a better sense of presence—being together in a VR space—which may enhance student engagement. Finally, we explored the feasibility of generating animated FBX files to show molecular motion in VR, which may further aid in understanding biochemical mechanisms. To evaluate the impact of these tools, we performed a survey of students, administered a voluntary quiz, and compared performance on the normal course exams between students in the VR recitation and the Zoom recitation. The study yielded valuable practical insights about the effective use of these tools; furthermore, the data provided preliminary evidence that the VR experience enhances student learning and strong evidence that it is more appealing to students than the Zoom recitation format. In Fall 2021, we replicated the previous study with two key modifications: (1) the VR condition was now compared to an in-person, rather than Zoom-based, recitation section. This development reflects the evolving situation of the Covid-19 pandemic; with a return to in-person instruction now possible, it is crucial to evaluate the efficacy of VR experiences relative to the appropriate “default” setting. (2) Instead of a single VR recitation section led by Professor Stockwell, the course will now feature two or three separate VR recitation sections with each led by a teaching assistant who has been specially trained in VR instruction. We hypothesized that the advantages of VR relative to in-person recitations would be manifested in student performance on assessments and survey responses. Data analysis is ongoing.

Access the project proposal here.

*********************************************************************************************************************

Title: Measuring the Learning Outcomes of Self-Directed Rhetorical Grammar Modules for Multilingual Writers

Principal Investigator: Vanessa Guide Mesina, Lecturer, Undergraduate Writing Program and English & Comparative Literature

Award Date: Summer 2021

Status: In progress

Summary: This study aims to measure the efficacy and learning outcomes of a series of self-directed rhetorical grammar modules designed for multilingual students enrolled in International Student sections of the first-year writing course at Columbia, University Writing (UWIS). The six “mini-lesson” modules, originally created with the assistance of a Hybrid Learning Redesign grant from the Provost’s office, currently focus on first person pronoun use; reporting verbs; punctuation for complex sentences; passive vs. active voice; hedging words; and nominalizations. The modules are housed on each UWIS’s Canvas course site. Lessons center on specific English language “rules” and conventions that international students encounter in the American academic setting, and ask students to consider how context, convention, and intent inform writers’ choices. Students are required to complete three modules during semester, however, the online component of the course is semi-synchronous and self-paced. Students select which three modules to complete, allowing them to decide which topics are of the greatest interest and urgency to them. This project therefore accommodates students of varying levels of English language proficiency and frees up valuable class time for students to practices these skills with their peers and instructors. Each module consists of a preliminary intake questionnaire, and then a mix of screencasted “mini-lectures” on target forms, on-screen annotations of texts, readings, on-screen quizzes, and reflective writing exercises. Assessment of this project’s efficacy will be comprised of three components: 1) student “exit ticket” surveys; 2) rubric-based rating of anonymized participant essays; 3) rubric-based assessment of participant peer feedback letters. Participating students will be randomly assigned to complete modules in one of two forms: fully interactive modules (as described above) or passive modules (video components only); it is hypothesized that students completing fully interactive modules will exhibit the strongest outcomes.

Access the project proposal here.

*********************************************************************************************************************

Title: Nudges informed by past student data to increase current students’ engagement with course materials

Principal Investigator: Samantha Garbers, Associate Professor, Population and Family Health 

Award Date: Summer 2021

Status: Data collection complete; analysis in progress

Summary: This project will develop an evidence-based nudge strategy to make explicit to students the interconnection between course materials (lectures, readings, learning checks) and course assessments (homeworks, papers, group projects, tests). The objective of the intervention is to increase student engagement with course content, and in turn, increase and improve: 1) student experience with learning; 2) demonstration of achievement of learning outcomes on specific assignments, the course overall, and the entire Core; and perceived ability to demonstrate competencies; and 3) metacognition, specifically the awareness of links between course content and assessments. Using evidence of past course student behaviors to influence current course student behaviors is a novel approach to the application of actionable insights from learning analytics that is low-intensity, low-risk, and easily replicable in other courses. The impact of this intervention will be evaluated by comparing student engagement and learning outcomes between those who receive the nudge and those who do not.

Access the project proposal here.

*********************************************************************************************************************

Title: Comparing the effects of onsite and online simulation-based education
on the development of clinical reasoning in student physical therapists: a crossover study

Principal Investigator: Wing Fu, Assistant Professor, Rehabilitation and Regenerative Medicine

Award Date: Summer 2021

Status: Data collection complete; analysis in progress

Summary: The COVID-19 pandemic has forced many educators to replace onsite education with online education. As the end of the pandemic is expectantly approaching, it is imperative to compare the learning outcomes of onsite and online education in order to make informed decisions on selecting educational interventions appropriate for specific learning objectives in a post-confinement world. The overarching objective of this proposed crossover study is to compare the effects of onsite and online simulation-based education on the development of clinical reasoning (CR) in student physical therapists (SPTs). Clinical reasoning is a critical attribute that physical therapists must possess. Simulated patients (one form of simulated-based education) appear to have an effect on developing physical therapy clinical practice competencies including CR. They are typically used as onsite trainings. Despite the existence of virtual simulated patients, they are not in the form of recorded videos of simulated patient sessions, which the principal investigator used during the recent lockdown to replace the pre-lockdown onsite simulations in her course. To the principal investigator’s knowledge no studies have been done to compare the effects of the pre-lockdown educational intervention and the in-lockdown educational intervention on the development of SPTs’ CR. The effects will be assessed at levels I and II of the Kirkpatrick model, including SPTs’ clinical reasoning competency, self-perceived clinical reasoning gain and their satisfaction with the respective educational interventions. The proposed study can help promote the implementation of evidence-based education, which is much needed, as we are about to enter the transformed post-pandemic educational era.

Access the project proposal here.

*********************************************************************************************************************

Title: Investigating bias in standardized patient assessments of medical students 

Principal Investigator: Beth Barron, Associate Professor, Medicine

Award Date: Summer 2021

Co-Sponsor: Provost's Large Scale Teaching and Learning Grant

Status: In progress

Summary: This project will contribute to understanding how systemic racism in the form of implicit bias adversely impacts medical students of color. The Jaharis Simulation Center at Vagelos College of Physicians and Surgeons (VP&S) provides learner assessments utilizing standardized patient (SP) objective structured clinical examinations (OSCEs) throughout medical school and residency. We have noted that Underrepresented in Medicine (URiM) students are consistently rated 5-10% lower than white students on communication skills. This is consistent with literature demonstrating that students of color and women are given lower OSCE scores in the areas of communication and demonstrated empathy and raises concern for implicit bias in OSCE grading. We aim to contribute by probing how this disparity develops and to investigate methods that can ameliorate it. The project addresses two objectives: (1) investigate if and how SP biases may contribute to lower OSCE scores through review of prior data and implementation of a controlled comparison intervention; and (2) develop and evaluate interventions to reduce bias in SP grading. This project is of critical importance as assessment biases may unfairly damage URiM students’ long-term career goals, self-confidence, and trust in us. Furthermore, as medical educators, it is our responsibility to foster more accurate, unbiased assessments for all students. VP&S Dean of Education, Director of Equity and Justice in curricular affairs, simulation center leadership, and the Columbia Center for Educational Research and Evaluation (CERE) will support and assist with sustainability and academic dissemination of study results; furthermore, this project was awarded a 2021 Large-Scale Provost’s Teaching and Learning Grant and will receive support from the Columbia Center for Teaching and Learning (CTL) in instructional design, accessibility compliance, and more.

Access the project proposal here.

Title: Evaluation of virtual reality for learning biochemistry and enhancing student engagement

Principal Investigator: Brent Stockwell, Professor, Biological Sciences

Co-Sponsor: CUIT Emerging Technology Grant

Award Date: Summer 2020

Status: Completed (Fall 2020)

Summary: In Professor Stockwell's Biochemistry I course, students were recruited to participate in weekly 30-minute small group sessions, similar to office hours. Half of the students were randomly assigned to the virtual reality (VR) condition, and the other half were assigned to a Zoom-based control condition. The students in the VR condition used Oculus Quest headsets to convene on the Spatial.io virtual meeting platform, where they interacted with 3D models of molecules and biochemical pathways. The students in the Zoom condition interacted with similar materials in a 2D format. At the end of the semester, the two groups were compared with respect to student performance on assessments (e.g., exams) and responses to surveys that queried the students' engagement and attitudes about the suitability of the platform for learning biochemistry. It was predicted that students in the VR group would exhibit enhanced learning outcomes and more positive attitudes relative to the control group. The project provided an instructive foray into the use of VR for biochemistry education, with key insights centering around the effective implementation of 3D assets. Although no enhancements to student outcomes were detected, students provided many positive and constructive comments that will guide future iterations of this line of inquiry. 

Access the project proposal here.

Access the project final report here

*********************************************************************************************************************

Title: Imaging the brain activity of students studying brain imaging

Principal Investigators: 

  • Alfredo Spagna, Lecturer & Director of Undergraduate Studies in Neuroscience & Behavior, Department of Psychology
  • Xiaofu He, Assistant Professor of Clinical Neurobiology, Department of Psychiatry (CUIMC), Data Science Institute Affiliate

Co-Sponsor: Data Science Institute Collaboratory Grant

Award Date: Summer 2020

Status: In preparation (experiments planned for Fall 2021; see note below)

Summary: Portable electroencephalography (EEG) headsets will be used to record students’ neural activity during Fundamentals of Human Brain Imaging seminars. Students will process and analyze their own EEG data as part of the curriculum. The research design features two components: (1) an observational cognitive neuroscience study and (2) a comparison of student learning and attitude outcomes to a control course, Consciousness & Attention. For Component 1, classroom recordings will temporally align seminar content to EEG data. Brain wave frequencies associated with on-task as opposed to mind-wandering attentional states will be extracted from EEG data. The extent to which the same attentional states are indicated by EEG data, observed behavioral measures, and students' self reports will be determined. For each student, correctly and incorrectly answered questions on in-class assessments will be compared with respect to the attentional states (as inferred from EEG data and other measures) during the relevant lecture/discussions in class. It is predicted that correctly answered questions will be associated with on-task attentional states whereas incorrectly answered questions will be associated with mind-wandering states. For Component 2, the control course has similar learning objectives but involves no EEG. It is predicted that students in experimental course will exhibit similar or enhanced learning and attitude outcomes relative to the control course.

Note: due to the ongoing COVID-19 pandemic, the primary experiments originally planned for Spring 2021 were postponed until the resumption of fully in-person classes in Fall 2021. Data collection in Fall 2021 was successful; analysis is ongoing. 

Access the project proposal here.

Access the project final report [coming soon].