Comparing the effects of onsite and online simulation-based education on the development of clinical reasoning in student physical therapists: a crossover study

Principal Investigator: Wing Fu, Assistant Professor, Rehabilitation and Regenerative Medicine
Award Date: Summer 2021
Status: Data collection complete; analysis in progress
Summary
The COVID-19 pandemic has forced many educators to replace onsite education with online education. As the end of the pandemic is expectantly approaching, it is imperative to compare the learning outcomes of onsite and online education in order to make informed decisions on selecting educational interventions appropriate for specific learning objectives in a post-confinement world. The overarching objective of this proposed crossover study is to compare the effects of onsite and online simulation-based education on the development of clinical reasoning (CR) in student physical therapists (SPTs). Clinical reasoning is a critical attribute that physical therapists must possess. Simulated patients (one form of simulated-based education) appear to have an effect on developing physical therapy clinical practice competencies including CR. They are typically used as onsite trainings. Despite the existence of virtual simulated patients, they are not in the form of recorded videos of simulated patient sessions, which the principal investigator used during the recent lockdown to replace the pre-lockdown onsite simulations in her course. To the principal investigator’s knowledge no studies have been done to compare the effects of the pre-lockdown educational intervention and the in-lockdown educational intervention on the development of SPTs’ CR. The effects will be assessed at levels I and II of the Kirkpatrick model, including SPTs’ clinical reasoning competency, self-perceived clinical reasoning gain and their satisfaction with the respective educational interventions. The proposed study can help promote the implementation of evidence-based education, which is much needed, as we are about to enter the transformed post-pandemic educational era.
Access the project proposal here.
Measuring the Learning Outcomes of Self-Directed Rhetorical Grammar Modules for Multilingual Writers
Summary
This study aims to measure the efficacy and learning outcomes of a series of self-directed rhetorical grammar modules designed for multilingual students enrolled in International Student sections of the first-year writing course at Columbia, University Writing (UWIS). The six “mini-lesson” modules, originally created with the assistance of a Hybrid Learning Redesign grant from the Provost’s office, currently focus on first person pronoun use; reporting verbs; punctuation for complex sentences; passive vs. active voice; hedging words; and nominalizations. The modules are housed on each UWIS’s Canvas course site. Lessons center on specific English language “rules” and conventions that international students encounter in the American academic setting, and ask students to consider how context, convention, and intent inform writers’ choices. Students are required to complete three modules during semester, however, the online component of the course is semi-synchronous and self-paced. Students select which three modules to complete, allowing them to decide which topics are of the greatest interest and urgency to them. This project therefore accommodates students of varying levels of English language proficiency and frees up valuable class time for students to practices these skills with their peers and instructors. Each module consists of a preliminary intake questionnaire, and then a mix of screencasted “mini-lectures” on target forms, on-screen annotations of texts, readings, on-screen quizzes, and reflective writing exercises. Assessment of this project’s efficacy will be comprised of three components: 1) student “exit ticket” surveys; 2) rubric-based rating of anonymized participant essays; 3) rubric-based assessment of participant peer feedback letters. Participating students will be randomly assigned to complete modules in one of two forms: fully interactive modules (as described above) or passive modules (video components only); it is hypothesized that students completing fully interactive modules will exhibit the strongest outcomes.
Access the project proposal here.

Principal Investigator: Vanessa Guide Mesina, Lecturer, Undergraduate Writing Program and English & Comparative Literature
Award Date: Summer 2021
Status: In progress
Nudges informed by past student data to increase current students’ engagement with course materials

Principal Investigator: Samantha Garbers, Associate Professor, Population and Family Health
Award Date: Summer 2021
Status: Data collection complete; analysis in progress
Summary
This project will develop an evidence-based nudge strategy to make explicit to students the interconnection between course materials (lectures, readings, learning checks) and course assessments (homeworks, papers, group projects, tests). The objective of the intervention is to increase student engagement with course content, and in turn, increase and improve: 1) student experience with learning; 2) demonstration of achievement of learning outcomes on specific assignments, the course overall, and the entire Core; and perceived ability to demonstrate competencies; and 3) metacognition, specifically the awareness of links between course content and assessments. Using evidence of past course student behaviors to influence current course student behaviors is a novel approach to the application of actionable insights from learning analytics that is low-intensity, low-risk, and easily replicable in other courses. The impact of this intervention will be evaluated by comparing student engagement and learning outcomes between those who receive the nudge and those who do not.
Access the project proposal here.
Evaluating Oculus Quest 2 and Glue as a Virtual Reality (VR) Platform for Learning Biochemistry
Summary
Learning biochemistry is challenging for students using chalkboards, Powerpoint and Zoom, due to 3D aspects of molecular conformations. In all environments, learning can be compromised by a lack of attention. VR offers a solution to both issues—it can be more immersive and enhance attention due to the lack of distractions, and it allows engagement with 3D objects, such as proteins, that cannot be directly experienced in the real world. We previously performed a pilot study using VR to teach biochemistry using Oculus Quest 1 headsets and Spatial.io, meeting with students to discuss 3D aspects of biochemistry. In Summer 2021, we improved upon that experience by upgrading the tools: Quest 2 headsets allowed up to 40 people to meet together in the online platform Glue, whereas Quest 1 headsets only allowed 15 people to meet. Glue offers more realistic animated avatars that provide a better sense of presence—being together in a VR space—which may enhance student engagement. Finally, we explored the feasibility of generating animated FBX files to show molecular motion in VR, which may further aid in understanding biochemical mechanisms. To evaluate the impact of these tools, we performed a survey of students, administered a voluntary quiz, and compared performance on the normal course exams between students in the VR recitation and the Zoom recitation. The study yielded valuable practical insights about the effective use of these tools; furthermore, the data provided preliminary evidence that the VR experience enhances student learning and strong evidence that it is more appealing to students than the Zoom recitation format. In Fall 2021, we replicated the previous study with two key modifications: (1) the VR condition was now compared to an in-person, rather than Zoom-based, recitation section. This development reflects the evolving situation of the Covid-19 pandemic; with a return to in-person instruction now possible, it is crucial to evaluate the efficacy of VR experiences relative to the appropriate “default” setting. (2) Instead of a single VR recitation section led by Professor Stockwell, the course will now feature two or three separate VR recitation sections with each led by a teaching assistant who has been specially trained in VR instruction. We hypothesized that the advantages of VR relative to in-person recitations would be manifested in student performance on assessments and survey responses. Data analysis is ongoing.
Access the project proposal here.

Principal Investigator: Brent Stockwell, Professor, Biological Sciences
Co-Sponsor: CUIT Emerging Technology Grant
Award Date: Summer 2021
Status: Data collection complete; analysis in progress
Investigating bias in standardized patient assessments of medical students

Principal Investigator: Beth Barron, Associate Professor, Medicine
Co-Sponsor: Provost's Large Scale Teaching and Learning Grant
Award Date: Summer 2021
Status: In progress
Summary
This project will contribute to understanding how systemic racism in the form of implicit bias adversely impacts medical students of color. The Jaharis Simulation Center at Vagelos College of Physicians and Surgeons (VP&S) provides learner assessments utilizing standardized patient (SP) objective structured clinical examinations (OSCEs) throughout medical school and residency. We have noted that Underrepresented in Medicine (URiM) students are consistently rated 5-10% lower than white students on communication skills. This is consistent with literature demonstrating that students of color and women are given lower OSCE scores in the areas of communication and demonstrated empathy and raises concern for implicit bias in OSCE grading. We aim to contribute by probing how this disparity develops and to investigate methods that can ameliorate it. The project addresses two objectives: (1) investigate if and how SP biases may contribute to lower OSCE scores through review of prior data and implementation of a controlled comparison intervention; and (2) develop and evaluate interventions to reduce bias in SP grading. This project is of critical importance as assessment biases may unfairly damage URiM students’ long-term career goals, self-confidence, and trust in us. Furthermore, as medical educators, it is our responsibility to foster more accurate, unbiased assessments for all students. VP&S Dean of Education, Director of Equity and Justice in curricular affairs, simulation center leadership, and the Columbia Center for Educational Research and Evaluation (CERE) will support and assist with sustainability and academic dissemination of study results; furthermore, this project was awarded a 2021 Large-Scale Provost’s Teaching and Learning Grant and will receive support from the Columbia Center for Teaching and Learning (CTL) in instructional design, accessibility compliance, and more.