The projects highlighted below received Provost's SOLER Seed Grants (PSSGs).
Title: Evaluating augmented reality (AR) using Microsoft Mesh and Hololens 2 for teaching deep expertise in introductory biochemistry
Principal Investigator: Brent Stockwell, Professor, Biological Sciences
Co-Sponsor: CUIT Emerging Technology Teaching and Learning Grant
Award Date: Summer 2022
Status: In development
Summary: Learning biochemistry is challenging for students using chalkboards, PowerPoint or Zoom, due to 3D aspects of molecular conformations. AR/VR offer a solution—they can be more immersive and enhance attention due to the lack of distractions, and allow engagement with 3D objects, such as proteins, that cannot be directly experienced in the real world. We previously performed studies with support from the Provost’s Office and from SOLER using VR to teach biochemistry using Oculus Quest 1 headsets and Spatial.io, meeting with students to discuss 3D aspects of biochemistry, and Quest 2 headsets with the online platform Glue. We found that using VR was impactful in terms of engagement and learning improvements. We found that many students are highly engaged during the in- person experience as well as the VR experience, but that rendering detailed proteins and their motions is challenging with the limited processing power of the Quest 2 headsets. Most critically, we found that students need to be able to translate the insights from interacting with 3D models of proteins back to representing proteins in two dimensions in order to build on their knowledge with subsequent reading of papers, textbooks, and other 2D representations of 3D molecules. This translation from 3D back to 2D is difficult to do in VR because it is difficult to simultaneously view 3D and 2D representations of molecules with high fidelity. We hypothesize that Augmented Reality (AR) will provide the missing link in translating deep insights into the 3D structure-function relationship in protein molecules into 2D representations of these molecules that can be used outside of the AR/VR format.
*********************************************************************************************************************
Title: Neuro-DBER: A flexible neuroscience- and technology-driven framework for Discipline-Based Education Research
Principal Investigator: Alfredo Spagna, Lecturer, Psychology
Co-Sponsor: CUIT Emerging Technology Research Grant
Award Date: Summer 2022
Status: In progress
Summary: What predicts student success? The classroom is a dynamic setting where students and instructors of diverse backgrounds exchange ideas in the pursuit of learning. Many variables complicate this collaborative paradigm: composition changes due to student absence; content difficulty increases as the course deepens; student fatigue accumulates as the semester progresses. While the relative contribution of complicating variables has been studied in isolation, a comprehensive understanding of their synergistic effects remains elusive. What can be done to help instructors and students integrate solutions to these well-known challenges into a cohesive framework? This project aims to develop a flexible neuroscience and technology-driven framework for Discipline-Based Education Research (Neuro-DBER) to revolutionize teaching and learning in the classroom. Our system merges cognitive neuroscience, education, and innovative technology to enable investigations of the relationships between subjective experience, body language, , real-time learning, and neural measures of in-class activities. Neuro-DBER is a flexible technological framework suitable to any field of study to address critical questions about classroom experiences, such as the gap between a student's subjective feelings versus actual learning measured. Featuring integration of advanced research technology, our framework facilitates thoughtful design of in-class activities that will thereby transform the way we teach and learn, yielding an inclusive environment conducive to effective teaching and addressing students’ distinctive needs.
*********************************************************************************************************************
Title: Increasing student engagement with course content and learning supports through mixed-methods mapping of motivation and learning behaviors
Principal Investigator: Samantha Garbers, Associate Professor, Population and Family Health
Award Date: Summer 2022
Status: In development
Summary: This project aims to conceptualize, quantify, and explore qualitatively student engagement and motivations in an interdisciplinary biostatistics and epidemiology Core course. At the Mailman School of Public Health, all incoming students spend their first semester participating in an interdisciplinary Core curriculum. The most challenging course in the curriculum, Research Methods & Application - Quantitative Data Analysis, is taught in a hybrid format with synchronous and asynchronous content. Student supports have been deployed in recent years, but some students continue to face challenges in successful completion of this course. As revealed by our prior SOLER-funded project – a randomized trial of behavioral nudges – student engagement with Core course content is low and varies substantially by week; most troublingly, engagement with learning supports for task management declined steadily over the course of the semester. Academic engagement is a critical construct known to improve course persistence, predict student success (including grades, degree completion), educational attainment, and occupational attainment. Interventions are needed to increase student engagement across the board, for all students, but particularly for students whose baseline assessments indicate the need for academic support. These interventions should aim to increase student engagement with both course content and task management learning supports. Intervention strategies should be tailored to students’ academic needs, baseline competencies, and informed by evidence on how students of varying baseline competency levels engage with these course elements. More nuanced data on how students of different competency levels engage with course elements – including qualitative data on student motivations for engagement – are needed to guide the development and deployment of interventions that can be tested using rigorous designs. In Phase 1, we will replicate student engagement analyses using learning management system learning analytics data for the 2021 course, stratifying by baseline competency level. In Phase 2, we will use rigorous qualitative methods as one-on-one interviews to explore student learning behaviors and motivations that underlie engagement. Our prior study did not demonstrate efficacy of behavioral nudges most likely because the nudges, though evidence-based, were not informed by data on what motivates students. In the final phase of the project, we will quantitatively explore student perspectives of motivating behavioral nudges using forced-choice paired comparison methods that assess preferences across a range of dimensions, including messaging, placement/delivery, timing, and design. This proposed work is an essential step before deploying a larger-scale trial testing nudge interventions with more complex features, such as tailored nudges based on baseline characteristics.
*********************************************************************************************************************
Title: An exploration of the anti-oppressive practice of 'un-grading' and its effects on student learning
Principal Investigator: Amy Werman, Lecturer, Social Work
Award Date: Summer 2022
Status: In development
Summary: A significant challenge facing instructors at Columbia School of Social Work is the fair, equitable, and accurate assessment of students. Instructors use letter grades to evaluate student outcomes, but there is little agreement as to what grades indicate. There is, however, a shared implicit assumption that grades measure learning. The current study interrogates that assumption by exploring the implementation of an "ungrading" pedagogy in a Program Evaluation course. Ungrading is an innovative practice in which students receive feedback on assignments and revise their work based on that feedback; however, their work is not graded. Rather, the student is asked to reflect on what they learned and on the process of learning (their motivation, effort, curiosity), and then give themselves a grade on the assignment. Anecdotal evidence suggests that ungrading fosters learning because students focus on the process of learning instead of the outcome of a grade. This study aims to examine objective and subjective student experiences with ungrading relative to traditional alternatives.
*********************************************************************************************************************
Title: How teacher-student connection improves abstract thinking
Principal Investigator: Tugce Bilgin, Lecturer, Ecology, Evolution, and Environmental Biology
Award Date: Summer 2022
Status: In development
Summary: The ‘social breathing’ and ‘interactive brain’ hypotheses as well as models of learning dynamics, joint action, and teacher-student interactions posit that low-level perceiving and signaling systems in nonverbal behavior scaffold and support interpersonal coordination and performance in joint tasks, such as teaching and learning. This study attempts to quantify this relationship. Teacher-student dyads recruited from various sections of the Columbia College Core course 'Frontiers of Science' will use a 1-on-1 online portal to collaboratively complete simple and complex tasks. We will examine the relationships between nonverbal behavior, task complexity and performance, and dyad familiarity (i.e., whether or not the student and instructor were drawn from the same section.)
*********************************************************************************************************************
Title: Interactive simulations to support inquiry-based statistics instruction in Frontiers of Science
Principal Investigator: Nicholas Bock, Lecturer, Earth & Environmental Sciences
Award Date: Summer 2022
Status: In development
Summary: Frontiers of Science is a required course for all first-year Columbia College students. The goal of the course is to provide an introduction to current research across four different scientific disciplines while helping students develop a framework for approaching, analyzing, and interpreting quantitative information (hereafter referred to as the scientific "habits of mind"). Students come to Frontiers of Science with a wide range of prior science coursework and attitudes/interests/experiences in science. Therefore, a central challenge in planning individual seminar sessions is identifying learning activities that address course objectives while serving the learning needs of all students in the classroom. This challenge can be particularly acute when introducing statistical concepts. Frontiers of Science students are introduced to basic statistical concepts (e.g., measures of central tendency and dispersion) at the start of the semester. During the first three weeks of the semester, students are expected to develop an understanding of how these concepts can be used to formulate statistically supported inferences (e.g., the use of standard error to evaluate confidence intervals and to support hypothesis tests). However, based on an introductory survey completed by students in seminar sections during the spring 2022 semester, only 50 percent of students reported having completed previous coursework in statistics. Therefore, there is a clear need within Frontiers of Science for instructional tools that are appropriate and challenging for students both with and without prior experience in statistics. We will develop a series of interactive simulations to support inquiry-based learning of basic statistical concepts in Frontiers of Science seminar sessions. We additionally propose to work with SOLER Initiative faculty to investigate the impact of the proposed instructional materials on learning outcomes in Frontiers of Science seminar sections. Because all incoming first-year students enroll in Frontiers of Science, student interactions with these simulations will impact learning outcomes across Columbia College. Moreover, because the statistical concepts to be introduced are foundational to data analysis across many different fields of study, these simulations will be of use to faculty across the university; for example, the simulations may be adapted to introduce Earth Science students to statistical analysis of environmental data. The simulations will additionally expand upon interactive simulations previously developed by faculty in the School of International and Public Affairs (SIPA) as part of the Stats Interactives website.
Access the project proposal here.
Title: Evaluating Oculus Quest 2 and Glue as a Virtual Reality (VR) Platform for Learning Biochemistry
Principal Investigator: Brent Stockwell, Professor, Biological Sciences
Co-Sponsor: CUIT Emerging Technology Grant
Award Date: Summer 2021
Status: Data collection complete; analysis in progress
Summary: Learning biochemistry is challenging for students using chalkboards, Powerpoint and Zoom, due to 3D aspects of molecular conformations. In all environments, learning can be compromised by a lack of attention. VR offers a solution to both issues—it can be more immersive and enhance attention due to the lack of distractions, and it allows engagement with 3D objects, such as proteins, that cannot be directly experienced in the real world. We previously performed a pilot study using VR to teach biochemistry using Oculus Quest 1 headsets and Spatial.io, meeting with students to discuss 3D aspects of biochemistry. In Summer 2021, we improved upon that experience by upgrading the tools: Quest 2 headsets allowed up to 40 people to meet together in the online platform Glue, whereas Quest 1 headsets only allowed 15 people to meet. Glue offers more realistic animated avatars that provide a better sense of presence—being together in a VR space—which may enhance student engagement. Finally, we explored the feasibility of generating animated FBX files to show molecular motion in VR, which may further aid in understanding biochemical mechanisms. To evaluate the impact of these tools, we performed a survey of students, administered a voluntary quiz, and compared performance on the normal course exams between students in the VR recitation and the Zoom recitation. The study yielded valuable practical insights about the effective use of these tools; furthermore, the data provided preliminary evidence that the VR experience enhances student learning and strong evidence that it is more appealing to students than the Zoom recitation format. In Fall 2021, we replicated the previous study with two key modifications: (1) the VR condition was now compared to an in-person, rather than Zoom-based, recitation section. This development reflects the evolving situation of the Covid-19 pandemic; with a return to in-person instruction now possible, it is crucial to evaluate the efficacy of VR experiences relative to the appropriate “default” setting. (2) Instead of a single VR recitation section led by Professor Stockwell, the course will now feature two or three separate VR recitation sections with each led by a teaching assistant who has been specially trained in VR instruction. We hypothesized that the advantages of VR relative to in-person recitations would be manifested in student performance on assessments and survey responses. Data analysis is ongoing.
Access the project proposal here.
*********************************************************************************************************************
Title: Measuring the Learning Outcomes of Self-Directed Rhetorical Grammar Modules for Multilingual Writers
Principal Investigator: Vanessa Guide Mesina, Lecturer, Undergraduate Writing Program and English & Comparative Literature
Award Date: Summer 2021
Status: In progress
Summary: This study aims to measure the efficacy and learning outcomes of a series of self-directed rhetorical grammar modules designed for multilingual students enrolled in International Student sections of the first-year writing course at Columbia, University Writing (UWIS). The six “mini-lesson” modules, originally created with the assistance of a Hybrid Learning Redesign grant from the Provost’s office, currently focus on first person pronoun use; reporting verbs; punctuation for complex sentences; passive vs. active voice; hedging words; and nominalizations. The modules are housed on each UWIS’s Canvas course site. Lessons center on specific English language “rules” and conventions that international students encounter in the American academic setting, and ask students to consider how context, convention, and intent inform writers’ choices. Students are required to complete three modules during semester, however, the online component of the course is semi-synchronous and self-paced. Students select which three modules to complete, allowing them to decide which topics are of the greatest interest and urgency to them. This project therefore accommodates students of varying levels of English language proficiency and frees up valuable class time for students to practices these skills with their peers and instructors. Each module consists of a preliminary intake questionnaire, and then a mix of screencasted “mini-lectures” on target forms, on-screen annotations of texts, readings, on-screen quizzes, and reflective writing exercises. Assessment of this project’s efficacy will be comprised of three components: 1) student “exit ticket” surveys; 2) rubric-based rating of anonymized participant essays; 3) rubric-based assessment of participant peer feedback letters. Participating students will be randomly assigned to complete modules in one of two forms: fully interactive modules (as described above) or passive modules (video components only); it is hypothesized that students completing fully interactive modules will exhibit the strongest outcomes.
Access the project proposal here.
*********************************************************************************************************************
Title: Nudges informed by past student data to increase current students’ engagement with course materials
Principal Investigator: Samantha Garbers, Associate Professor, Population and Family Health
Award Date: Summer 2021
Status: Data collection complete; analysis in progress
Summary: This project will develop an evidence-based nudge strategy to make explicit to students the interconnection between course materials (lectures, readings, learning checks) and course assessments (homeworks, papers, group projects, tests). The objective of the intervention is to increase student engagement with course content, and in turn, increase and improve: 1) student experience with learning; 2) demonstration of achievement of learning outcomes on specific assignments, the course overall, and the entire Core; and perceived ability to demonstrate competencies; and 3) metacognition, specifically the awareness of links between course content and assessments. Using evidence of past course student behaviors to influence current course student behaviors is a novel approach to the application of actionable insights from learning analytics that is low-intensity, low-risk, and easily replicable in other courses. The impact of this intervention will be evaluated by comparing student engagement and learning outcomes between those who receive the nudge and those who do not.
Access the project proposal here.
*********************************************************************************************************************
Title: Comparing the effects of onsite and online simulation-based education
on the development of clinical reasoning in student physical therapists: a crossover study
Principal Investigator: Wing Fu, Assistant Professor, Rehabilitation and Regenerative Medicine
Award Date: Summer 2021
Status: Data collection complete; analysis in progress
Summary: The COVID-19 pandemic has forced many educators to replace onsite education with online education. As the end of the pandemic is expectantly approaching, it is imperative to compare the learning outcomes of onsite and online education in order to make informed decisions on selecting educational interventions appropriate for specific learning objectives in a post-confinement world. The overarching objective of this proposed crossover study is to compare the effects of onsite and online simulation-based education on the development of clinical reasoning (CR) in student physical therapists (SPTs). Clinical reasoning is a critical attribute that physical therapists must possess. Simulated patients (one form of simulated-based education) appear to have an effect on developing physical therapy clinical practice competencies including CR. They are typically used as onsite trainings. Despite the existence of virtual simulated patients, they are not in the form of recorded videos of simulated patient sessions, which the principal investigator used during the recent lockdown to replace the pre-lockdown onsite simulations in her course. To the principal investigator’s knowledge no studies have been done to compare the effects of the pre-lockdown educational intervention and the in-lockdown educational intervention on the development of SPTs’ CR. The effects will be assessed at levels I and II of the Kirkpatrick model, including SPTs’ clinical reasoning competency, self-perceived clinical reasoning gain and their satisfaction with the respective educational interventions. The proposed study can help promote the implementation of evidence-based education, which is much needed, as we are about to enter the transformed post-pandemic educational era.
Access the project proposal here.
*********************************************************************************************************************
Title: Investigating bias in standardized patient assessments of medical students
Principal Investigator: Beth Barron, Associate Professor, Medicine
Award Date: Summer 2021
Co-Sponsor: Provost's Large Scale Teaching and Learning Grant
Status: In progress
Summary: This project will contribute to understanding how systemic racism in the form of implicit bias adversely impacts medical students of color. The Jaharis Simulation Center at Vagelos College of Physicians and Surgeons (VP&S) provides learner assessments utilizing standardized patient (SP) objective structured clinical examinations (OSCEs) throughout medical school and residency. We have noted that Underrepresented in Medicine (URiM) students are consistently rated 5-10% lower than white students on communication skills. This is consistent with literature demonstrating that students of color and women are given lower OSCE scores in the areas of communication and demonstrated empathy and raises concern for implicit bias in OSCE grading. We aim to contribute by probing how this disparity develops and to investigate methods that can ameliorate it. The project addresses two objectives: (1) investigate if and how SP biases may contribute to lower OSCE scores through review of prior data and implementation of a controlled comparison intervention; and (2) develop and evaluate interventions to reduce bias in SP grading. This project is of critical importance as assessment biases may unfairly damage URiM students’ long-term career goals, self-confidence, and trust in us. Furthermore, as medical educators, it is our responsibility to foster more accurate, unbiased assessments for all students. VP&S Dean of Education, Director of Equity and Justice in curricular affairs, simulation center leadership, and the Columbia Center for Educational Research and Evaluation (CERE) will support and assist with sustainability and academic dissemination of study results; furthermore, this project was awarded a 2021 Large-Scale Provost’s Teaching and Learning Grant and will receive support from the Columbia Center for Teaching and Learning (CTL) in instructional design, accessibility compliance, and more.
Access the project proposal here.
Title: Evaluation of virtual reality for learning biochemistry and enhancing student engagement
Principal Investigator: Brent Stockwell, Professor, Biological Sciences
Co-Sponsor: CUIT Emerging Technology Grant
Award Date: Summer 2020
Status: Completed (Fall 2020)
Summary: In Professor Stockwell's Biochemistry I course, students were recruited to participate in weekly 30-minute small group sessions, similar to office hours. Half of the students were randomly assigned to the virtual reality (VR) condition, and the other half were assigned to a Zoom-based control condition. The students in the VR condition used Oculus Quest headsets to convene on the Spatial.io virtual meeting platform, where they interacted with 3D models of molecules and biochemical pathways. The students in the Zoom condition interacted with similar materials in a 2D format. At the end of the semester, the two groups were compared with respect to student performance on assessments (e.g., exams) and responses to surveys that queried the students' engagement and attitudes about the suitability of the platform for learning biochemistry. It was predicted that students in the VR group would exhibit enhanced learning outcomes and more positive attitudes relative to the control group. The project provided an instructive foray into the use of VR for biochemistry education, with key insights centering around the effective implementation of 3D assets. Although no enhancements to student outcomes were detected, students provided many positive and constructive comments that will guide future iterations of this line of inquiry.
Access the project proposal here.
Access the project final report here.
*********************************************************************************************************************
Title: Imaging the brain activity of students studying brain imaging
Principal Investigators:
- Alfredo Spagna, Lecturer & Director of Undergraduate Studies in Neuroscience & Behavior, Department of Psychology
- Xiaofu He, Assistant Professor of Clinical Neurobiology, Department of Psychiatry (CUIMC), Data Science Institute Affiliate
Co-Sponsor: Data Science Institute Collaboratory Grant
Award Date: Summer 2020
Status: In preparation (experiments planned for Fall 2021; see note below)
Summary: Portable electroencephalography (EEG) headsets will be used to record students’ neural activity during Fundamentals of Human Brain Imaging seminars. Students will process and analyze their own EEG data as part of the curriculum. The research design features two components: (1) an observational cognitive neuroscience study and (2) a comparison of student learning and attitude outcomes to a control course, Consciousness & Attention. For Component 1, classroom recordings will temporally align seminar content to EEG data. Brain wave frequencies associated with on-task as opposed to mind-wandering attentional states will be extracted from EEG data. The extent to which the same attentional states are indicated by EEG data, observed behavioral measures, and students' self reports will be determined. For each student, correctly and incorrectly answered questions on in-class assessments will be compared with respect to the attentional states (as inferred from EEG data and other measures) during the relevant lecture/discussions in class. It is predicted that correctly answered questions will be associated with on-task attentional states whereas incorrectly answered questions will be associated with mind-wandering states. For Component 2, the control course has similar learning objectives but involves no EEG. It is predicted that students in experimental course will exhibit similar or enhanced learning and attitude outcomes relative to the control course.
Note: due to the ongoing COVID-19 pandemic, the primary experiments originally planned for Spring 2021 were postponed until the resumption of fully in-person classes in Fall 2021. Data collection in Fall 2021 was successful; analysis is ongoing.
Access the project proposal here.
Access the project final report [coming soon].