During the COVID-19 pandemic, many schools and universities had to pivot to online learning. However, there is little research on the causal effects of online learning compared to in-person instruction because most colleges and school districts choose to go either all in-person or online, or a hybrid for each student.
A recent IZA discussion paper by Michael S. Kofoed, Lucas Gebhart, Dallas Gilmore, Ryan Moschitto exploits a unique experimental setting at the United States Military Academy at West Point to examine the effects of online learning on student performance. West Point randomizes students into class sections, class hours, and instructors. As a response to the pandemic, the authors were allowed to use this variation to randomly assign students into either an online or in-person class section in a Principles of Economics class.
Large learning gaps for academically at-risk students
Courses at West Point have uniform graded events and lesson plans across instructors, and each instructor agreed to teach half of their teaching load in both modalities. The authors randomized 550 students across twelve instructors in 36 different classrooms. The analysis shows that students assigned to an online classroom received a final grade that was 0.20 standard deviations less than those assigned to an in-person class. This corresponds to a reduction by a half +/- grade.
Demographic data reveal that students who previously had below-average academic ability have the largest learning gaps, including those who were formerly enlisted in the United States Army or attended the preparatory school (similar to a community college). The result is also consistent across NCAA athletes and non-athletes.
Finally, the authors use a post-course survey to explore mechanisms. The responses indicate that online learning greatly reduced concentration, but also whether a student felt connected to their instructor and whether their instructor cared about them. Students also felt less connected to their peers. In sum, the study shows that online coursework reduced academic achievement, increased inequality, and deteriorated the classroom experience during the pandemic.
Read more about the paper in a Twitter thread by @mikekofoed.