(Tap, tap) Is this thing on?
After lengthy delays after lengthy delays, with causes ranging from passwords to pandemics, we are almost finished with data collection for the ManyClasses 1 experiment. This would not have been possible without support from the Unizin Consortium, whose Unizin Data Platform (UDP) made it possible to collect data directly from Canvas data feeds from participating class sites for consenting participants.
This also would not have been possible without the contributions from an amazing set of collaborating instructors. All 38 classes that began the study completed the study, and all instructors have provided thorough reports of student performance on outcome assessments (“posttest” scores after students receive different feedback treatments). We cannot express our appreciation for the efforts of these instructors, and we are so looking forward to joining them in authorship of our final report.
We have now publicly posted de-identified data about participating classes and their institutions to OSF:
-
manyclasses_class_moderators.csv - https://osf.io/tjkyr/
-
manyclasses_institution_moderators.csv - https://osf.io/e8d5f/
…Which contain all preregistered moderator values at the class- and institution-level. De-identified student moderators and final outcome scores will also be posted shortly, as we are continuing to audit and validate these data.
Our preprocessing pipeline is described here: https://osf.io/p2csf/
If you’ve read this far, you’re probably wondering what our findings are. We eagerly look forward to presenting the results as soon as possible, once we’re confident in the source data and in our analysis models. In the meantime, based on our very preliminary runs through not-yet-totally-validated data, I’ll just say that the ManyClasses approach seems to be revealing systematic patterns of dependencies across classroom implementations. These dependencies would not be evident in a typical experimental design, and they reveal fascinating properties of the generalizability of instructional strategies across contexts.