ABSTRACT
The printing press long ago and the computer today have made widespread access to information possible. Learning theorists have suggested, however, that mere information is a poor way to learn. Instead, more effective learning comes through doing. While the most popularized element of today's MOOCs are the video lectures, many MOOCs also include interactive activities that can afford learning by doing. This paper explores the learning benefits of the use of informational assets (e.g., videos and text) in MOOCs, versus the learning by doing opportunities that interactive activities provide. We find that students doing more activities learn more than students watching more videos or reading more pages. We estimate the learning benefit from extra doing (1 SD increase) to be more than six times that of extra watching or reading. Our data, from a psychology MOOC, is correlational in character, however we employ causal inference mechanisms to lend support for the claim that the associations we find are causal.
- Aleven, V., Beal, C. R., & Graesser, A. C. (2013). Introduction to the special issue on advanced learning technologies. Journal of Educational Psychology, 105(4), 929--931.Google ScholarCross Ref
- Anderson, J. R., Corbett, A. T., Koedinger, K. R., & Pelletier, R. (1995) . Cognitive tutors:Lessons learned. The Journal of the Learning Sciences, 4, 167--207.Google ScholarCross Ref
- Anzai, Y., & Simon, H. A. (1979). The theory of learning by doing. Psychological Review, 86, 124--140.Google ScholarCross Ref
- Barab, S.A. & Squire, K.D. (2004). Design-based research: Putting a stake in the ground. Journal of the Learning Science, 13 (1) 1--14.Google ScholarCross Ref
- Bichsel, J. (2013). The State of E-Learning in Higher Education: An Eye toward Growth and Increased Access (Research Report), Louisville, CO: EDUCAUSE Center for Analysis and Research. Available online: http://www.educause.edu/ecar.Google Scholar
- Bowen, W. G., Chingos, M. M., Lack, K. A., & Nygren, T. I. (2013). Interactive learning online at public universities: Evidence from a six-campus randomized trial. Journal of Policy Analysis and Management, 33(1), 94--111.Google ScholarCross Ref
- Champaign, J., Fredericks, C., Colvin, K., Seaton, D., Liu, A. & Pritchard, D. Correlating skill and improvement in 2 MOOCs with a student's time on task. In Proc. Learning@Scale 2014, Retrieved from: http://dx.doi.org/10.1145/2556325.2566250. Google ScholarDigital Library
- Clark, R.E., Feldon, D., van Merriënboer, J., Yates, K., & Early, S. (2007). Cognitive task analysis. In J.M. Spector, M.D. Merrill, J.J.G. van Merriënboer, & M.P. Driscoll (Eds.), Handbook of research on educational communications and technology (3rd ed., pp. 577--593). Mahwah, NJ: Lawrence Erlbaum Associates.Google Scholar
- Collins, E.D. (2013) "SJSU Plus augmented online learning environment: Pilot project report." The Research and Planning Group for California Community Colleges. Available online:http://www.sjsu.edu/chemistry/People/Faculty/Collins_Research_Page/AOLE%20Report%20Final%20Version_Jan%201_2014.pdfGoogle Scholar
- Collins, A., Brown, J. S., & Newman, S. E. (1989). Cognitive apprenticeship: Teaching the crafts of reading, writing, and mathematics. In L. B. Resnick. Knowing, Learning, and Instruction: Essays in Honor of Robert Glaser (pp. 453--494). Hillsdale, NJ: Erlbaum.Google Scholar
- Dewey, J. (1916), (2007 edition). Democracy and Education, Teddington: Echo Library.Google Scholar
- Hill, P. (2013). Emerging Student Patterns in MOOCs: A (Revised) Graphical View. e-literate. Available online: http://mfeldstein.com/emerging-student-patterns-in-moocs-a-revised-graphical-view/Google Scholar
- Kaufman, J.; Ryan, R.; Thille, C. and Bier, N. (2013) Open Learning Initiative Courses in Community Colleges: Evidence on Use and Effectiveness. Mellon University, Pittsburgh, PA. Available online: http://www.hewlett.org/sites/default/files/CCOLI_Report_Final_1.pdfGoogle Scholar
- Koedinger, K.R., & Aleven V. (2007). Exploring the assistance dilemma in experiments with Cognitive Tutors. Educational Psychology Review, 19(3), 239--264.Google ScholarCross Ref
- Koedinger, K.R., Baker, R.S.J.d., Cunningham, K., Skogsholm, A., Leber, B., Stamper, J. (2010). A Data Repository for the EDM community: The PSLC DataShop. In Romero, C., Ventura, S., Pechenizkiy, M., Baker, R.S.J.d. (Eds.) Handbook of Educational Data Mining. Boca Raton, FL: CRC Press.Google ScholarCross Ref
- Koedinger, K.R., Booth, J. L., & Klahr, D. (2013). Instructional Complexity and the Science to Constrain It. Science, 342(6161), 935--937.Google ScholarCross Ref
- Koedinger, K. R., Corbett, A. C., & Perfetti, C. (2012). The Knowledge-Learning-Instruction (KLI) framework: Bridging the science-practice chasm to enhance robust student learning. Cognitive Science, 36 (5), 757--798.Google ScholarCross Ref
- Koedinger, K. R., McLaughlin, E. A., & Stamper, J. C. (2014). Ubiquity symposium: MOOCs and technology to advance learning and learning research: Data-driven learner modeling to understand and improve online learning. Ubiquity, Volume 2014, Number May (2014), Pages 1--13. ACM New York, NY, USA. DOI:10.1145/2591682 Google ScholarDigital Library
- Koedinger, K. R., & Sueker, E. L. F. (2014). Monitored design of an effective learning environment for algebraic problem solving. Technical report Carnegie Mellon University-HCII-14-102.Google Scholar
- Lovett, M., Meyer, O., Thille, C. (2008). The Open Learning Initiative: Measuring the effectiveness of the OLI learning course in accelerating student learning. Journal of Interactive Media in Education. http://jime.open.ac.uk/2008/14/.Google Scholar
- Meyer, O., & Lovett, M. C. (2002). Implementing a computerized tutor in a statistical reasoning Course: Getting the big picture. In B. Phillips (Ed.) Proc. of the Sixth International Conference on Teaching Statistics.Google Scholar
- Morote, E. & Pritchard, D. E. (2002). What Course Elements Correlate with Improvement on Tests in Introductory Newtonian Mechanics? National Association for Research in Science Teaching -- NARST- 2002 Conference.Google Scholar
- Ng, A. and Widom J. (2014). Origins of the Modern MOOC. http://www.cs.stanford.edu/people/ang/papers/mooc14-OriginsOfModernMOOC.pdfGoogle Scholar
- Pane, J.F., Griffin, B., McCaffrey, D.F. & Karam, R. (2014). Effectiveness of Cognitive Tutor Algebra I at Scale. Educational Evaluation and Policy Analysis, 36 (2), 127--144.Google ScholarCross Ref
- Pappano, L. (2012, November 2). The year of the MOOC. The New York Times.Google Scholar
- Pashler, H., Bain, P., Bottge, B., Graesser, A., Koedinger, K., McDaniel, M., & Metcalfe, J. (2007). Organizing Instruction and Study to Improve Student Learning (NCER 2007-2004). Washington, DC: National Center for Education Research, Institute of Education Sciences, U.S. Department of Education.Google Scholar
- Ritter S., Anderson, J. R., Koedinger, K. R., & Corbett, A. (2007). Cognitive tutor: Applied research in mathematics education. Psychonomic Bulletin & Review, 14 (2):249--255.Google ScholarCross Ref
- Schank, R. C., Berman, T. R. & Macperson, K. A. (1999). Learning by doing. In C. M. Reigeluth (Ed.), Instructional Design Theories and Models: A New Paradigm of Instructional Theory (Vol. II) (pp.161--181). Mahwah, NJ: Lawrence Erlbaum Associates.Google Scholar
- Singer, S.R. & Bonvillian, W.B. (2013). Two Revolutions in Learning. Science 22, Vol. 339 no. 6126, p.1359.Google ScholarCross Ref
- Sinha, T., Jermann, P., Li, N., Dillenbourg, P. (2014). Your click decides your fate: Inferring Information Processing and Attrition Behavior from MOOC Video Clickstream Interactions. Proc. of the 2014 Empirical Methods in Natural Language Processing Workshop on Modeling Large Scale Social Interaction in Massively Open Online Courses.Google ScholarCross Ref
- Strader, R. & Thille, C. (2012). The Open Learning Initiative: Enacting Instruction Online. In Oblinger, D.G. (Ed.) Game Changers: Education and Information Technologies (201--213). Educause.Google Scholar
- Tetrad IV. http://www.phil.cmu.edu/tetrad.Google Scholar
- Thille, C. (2014). Ubiquity symposium: MOOCs and technology to advance learning and learning research: opening statement. Ubiquity, Volume 2014, Number April (2014), Pages 1--7. ACM New York, NY, USA. DOI: 10.1145/2601337. Google ScholarDigital Library
- VanLehn, K. (2006). The behavior of tutoring systems. International Journal of Artificial Intelligence in Education, 16(3), 227--265. Google ScholarDigital Library
- VanLehn, K. (2011). The relative effectiveness of human tutoring, intelligent tutoring systems, and other tutoring systems. Educational Psychologist, 46(4), 197--221.Google ScholarCross Ref
- Zhu X., Lee Y., Simon H.A., & Zhu, D. (1996). Cue recognition and cue elaboration in learning from examples. In Proc. of the National Academy of Sciences 93, (pp. 1346--1351).Google ScholarCross Ref
Index Terms
Learning is Not a Spectator Sport: Doing is Better than Watching for Learning from a MOOC
Recommendations
HCI and the Educational Technology Revolution
AVI '16: Proceedings of the International Working Conference on Advanced Visual InterfacesWhile educational technology has a long pedigree, the last few years have seen dramatic changes. These have included the rise and institutionalisation of MOOCs, and other web-based initiatives such as Kahn Academy and Peer-to-Peer University (P2PU). ...
Engaging with open education
WikiSym '10: Proceedings of the 6th International Symposium on Wikis and Open CollaborationThis panel will address key issues and provide empirically-grounded insights on learning about, with, and through open educational resources (OER). Open educational resources are educational materials that are freely shared by those who seek to learn ...
Open Educational Resources for Language Education: Towards the Development of an e-Toolkit
Learning and Collaboration Technologies: New Challenges and Learning ExperiencesAbstractThe Open Educational Resources (OER) movement is overgrowing over the last 15 years, as many institutions adopt the idea of openness and universal access to educational content. Thus, the research and educational material must be open to everyone ...
Comments