skip to main content
10.1145/2724660.2724681acmconferencesArticle/Chapter ViewAbstractPublication Pagesl-at-sConference Proceedingsconference-collections
research-article
Open Access

Learning is Not a Spectator Sport: Doing is Better than Watching for Learning from a MOOC

Published:14 March 2015Publication History

ABSTRACT

The printing press long ago and the computer today have made widespread access to information possible. Learning theorists have suggested, however, that mere information is a poor way to learn. Instead, more effective learning comes through doing. While the most popularized element of today's MOOCs are the video lectures, many MOOCs also include interactive activities that can afford learning by doing. This paper explores the learning benefits of the use of informational assets (e.g., videos and text) in MOOCs, versus the learning by doing opportunities that interactive activities provide. We find that students doing more activities learn more than students watching more videos or reading more pages. We estimate the learning benefit from extra doing (1 SD increase) to be more than six times that of extra watching or reading. Our data, from a psychology MOOC, is correlational in character, however we employ causal inference mechanisms to lend support for the claim that the associations we find are causal.

References

  1. Aleven, V., Beal, C. R., & Graesser, A. C. (2013). Introduction to the special issue on advanced learning technologies. Journal of Educational Psychology, 105(4), 929--931.Google ScholarGoogle ScholarCross RefCross Ref
  2. Anderson, J. R., Corbett, A. T., Koedinger, K. R., & Pelletier, R. (1995) . Cognitive tutors:Lessons learned. The Journal of the Learning Sciences, 4, 167--207.Google ScholarGoogle ScholarCross RefCross Ref
  3. Anzai, Y., & Simon, H. A. (1979). The theory of learning by doing. Psychological Review, 86, 124--140.Google ScholarGoogle ScholarCross RefCross Ref
  4. Barab, S.A. & Squire, K.D. (2004). Design-based research: Putting a stake in the ground. Journal of the Learning Science, 13 (1) 1--14.Google ScholarGoogle ScholarCross RefCross Ref
  5. Bichsel, J. (2013). The State of E-Learning in Higher Education: An Eye toward Growth and Increased Access (Research Report), Louisville, CO: EDUCAUSE Center for Analysis and Research. Available online: http://www.educause.edu/ecar.Google ScholarGoogle Scholar
  6. Bowen, W. G., Chingos, M. M., Lack, K. A., & Nygren, T. I. (2013). Interactive learning online at public universities: Evidence from a six-campus randomized trial. Journal of Policy Analysis and Management, 33(1), 94--111.Google ScholarGoogle ScholarCross RefCross Ref
  7. Champaign, J., Fredericks, C., Colvin, K., Seaton, D., Liu, A. & Pritchard, D. Correlating skill and improvement in 2 MOOCs with a student's time on task. In Proc. Learning@Scale 2014, Retrieved from: http://dx.doi.org/10.1145/2556325.2566250. Google ScholarGoogle ScholarDigital LibraryDigital Library
  8. Clark, R.E., Feldon, D., van Merriënboer, J., Yates, K., & Early, S. (2007). Cognitive task analysis. In J.M. Spector, M.D. Merrill, J.J.G. van Merriënboer, & M.P. Driscoll (Eds.), Handbook of research on educational communications and technology (3rd ed., pp. 577--593). Mahwah, NJ: Lawrence Erlbaum Associates.Google ScholarGoogle Scholar
  9. Collins, E.D. (2013) "SJSU Plus augmented online learning environment: Pilot project report." The Research and Planning Group for California Community Colleges. Available online:http://www.sjsu.edu/chemistry/People/Faculty/Collins_Research_Page/AOLE%20Report%20Final%20Version_Jan%201_2014.pdfGoogle ScholarGoogle Scholar
  10. Collins, A., Brown, J. S., & Newman, S. E. (1989). Cognitive apprenticeship: Teaching the crafts of reading, writing, and mathematics. In L. B. Resnick. Knowing, Learning, and Instruction: Essays in Honor of Robert Glaser (pp. 453--494). Hillsdale, NJ: Erlbaum.Google ScholarGoogle Scholar
  11. Dewey, J. (1916), (2007 edition). Democracy and Education, Teddington: Echo Library.Google ScholarGoogle Scholar
  12. Hill, P. (2013). Emerging Student Patterns in MOOCs: A (Revised) Graphical View. e-literate. Available online: http://mfeldstein.com/emerging-student-patterns-in-moocs-a-revised-graphical-view/Google ScholarGoogle Scholar
  13. Kaufman, J.; Ryan, R.; Thille, C. and Bier, N. (2013) Open Learning Initiative Courses in Community Colleges: Evidence on Use and Effectiveness. Mellon University, Pittsburgh, PA. Available online: http://www.hewlett.org/sites/default/files/CCOLI_Report_Final_1.pdfGoogle ScholarGoogle Scholar
  14. Koedinger, K.R., & Aleven V. (2007). Exploring the assistance dilemma in experiments with Cognitive Tutors. Educational Psychology Review, 19(3), 239--264.Google ScholarGoogle ScholarCross RefCross Ref
  15. Koedinger, K.R., Baker, R.S.J.d., Cunningham, K., Skogsholm, A., Leber, B., Stamper, J. (2010). A Data Repository for the EDM community: The PSLC DataShop. In Romero, C., Ventura, S., Pechenizkiy, M., Baker, R.S.J.d. (Eds.) Handbook of Educational Data Mining. Boca Raton, FL: CRC Press.Google ScholarGoogle ScholarCross RefCross Ref
  16. Koedinger, K.R., Booth, J. L., & Klahr, D. (2013). Instructional Complexity and the Science to Constrain It. Science, 342(6161), 935--937.Google ScholarGoogle ScholarCross RefCross Ref
  17. Koedinger, K. R., Corbett, A. C., & Perfetti, C. (2012). The Knowledge-Learning-Instruction (KLI) framework: Bridging the science-practice chasm to enhance robust student learning. Cognitive Science, 36 (5), 757--798.Google ScholarGoogle ScholarCross RefCross Ref
  18. Koedinger, K. R., McLaughlin, E. A., & Stamper, J. C. (2014). Ubiquity symposium: MOOCs and technology to advance learning and learning research: Data-driven learner modeling to understand and improve online learning. Ubiquity, Volume 2014, Number May (2014), Pages 1--13. ACM New York, NY, USA. DOI:10.1145/2591682 Google ScholarGoogle ScholarDigital LibraryDigital Library
  19. Koedinger, K. R., & Sueker, E. L. F. (2014). Monitored design of an effective learning environment for algebraic problem solving. Technical report Carnegie Mellon University-HCII-14-102.Google ScholarGoogle Scholar
  20. Lovett, M., Meyer, O., Thille, C. (2008). The Open Learning Initiative: Measuring the effectiveness of the OLI learning course in accelerating student learning. Journal of Interactive Media in Education. http://jime.open.ac.uk/2008/14/.Google ScholarGoogle Scholar
  21. Meyer, O., & Lovett, M. C. (2002). Implementing a computerized tutor in a statistical reasoning Course: Getting the big picture. In B. Phillips (Ed.) Proc. of the Sixth International Conference on Teaching Statistics.Google ScholarGoogle Scholar
  22. Morote, E. & Pritchard, D. E. (2002). What Course Elements Correlate with Improvement on Tests in Introductory Newtonian Mechanics? National Association for Research in Science Teaching -- NARST- 2002 Conference.Google ScholarGoogle Scholar
  23. Ng, A. and Widom J. (2014). Origins of the Modern MOOC. http://www.cs.stanford.edu/people/ang/papers/mooc14-OriginsOfModernMOOC.pdfGoogle ScholarGoogle Scholar
  24. Pane, J.F., Griffin, B., McCaffrey, D.F. & Karam, R. (2014). Effectiveness of Cognitive Tutor Algebra I at Scale. Educational Evaluation and Policy Analysis, 36 (2), 127--144.Google ScholarGoogle ScholarCross RefCross Ref
  25. Pappano, L. (2012, November 2). The year of the MOOC. The New York Times.Google ScholarGoogle Scholar
  26. Pashler, H., Bain, P., Bottge, B., Graesser, A., Koedinger, K., McDaniel, M., & Metcalfe, J. (2007). Organizing Instruction and Study to Improve Student Learning (NCER 2007-2004). Washington, DC: National Center for Education Research, Institute of Education Sciences, U.S. Department of Education.Google ScholarGoogle Scholar
  27. Ritter S., Anderson, J. R., Koedinger, K. R., & Corbett, A. (2007). Cognitive tutor: Applied research in mathematics education. Psychonomic Bulletin & Review, 14 (2):249--255.Google ScholarGoogle ScholarCross RefCross Ref
  28. Schank, R. C., Berman, T. R. & Macperson, K. A. (1999). Learning by doing. In C. M. Reigeluth (Ed.), Instructional Design Theories and Models: A New Paradigm of Instructional Theory (Vol. II) (pp.161--181). Mahwah, NJ: Lawrence Erlbaum Associates.Google ScholarGoogle Scholar
  29. Singer, S.R. & Bonvillian, W.B. (2013). Two Revolutions in Learning. Science 22, Vol. 339 no. 6126, p.1359.Google ScholarGoogle ScholarCross RefCross Ref
  30. Sinha, T., Jermann, P., Li, N., Dillenbourg, P. (2014). Your click decides your fate: Inferring Information Processing and Attrition Behavior from MOOC Video Clickstream Interactions. Proc. of the 2014 Empirical Methods in Natural Language Processing Workshop on Modeling Large Scale Social Interaction in Massively Open Online Courses.Google ScholarGoogle ScholarCross RefCross Ref
  31. Strader, R. & Thille, C. (2012). The Open Learning Initiative: Enacting Instruction Online. In Oblinger, D.G. (Ed.) Game Changers: Education and Information Technologies (201--213). Educause.Google ScholarGoogle Scholar
  32. Tetrad IV. http://www.phil.cmu.edu/tetrad.Google ScholarGoogle Scholar
  33. Thille, C. (2014). Ubiquity symposium: MOOCs and technology to advance learning and learning research: opening statement. Ubiquity, Volume 2014, Number April (2014), Pages 1--7. ACM New York, NY, USA. DOI: 10.1145/2601337. Google ScholarGoogle ScholarDigital LibraryDigital Library
  34. VanLehn, K. (2006). The behavior of tutoring systems. International Journal of Artificial Intelligence in Education, 16(3), 227--265. Google ScholarGoogle ScholarDigital LibraryDigital Library
  35. VanLehn, K. (2011). The relative effectiveness of human tutoring, intelligent tutoring systems, and other tutoring systems. Educational Psychologist, 46(4), 197--221.Google ScholarGoogle ScholarCross RefCross Ref
  36. Zhu X., Lee Y., Simon H.A., & Zhu, D. (1996). Cue recognition and cue elaboration in learning from examples. In Proc. of the National Academy of Sciences 93, (pp. 1346--1351).Google ScholarGoogle ScholarCross RefCross Ref

Index Terms

  1. Learning is Not a Spectator Sport: Doing is Better than Watching for Learning from a MOOC

    Recommendations

    Reviews

    David E. Goldfarb

    This paper explores the question of whether students learn better by doing activities or by just passively watching videos of lectures. The authors are part of the Open Learning Initiative at Carnegie Mellon, which "focus[es] on rich and interactive learn-by-doing activities," as compared to the "video-based lectures and ... discussion forums" of other massive open online course (MOOC) programs. In eight pages of detailed statistical analysis, the authors examine the results of one online course (12 weeks; 27,720 students started the course; 1,154 completed). They discover-no great surprise-that active participation leads to better results. This paper may be of some interest to readers interested in the particular statistics or in the methodologies used. However, I cannot recommend it as relevant to any wider audience. Online Computing Reviews Service

    Access critical reviews of Computing literature here

    Become a reviewer for Computing Reviews.

    Comments

    Login options

    Check if you have access through your login credentials or your institution to get full access on this article.

    Sign in
    • Published in

      cover image ACM Conferences
      L@S '15: Proceedings of the Second (2015) ACM Conference on Learning @ Scale
      March 2015
      438 pages
      ISBN:9781450334112
      DOI:10.1145/2724660

      Copyright © 2015 Owner/Author

      Permission to make digital or hard copies of part or all of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for third-party components of this work must be honored. For all other uses, contact the Owner/Author.

      Publisher

      Association for Computing Machinery

      New York, NY, United States

      Publication History

      • Published: 14 March 2015

      Check for updates

      Qualifiers

      • research-article

      Acceptance Rates

      L@S '15 Paper Acceptance Rate23of90submissions,26%Overall Acceptance Rate117of440submissions,27%

    PDF Format

    View or Download as a PDF file.

    PDF

    eReader

    View online with eReader.

    eReader