René F. Kizilcec
Chris Piech
ABSTRACT
As MOOCs grow in popularity, the relatively low completion rates of learners has been a central criticism. This focus on completion rates, however, reflects a monolithic view of disengagement that does not allow MOOC designers to target interventions or develop adaptive course features for particular subpopulations of learners. To address this, we present a simple, scalable, and informative classification method that identifies a small number of longitudinal engagement trajectories in MOOCs. Learners are classified based on their patterns of interaction with video lectures and assessments, the primary features of most MOOCs to date.
In an analysis of three computer science MOOCs, the classifier consistently identifies four prototypical trajectories of engagement. The most notable of these is the learners who stay engaged through the course without taking assessments. These trajectories are also a useful framework for the comparison of learner engagement between different course structures or instructional approaches. We compare learners in each trajectory and course across demographics, forum participation, video access, and reports of overall experience. These results inform a discussion of future interventions, research, and design directions for MOOCs. Potential improvements to the classification mechanism are also discussed, including the introduction of more fine-grained analytics.
- S. Amershi and C. Conati. Automatic recognition of learner types in exploratory learning environments. Handbook of Educational Data Mining, page 213, 2010.Google Scholar
- P. Bahr. The bird's eye view of community colleges: A behavioral typology of first-time students based on cluster analytic classification. Research in Higher Education, 51(8):724--749, 2010.Google ScholarCross Ref
- P. Black and D. Wiliam. Assessment and classroom learning. Assessment in education, 5(1):7--74, 1998.Google ScholarCross Ref
- J. Bransford, A. Brown, and R. Cocking. How people learn: Brain, mind, experience, and school. National Academies Press, 2000.Google Scholar
- P. Brusilovsky and E. Millán. User models for adaptive hypermedia and adaptive educational systems. The adaptive web, pages 3--53, 2007. Google ScholarDigital Library
- J. Cohen. Statistical power analysis for the behavioral sciences. Lawrence Erlbaum, 1988.Google Scholar
- B. De Wever, T. Schellens, M. Valcke, and H. Van Keer. Content analysis schemes to analyze transcripts of online asynchronous discussion groups: A review. Computers & Education, 46(1):6--28, 2006. Google ScholarDigital Library
- S. Downes. New technology supporting informal learning. Journal of Emerging Technologies in Web Intelligence, 2(1):27--33, 2010.Google ScholarCross Ref
- C. Dweck. Mindset: The new psychology of success. Ballantine Books, 2007.Google Scholar
- C. Dweck, G. Walton, and G. Cohen. Academic tenacity: Mindset and skills that promote long-term learning. Gates Foundation. Seattle, WA: Bill & Melinda Gates Foundation, 2011.Google Scholar
- B. Everitt and T. Hothorn. A handbook of statistical analyses using R. Chapman & Hall/CRC, 2009. Google ScholarDigital Library
- C. Farrington, M. Roderick, E. Allensworth, J. Nagaoka, T. Keyes, D. Johnson, and N. Beechum. Teaching adolescents to become learners: The role of noncognitive factors in shaping school performance: A critical literature review. Chicago: University of Chicago Consortium on Chicago School Research, 2012.Google Scholar
- B. Fogg. A behavior model for persuasive design. In Proceedings of the 4th international Conference on Persuasive Technology, page 40. ACM, 2009. Google ScholarDigital Library
- S. Goldrick-Rab. Challenges and opportunities for improving community college student success. Review of Educational Research, 80(3):437--469, 2010.Google ScholarCross Ref
- B. Kim and T. Reeves. Reframing research on learning with technology: in search of the meaning of cognitive tools. Instructional Science, 35(3):207--256, 2007.Google ScholarCross Ref
- K. Koedinger, A. Corbett, et al. Cognitive tutors: Technology bringing learning science to the classroom. The Cambridge handbook of the learning sciences, pages 61--78, 2006.Google Scholar
- R. Kraut, P. Resnick, S. Kiesler, Y. Ren, Y. Chen, M. Burke, N. Kittur, J. Riedl, and J. Konstan. Building successful online communities: Evidence-based social design. The MIT Press, 2012. Google ScholarDigital Library
- M. McDaniel, H. Roediger, and K. McDermott. Generalizing test-enhanced learning from the laboratory to the classroom. Psychonomic Bulletin & Review, 14(2):200--206, 2007.Google ScholarCross Ref
- E. Ophir, C. Nass, and A. Wagner. Cognitive control in media multitaskers. Proceedings of the National Academy of Sciences, 106(37):15583--15587, 2009.Google ScholarCross Ref
- C. Rodriguez. Moocs and the ai-stanford like courses: Two successful and distinct course formats for massive open online courses. Learning, 2012.Google Scholar
- H. Roediger III and J. Karpicke. Test-enhanced learning taking memory tests improves long-term retention. Psychological Science, 17(3):249--255, 2006.Google ScholarCross Ref
- P. J. Rousseeuw. Silhouettes: A graphical aid to the interpretation and validation of cluster analysis. Journal of Computational and Applied Mathematics, 20(0):53--65, 1987. Google ScholarDigital Library
- V. Saenz, D. Hatch, B. Bukoski, S. Kim, K. Lee, and P. Valdez. Community college student engagement patterns a typology revealed through exploratory cluster analysis. Community College Review, 39(3):235--267, 2011.Google ScholarCross Ref
- C. Shirky. Cognitive surplus: Creativity and generosity in a connected age. ePenguin, 2010.Google Scholar
- G. Siemens. Connectivism: A learning theory for the digital age. 2004.Google Scholar
- S. Spencer, C. Steele, and D. Quinn. Stereotype threat and women's math performance. Journal of Experimental Social Psychology, 35(1):4--28, 1999.Google ScholarCross Ref
- G. Stahl, T. Koschmann, and D. Suthers. Computer-supported collaborative learning: An historical perspective. In R. K. Sawyer, editor, Cambridge handbook of the learning sciences, pages 409--426. Cambridge, UK: Cambridge University Press, 2006.Google Scholar
- C. Steele, S. Spencer, and J. Aronson. Contending with group image: The psychology of stereotype and social identity threat. Advances in experimental social psychology, 34:379--440, 2002.Google Scholar
- United Nations Development Programme. 2011 human development report. Retrieved from http://hdr.undp.org/en/media/HDR_2011_Statistical_Tables.xls. (Accessed 18-Jan-2013).Google Scholar
Index Terms
- Deconstructing disengagement: analyzing learner subpopulations in massive open online courses
Recommendations
Examining engagement: analysing learner subpopulations in massive open online courses (MOOCs)
LAK '15: Proceedings of the Fifth International Conference on Learning Analytics And KnowledgeMassive open online courses (MOOCs) are now being used across the world to provide millions of learners with access to education. Many learners complete these courses successfully, or to their own satisfaction, but the high numbers who do not finish ...
MOOCs and the funnel of participation
LAK '13: Proceedings of the Third International Conference on Learning Analytics and KnowledgeMassive Online Open Courses (MOOCs) are growing substantially in numbers, and also in interest from the educational community. MOOCs offer particular challenges for what is becoming accepted as mainstream practice in learning analytics.
Partly for this ...
Moving Through MOOCS: Pedagogy, Learning Design and Patterns of Engagement
Design for Teaching and Learning in a Networked WorldAbstractMassive open online courses (MOOCs) are part of the lifelong learning experience of people worldwide. Many of these learners participate fully. However, the high levels of dropout on most of these courses are a cause for concern. Previous studies ...
Comments