René F. Kizilcec
Emily Schneider
Abstract
Open online learning environments attract an audience with diverse motivations who interact with structured courses in several ways. To systematically describe the motivations of these learners, we developed the Online Learning Enrollment Intentions (OLEI) scale, a 13-item questionnaire derived from open-ended responses to capture learners' authentic perspectives. Although motivations varied across courses, we found that each motivation predicted key behavioral outcomes for learners (N = 71, 475 across 14 courses).
From learners' motivational and behavioral patterns, we infer a variety of needs that they seek to gratify by engaging with the courses, such as meeting new people and learning English. To meet these needs, we propose multiple design directions, including virtual social spaces outside any particular course, improved support for local groups of learners, and modularization to promote accessibility and organization of course content. Motivations thus provide a lens for understanding online learners and designing online courses to better support their needs.
Supplemental Material
Available for Download
Supplemental movie, appendix, image and software files for, Motivation as a Lens to Understand Online Learners: Toward Data-Driven Design with the OLEI Scale
- Ashton Anderson, Daniel Huttenlocher, Jon Kleinberg, and Jure Leskovec. 2014. Engaging with massive online courses. In Proceedings of the 23rd International Conference on World Wide Web. 687--698. Google Scholar
Digital Library
- Joshua D. Angrist and Jörn-Steffen Pischke. 2008. Mostly Harmless Econometrics: An Empiricist's Companion. Princeton University Press.Google Scholar
- Tara S. Behrend, David J. Sharek, Adam W. Meade, and Eric N. Wiebe. 2011. The viability of crowdsourcing for survey research. Behavior Research Methods 43, 3, 800--813.Google ScholarCross Ref
- Marie Bienkowski, Mingyu Feng, and Barbara Means. 2012. Enhancing Teaching and Learning through Educational Data Mining and Learning Analytics: An Issue Brief. U.S. Department of Education, Office of Educational Technology.Google Scholar
- Jay G. Blumler and Elihu Katz. 1974. The Uses of Mass Communications: Current Perspectives on Gratifications Research, Vol. 3. Sage, Beverly Hills, CA.Google Scholar
- John D. Bransford, Ann L. Brown, and Rodney R. Cocking (Eds). 2000. How People Learn. National Academies Press, Washington, DC.Google Scholar
- Derek O. Bruff, Douglas H. Fisher, Kathryn E. McEwen, and Blaine E. Smith. 2013. Wrapping a MOOC: Student perceptions of an experiment in blended learning. Journal of Online Learning and Teaching 9, 2.Google Scholar
- Vannevar Bush. 1945. As we may think. Atlantic Monthly 176, 1, 101--108.Google Scholar
- Martin Bygate, Merrill Swain, and Peter Skehan. 2013. Researching Pedagogic Tasks: Second Language Learning, Teaching, and Testing. Routledge.Google Scholar
- Julia Cambre, Chinmay Kulkarni, Michael S. Bernstein, and Scott R. Klemmer. 2014. Talkabout: Small-group discussions in massive global classes. In Proceedings of the 1st ACM Conference on Learning @ Scale. ACM, New York, NY, 161--162. Google ScholarDigital Library
- Justin Cheng, Chinmay Kulkarni, and Scott Klemmer. 2013. Tools for predicting drop-off in large online classes. In Proceedings of the 2013 Conference on Computer Supported Cooperative Work Companion. ACM, New York, NY, 121--124. Google ScholarDigital Library
- Derrick Coetzee, Armando Fox, Marti A. Hearst, and Björn Hartmann. 2014. Should your MOOC forum use a reputation system? In Proceedings of the 17th ACM Conference on Computer Supported Cooperative Work and Social Computing. ACM, New York, NY, 1176--1187. Google ScholarDigital Library
- David Colton and Robert W. Covert. 2007. Designing and Constructing Instruments for Social Research and Evaluation. John Wiley and Sons.Google Scholar
- Catherine H. Crouch and Eric Mazur. 2001. Peer instruction: Ten years of experience and results. American Journal of Physics 69, 9, 970--977.Google ScholarCross Ref
- Linda Darling-Hammond, Brigid Barron, P. David Pearson, Alan H. Schoenfeld, Elizabeth K. Stage, Timothy D. Zimmerman, Gina N. Cervetti, and Jennifer L. Tilson. 2008. Powerful Learning: What We Know about Teaching for Understanding. Jossey-Bass, San Francisco, CA.Google Scholar
- Jennifer DeBoer, Andrew D. Ho, Glenda S. Stump, and Lori Breslow. 2014. Changing “course”: Reconceptualizing educational variables for massive open online courses. Educational Researcher 43, 2, 74--84.Google ScholarCross Ref
- Frank Fischer, Ingo Kollar, Karsten Stegmann, and Christof Wecker. 2013. Toward a script theory of guidance in computer-supported collaborative learning. Educational Psychologist 48, 1, 56--66.Google ScholarCross Ref
- D. Randy Garrison. 1997. Self-directed learning: Toward a comprehensive model. Adult Education Quarterly 48, 1, 18--33.Google ScholarCross Ref
- Sara Goldrick-Rab. 2010. Challenges and opportunities for improving community college student success. Review of Educational Research 80, 3, 437--469.Google ScholarCross Ref
- Philip J. Guo and Katharina Reinecke. 2014. Demographic differences in how students navigate through MOOCs. In Proceedings of the 1st ACM Conference on Learning @ Scale. ACM, New York, NY, 21--30. Google ScholarDigital Library
- Laura Horn, Stephanie Nevill, and James Griffith. 2006. Profile of Undergraduates in U.S. Postsecondary Education Institutions: 2003--04. Statistical Analysis Report NCES 2006-184. U.S. Department of Education, National Center for Education Statistics, U.S. Government Printing Office, Washington, DC.Google Scholar
- David W. Johnson, Roger T. Johnson, and Mary Beth Stanne. 2000. Cooperative Learning Methods: A Meta-Analysis. Retrieved February 3, 2015, from http://www.ccsstl.com/sites/default/files/Cooperative%20Learning%20Research%20.pdf.Google Scholar
- Slava Kalyuga, Paul Ayres, Paul Chandler, and John Sweller. 2003. The expertise reversal effect. Educational Psychologist 38, 1, 23--31.Google ScholarCross Ref
- Wendy Kicken, Saskia Brand-Gruwel, and Jeroen J. G. van Merriënboer. 2008. Scaffolding advice on task selection: A safe path toward self-directed learning in on-demand education. Journal of Vocational Education and Training 60, 3, 223--239.Google ScholarCross Ref
- René F. Kizilcec. 2013. Collaborative learning in geographically distributed and in-person groups. In Proceedings of the Workshops at the 16th International Conference on Artificial Intelligence in Education (AIED'13). 67--74.Google Scholar
- René F. Kizilcec. 2014. Reducing non-response bias with survey reweighting: Applications for online learning researchers. In Proceedings of the 1st ACM Conference on Learning @ Scale. ACM, New York, NY, 143--144. Google ScholarDigital Library
- René F. Kizilcec, Chris Piech, and Emily Schneider. 2013. Deconstructing disengagement: Analyzing learner subpopulations in massive open online courses. In Proceedings of the 3rd International Conference on Learning Analytics and Knowledge. ACM, New York, NY, 170--179. Google ScholarDigital Library
- René F. Kizilcec, Emily Schneider, Geoff Cohen, and Daniel McFarland. 2014. Encouraging forum participation in online courses with collectivist, individualist, and neutral motivational framings. eLearning Papers 37, 13--22.Google Scholar
- Malcolm S. Knowles. 1975. Self-Directed Learning. Association Press, New York, NY.Google Scholar
- Geza Kovacs. 2013. Smart subtitles for language learning. In CHI'13 Extended Abstracts on Human Factors in Computing Systems. ACM, New York, NY, 2719--2724. Google ScholarDigital Library
- Robert E. Kraut, Paul Resnick, Sara Kiesler, Moira Burke, Yan Chen, Niki Kittur, Joseph Konstan, Yuqing Ren, and John Riedl. 2012. Building Successful Online Communities: Evidence-Based Social Design. MIT Press, Cambridge, MA.Google Scholar
- Jon A. Krosnick. 1999. Survey research. Annual Review of Psychology 50, 1, 537--567.Google ScholarCross Ref
- Mark R. Lepper, David Greene, and Richard E. Nisbett. 1973. Undermining children's intrinsic interest with extrinsic reward: A test of the “overjustification” hypothesis. Journal of Personality and Social Psychology 28, 1, 129.Google ScholarCross Ref
- Seongtaek Lim, Derrick Coetzee, Bjoern Hartmann, Armando Fox, and Marti A. Hearst. 2014. Initial experiences with small group discussions in MOOCs. In Proceedings of the 1st ACM Conference on Learning @ Scale. ACM, New York, NY, 151--152. DOI:http://dx.doi.org/10.1145/2556325.2567854 Google ScholarDigital Library
- Lena Mamykina, Bella Manoim, Manas Mittal, George Hripcsak, and Björn Hartmann. 2011. Design lessons from the fastest Q&A site in the west. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI'11). ACM, New York, NY, 2857--2866. DOI:http://dx.doi.org/10.1145/1978942.1979366 Google ScholarDigital Library
- Richard E. Mayer and Laura J. Massa. 2003. Three facets of visual and verbal learners: Cognitive ability, cognitive style, and learning preference. Journal of Educational Psychology 95, 4, 833.Google ScholarCross Ref
- Sandra D. Mitchell. 2009. Unsimple Truths: Science, Complexity, and Policy. University of Chicago Press.Google ScholarCross Ref
- Tommy Mullaney and Justin Reich. 2014. Staggered versus all-at-once content release in massive open online courses: Evaluating a natural experiment. Retrieved February 3, 2015, from http://harvardx.harvard.edu/files/harvardx/files/ssrn-id2499729.pdfGoogle Scholar
- Hendrik Müller, Aaron Sedley, and Elizabeth Ferrall-Nunge. 2014. Survey research in HCI. In Ways of Knowing in HCI. Springer, 229--266.Google Scholar
- National Science Foundation. 2013. Women, Minorities, and Persons with Disabilities in Science and Engineering: 2013. Special Report NSF 13-304. Arlington, VA.Google Scholar
- James W. Pellegrino and Margaret L. Hilton (Eds.). Committee on Defining Deeper Learning and 21st Century Skills; Center for Education; Division on Behavioral and Social Sciences and Education; and National Research Council. 2013. Education for Life and Work: Developing Transferable Knowledge and Skills in the 21st Century. National Academies Press, Washington, DC.Google Scholar
- Paul R. Pintrich. 2003. A motivational science perspective on the role of student motivation in learning and teaching contexts. Journal of Educational Psychology 95, 4, 667.Google ScholarCross Ref
- Paul R. Pintrich, David A. F. Smith, Teresa García, and Wilbert J. McKeachie. 1993. Reliability and predictive validity of the Motivated Strategies for Learning Questionnaire (MSLQ). Educational and Psychological Measurement 53, 3, 801--813.Google ScholarCross Ref
- Richard M. Ryan and Edward L. Deci. 2000. Self-determination theory and the facilitation of intrinsic motivation, social development, and well-being. American Psychologist 55, 1, 68.Google ScholarCross Ref
- Dale H. Schunk and Barry J. Zimmerman. 2012. Motivation and Self-Regulated Learning: Theory, Research, and Applications. Routledge.Google Scholar
- Daniel T. Seaton, Yoav Bergner, Isaac Chuang, Piotr Mitros, and David E. Pritchard. 2014. Who does what in a massive open online course? Communications of the ACM 57, 4, 58--65. Google ScholarDigital Library
- Michelle K. Smith, William B. Wood, Wendy K. Adams, Carl Wieman, Jennifer K. Knight, Nancy Guild, and Tin Tin Su. 2009. Why peer discussion improves student performance on in-class concept questions. Science 323, 5910, 122--124.Google Scholar
- Leonard Springer, Mary Elizabeth Stanne, and Samuel S. Donovan. 1999. Effects of small-group learning on undergraduates in science, mathematics, engineering, and technology: A meta-analysis. Review of Educational Research 69, 1, 21--51.Google ScholarCross Ref
- Thomas F. Stafford. 2005. Understanding motivations for Internet use in distance education. IEEE Transactions on Education 48, 2, 301--306. Google ScholarDigital Library
- Vincent Tinto. 1997. Classrooms as communities. Journal of Higher Education 68, 6, 599--623.Google Scholar
- Roger Tourangeau, Kenneth A. Rasinski, and Norman Bradburn. 1991. Measuring happiness in surveys: A test of the subtraction hypothesis. Public Opinion Quarterly 55, 2, 255--266.Google ScholarCross Ref
- David Tyack and William Tobin. 1994. The grammar of schooling: Why has it been so hard to change? American Educational Research Journal 31, 3, 453--479.Google Scholar
- Lev S. Vygotsky. 1934 (1986). Thought and Language. MIT Press.Google Scholar
- Julia Wilkowski, Amit Deutsch, and Daniel M. Russell. 2014. Student skill and goal achievement in the mapping with Google MOOC. In Proceedings of the 1st ACM Conference on Learning @ Scale. ACM, New York, NY, 3--10. Google ScholarDigital Library
Index Terms
- Motivation as a Lens to Understand Online Learners: Toward Data-Driven Design with the OLEI Scale
Recommendations
Understanding Student Motivation, Behaviors and Perceptions in MOOCs
CSCW '15: Proceedings of the 18th ACM Conference on Computer Supported Cooperative Work & Social ComputingMassive Open Online Courses (MOOCs) have recently experienced rapid development and garnered significant attention from various populations. Despite the wide recognition of MOOCs as an important opportunity within educational practices, there are still ...
Understanding Learners' General Perception Towards Learning with MOOC Classmates: An Exploratory Study
L@S '15: Proceedings of the Second (2015) ACM Conference on Learning @ ScaleIn this work-in-progress, we present our preliminary findings from an exploratory study on understanding learners' general behavior and perception towards learning with classmates in MOOCs. One-on-one semi-structured interview designed with grounded ...
Ask the Instructors: Motivations and Challenges of Teaching Massive Open Online Courses
CSCW '16: Proceedings of the 19th ACM Conference on Computer-Supported Cooperative Work & Social ComputingMassive Open Online Courses (MOOCs) have experienced rapid growth and attracted significant attention within aca-demia. However, despite widespread acceptance of MOOCs as a unique opportunity to transform educational practices, many questions remain ...
Reviews
M. Sasikumar
Massive open online courses (MOOCs) pose fresh challenges in e-learning. They report very large and global enrollments-across cultures, time zones, and goals. This necessitates new studies on learner behavior and effective teaching/learning practices. In this context, this paper makes a good attempt to understand the variety in such learners and links this variety to learner behavior online. The first part of this paper discusses evolving an online learning enrollment intention (OLEI) scale to capture the various possible motivations for MOOC learners. The process of evolution is discussed to demonstrate its usefulness and adequacy. From open-ended responses, generalized items are created manually. These were cross-validated for adequacy and ambiguity and then refined over a series of iterations. The final scale includes entries such as "to improve English," "general interest," and "relevant to job." These motivations were analyzed for correlations and variations across gender and age using a large dataset of MOOC learners. The rest of the paper postulates a series of research questions mapping learner behavior, including forum posts, fraction of videos seen, and assignments done to different motivations. For example, those who are motivated by the need to strengthen academic records would tend to submit all of the required assignments. All studies are presented well and follow sound analysis methodology appropriate to the specific type of relation considered. The results of the studies will be of interest to all those involved in e-learning, specifically the buzz of MOOCs. Overall, this is a good paper. Online Computing Reviews Service
Access critical reviews of Computing literature here
Become a reviewer for Computing Reviews.
Comments