Nesra Yannier
Scott E. Hudson
Kenneth R. Koedinger
Skip Abstract Section
Skip Supplemental Material SectionAbstract
Can experimenting with three-dimensional (3D) physical objects in mixed-reality environments produce better learning and enjoyment than flat-screen two-dimensional (2D) interaction? We explored this question with EarthShake: a mixed-reality game bridging physical and virtual worlds via depth-camera sensing, designed to help children learn basic physics principles. In this paper, we report on a controlled experiment with 67 children, 4--8 years old, that examines the effect of observing physical phenomena and collaboration (pairs vs. solo). A follow-up experiment with 92 children tests whether adding simple physical control, such as shaking a tablet, improves learning and enjoyment. Our results indicate that observing physical phenomena in the context of a mixed-reality game leads to significantly more learning and enjoyment compared to screen-only versions. However, there were no significant effects of adding simple physical control or having students play in pairs vs. alone. These results and our gesture analysis provide evidence that children's science learning can be enhanced through experiencing physical phenomena in a mixed-reality environment.
Supplemental Material
- Dor Abrahamson, Rosa G. Lee, Andrea G. Negrete, and José F. Gutiérrez. 2014. Coordinating visualizations of polysemous action: Values added for grounding proportion. ZDM 46, 1 (2014), 79--93.Google Scholar
Cross Ref
- Vincent A. W. M. M. Aleven and Kenneth R. Koedinger. 2000. The need for tutorial dialog to support self-explanation. In Paper Presented at the AAAI Fall Symposium on Building Dialogue Systems for Tutorial Applications. 65--73.Google Scholar
- Alissa N. Antle. 2013. Research opportunities: Embodied child--computer interaction. Int. J. Child-Comput. Interact. 1, 1 (2013), 30--36. DOI:http://dx.doi.org/10.1016/j.ijcci.2012.08.001Google ScholarCross Ref
- Alissa N. Antle, Milena Droumeva, and Daniel Ha. 2009. Hands on what?: Comparing children's mouse-based and tangible-based interaction. In Proceedings of the 8th International Conference on Interaction Design and Children. 80--88. Google ScholarDigital Library
- Paul Ayres and John Sweller. 2014. The split-attention principle in multimedia learning. Cambridge Handb. Multimed. Learn. 2 (2014).Google ScholarCross Ref
- Margarita Azmitia and Kevin Crowley. 2001. The rhythms of scientific thinking: A study of collaboration in an earthquake microworld. Des. Sci. Implic. from everyday, classroom, Prof. settings (2001), 51--81.Google Scholar
- Maribeth Back, Jonathan Cohen, Rich Gold, Steve Harrison, and Scott Minneman. 2001. Listen reader: An electronically augmented paper-based book. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems. 23--29. Google ScholarDigital Library
- Saskia Bakker, Elise van den Hoven, and Alissa N. Antle. 2011. MoSo tangibles: Evaluating embodied learning. In Proceedings of the 5th International Conference on Tangible, Embedded, and Embodied Interaction. 85--92. Google ScholarDigital Library
- Rachel Barr. 2010. Transfer of learning between 2D and 3D sources during infancy: Informing theory and practice. Dev. Rev. 30, 2 (2010), 128--154.Google ScholarCross Ref
- Hrvoje Benko, Ricardo Jota, and Andrew Wilson. 2012. MirageTable: freehand interaction on a projected augmented reality tabletop. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems. 199--208. Google ScholarDigital Library
- Robert M. Carini, George D. Kuh, and Stephen P. Klein. 2006. Student engagement and student learning: Testing the linkages. Res. High. Educ. 47, 1 (2006), 1--32.Google ScholarCross Ref
- Catherine C. Chase, Jonathan T. Shemwell, and Daniel L. Schwartz. 2010. Explaining across contrasting cases for deep understanding in science: An example using interactive simulations. In Proceedings of the 9th International Conference of the Learning Sciences, Vol. 1. 153--160. Google ScholarDigital Library
- Michael G. Christel, Scott M. Stevens, Bryan S. Maher, Sean Brice, Matt Champer, Luke Jayapalan, Qiaosi Chen, Jing Jin, Daniel Hausmann, Nora Bastida, and Xun Zhang. 2012. RumbleBlocks: Teaching science concepts to young children through a Unity game. In Proceedings of the 17th International Conference on Computer Games (CGAMES’17). 162--166. Google ScholarDigital Library
- Albert T. Corbett and John R. Anderson. 2001. Locus of feedback control in computer-based tutoring: Impact on learning rate, achievement and attitudes. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems. 245--252. Google ScholarDigital Library
- Brent Davis. 2015. Spatial Reasoning in the Early Years: Principles, Assertions, and Speculations, Routledge.Google Scholar
- Johannes Engelkamp and Hubert D. Zimmer. 1989. Memory for action events: A new field of research. Psychol. Res. 51, 4 (1989), 153--157.Google ScholarCross Ref
- Taciana Pontual Falcão and Sara Price. 2009. What have you done! the role of’interference’in tangible environments for supporting collaborative learning. In Proceedings of the 9th International Conference on Computer Supported Collaborative Learning, Vol. 1. 325--334. Google ScholarDigital Library
- George W. Fitzmaurice, Hiroshi Ishii, and William A. S. Buxton. 1995. Bricks: Laying the foundations for graspable user interfaces. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems. 442--449. Google ScholarDigital Library
- Arthur M. Glenberg. 1997. What memory is for: Creating meaning in the service of action. Behav. Brain Sci. 20, 01 (1997), 41--50.Google ScholarCross Ref
- Anuradha A. Gokhale. 1995. Collaborative learning enhances critical thinking. Journal of Technology Education.Google ScholarCross Ref
- Ankit Gupta, Dieter Fox, Brian Curless, and Michael Cohen. 2012. DuploTrack: A real-time system for authoring and guiding duplo block assembly. In Proceedings of the 25th Annual ACM Symposium on User Interface Software and Technology. 389--402. Google ScholarDigital Library
- Harlene Hayne, Jane Herbert, and Gabrielle Simcock. 2003. Imitation from television by 24-and 30-month-olds. Dev. Sci. 6, 3 (2003), 254--261.Google ScholarCross Ref
- Philip H. Henning. 2004. Everyday cognition and situated learning. Handb. Res. Educ. Commun. Technol. (2004), 143--168.Google Scholar
- Eva Hornecker and Jacob Buur. 2006. Getting a grip on tangible interaction: A framework on physical space and social interaction. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems. 437--446. Google ScholarDigital Library
- Autumn B. Hostetter and Martha W. Alibali. 2008. Visible embodiment: Gestures as simulated action. Psychon. Bull. Rev. 15, 3 (2008), 495--514.Google ScholarCross Ref
- David Klahr, Lara M. Triona, and Cameron Williams. 2007. Hands on what? The relative effectiveness of physical versus virtual materials in an engineering design project by middle school children. J. Res. Sci. Teach. 44, 1 (2007), 183--203.Google ScholarCross Ref
- Kenneth R. Koedinger, Elizabeth A. McLaughlin, and Neil T. Heffernan. 2010. A quasi-experimental evaluation of an on-line formative assessment and tutoring system. J. Educ. Comput. Res. 43, 4 (2010), 489--510.Google ScholarCross Ref
- Harlan Lebo. 2007. The 2007 Digital Future Report: Surveying the Digital Future, Year Six. Center for the Digital Future, USC Annenberg School.Google Scholar
- Mark W. Lipsey, Kelly Puzio, Cathy Yun, Michael A. Hebert, Kasia Steinka-Fry, Mikel W. Cole, Megan Roberts, Karen S. Anthony, and Matthew D. Busick. 2012. Translating the Statistical Representation of the Effects of Education Interventions into More Readily Interpretable Forms. National Center for Special Education Research.Google Scholar
- Andrew Manches. 2011. Digital manipulatives: Tools to transform early learning experiences. Int. J. Technol. Enhanc. Learn. 3, 6 (2011), 608--626. Google ScholarDigital Library
- Andrew Manches and Sara Price. 2011. Designing learning representations around physical manipulation: hands and objects. In Proceedings of the 10th International Conference on Interaction Design and Children. 81--89. Google ScholarDigital Library
- Paul Marshall, Peter C. H. Cheng, and Rosemary Luckin. 2010. Tangibles in the balance: A discovery learning task with physical or graphical materials. In Proceedings of the Fourth International Conference on Tangible, Embedded, and Embodied Interaction. 153--160. Google ScholarDigital Library
- Taylor Martin and Daniel L. Schwartz. 2005. Physically distributed learning: Adapting and reinterpreting physical environments in the development of fraction concepts. Cogn. Sci. 29, 4 (2005), 587--625.Google ScholarCross Ref
- Maria Montessori. 1964. The Montessori Method. Transaction publishers.Google Scholar
- Raymond S. Nickerson. 1998. Confirmation bias: A ubiquitous phenomenon in many guises. Rev. Gen. Psychol. 2, 2 (1998), 175.Google ScholarCross Ref
- Claire O'Malley and Danae Stanton Fraser. 2004. Literature Review in Learning with Tangible Technologies. A NESTA Futurelab Research report - report 12.Google Scholar
- Georgios Olympiou and Zacharias C. Zacharia. 2012. Blending physical and virtual manipulatives: An effort to improve students’ conceptual understanding through science laboratory experimentation. Sci. Educ. 96, 1 (2012), 21--47. DOI:http://dx.doi.org/10.1002/sce.20463Google Scholar
- Karen J. Pine, Nicola Lufkin, and David Messer. 2004. More gestures than answers: Children learning about balance. Dev. Psychol. 40, 6 (2004), 1059.Google ScholarCross Ref
- Sara Price, George Roussos, Taciana Pontual Falcão, and Jennifer G. Sheridan. 2009. Technology and embodiment: Relationships and implications for knowledge, creativity and communication. Beyond Curr. Horiz. (2009), 85--92.Google Scholar
- Laurel Puchner, Robyn Rapoport, and Suzanne Gaskins. 2001. Learning in children's museums: Is it really happening? Curator Mus. J. 44, 3 (2001), 237--259.Google ScholarCross Ref
- Helen Quinn, Heidi Schweingruber, Thomas Keller, and others. 2012. A Framework for K-12 Science Education: Practices, Crosscutting Concepts, and Core Ideas. National Academies Press.Google Scholar
- Mitchel Resnick, et al. 1998a. Digital manipulatives. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI’98). 281--287. DOI:http://dx.doi.org/10.1145/274644.274684 Google ScholarDigital Library
- Mitchel Resnick, Fred Martin, Robert Berg, Rick Borovoy, Vanessa Colella, Kwin Kramer, and Brian Silverman. 1998b. Digital manipulatives: New toys to think with. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI’98). ACM Press/Addison-Wesley Publishing Co., New York, NY, USA, 281--287. DOI:http://dx.doi.org/10.1145/274644.274684 Google ScholarDigital Library
- John J. Rieser, Anne E. Garing, and Michael F. Young. 1994. Imagery, action, and young children's spatial orientation: It's not being there that counts, it's what one has in mind. Child Dev. 65, 5 (1994), 1262--1278.Google ScholarCross Ref
- Keith Roe and Daniel Mujis. 1998. Children and computer games -- A profile of the heavy user. Eur. J. Commun. 2, 13 (1998), 181--200.Google ScholarCross Ref
- Yvonne Rogers, Mike Scaife, Silvia Gabrielli, Hilary Smith, and Eric Harris. 2002. A conceptual framework for mixed reality environments: Designing novel learning activities for young children. Presence 11, 6 (2002), 677--686. DOI:http://dx.doi.org/10.1162/105474602321050776 Google ScholarDigital Library
- Kimiko Ryokai and Justine Cassell. 1999. StoryMat: A play space for collaborative storytelling. In Extended Abstracts on Human Factors in Computing Systems (CHI EA’99). ACM, New York, NY, USA, 272--273. DOI:http://dx.doi.org/10.1145/632716.632883 Google ScholarDigital Library
- Bertrand Schneider, Patrick Jermann, Guillaume Zufferey, and Pierre Dillenbourg. 2011. Benefits of a tangible interface for collaborative learning and interaction. IEEE Trans. Learn. Technol. 4, 3 (2011), 222--232. DOI:http://dx.doi.org/10.1109/TLT.2010.36 Google ScholarDigital Library
- Tia Shelley, Leilah Lyons, Moira Zellner, and Emily Minor. 2011. Evaluating the embodiment benefits of a paper-based tui for educational simulations. In Extended Abstracts on Human Factors in Computing Systems (CHI EA’11). ACM, New York, NY, USA, 1375--1380. DOI:http://dx.doi.org/10.1145/1979742.1979777 Google ScholarDigital Library
- D. Stanton, C. O’Malley, K. Huing, M. Fraser, and S. Benford. 2003. Situating historical events through mixed reality. In Barbara Wasson, Sten Ludvigsen, and Ulrich Hoppe (Eds.). Designing for Change in Networked Learning Environments: Proceedings of the International Conference on Computer Support for Collaborative Learning. Springer Netherlands, Dordrecht, 293--302. DOI:http://dx.doi.org/10.1007/978-94-017-0195-2_37Google Scholar
- Tiffany Tseng, Coram Bryant, and Paulo Blikstein. 2011. Collaboration through documentation: automated capturing of tangible constructions to support engineering design. In Proceedings of the 10th International Conference on Interaction Design and Children (IDC’11). ACM, New York, NY, USA, 118--126. DOI:http://dx.doi.org/10.1145/1999030.1999044 Google ScholarDigital Library
- B. Ullmer and H. Ishii. 2000. Emerging frameworks for tangible user interfaces. IBM Syst. J. 39, 3--4 (Jul. 2000), 915--931. DOI:http://dx.doi.org/10.1147/sj.393.0915 Google ScholarDigital Library
- David H. Uttal and Katherine O’ Doherty. 2008. Comprehending and learning from ‘visualizations’: A developmental perspective. In John K. Gilbert, Miriam Reiner, and Mary Nakhleh (Eds.). Visualization: Theory and Practice in Science Education. Dordrecht: Springer Netherlands, 53--72. DOI:http://dx.doi.org/10.1007/978-1-4020-5267-5_3Google Scholar
- Consuelo Valdes, Michelle Ferreirae, Taili Feng, Heidi Wang, Kelsey Tempel, Sirui Liu, and Orit Shaer. 2012. A collaborative environment for engaging novices in scientific inquiry. In Proceedings of the ACM International Conference on Interactive Tabletops and Surfaces (ITS’12). ACM, New York, NY, USA, 109--118. DOI:http://dx.doi.org/10.1145/2396636.2396654 Google ScholarDigital Library
- Lev Vygotsky. 1978. Interaction between learning and development. Read. Dev. Child. 23, 3 (1978), 34--41.Google Scholar
- Erin Walker and Winslow Burleson. 2012. Using need validation to design an intelligent tangible learning environment. Proc. ACM Annual Conference Extended Abstracts on Human Factors in Computing Systems (CHI EA’12), 2123. DOI:http://dx.doi.org/10.1145/2212776.2223763 Google ScholarDigital Library
- Nesra Yannier, Cagatay Basdogan, Serdar Tasiran, and Omer L. Sen. 2008. Using haptics to convey cause-and-effect relations in climate visualization. IEEE Trans. Haptics 1, 2 (2008), 130--141. DOI:http://dx.doi.org/10.1109/TOH.2008.16 Google ScholarDigital Library
- Nesra Yannier, Kenneth R. Koedinger, and Scott E. Hudson. 2015. Learning from mixed-reality games: Is shaking a tablet as effective as physical observation? In Proceedings of the 33rd Annual ACM Conference on Human Factors in Computing Systems (CHI’15). ACM, New York, NY, USA, 1045--1054. DOI:http://dx.doi.org/10.1145/2702123.2702397 Google ScholarDigital Library
- Nesra Yannier, Kenneth R. Koedinger, and Scott E. Hudson. 2013. Tangible collaborative learning with a mixed-reality game: EarthShake. In H. Chad Lane, Kalina Yacef, Jack Mostow, and Philip Pavlik, (Eds.). Artificial Intelligence in Education: Proceedings of the 16th International Conference (AIED’13) Memphis, TN, USA, July 9--13, 2013. Springer, Berlin, 131--140. DOI:http://dx.doi.org/10.1007/978-3-642-39112-5_14Google Scholar
- Nicola Yuill and Yvonne Rogers. 2012. Mechanisms for collaboration: A design and evaluation framework for multi-user interfaces. ACM Trans. Comput. Interact. 19, 1 (May 2012), 1:1--1:25. DOI:http://dx.doi.org/10.1145/2147783.2147784 Google ScholarDigital Library
Recommendations
Learning from Mixed-Reality Games: Is Shaking a Tablet as Effective as Physical Observation?
CHI '15: Proceedings of the 33rd Annual ACM Conference on Human Factors in Computing SystemsThe possibility of leveraging technology to support children's learning in the real world is both appealing and technically challenging. We have been exploring factors in tangible games that may contribute to both learning and enjoyment with an eye ...
Providing both physical and perceived affordances using physical games pieces on touch based tablets
IE '12: Proceedings of The 8th Australasian Conference on Interactive Entertainment: Playing the SystemWhilst capacitive touch screen phones and tablets, such as the iPhone and iPad, are increasingly becoming one of the main forms of gaming platform, the nature of the touch interface and the lack of physical feedback are seen as limitations. In this ...
Children as teachers: a tangible approach
ACE '11: Proceedings of the 8th International Conference on Advances in Computer Entertainment TechnologyThis paper describes an interactive tool that enables children to build their own educational games, based on physical objects with which they usually interact. The T-Games (Tangible Games) authoring tool allows children to use tangible objects for ...
Comments