Abstract
The benefits of personalized social robots must be evaluated in real-world educational contexts over periods of time longer than a single session to understand their full potential to impact learning outcomes. In this work, we describe a personalization system designed for longer-term personalization that orders curriculum based on an adaptive Hidden Markov Model (HMM) that evaluates students’ skill proficiencies. We present a study investigating the effectiveness of this system in a five-session interaction with a robot tutor, taking place over the course of 2 weeks. Our system is evaluated in the context of native Spanish-speaking first-graders interacting with a social robot tutor while completing an English Language Learning educational task. Participants either received lessons: (1) ordered by our adaptive HMM personalization system which selects a lesson based on a skill that the individual participant needs more practice with (“personalized condition”) or (2) ordered randomly from among the lessons the participant had not yet seen (“non-personalized condition”). We found that participants who received personalized lessons from the robot tutor outperformed participants who received non-personalized lessons on a post-test by 2.0 standard deviations on average, corresponding to a mean learning gain in the 98th percentile.
- Minoo Alemi, Ali Meghdari, and Maryam Ghazisaedy. 2014. Employing humanoid robots for teaching English language in Iranian junior high-schools. Int. J. Human. Robot. 11, 3 (2014).Google ScholarCross Ref
- Wilma A. Bainbridge, Justin Hart, Elizabeth S. Kim, and Brian Scassellati. 2008. The effect of presence on human-robot interaction. In Proceedings of the 17th IEEE International Symposium on Robot and Human Interactive Communication (RO-MAN’08). IEEE, 701--706.Google ScholarCross Ref
- R. S. Baker, A. T. Corbett, and K. R. Koedinger. 2004. Detecting student misuse of intelligent tutoring systems. In Intelligent Tutoring Systems. Springer, 531--540.Google Scholar
- Paul Baxter, Emily Ashurst, Robin Read, James Kennedy, and Tony Belpaeme. 2017. Robot education peers in a situated primary school study: Personalisation promotes child learning. PLOS ONE 12, 5 (2017), e0178126.Google ScholarCross Ref
- Tony Belpaeme, James Kennedy, Aditi Ramachandran, Brian Scassellati, and Fumihide Tanaka. 2018. Social robots for education: A review. Sci. Robot. 3, 21 (2018). http://robotics.sciencemag.org/content/3/21/eaat5954.Google Scholar
- Tony Belpaeme, Paul Vogt, Rianne van den Berghe, Kirsten Bergmann, Tilbe Göksun, Mirjam de Haas, Junko Kanero, James Kennedy, Aylin C Küntay, Ora Oudgenoeg-Paz, et al. 2018. Guidelines for designing social robots as second language tutors. Int. J. Soc. Robot. 10, 3 (2018), 325--341.Google ScholarCross Ref
- R. M. Callahan. 2005. Tracking and high school English learners: Limiting opportunity to learn. Am. Educ. Res. J. 42, 2 (2005), 305--328.Google ScholarCross Ref
- M. Chen and K. Zechner. 2011. Computing and evaluating syntactic complexity features for automated scoring of spontaneous non-native speech. In Proceedings of the 49th Annual Meeting of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (HLT’11). Association for Computational Linguistics, Stroudsburg, PA, 722--731. http://dl.acm.org/citation.cfm?id=2002472.2002564 Google ScholarDigital Library
- Caitlyn Clabaugh, Gisele Ragusa, Fei Sha, and Maja Matarić. 2015. Designing a socially assistive robot for personalized number concepts learning in preschool children. In Proceedings of the 2015 Joint IEEE International Conference on Development and Learning and Epigenetic Robotics (ICDL-EpiRob’15). IEEE, 314--319.Google ScholarCross Ref
- C. Conati and H. Maclaren. 2009. Empirically building and evaluating a probabilistic model of user affect. User Model. User-Adapt. Interact. 19, 3 (2009), 267--303. Google ScholarDigital Library
- Alexandre Coninx, Paul Baxter, Elletra Oleari, Sara Bellini, Bert Bierman, Olivier Blanson-Henkemans, Lola Canamero, Piero Cosi, Valentin Enescu, Raquel Ros, Antoine Hiolle, Remi Humbert, Bernd Kiefer, Ivana Kruijff-Korbayova, Rosemarijn Looije, Marco Mosconi, Mark Neerincx, Giulio Paci, Georgios Patsis, Clara Pozzi, Francesca Sacchitelli, Hichem Sahli, Alberto Sanna, Giacomo Sommavilla, Fabio Tesser, Yiannis Demiris, and Tony Belpaeme. 2016. Towards long-term social child-robot interaction: Using multi-activity switching to engage young users. J. Hum.-Robot Interact. 5, 1 (2016), 32--67. Google ScholarDigital Library
- Albert T. Corbett and John R. Anderson. 1994. Knowledge tracing: Modeling the acquisition of procedural knowledge. User Model. User-Adapt. Interact. 4, 4 (1994), 253--278.Google ScholarCross Ref
- Jan de Wit, Thorsten Schodde, Bram Willemsen, Kirsten Bergmann, Mirjam de Haas, Stefan Kopp, Emiel Krahmer, and Paul Vogt. 2018. The effect of a robot’s gestures and adaptive tutoring on children’s acquisition of second language vocabularies. In Proceedings of the 2018 ACM/IEEE International Conference on Human-Robot Interaction. ACM, 50--58. Google ScholarDigital Library
- M. C. Desmarais and R. S. Baker. 2012. A review of recent advances in learner and skill modeling in intelligent learning environments. User Model. User-Adapt.Interact. 22, 1--2 (Apr. 2012), 9--38. Google ScholarDigital Library
- S. D’Mello. 2012. Monitoring affective trajectories during complex learning. In Encyclopedia of the Sciences of Learning. Springer, 2325--2328.Google Scholar
- M. Eskenazi. 2009. An overview of spoken language technology for education. Speech Commun. 51, 10 (2009), 832--844. Google ScholarDigital Library
- G. D. Forney Jr. 1973. The Viterbi algorithm. Proc. IEEE 61, 3 (1973), 268--278.Google ScholarCross Ref
- Goren Gordon and Cynthia Breazeal. 2015. Bayesian active learning-based robot tutor for children’s word-reading skills. In Proceedings of the AAAI Conference on Artificial Intelligence (AAAI’15). 1343--1349. Google ScholarDigital Library
- Goren Gordon, Samuel Spaulding, Jacqueline Kory Westlund, Jin Joo Lee, Luke Plummer, Marayna Martinez, Madhurima Das, and Cynthia Breazeal. 2016. Affective personalization of a social robot tutor for children’s second language skills. In Proceedings of the AAAI Conference on Artificial Intelligence (AAAI’16). 3951--3957. Google ScholarDigital Library
- Jillian Greczek, Elaine Short, Caitlyn E. Clabaugh, Katelyn Swift-Spong, and Maja Mataric. 2014. Socially assistive robotics for personalized education for children. In Proceedings of the AAAI Fall Symposium on Artificial Intelligence and Human-Robot Interaction (AI-HRI’14).Google Scholar
- K. E. Hogan and M. E. Pressley. 1997. Scaffolding Student Learning: Instructional Approaches and Issues. Brookline Books.Google Scholar
- Iris Howley, Takayuki Kanda, Kotaro Hayashi, and Carolyn Rosé. 2014. Effects of social presence and social role on help-seeking and learning. In Proceedings of the 2014 ACM/IEEE International Conference on Human-Robot Interaction. ACM, 415--422. Google ScholarDigital Library
- K. Humes, N. A. Jones, and R. R. Ramirez. 2011. Overview of Race and Hispanic Origin, 2010. U.S. Department of Commerce.Google Scholar
- J. S. Johnson and E. L. Newport. 1989. Critical period effects in second language learning: The influence of maturational state on the acquisition of English as a second language. Cogn. Psychol. 21, 1 (1989), 60--99.Google ScholarCross Ref
- Aidan Jones and Ginevra Castellano. 2018. Adaptive robotic tutors that support self-regulated learning: A longer-term investigation with primary school children. Int. J. Soc. Robot. 10, 3 (2018), 357--370.Google ScholarCross Ref
- Takayuki Kanda, Takayuki Hirano, Daniel Eaton, and Hiroshi Ishiguro. 2004. Interactive robots as social partners and peer tutors for children: A field trial. Hum.-Comput. Interact. 19, 1 (2004), 61--84. Google ScholarDigital Library
- James Kennedy, Paul Baxter, Emmanuel Senft, and Tony Belpaeme. 2015. Higher nonverbal immediacy leads to greater learning gains in child-robot tutoring interactions. In Proceedings of the International Conference on Social Robotics. Springer, 327--336.Google ScholarCross Ref
- James Kennedy, Paul Baxter, Emmanuel Senft, and Tony Belpaeme. 2016. Social robot tutoring for child second language learning. In Proceedings of the 11th ACM/IEEE International Conference on Human-Robot Interaction (HRI’16). IEEE, 231--238. Google ScholarDigital Library
- Kenneth R. Koedinger. 2002. Toward evidence for instructional design principles: Examples from cognitive tutor math 6. In Proceedings of the Annual Meeting of the North American Chapter of the International Group for the Psychology of Mathematics Education. ERIC, 21--49.Google Scholar
- C. C. Kulik, J. A. Kulik, and R. L. Bangert-Drowns. 1990. Effectiveness of mastery learning programs: A meta-analysis. Rev. Educ. Res. 60, 2 (1990), 265--299.Google ScholarCross Ref
- Sungjin Lee, Hyungjong Noh, Jonghoon Lee, Kyusong Lee, Gary Geunbae Lee, Seongdae Sagong, and Munsang Kim. 2011. On the effectiveness of robot-assisted language learning. ReCALL 23, 1 (2011), 25--58. Google ScholarDigital Library
- Iolanda Leite, Ginevra Castellano, Andre Pereira, Carlos Martinho, and Ana Paiva. 2012. Long-term interactions with empathic robots: Evaluating perceived support in children. In International Conference on Social Robotics. Springer, 298--307. Google ScholarDigital Library
- Iolanda Leite, Ginevra Castellano, André Pereira, Carlos Martinho, and Ana Paiva. 2014. Empathic robots for long-term interaction. Int. J. Soc. Robot. 6, 3 (2014), 329--341.Google ScholarCross Ref
- M. Levy. 1997. Computer-Assisted Language Learning: Context and Conceptualization. ERIC.Google Scholar
- M. Levy and G. Stockwell. 2013. CALL Dimensions: Options and Issues in Computer-assisted Language Learning. Routledge.Google Scholar
- Daniel Leyzberg, Samuel Spaulding, and Brian Scassellati. 2014. Personalizing robot tutors to individuals’ learning differences. In Proceedings of the 2014 ACM/IEEE International Conference on Human-Robot Interaction. ACM, 423--430. Google ScholarDigital Library
- D. Leyzberg, S. Spaulding, M. Toneva, and B. Scassellati. 2012. The physical presence of a robot tutor increases cognitive learning gains. In Proceedings of the 34th Annual Conference of the Cognitive Science Society. Cognitive Science Society, Austin, TX.Google Scholar
- Robert V. Lindsey, Jeffery D. Shroyer, Harold Pashler, and Michael C. Mozer. 2014. Improving students’ long-term knowledge retention through personalized review. Psychol. Sci. 25, 3 (2014), 639--647.Google ScholarCross Ref
- K. Littleton and P. Light. 1999. Learning with Computers: Analysing Productive Interaction. Psychology Press. Google ScholarDigital Library
- D. C. Merrill, R. J. Reiser, M. Ranney, and J. G. Trafton. 1992. Effective tutoring techniques: A comparison of human tutors and intelligent tutoring systems. J. Learn. Sci. 2, 3 (1992), pp. 277--305.Google ScholarCross Ref
- Philip I. Pavlik and John R. Anderson. 2005. Practice and forgetting effects on vocabulary memory: An activation-based model of the spacing effect. Cogn. Sci. 29, 4 (2005), 559--586.Google ScholarCross Ref
- Philip I. Pavlik and John R. Anderson. 2008. Using a model to compute the optimal schedule of practice.J. Exp. Psychol. Appl. 14, 2 (2008), 101.Google ScholarCross Ref
- André Pereira, Carlos Martinho, Iolanda Leite, and Ana Paiva. 2008. iCat, the chess player: The influence of embodiment in the enjoyment of a game. In Proceedings of the 7th International Joint Conference on Autonomous Agents and Multiagent Systems, Volume 3. International Foundation for Autonomous Agents and Multiagent Systems, 1253--1256. Google ScholarDigital Library
- Aaron Powers, Sara Kiesler, Susan Fussell, and Cristen Torrey. 2007. Comparing a computer agent with a humanoid robot. In Proceedings of the 2nd ACM/IEEE International Conference on Human-Robot Interaction (HRI’07). IEEE, 145--152. Google ScholarDigital Library
- Aditi Ramachandran, Chien-Ming Huang, and Brian Scassellati. 2017. Give me a break!: Personalized timing strategies to promote learning in robot-child tutoring. In Proceedings of the 2017 ACM/IEEE International Conference on Human-Robot Interaction. ACM, 146--155. Google ScholarDigital Library
- Aditi Ramachandran, Alexandru Litoiu, and Brian Scassellati. 2016. Shaping productive help-seeking behavior during robot-child tutoring interactions. In Proceedings of the 11th ACM/IEEE International Conference on Human-Robot Interaction. IEEE Press, 247--254. Google ScholarDigital Library
- G. Saunders. 1988. Bilingual Children: From Birth to Teens. ERIC.Google Scholar
- Thorsten Schodde, Kirsten Bergmann, and Stefan Kopp. 2017. Adaptive robot language tutoring based on Bayesian knowledge tracing and predictive decision-making. In Proceedings of the 2017 ACM/IEEE International Conference on Human-Robot Interaction. ACM, 128--136. Google ScholarDigital Library
- Sofia Serholt, Christina Anne Basedow, Wolmet Barendregt, and Mohammad Obaid. 2014. Comparing a humanoid tutor to a human tutor delivering an instructional task to children. In Proceedings of the 14th IEEE-RAS International Conference on Humanoid Robots (Humanoids’14). IEEE, 1134--1141.Google ScholarCross Ref
- Daniel Szafir and Bilge Mutlu. 2012. Pay attention!: Designing adaptive agents that monitor and improve user engagement. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI’12). ACM, New York, NY, 11--20. Google ScholarDigital Library
- Fumihide Tanaka, Aaron Cicourel, and Javier R. Movellan. 2007. Socialization between toddlers and robots at an early childhood education center. In Proceedings of the National Academy of Sciences 104, 46 (2007), 17954--17958.Google ScholarCross Ref
- Muhammad Tanveer. 2007. Investigation of the factors that cause language anxiety for ESL/EFL learners in learning speaking skills and the influence it casts on communication in the target language. Master's thesis. University of Glasgow, Scotland, UK.Google Scholar
- Adriana Tapus, Cristian Tapus, and Maja Mataric. 2009. The role of physical embodiment of a therapist robot for individuals with cognitive impairments. In Proceedings of the 18th IEEE International Symposium on Robot and Human Interactive Communication (RO-MAN’09). IEEE, 103--107.Google ScholarCross Ref
- Wayne P. Thomas and Virginia P. Collier. 2002. A National Study of School Effectiveness for Language Minority Students' Long-Term Academic Achievement. Technical Report. University of California at Santa Cruz, Center for Research on Education, Diversity, and Excellence.Google Scholar
- U.S. Census Bureau. 2011. 2010 Census. U.S. Department of Commerce.Google Scholar
- Joshua Wainer, David J. Feil-Seifer, Dylan A. Shell, and Maja J. Mataric. 2007. Embodiment and human-robot interaction: A task-based perspective. In Proceedings of the 16th IEEE International Symposium on Robot and Human Interactive Communication (RO-MAN’07). IEEE, 872--877.Google Scholar
- L. R. Welch. 2003. Hidden Markov models and the Baum-Welch algorithm. IEEE Inf. Theory Soc. Newslett. 53, 4 (2003), 10--13.Google Scholar
- S. M. Williams, D. Nix, and P. Fairweather. 2013. Using speech recognition technology to enhance literacy instruction for emerging readers. In Proceedings of the 4th International Conference of the Learning Sciences. 115--120.Google Scholar
Index Terms
- The Effect of Personalization in Longer-Term Robot Tutoring
Recommendations
Personalizing robot tutors to individuals' learning differences
HRI '14: Proceedings of the 2014 ACM/IEEE international conference on Human-robot interactionIn education research, there is a widely-cited result called "Bloom's two sigma" that characterizes the differences in learning outcomes between students who receive one-on-one tutoring and those who receive traditional classroom instruction. Tutored ...
Thinking Aloud with a Tutoring Robot to Enhance Learning
HRI '18: Proceedings of the 2018 ACM/IEEE International Conference on Human-Robot InteractionThinking aloud, while requiring extra mental effort, is a metacognitive technique that helps students navigate through complex problem-solving tasks. Social robots, bearing embodied immediacy that fosters engaging and compliant interactions, are a ...
Toward Effective Robot--Child Tutoring: Internal Motivation, Behavioral Intervention, and Learning Outcomes
Personalized learning environments have the potential to improve learning outcomes for children in a variety of educational domains, as they can tailor instruction based on the unique learning needs of individuals. Robot tutoring systems can further ...
Comments