Why doesn't it work?: voice-driven interfaces and young children's communication repair strategies

ABSTRACT

In this study, we examine the conversational repair strategies that preschoolers use to correct communication breakdowns with a voice-driven interface. We conducted a two-week deployment in the homes of 14 preschoolers of a tablet game that included a broken voice-driven mini-game. We collected 107 audio samples of these children's (unsuccessful) attempts to communicate with the mini-game. We found that children tried a common set of repair strategies, including repeating themselves and experimenting with the tone and pronunciation of their words. Children were persistent, rarely giving up on the interaction, asking for help, or showing frustration. When parents participated in the interaction, they moved through four phases of engagement: first making suggestions, then intervening, then making statements of resignation, and finally pronouncing that the interaction could not be repaired. Designers should anticipate that in this context, children will borrow behaviors from person-to-person communication, such as pivoting strategies to probe the source of failed communication and structuring communication into turn-taking attempts.

References

1. Sean Andrist, Iolanda Leite, and Jill Lehman. 2013. Fun and fair. In Proceedings of the 12th International Conference on Interaction Design and Children - IDC '13, 352--355.
2. Bonnie Brinton, Martin Fujiki, Diane Frome Loeb, and Erika Winkler. 1986. Development of conversational repair strategies in response to requests for clarification. Journal of Speech and Hearing Research 29, 1: 75--81.
3. Cga. Let's CoPlay! 2016 A Developer's Guide to Family Cooperative Mobile Play.
4. Eve V Clark. 2009. First language acquisition. Cambridge University Press.
5. Michael A. Forrester and Sarah M. Cherington. 2009. The development of other-related conversational skills: A case study of conversational repair during the early years. First Language 29, 2: 166--191.
6. Marina Fridin. 2014. Storytelling by a kindergarten social assistive robot: A tool for constructive learning in preschool education. Computers & Education 70: 53--64.
7. Tanya Gallagher. 1977. Revision behaviors in the speech of normal children developing language. Journal of Speech and Hearing Research 20, 2: 303--318.
8. Roberta Michnick Golinkoff. 1986. "I beg your pardon?": the preverbal negotiation of failed messages. Journal of Child Language 13, 3: 455--476.
9. Philip J. Hayes and D. Raj Reddy. 1983. Steps toward graceful interaction in spoken and written man-machine communication. International Journal of Man-Machine Studies 19, 3: 231--284.
10. Alexis Hiniker, Bongshin Lee, Kiley Sobel, and Eun Kyoung Choe. 2017. Plan & play: supporting intentional media use in early childhood. In Proceedings of the 2017 Conference on Interaction Design and Children - IDC '17, 85--95.
11. Hilary Hutchinson, Heiko Hansen, Nicolas Roussel, Björn Eiderbäck, Wendy Mackay, Bo Westerlund, Benjamin B. Bederson, Allison Druin, Catherine Plaisant, Michel Beaudouin-Lafon, Stéphane Conversy, and Helen Evans. 2003. Technology probes. In Proceedings of the conference on Human factors in computing systems - CHI '03, 17.
12. Peter H. Kahn, Takayuki Kanda, Hiroshi Ishiguro, Nathan G. Freier, Rachel L. Severson, Brian T. Gill, Jolina H. Ruckert, and Solace Shen. 2012. "Robovie, you'll have to go into the closet now": Children's social and moral relationships with a humanoid robot. Developmental Psychology 48, 2: 303--314.
13. Manja Lohse, Katharina J. Rohlfing, Britta Wrede, and Gerhard Sagerer. 2008. "Try something else!" --- When users change their discursive behavior in human-robot interaction. In 2008 IEEE International Conference on Robotics and Automation, 3481--3486.
14. Silvia Lovato and Anne Marie Piper. 2015. "Siri, is this you?" In Proceedings of the 14th International Conference on Interaction Design and Children - IDC '15, 335--338.
15. Ernst Moerk. Principles of interaction in language learning. Merrill-Palmer Quarterly of Behavior and Development 18, 229--257.
16. Clifford Nass and Youngme Moon. 2000. Machines and Mindlessness: Social Responses to Computers. Journal of Social Issues 56, 1: 81--103.
17. Dave Paresh. 2018. Which voice in your fridge? Makers pick virtual assistants. Thomson Reuters. Retrieved January 16, 2018 from https://www.reuters.com/article/us-tech-ces-assistants/which-voice-in-your-fridge-makers-pick-virtual-assistants-idUSKBN1F12OQ
18. Karen Petty. Developmental milestones of Young Children, Revised Edition. Retrieved April 2, 2018 from https://www.redleafpress.org/Developmental-Milestones-of-Young-Children-Revised-Edition-P1411.aspx
19. L. Plowman, O. Stevenson, J. McPake, C. Stephen, and C. Adey. 2011. Parents, pre-schoolers and learning with technology at home: some implications for policy. Journal of Computer Assisted Learning 27, 4: 361--371.
20. Amanda Purington, Jessie G. Taft, Shruti Sannon, Natalya N. Bazarova, and Samuel Hardman Taylor. 2017. "Alexa is my new BFF." In Proceedings of the 2017 CHI Conference Extended Abstracts on Human Factors in Computing Systems - CHI EA '17, 2853--2859.
21. Eric E Rasmussen, Autumn Shafer, Malinda J. Colwell, Shawna White, Narissra Punyanunt-Carter, Rebecca L. Densley, and Holly Wright. 2016. Relation between active mediation, exposure to Daniel Tiger's Neighborhood, and US preschoolers' social and emotional development. Journal of Children and Media 10, 4: 443--461.
22. Karen Rust, Meethu Malu, Lisa Anthony, and Leah Findlater. 2014. Understanding childdefined gestures and children's mental models for touchscreen tabletop interaction. In Proceedings of the 2014 conference on Interaction design and children - IDC '14, 201--204. Retrieved from http://dl.acm.org/citation.cfm?id=2593968.2610452
23. Harvey Sacks, Emanuel A. Schegloff, and Gail Jefferson. 1974. A Simplest Systematics for the Organization of Turn-Taking for Conversation. Language 50, 4: 696.
24. D. N. Stern, S. Spieker, R. K. Barnett, and K. MacKain. 1983. The prosody of maternal speech: infant age and context related changes. Journal of Child Language 10, 1: 1--15.
25. Lucy A Suchman. 1987. Plans and situated actions: The problem of human-machine communication. Cambridge university press.
26. Lori Takeuchi, Reed Stevens, and others. 2011. The new coviewing: Designing for learning through joint media engagement. In New York, NY: The Joan Ganz Cooney Center at Sesame Workshop.
27. Yumiko Tamura, Mitsuhiko Kimoto, Masahiro Shiomi, Takamasa Iio, Katsunori Shimohara, and Norihiro Hagita. 2017. Effects of a Listener Robot with Children in Storytelling. In Proceedings of the 5th International Conference on Human Agent Interaction - HAI '17, 35--43.
28. Michael Tomasello, Michael Jeffrey Farrar, and Jennifer Dines. 1984. Children's Speech Revisions for a Familiar and an Unfamiliar Adult. Journal of Speech Language and Hearing Research 27, 3: 359.
29. Amy M Wetherby and Barry M Prizant. 1993. Communication and symbolic behavior scales: Manual. Riverside Publishing.
30. John M. Wiemann and Mark L. Knapp. 1975. Turn-taking in Conversations. Journal of Communication 25, 2: 75--92.
31. Jacob O. Wobbrock, Meredith Ringel Morris, and Andrew D. Wilson. 2009. User-defined gestures for surface computing. In Proceedings of the 27th international conference on Human factors in computing systems - CHI 09, 1083.
32. Selma Yilmazyildiz, Robin Read, Tony Belpeame, and Werner Verhelst. 2016. Review of Semantic-Free Utterances in Social Human-Robot Interaction. International Journal of Human-Computer Interaction 32, 1: 63--85.
33. Amazon.com: The Magic Door: Alexa Skills. Retrieved January 16, 2018 from https://www.amazon.com/The-Magic-Door-LLC/dp/B01BMUU6JQ
34. Cookie Monster's Challenge Mobile Downloads | PBS KIDS. Retrieved January 14, 2018 from http://pbskids.org/apps/cookie-monsters-challenge.html