Spontaneous Interactions with a Virtually Embodied Intelligent Assistant in Minecraft

ABSTRACT

An increasing number of our technological interactions are mediated through virtually embodied characters and software agents powered by machine learning. To understand how users relate to and evaluate these types of interfaces, we designed a Wizard of Oz prototype of an embodied agent in Minecraft that learns from users' actions, and conducted a user study with 18 school-aged Minecraft players. We categorised nine main ways users spontaneously attempted to interact with and teach the agent: four using game controls, and five using natural language text input. This study lays groundwork for a better understanding of human interaction with learning agents in virtual worlds.

References

1. Saleema Amershi, Maya Cakmak, W Bradley Knox, and Todd Kulesza. 2014. Power to the People: The Role of Humans in Interactive Machine Learning. AI Magazine 35, 4.
2. Jasmin Bernotat, Birte Schiffhauer, Friederike Eyssel, et al. 2016. Welcome to the Future -- How Naïve Users Intuitively Address an Intelligent Robotics Apartment. In A. Agah, J.-J. Cabibihan, A.M. Howard, M.A. Salichs, and H. He, eds., Social Robotics: 8th International Conference, ICSR 2016, Kansas City, MO, USA, November 1--3, 2016 Proceedings. Springer International Publishing, Cham, 982--992.
3. Roland Buchner, Daniela Wurhofer, Astrid Weiss, and Manfred Tscheligi. 2013. Robots in Time: How User Experience in Human-Robot Interaction Changes over Time. Proceedings of the 5th International Conference on Social Robotics, Springer, 138--147.
4. Kerstin Fischer, Katrin S Lohan, Chrystopher Nehaniv, and Hagen Lehmann. 2013. Effects of Different Kinds of Robot Feedback. Proceedings of the 5th International Conference on Social Robotics, Springer, 260--269.
5. Susanne Frennert, Håkan Eftring, and Britt Östlund. 2013. What Older People Expect of Robots: A Mixed Methods Approach. Proceedings of the 5th International Conference on Social Robotics, Springer, 19--29.
6. James Paul Gee and Judith L. Green. 1998. Discourse Analysis, Learning, and Social Practice: A Methodological Study. Review of Research in Education 23: 119--169.
7. Michael A. Goodrich and Alan C. Schultz. 2008. Human--Robot Interaction: A Survey. Foundations and Trends® in Human--Computer Interaction 1, 3: 203--275.
8. Kristina Höök. 2000. Steps to Take Before Intelligent User Interfaces Become Real. Interacting with Computers 12, 4: 409--426.
9. Matthew Johnson, Katja Hofmann, Tim Hutton, and David Bignell. 2016. The Malmo Platform for Artificial Intelligence Experimentation. Proceedings of the Twenty-Fifth International Joint Conference on Artificial Intelligence (IJCAI-16), 4246--4247.
10. Tasneem Kaochar, Raquel Torres Peralta, Clayton T. Morrison, Ian R. Fasel, Thomas J. Walsh, and Paul R. Cohen. 2011. Towards Understanding How Humans Teach Robots. Proceedings of the 19th International Conference on User Modeling, Adaption, and Personalization, Springer-Verlag, 347--352.
11. W. Bradley Knox, Brian D. Glass, Bradley C. Love, W. Todd Maddox, and Peter Stone. 2012. How Humans Teach Agents. International Journal of Social Robotics 4, 4: 409--421.
12. Nathan Koenig, Leila Takayama, and Maja Matarić. 2010. Communication and Knowledge Sharing in Human--Robot Interaction and Learning from Demonstration. Neural Networks 23, 8--9: 1104-- 1112.
13. Jean Lave and Etienne Wenger. 1991. Situated Learning: Legitimate Peripheral Participation. Cambridge University Press.
14. Pattie Maes. 1994. Agents That Reduce Work and Information Overload. Communications of the ACM 37, 30--40.
15. David Maulsby, Saul Greenberg, and Richard Mander. 1993. Prototyping an Intelligent Agent through Wizard of Oz. Proceedings of the INTERACT '93 and CHI '93 Conference on Human Factors in Computing Systems, ACM, 277--284.
16. Donald A. Norman. 1994. How Might People Interact with Agents. Communications of the ACM 37, 7: 68--71.
17. Laurel Riek. 2012. Wizard of Oz Studies in HRI: A Systematic Review and New Reporting Guidelines. Journal of Human-Robot Interaction 1, 1: 119-- 136.
18. Ben Shneiderman. 1982. The Future of Interactive Systems and the Emergence of Direct Manipulation. Behaviour & Information Technology 1, 3: 237--256.
19. Gerry Stahl. 2013. Theories of Cognition in Collaborative Learning. In C.E. Hmelo-Silver, ed., The International Handbook of Collaborative Learning. Routledge, New York, NY, USA, 74--90.
20. Yong Xu, Yoshimasa Ohmoto, Kazuhiro Ueda, et al. 2011. Active Adaptation in Human-Agent Collaborative Interaction. Journal of Intelligent Information Systems 37, 1: 23--38.

Index Terms

• Human-centered computing

  • Human computer interaction (HCI)