Inferring Cognitive Models from Data using Approximate Bayesian Computation

ABSTRACT

An important problem for HCI researchers is to estimate the parameter values of a cognitive model from behavioral data. This is a difficult problem, because of the substantial complexity and variety in human behavioral strategies. We report an investigation into a new approach using approximate Bayesian computation (ABC) to condition model parameters to data and prior knowledge. As the case study we examine menu interaction, where we have click time data only to infer a cognitive model that implements a search behaviour with parameters such as fixation duration and recall probability. Our results demonstrate that ABC (i) improves estimates of model parameter values, (ii) enables meaningful comparisons between model variants, and (iii) supports fitting models to individual users. ABC provides ample opportunities for theoretical HCI research by allowing principled inference of model parameter values and their uncertainty.

References

1. John R. Anderson. 1991. Is human cognition adaptive? Behavioral and Brain Sciences 14, 3 (1991), 471--485. DOI:http://dx.doi.org/10.1017/S0140525X00070801
2. John R. Anderson, Michael Matessa, and Christian Lebiere. 1997. ACT-R: A theory of higher level cognition and its relation to visual attention. Human--Computer Interaction 12, 4 (1997), 439--462. DOI: http://dx.doi.org/10.1207/s15327051hci1204_5
3. Leif Azzopardi. 2014. Modelling interaction with economic models of search. In Proceedings of the 37th International ACM SIGIR Conference on Research & Development in Information Retrieval. ACM, 3--12. DOI: http://dx.doi.org/10.1145/2600428.2609574
4. Myroslav Bachynskyi, Gregorio Palmas, Antti Oulasvirta, and Tino Weinkauf. 2015. Informing the design of novel input methods with muscle coactivation clustering. ACM Transactions on Computer-Human Interaction 21, 6 (2015), 30:1--30:25. DOI: http://dx.doi.org/10.1145/2687921
5. Gilles Bailly, Antti Oulasvirta, Duncan P. Brumby, and Andrew Howes. 2014. Model of visual search and selection time in linear menus. In Proceedings of the 32nd Annual ACM Conference on Human Factors in Computing Systems. ACM, 3865--3874. DOI: http://dx.doi.org/10.1145/2556288.2557093
6. Robert W. Baloh, Andrew W. Sills, Warren E. Kumley, and Vicente Honrubia. 1975. Quantitative measurement of saccade amplitude, duration, and velocity. Neurology 25, 11 (1975), 1065--1065. DOI: http://dx.doi.org/10.1212/WNL.25.11.1065
7. James L. Beck and Ka-Veng Yuen. 2004. Model selection using response measurements: Bayesian probabilistic approach. Journal of Engineering Mechanics 130, 2 (2004), 192--203. DOI:http: //dx.doi.org/10.1061/(ASCE)0733--9399(2004)130:2(192)
8. Duncan P. Brumby, Anna L. Cox, Jacqueline Chung, and Byron Fernandes. 2014. How does knowing what you are looking for change visual search behavior?. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems. ACM, 3895--3898. DOI: http://dx.doi.org/10.1145/2556288.2557064
9. Michael D. Byrne. 2001. ACT-R/PM and menu selection: Applying a cognitive architecture to HCI. International Journal of Human--Computer Studies 55, 1 (2001), 41--84. DOI:http://dx.doi.org/10.1006/ijhc.2001.0469
10. Colin Camerer. 2003. Behavioral game theory: Experiments in strategic interaction. Princeton University Press.
11. Stuart K. Card, Allen Newell, and Thomas P. Moran. 1983. The psychology of human--computer interaction. L. Erlbaum Associates Inc.
12. Nick Chater and Mike Oaksford. 1999. Ten years of the rational analysis of cognition. Trends in Cognitive Sciences 3, 2 (1999), 57--65. DOI: http://dx.doi.org/10.1016/S1364--6613(98)01273-X
13. Xiuli Chen, Gilles Bailly, Duncan P. Brumby, Antti Oulasvirta, and Andrew Howes. 2015. The Emergence of Interactive Behavior: A Model of Rational Menu Search. In Proceedings of the 33rd Annual ACM Conference on Human Factors in Computing Systems. ACM, 4217--4226. DOI:http://dx.doi.org/10.1145/2702123.2702483
14. Andy Cockburn, Per O. Kristensson, Jason Alexander, and Shumin Zhai. 2007. Hard lessons: Effort-inducing interfaces benefit spatial learning. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems. ACM, 1571--1580. DOI: http://dx.doi.org/10.1145/1240624.1240863
15. Katalin Csilléry, Michael G.B. Blum, Oscar E. Gaggiotti, and Olivier François. 2010. Approximate Bayesian computation (ABC) in practice. Trends in Ecology & Evolution 25, 7 (2010), 410--418. DOI: http://dx.doi.org/10.1016/j.tree.2010.04.001
16. Wai-Tat Fu and Peter Pirolli. 2007. SNIF-ACT: A cognitive model of user navigation on the World Wide Web. Human--Computer Interaction 22, 4 (2007), 355--412. http://www.tandfonline.com/doi/abs/10.1080/ 07370020701638806
17. Samuel J. Gershman, Eric J. Horvitz, and Joshua B. Tenenbaum. 2015. Computational rationality: A converging paradigm for intelligence in brains, minds, and machines. Science 349, 6245 (2015), 273--278. DOI: http://dx.doi.org/10.1126/science.aac6076
18. Michael U. Gutmann and Jukka Corander. 2016. Bayesian optimization for likelihood-free inference of simulator-based statistical models. Journal of Machine Learning Research 17, 125 (2016), 1--47. http://jmlr.org/papers/v17/15-017.html
19. Tim Halverson and Anthony J. Hornof. 2011. A computational model of "active vision" for visual search in human--computer interaction. Human--Computer Interaction 26, 4 (2011), 285--314. DOI: http://dx.doi.org/10.1080/07370024.2011.625237
20. Mary Hayhoe and Dana Ballard. 2014. Modeling task control of eye movements. Current Biology 24, 13 (2014), R622--R628. DOI: http://dx.doi.org/10.1016/j.cub.2014.05.020
21. Anthony J. Hornof. 2004. Cognitive strategies for the visual search of hierarchical computer displays. Human--Computer Interaction 19, 3 (2004), 183--223. DOI:http://dx.doi.org/10.1207/s15327051hci1903_1
22. Andrew Howes, Richard L. Richard L Lewis, and Alonso Vera. 2009. Rational adaptation under task and processing constraints: Implications for testing theories of cognition and action. Psychological Review 116, 4 (2009), 717--751. DOI:http://dx.doi.org/10.1037/a0017187
23. Richard J. Jagacinski and John M. Flach. 2003. Control theory for humans: Quantitative approaches to modeling performance. CRC Press.
24. Antti Kangasrääsiö, Jarno Lintusaari, Kusti Skytén, Marko Järvenpää, Henri Vuollekoski, Michael Gutmann, Aki Vehtari, Jukka Corander, and Samuel Kaski. 2016. ELFI: Engine for likelihood-free inference (extended abstract). In NIPS 2016 Workshop on Advances in Approximate Bayesian Inference. https://github.com/hiit/elfi
25. David E. Kieras and Anthony J. Hornof. 2014. Towards accurate and practical predictive models of active-vision-based visual search. In Proceedings of the 32nd Annual ACM Conference on Human Factors in Computing Systems. ACM, 3875--3884. DOI: http://dx.doi.org/10.1145/2556288.2557324
26. David E. Kieras and David E. Meyer. 1997. An overview of the EPIC architecture for cognition and performance with application to human--computer interaction. Human--Computer Interaction 12, 4 (1997), 391--438. DOI:http://dx.doi.org/10.1207/s15327051hci1204_4
27. David E. Kieras, David E. Meyer, James A. Ballas, and Erick J. Lauber. 2000. Modern computational perspectives on executive mental processes and cognitive control: Where to from here? Control of Cognitive Processes: Attention and Performance XVIII (2000), 681--712.
28. Richard L. Lewis, Andrew Howes, and Satinder Singh. 2014. Computational rationality: Linking mechanism and behavior through bounded utility maximization. Topics in Cognitive Science 6, 2 (2014), 279--311. DOI: http://dx.doi.org/10.1111/tops.12086
29. Craig S. Miller and Roger W. Remington. 2004. Modeling information navigation: Implications for information architecture. Human--Computer Interaction 19, 3 (2004), 225--271. DOI: http://dx.doi.org/10.1207/s15327051hci1903_2
30. Lewis R. L. Howes A. Myers, C. W. 2013. Bounded optimal state estimation and control in visual search: explaining distractor ratio effects. In Proceedings of the 35th Annual Conference of the Cognitive Science Society. Cognitive Science Society.
31. Jay I. Myung and Mark A. Pitt. 2016. Model comparison in psychology. In The Stevens' Handbook of Experimental Psychology and Cognitive Neuroscience (Fourth Edition), Volume 5: Methodology. John Wiley & Sons. To appear.
32. Andrew Y. Ng and Stuart J. Russell. 2000. Algorithms for inverse reinforcement learning. In Proceedings of the Seventeenth International Conference on Machine Learning. 663--670.
33. Erik Nilsen and Jake Evans. 1999. Exploring the divide between two unified theories of cognition: Modeling visual attention in menu selection. In CHI'99 Extended Abstracts on Human Factors in Computing Systems. ACM, 288--289. DOI: http://dx.doi.org/10.1145/632716.632893
34. Jose Nunez-Varela and Jeremy L. Wyatt. 2013. Models of gaze control for manipulation tasks. ACM Transactions on Applied Perception 10, 4 (2013), 20:1--20:22. DOI: http://dx.doi.org/10.1145/2536764.2536767
35. Mike Oaksford and Nick Chater. 1994. A rational analysis of the selection task as optimal data selection. Psychological Review 101, 4 (1994), 608--631. DOI: http://dx.doi.org/10.1037/0033--295X.101.4.608
36. Stephen J. Payne and Andrew Howes. 2013. Adaptive interaction: A utility maximization approach to understanding human interaction with technology. Synthesis Lectures on Human-Centered Informatics 6, 1 (2013), 1--111. DOI: http://dx.doi.org/10.2200/S00479ED1V01Y201302HCI016
37. Peter Pirolli. 2005. Rational analyses of information foraging on the web. Cognitive Science 29, 3 (2005), 343--373. DOI: http://dx.doi.org/10.1207/s15516709cog0000_20
38. Peter Pirolli and Stuart Card. 1999. Information foraging. Psychological Review 106, 4 (1999), 643--675. DOI: http://dx.doi.org/10.1037/0033--295X.106.4.643
39. Deepak Ramachandran and Eyal Amir. 2007. Bayesian inverse reinforcement learning. In Proceedings of the Twentieth International Joint Conference on Artificial Intelligence. 2586--2591.
40. Keith Rayner. 1998. Eye movements in reading and information processing: 20 years of research. Psychological Bulletin 124, 3 (1998), 372--422. DOI: http://dx.doi.org/10.1037/0033--2909.124.3.372
41. David E. Rumelhart and Donald A. Norman. 1982. Simulating a skilled typist: A study of skilled cognitive-motor performance. Cognitive Science 6, 1 (1982), 1--36.
42. Mikael Sunnåker, Alberto G. Busetto, Elina Numminen, Jukka Corander, Matthieu Foll, and Christophe Dessimoz. 2013. Approximate Bayesian computation. PLoS Computational Biology 9, 1 (2013), e1002803. DOI: http://dx.doi.org/10.1371/journal.pcbi.1002803
43. Richard S. Sutton and Andrew G. Barto. 1998. Reinforcement learning: An introduction. MIT press.
44. Yuan-Chi Tseng and Andrew Howes. 2015. The adaptation of visual search to utility, ecology and design. International Journal of Human--Computer Studies 80 (2015), 45--55. DOI: http://dx.doi.org/10.1016/j.ijhcs.2015.03.005
45. Brian D. Ziebart, Andrew Maas, J. Andrew Bagnell, and Anind K. Dey. 2008. Maximum entropy inverse reinforcement learning. In Proceedings of the Association for the Advancement of Artificial Intelligence Conference on Artificial Intelligence. 1433--1438.

Index Terms

• Computing methodologies

  • Artificial intelligence

    • Philosophical/theoretical foundations of artificial intelligence

• Human-centered computing

  • Human computer interaction (HCI)