Tasty tech: human-food interaction and multimodal interfaces

ABSTRACT

The perception of food involves, not just taste, but all senses. We describe several interactive and novel mixed reality systems designed to enhance or change a dining experience by using computer generated visual, audio, and tactile stimuli. We describe how ideas from these playful designs can influence the perception of, and the interaction with food, and can inform the design of studies into human-food interaction.

References

1. A. Anonymous, B. Anonymous, and C. Anonymous. A moving feast: Effects of color, shape and animation on taste associations and taste perceptions. In proceedings of Advances in Computer Entertainment (in press), 2016.
2. M. Auvray and C. Spence. The multisensory perception of flavor. Consciousness and cognition, 17(3):1016--1031, 2008.
3. P. S. Bordnick, B. L. Carter, and A. C. Traylor. What virtual reality research in addictions can tell us about the future of obesity assessment and treatment. Journal of diabetes science and technology, 5(2):265--271, 2011.
4. S. N. Cheong, W. W. Chiew, and W. J. Yap. Design and development of multi-touchable e-restaurant management system. In Science and Social Research (CSSR), 2010 International Conference on, pages 680--685. IEEE, 2010.
5. J. Delwiche. The impact of perceptual interactions on perceived flavor. Food Quality and preference, 15(2):137--146, 2004.
6. J. F. Delwiche. You eat with your eyes first. Physiology & behavior, 107(4):502--504, 2012.
7. S. Ganesh, P. Marshall, Y. Rogers, and K. O'Hara. Foodworks: tackling fussy eating by digitally augmenting children's meals. In Proceedings of the 8th Nordic Conference on Human-Computer Interaction: Fun, Fast, Foundational, pages 147--156. ACM, 2014.
8. L. L. Garber Jr, E. M. Hyatt, and R. G. Starr Jr. Placing food color experimentation into a valid consumer context. Journal of food products Marketing, 7(3):3--24, 2001.
9. V. Halupka, A. Almahr, Y. Pan, and A. D. Cheok. Chop chop: a sound augmented kitchen prototype. In Advances in Computer Entertainment, pages 494--497. Springer, 2012.
10. G. Hanson-Vaux, A. S. Crisinel, and C. Spence. Smelling shapes: Crossmodal correspondences between odors and shapes. Chemical Senses, 38(2):161--166, 2013.
11. V. Harrar and C. Spence. The taste of cutlery: how the taste of food is affected by the weight, size, shape, and colour of the cutlery used to eat it. Flavour, 2(1):21, 2013.
12. S. Hermsen, J. H. Frost, E. Robinson, S. Higgs, M. Mars, and R. C. Hermans. Evaluation of a smart fork to decelerate eating rate. Journal of the Academy of Nutrition and Dietetics, 2016.
13. R. Hurling and R. Shepherd. Eating with your eyes: effect of appearance on expectations of liking. Appetite, 41(2):167--174, 2003.
14. A. Kadomura, C.-Y. Li, Y.-C. Chen, K. Tsukada, I. Siio, and H.-h. Chu. Sensing fork: eating behavior detection utensil and mobile persuasive game. In CHI'13 Extended Abstracts on Human Factors in Computing Systems, pages 1551--1556. ACM, 2013.
15. R. A. Khot, J. Lee, D. Aggarwal, L. Hjorth, and F. Mueller. Tastybeats: Designing palatable representations of physical activity. In Proceedings of the 33rd Annual ACM Conference on Human Factors in Computing Systems, pages 2933--2942. ACM, 2015.
16. Y. Kita and J. Rekimoto. Spot-Light: Multimodal Projection Mapping on Food. In C. Stephanidis, editor, HCI International 2013 - Posters' Extended Abstracts: International Conference, HCI International 2013, Las Vegas, NV, USA, July 21-26, 2013, Proceedings, Part II, pages 652--655. Springer Berlin Heidelberg, 2013.
17. N. Koizumi, H. Tanaka, Y. Uema, and M. Inami. Chewing Jockey: Augmented Food Texture by Using Sound Based on the Cross-modal Effect. In Proceedings of the 8th International Conference on Advances in Computer Entertainment Technology, 2011.
18. L. G. Kurland and E. Gilbert. Integrated interactive restaurant communication system for food and entertainment processing, Nov. 12 1985. US Patent 4,553,222.
19. C. A. Levitan, M. Zampini, R. Li, and C. Spence. Assessing the role of color cues and people's beliefs about color-flavor associations on the discrimination of the flavor of sugar-coated chocolates. Chemical Senses, 33(5):415--423, 2008.
20. S. Mennicken, T. Karrer, P. Russell, and J. Borchers. First-person cooking: a dual-perspective interactive kitchen counter. In CHI'10 Extended Abstracts on Human Factors in Computing Systems, pages 3403--3408. ACM, 2010.
21. H. Nakamura and H. Miyashita. Communication by change in taste. In CHI'11 Extended Abstracts on Human Factors in Computing Systems, pages 1999--2004. ACM, 2011.
22. Y. Ohta and H. Tamura. Mixed reality: merging real and virtual worlds. Springer Publishing Company, Incorporated, 2014.
23. W. J. S. Peschardt. Manufacture of artificial edible cherries, soft sheets, and the like, July 9 1946. US Patent 2,403,547.
24. B. Piqueras-Fiszman and C. Spence. Colour, pleasantness, and consumption behaviour within a meal. Appetite, 75:165--172, 2014.
25. N. Ranasinghe, A. Cheok, R. Nakatsu, and E. Y.-L. Do. Simulating the sensation of taste for immersive experiences. In Proceedings of the 2013 ACM international workshop on Immersive media experiences, pages 29--34. ACM, 2013.
26. F. Reinoso Carvalho, R. Van Ee, M. Rychtarikova, A. Touhafi, K. Steenhaut, D. Persoone, C. Spence, and M. Leman. Does music influence the multisensory tasting experience? Journal of Sensory Studies, 30(5):404--412, 2015.
27. H.-S. Seo, A. Arshamian, K. Schemmer, I. Scheer, T. Sander, G. Ritter, and T. Hummel. Cross-modal integration between odors and abstract symbols. Neuroscience letters, 478(3):175--178, 2010.
28. M. U. Shankar, C. A. Levitan, and C. Spence. Grape expectations: The role of cognitive influences in color-flavor interactions. Consciousness and Cognition, 19(1):380--390, 2010.
29. M. Snyder, J. Zimmerman, and J. Forlizzi. Your dinner's calling: supporting family dinnertime activities. In Proceedings of the 2007 conference on Designing pleasurable products and interfaces, pages 485--489. ACM, 2007.
30. C. Spence. Multisensory flavour perception. Current Biology, 23(9):R365--R369, 2013.
31. C. Spence, C. A. Levitan, M. U. Shankar, and M. Zampini. Does food color influence taste and flavor perception in humans? Chemosensory Perception, 3(1):68--84, 2010.
32. C. Spence, K. Okajima, A. D. Cheok, O. Petit, and C. Michel. Eating with our eyes: From visual hunger to digital satiation. Brain and Cognition, 2015.
33. C. Spence and B. Piqueras-Fiszman. The perfect meal: the multisensory science of food and dining. John Wiley & Sons, 2014.
34. C. Stangor. Social groups in action and interaction. Psychology Press, 2004.
35. Y. Takahashi, N. Hasegawa, K. Takahashi, and T. Hatakeyama. Human interface using pc display with head pointing device for eating assist robot and emotional evaluation by gsr sensor. In Robotics and Automation, 2001. Proceedings 2001 ICRA. IEEE International Conference on, volume 4, pages 3674--3679. IEEE, 2001.
36. C. Velasco, A. T. Woods, O. Deroy, and C. Spence. Hedonic mediation of the crossmodal correspondence between taste and shape. Food Quality and Preference, 41:151--158, 2015.
37. J. V. Verhagen and L. Engelen. The neurocognitive bases of human multimodal food perception: sensory integration. Neuroscience & biobehavioral reviews, 30(5):613--650, 2006.
38. J. Wei, X. Wang, R. L. Peiris, Y. Choi, X. R. Martinez, R. Tache, J. T. K. V. Koh, V. Halupka, and A. D. Cheok. Codine: an interactive multi-sensory system for remote dining. In Proceedings of the 13th international conference on Ubiquitous computing, pages 21--30. ACM, 2011.
39. M. Zampini, D. Sanabria, N. Phillips, and C. Spence. The multisensory perception of flavor: Assessing the influence of color cues on flavor discrimination responses. Food Quality and Preference, 18(7):975--984, 2007.

Index Terms

• Information systems

  • Information systems applications

    • Multimedia information systems