skip to main content
10.1145/1978942.1979396acmconferencesArticle/Chapter ViewAbstractPublication PageschiConference Proceedingsconference-collections
research-article

How to evaluate technologies for health behavior change in HCI research

Published:07 May 2011Publication History

ABSTRACT

New technologies for encouraging physical activity, healthy diet, and other types of health behavior change now frequently appear in the HCI literature. Yet, how such technologies should be evaluated within the context of HCI research remains unclear. In this paper, we argue that the obvious answer to this question - that evaluations should assess whether a technology brought about the intended change in behavior - is too limited. We propose that demonstrating behavior change is often infeasible as well as unnecessary for a meaningful contribution to HCI research, especially when in the early stages of design or when evaluating novel technologies. As an alternative, we suggest that HCI contributions should focus on efficacy evaluations that are tailored to the specific behavior-change intervention strategies (e.g., self-monitoring, conditioning) embodied in the system and studies that help gain a deep understanding of people's experiences with the technology.

References

  1. Abowd, G. D., Mynatt, E. D., and Rodden, T. The human experience {of ubiquitous computing}. IEEE pervasive computing 1, 1 (2002), 48--57. Google ScholarGoogle ScholarDigital LibraryDigital Library
  2. Anderson, I., Maitland, J., Sherwood, S., et al. Shakra: Tracking and Sharing Daily Activity Levels with Unaugmented Mobile Phones. Mobile Networks and Applications 12, 2-3 (2007), 185--199. Google ScholarGoogle ScholarDigital LibraryDigital Library
  3. Bargh, J. A., Gollwitzer, P. M., Lee-Chai, A., Barndollar, K., and Trötschel, R. The automated will: Nonconscious activation and pursuit of behavioral goals. Journal of Personality and Social Psychology 81, 6 (2001), 1014--1027.Google ScholarGoogle ScholarCross RefCross Ref
  4. Barkhuus, L. and Rode, J. From mice to men: 24 years of evaluation in CHI. ACM CHI '07 - Alt.CHI, (2007). Google ScholarGoogle ScholarDigital LibraryDigital Library
  5. van der Berg, M. H., Schoones, J. W., and Vliet Vlieland, T. P. M. Internet-based physical activity interventions: A systematic review of the literature. Journal of Medical Internet Research 9, 3 (2007).Google ScholarGoogle Scholar
  6. Bickmore, T. W., Caruso, L., and Clough-Gorr, K. Acceptance and usability of a relational agent interface by older urban results. Proceedings of CHI 2005, ACM Press (2005). Google ScholarGoogle ScholarDigital LibraryDigital Library
  7. Consolvo, S., Everitt, K. M., Smith, I., and Landay, J. A. Design requirements for technologies that encourage physical activity. Proceedings of CHI 2006, ACM Press (2006), 457--466. Google ScholarGoogle ScholarDigital LibraryDigital Library
  8. Consolvo, S., Klasnja, P., McDonald, D. W., et al. Flowers or a robot army?: encouraging awareness & activity with personal, mobile displays. Proceedings of the 10th international conference on Ubiquitous computing, ACM (2008), 54--63. Google ScholarGoogle ScholarDigital LibraryDigital Library
  9. Consolvo, S., McDonald, D. W., Toscos, T., et al. Activity sensing in the wild: a field trial of ubifit garden. Proceeding of CHI 2008, ACM (2008), 1797--1806. Google ScholarGoogle ScholarDigital LibraryDigital Library
  10. Csikszentmihalyi, M. and Larson, R. Validity and reliability of the Experience Sampling Method. Journal of Nervous and Mental Disease 175, 9 (1987), 526--536.Google ScholarGoogle ScholarCross RefCross Ref
  11. Deci, E. L. and Ryan, R. M. The "What" and "Why" of Goal Pursuits: Human Needs and the Self-Determination of Behavior. Psychological Inquiry: An International Journal for the Advancement of Psychological Theory 11, 4 (2000), 227--268.Google ScholarGoogle Scholar
  12. Denning, T., Andrew, A., Chaudhri, R., et al. BALANCE: towards a usable pervasive wellness application with accurate activity inference. Proceedings of the 10th workshop on Mobile Computing Systems & Applications, ACM (2009), 1--6. Google ScholarGoogle ScholarDigital LibraryDigital Library
  13. DeShazo, J., Harris, L., Turner, A., and Pratt, W. Grounded in theory, user-centered, and someplace else: Designing and remotely testing mobile diabetes video games. Journal of Telemedicine and Telecare 16, 7 (2010), 378--382.Google ScholarGoogle ScholarCross RefCross Ref
  14. Dixon, M. and Fogarty, J. Prefab: implementing advanced behaviors using pixel-based reverse engineering of interface structure. Proceedings of CHI 2010, ACM (2010), 1525--1534. Google ScholarGoogle ScholarDigital LibraryDigital Library
  15. Fogg, B. J. Persuasive Technology: Using Computers to Change What We Think and Do. Morgan Kaufmann, New York, 2003. Google ScholarGoogle ScholarCross RefCross Ref
  16. Froehlich, J., Chen, M. Y., Consolvo, S., Harrison, B. L., and Landay, J. A. MyExperience: A system for in situ tracing and capturing of user feedback on mobile phones. Proceedings of the 5th international conference on Mobile systems, applications and services (MobiSys '07), ACM Press (2007), 57--70. Google ScholarGoogle ScholarDigital LibraryDigital Library
  17. Gasser, R., Brodbeck, D., Degen, M., Luthiger, J., Wyss, R., and Reichlin, S. Persuasiveness of a Mobile Lifestyle Coaching Application Using Social Facilitation. Proceedings of the First International Conference on Persuasive Technology, Springer-Verlag (2006), 27--38. Google ScholarGoogle ScholarDigital LibraryDigital Library
  18. Greenberg, S. and Buxton, B. Usability evaluation considered harmful (some of the time). Proceeding of CHI 2008, ACM (2008), 111--120. Google ScholarGoogle ScholarDigital LibraryDigital Library
  19. Higgins, E. T. Knowledge activation: Accessibility, applicability, and salience. In E. T. Higgins and A. W. Kruglanski, eds., Social psychology: Handbook of basic principles. Guilford Press, New York, 1996, 133--168.Google ScholarGoogle Scholar
  20. Hoffman, C., Rice, D., and Sung, H. Y. Persons with chronic conditions. Their prevalence and costs. JAMA: The Journal of the American Medical Association 276, 18 (1996), 1473--1479.Google ScholarGoogle ScholarCross RefCross Ref
  21. Hurling, R., Catt, M., De Boni, M., et al. Using internet and mobile phone technology to deliver an automated physical activity program: Randomized controlled trial. Journal of Medical Internet Research 9, (2007), e7.Google ScholarGoogle ScholarCross RefCross Ref
  22. Kopp, J. Self-monitoring: A literature review of research and practice. Social Work Research & Abstracts 24, (1988), 8--20.Google ScholarGoogle ScholarCross RefCross Ref
  23. Lee, G., Tsai, C., Griswold, W. G., Raab, F., and Patrick, K. PmEB: a mobile phone application for monitoring caloric balance. CHI '06 extended abstracts on Human factors in computing systems, ACM (2006), 1013--1018. Google ScholarGoogle ScholarDigital LibraryDigital Library
  24. Lin, J. L., Mamykina, L., Lindtner, S., Delajoux, G., and Strub, H. B. Fish'n'Steps: Encouraging physical activity with an interactive computer game. Proceedings of Ubicomp 2006, Springer (2006), 261--278. Google ScholarGoogle ScholarDigital LibraryDigital Library
  25. Logue, A. W. Self-control and health behavior. In W. K. Bickel and R. E. Vuchinich, eds., Reframing health behavior change with behavioral economics. Lawrence Erlbaum Associates, Inc., Mahwah, NJ, 2000, 167--192.Google ScholarGoogle Scholar
  26. Mamykina, L., Mynatt, E., Davidson, P., and Greenblatt, D. MAHI: investigation of social scaffolding for reflective thinking in diabetes management. Proceeding of CHI 2008, ACM (2008), 477--486. Google ScholarGoogle ScholarDigital LibraryDigital Library
  27. Marcus, B. H., Dubbert, P. M., Forsyth, L. H., et al. Physical activity behavior change: issues in adoption and maintenance. Health Psychology: Official Journal of the Division of Health Psychology, American Psychological Association 19, 1 Suppl (2000), 32--41.Google ScholarGoogle Scholar
  28. Marlatt, G. A. and George, W. H. Relapse prevention and the maintenance of optimal health. In S. A. Shumaker, E. B. Schron, J. K. Ockene and W. L. McBee, eds., The handbook of health behavior change. Springer, New York, 1998, 33--58.Google ScholarGoogle Scholar
  29. Mattila, E., Pärkkä, J., Hermersdorf, M., et al. Mobile diary for wellness management - Results on usage and usability in two user studies. IEEE Transactions on Information Technology in Biomedicine 12, 4 (2008), 501--512. Google ScholarGoogle ScholarDigital LibraryDigital Library
  30. Moore, S. M., Dolansky, M. A., Ruland, C. M., Pashkow, F. J., and Blackburn, G. G. Predictors of women's exercise maintenance after cardiac rehabilitation. Journal of Cardiopulmonary Rehabilitation 23, 1 (2003), 40--49.Google ScholarGoogle ScholarCross RefCross Ref
  31. Morris, M., Kathawala, Q., Leen, T. K., et al. Mobile therapy: Case study evaluations of a cell phone application for emotional self-awareness. Journal of Medical Internet Research 12, 2 (2010), e10.Google ScholarGoogle ScholarCross RefCross Ref
  32. Muraven, M. and Baumeister, R. F. Self-regulation and depletion of limited resources: Does self-control resemble a muscle? Psychological Bulletin 126, 2 (2000), 247--259.Google ScholarGoogle ScholarCross RefCross Ref
  33. Muraven, M., Tice, D. M., and Baumeister, R. F. Self-control as limited resource: Regulatory depletion patterns. Journal of Personality and Social Psychology 74, 3 (1998), 774--789.Google ScholarGoogle ScholarCross RefCross Ref
  34. Nawyn, J., Intille, S. S., and Larson, K. Embedding behavior modification strategies into a consumer electronic device: a case study. Proceedings of UbiComp 2006, Springer (2006), 297--314. Google ScholarGoogle ScholarDigital LibraryDigital Library
  35. Nelson, R. O. Assessment and therapeutic functions of self-monitoring. In M. Hersen, R. M. Eisler and P. M. Miller, eds., Progress in behavior modification. Academic Press, New York, 1977.Google ScholarGoogle Scholar
  36. Prochaska, J. O., Johnson, S., and Lee, P. The transtheoretical model of behavior change. In S. A. Shumaker, E. B. Schron, J. K. Ockene and W. L. McBee, eds., The handbook of health behavior change. Springer, New York, 1998, 59--84.Google ScholarGoogle Scholar
  37. Radtke, K. L. Exercise compliance in cardiac rehabilitation. Rehabilitation Nursing: The Official Journal of the Association of Rehabilitation Nurses 14, 4 (1989), 182--186, 195.Google ScholarGoogle ScholarCross RefCross Ref
  38. Rogers, Y., Connelly, K., Tedesco, L., et al. Why It's Worth the Hassle: The Value of In-Situ Studies When Designing Ubicomp. In UbiComp 2007: Ubiquitous Computing. 2007, 336--353. Google ScholarGoogle ScholarDigital LibraryDigital Library
  39. Smith, B. K., Frost, J., Albayrak, M., and Sudhakar, R. Integrating glucometers and digital photography as experience capture tools to enhance patient understanding and communication of diabetes self-management practices. Personal Ubiquitous Computing 11, 4 (2007), 273--286. Google ScholarGoogle ScholarDigital LibraryDigital Library
  40. Tsai, C. C., Lee, G., Raab, F., et al. Usability and feasability of PmEB: A mobile phone application for monitoring real time caloric balance. Mobile Networks and Applications 12, 2-3 (2007), 173--184. Google ScholarGoogle ScholarDigital LibraryDigital Library
  41. Twardella, D., Merx, H., Hahmann, H., Wüsten, B., Rothenbacher, D., and Brenner, H. Long term adherence to dietary recommendations after inpatient rehabilitation: prospective follow up study of patients with coronary heart disease. Heart (British Cardiac Society) 92, 5 (2006), 635--640.Google ScholarGoogle Scholar
  42. Witkiewitz, K. and Marlatt, G. A. Relapse prevention for alcohol and drug problems: that was Zen, this is Tao. The American Psychologist 59, 4 (2004), 224--235.Google ScholarGoogle ScholarCross RefCross Ref

Index Terms

  1. How to evaluate technologies for health behavior change in HCI research
      Index terms have been assigned to the content through auto-classification.

      Recommendations

      Comments

      Login options

      Check if you have access through your login credentials or your institution to get full access on this article.

      Sign in
      • Published in

        cover image ACM Conferences
        CHI '11: Proceedings of the SIGCHI Conference on Human Factors in Computing Systems
        May 2011
        3530 pages
        ISBN:9781450302289
        DOI:10.1145/1978942

        Copyright © 2011 ACM

        Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

        Publisher

        Association for Computing Machinery

        New York, NY, United States

        Publication History

        • Published: 7 May 2011

        Permissions

        Request permissions about this article.

        Request Permissions

        Check for updates

        Qualifiers

        • research-article

        Acceptance Rates

        CHI '11 Paper Acceptance Rate410of1,532submissions,27%Overall Acceptance Rate6,199of26,314submissions,24%

        Upcoming Conference

        CHI '24
        CHI Conference on Human Factors in Computing Systems
        May 11 - 16, 2024
        Honolulu , HI , USA

      PDF Format

      View or Download as a PDF file.

      PDF

      eReader

      View online with eReader.

      eReader