Multimodal archiving, real-time annotation and information visualization in a biofeedback system for stroke patient rehabilitation

ABSTRACT

In this paper we present our work on a system to support real-time multimodal archiving, collaborative annotation and offline information visualization for a biofeedback stroke-rehabilitation application. Our archiving / annotation / visualization system can play a critical role in the long-term biofeedback stroke therapy by supporting cooperative data analysis and media feedback as well as by providing the therapist with insight into computing-supported therapy. There are three contributions of this paper: (a) the design of a robust archiving system that archives in real time parametric model data (motion capture, motion analysis and audio / visual synthesis parameters) as well as audio / video from the biofeedback environment. (b) a web-based annotation tool designed with low cognitive load (c) a hierarchical information visualization tool that enables the therapist and other team members to examine quantitative motion analysis of subject performance with the context of media feedback, thus enabling collaborative insights. Our user studies indicate that the system performs well.

References

1. Max/MSP (media tool). http://www.cycling74.com.]]
2. J. J. Cadiz, A. Gupta and J. Grudin (2000). Using Web annotations for asynchronous collaboration around documents, Proceedings of the 2000 ACM conference on Computer supported cooperative work, 1-58113-222-0, ACM Press, 309--318, Philadelphia, Pennsylvania, United States.]]
3. Y. Chen, H. Huang, W. Xu, R. Wallis, H. Sundaram, et al. (2006). The Design Of A Real-Time, Multimodal Biofeedback System For Stroke Patient Rehabilitation, Proc. SIG ACM Multimedia 2006, also AME-TR-2006-07, Oct. 2006, Santa Barbara, CA.]]
4. S. E. Ellis and D. P. Groth (2004). A Collaborative Annotation System for Data Visualization, Proc. of the working conference on Advanced Visual Interfaces.,]]
5. J. Gemmell, G. Bell, R. Lueder, S. Drucker and C. Wong (2002). MyLifeBits: fulfilling the Memex vision, Proceedings of the 10th ACM international conference on Multimedia, pp. 235--238, Dec. 2002, Juan Les-Pins, France.]]
6. S. Hodges, L. Williams, E. Berry, S. Izadi, J. Srinivasan, et al. SenseCam: a Retrospective Memory Aid, Proc. 8th International Conference on Ubicomp 2006, Sep. 2006, Orange County, CA.]]
7. M. Holden and T. Dyar (2002). Virtual environment training: a new tool for neurorehabilitation. Neurology Report 26(2): 62--72.]]
8. D. Jack, R. Boian, A. S. Merians, M. Tremaine, G. C. Burdea, et al. (2001). Virtual reality-enhanced stroke rehabilitation. IEEE Trans Neural Syst Rehabil Eng 9(3): 308--318.]]
9. R. E. Kraut, D. Gergle and S. R. Fussell (2002). The use of visual information in shared visual spaces: informing the development of virtual co-presence, Proceedings of the 2002 ACM conference on Computer supported cooperative work, 1-58113-560-2, ACM Press, 31--40, New Orleans, Louisiana, USA.]]
10. J. Ou, L. M. Oh, J. Yang and S. R. Fussell (2005). Effects of task properties, partner actions, and message content on eye gaze patterns in a collaborative task, Proceedings of the SIGCHI conference on Human factors in computing systems, 1-58113-998-5, ACM Press, 231--240, Portland, Oregon, USA.]]
11. C. Plaisant, B. Milash, A. Rose, S. Widoff and B. Shneiderman (1996). LifeLines: visualizing personal histories. Proceedings of the SIGCHI conference on Human factors in computing systems(eds), ACM Press: 221--ff.]]
12. D. Quan, D. Huynh and D. R. Karger (2003). Haystack: A Platform for Authoring End User Semantic Web Applications, 2nd International Semantic Web Conference, Oct. 2003, Sanibel Island, Florida, USA.]]
13. C. Weng and J. H. Gennari (2004). Asynchronous collaborative writing through annotations, Proceedings of the 2004 ACM conference on Computer supported cooperative work, 1-58113-810-5, ACM Press, 578-581, Chicago, Illinois, USA.]]
14. A. Wilhelm, Y. Takhteyev, R. Sarvas, N. V. House and M. Davis (2004). Photo Annotation on a Camera Phone, ACM Conference on Human Computer Interaction, Apr. 2004, Vienna, Austria.]]
15. S. H. You, S. H. Jang, Y. H. Kim, M. Hallett, S. H. Ahn, et al. (2005). Virtual reality-induced cortical reorganization and associated locomotor recovery in chronic stroke: an experimenter-blind randomized study. Stroke 36(6): 1166--1171.]]

Index Terms

• Applied computing

  • Life and medical sciences

    • Health care information systems

• Computing methodologies

  • Artificial intelligence

    • Philosophical/theoretical foundations of artificial intelligence

      • Cognitive science

• Human-centered computing

  • Human computer interaction (HCI)

    • Interaction devices

      • Touch screens

    • Interaction paradigms

    • Interaction techniques

      • Auditory feedback

  • Interaction design