skip to main content
research-article

USMART: An Unsupervised Semantic Mining Activity Recognition Technique

Published:13 November 2014Publication History
Skip Abstract Section

Abstract

Recognising high-level human activities from low-level sensor data is a crucial driver for pervasive systems that wish to provide seamless and distraction-free support for users engaged in normal activities. Research in this area has grown alongside advances in sensing and communications, and experiments have yielded sensor traces coupled with ground truth annotations about the underlying environmental conditions and user actions. Traditional machine learning has had some success in recognising human activities; but the need for large volumes of annotated data and the danger of overfitting to specific conditions represent challenges in connection with the building of models applicable to a wide range of users, activities, and environments. We present USMART, a novel unsupervised technique that combines data- and knowledge-driven techniques. USMART uses a general ontology model to represent domain knowledge that can be reused across different environments and users, and we augment a range of learning techniques with ontological semantics to facilitate the unsupervised discovery of patterns in how each user performs daily activities. We evaluate our approach against four real-world third-party datasets featuring different user populations and sensor configurations, and we find that USMART achieves up to 97.5% accuracy in recognising daily activities.

References

  1. Ling Bao and Stephen S. Intille. 2004. Activity recognition from user-annotated acceleration data. In Pervasive Computing. Lecture Notes in Computer Science, Vol. 3001. Springer, 1--17. DOI: http://dx.doi.org/10.1007/978-3-540-24646-6_1Google ScholarGoogle Scholar
  2. Oliver Brdiczka, James L. Crowley, and Patrick Reignier. 2007. Learning situation models for providing context-aware services. In Proceedings of the 4th International Conference on Universal Access in Human-Computer Interaction: Ambient Interaction (UAHCI’07). Springer-Verlag, Berlin, Heidelberg, 23--32. http://dl.acm.org/citation.cfm?id=1763296.1763300 Google ScholarGoogle ScholarDigital LibraryDigital Library
  3. Hung H. Bui, Svetha Venkatesh, and Geoff West. 2002. Tracking and surveillance in wide-area spatial environments using the abstract hidden Markov model. In Hidden Markov Models: Applications in Computer Vision. World Scientific Publishing, River Edge, NJ, 177--196. Google ScholarGoogle ScholarDigital LibraryDigital Library
  4. Liming Chen, Jesse Hoey, Chris D. Nugent, Diane J. Cook, and Zhiwen Yu. 2012a. Sensor-based activity recognition. IEEE Transactions on Systems, Man, and Cybernetics, Part C: Applications and Reviews 42, 6, 790--808. DOI: http://dx.doi.org/10.1109/TSMCC.2012.2198883 Google ScholarGoogle ScholarDigital LibraryDigital Library
  5. Liming Chen, Chris D. Nugent, and Hui Wang. 2012b. A knowledge-driven approach to activity recognition in smart homes. IEEE Transactions on Knowledge and Data Engineering 24, 961--974. DOI: http://dx.doi.org/10.1109/TKDE.2011.51 Google ScholarGoogle ScholarDigital LibraryDigital Library
  6. Diane Cook. 2012. Learning setting-generalized activity models for smart spaces. IEEE Intelligent Systems 27, 1, 32--38. DOI: http://dx.doi.org/10.1109/MIS.2010.112 Google ScholarGoogle ScholarDigital LibraryDigital Library
  7. Shai Fine, Yoram Singer, and Naftali Tishby. 1998. The hierarchical hidden Markov model: Analysis and applications. Machine Learning 32, 1, 41--62. Google ScholarGoogle ScholarDigital LibraryDigital Library
  8. Tao Gu, Shaxun Chen, Xianping Tao, and Jian Lu. 2010. An unsupervised approach to activity recognition and segmentation based on object-use fingerprints. Data and Knowledge Engineering 69, 6, 533--544. DOI: http://dx.doi.org/10.1016/j.datak.2010.01.004 Google ScholarGoogle ScholarDigital LibraryDigital Library
  9. Tao Gu, Hung Keng Pung, and Da Qing Zhang. 2005. A service-oriented middleware for building context-aware services. Journal of Network and Computer Applications 28, 1, 1--18. Google ScholarGoogle ScholarDigital LibraryDigital Library
  10. Tao Gu, Liang Wang, Zhanqing Wu, Xianping Tao, and Jian Lu. 2011. A pattern mining approach to sensor-based human activity recognition. IEEE Transactions on Knowledge and Data Engineering 23, 9, 1359--1372. DOI: http://dx.doi.org/10.1109/TKDE.2010.184 Google ScholarGoogle ScholarDigital LibraryDigital Library
  11. Tao Gu, Xiao Hang Wang, Hung Keng Pung, and Da Qing Zhang. 2004. An ontology-based context model in intelligent environments. In Proceedings of the International Conference on Communication Network and Distributed Systems (CNDS’04). 270--275.Google ScholarGoogle Scholar
  12. Tao Gu, Zhanqing Wu, Xianping Tao, Hung Keng Pung, and Jian Lu. 2009. epSICAR: An emerging patterns based approach to sequential, interleaved and concurrent activity recognition. In Proceedings of the IEEE International Conference on Pervasive Computing and Communications (PerCom’09). 1--9. Google ScholarGoogle ScholarDigital LibraryDigital Library
  13. Jiawei Han, Micheline Kamber, and Jian Pei. 2011. Data Mining: Concepts and Techniques (3rd ed.). Morgan Kaufmann. Google ScholarGoogle ScholarDigital LibraryDigital Library
  14. Xin Hong and Chris D. Nugent. 2009. Partitioning time series sensor data for activity recognition. In Proceedings of the 9th International Conference on Information Technology and Applications in Biomedicine (ITAB’09). 1--4. DOI: http://dx.doi.org/10.1109/ITAB.2009.5394306Google ScholarGoogle Scholar
  15. Clare J. Hooper, Anne Preston, Madeline Balaam, Paul Seedhouse, Daniel Jackson, Cuong Pham, Cassim Ladha, Karim Ladha, Thomas Plötz, and Patrick Olivier. 2012. The French kitchen: Task-based learning in an instrumented kitchen. In Proceedings of the 2012 ACM Conference on Ubiquitous Computing (UbiComp’12). ACM, New York, NY, 193--202. DOI: http://dx.doi.org/10.1145/2370216.2370246 Google ScholarGoogle ScholarDigital LibraryDigital Library
  16. Tâm Huỳnh, Ulf Blanke, and Bernt Schiele. 2007. Scalable recognition of daily activities with wearable sensors. In Proceedings of the 3rd International Conference on Location and Context Awareness (LoCA’07). 50--67. http://dl.acm.org/citation.cfm?id=1777235.1777241 Google ScholarGoogle ScholarDigital LibraryDigital Library
  17. Jay J. Jiang and David W. Conrath. 1997. Semantic similarity based on corpus statistics and lexical taxonomy. In Proceedings of the International Conference Research on Computational Linguistics. 9008+. http://adsabs.harvard.edu/cgi-bin/nph-bib_query?bibcode=1997cmp.lg....9008JGoogle ScholarGoogle Scholar
  18. Tim L. M. Kasteren, Gwenn Englebienne, and Ben J. A. Kröse. 2011. Human activity recognition from wireless sensor network data: Benchmark and software. In Activity Recognition in Pervasive Intelligent Environments. Atlantis Ambient and Pervasive Intelligence, Vol. 4. Springer, 165--186.Google ScholarGoogle Scholar
  19. Narayanan C. Krishnan and Diane J. Cook. 2014. Activity recognition on streaming sensor data. Pervasive and Mobile Computing 10, 138--154. DOI: http://dx.doi.org/10.1016/j.pmcj.2012.07.003 Google ScholarGoogle ScholarDigital LibraryDigital Library
  20. Javier Ortiz Laguna, Angel García Olaya, and Daniel Borrajo. 2011. A dynamic sliding window approach for activity recognition. In Proceedings of the 19th International Conference on User Modeling, Adaption, and Personalization (UMAP’11). Springer-Verlag, Berlin, Heidelberg, 219--230. http://dl.acm.org/citation.cfm?id=2021855.2021875 Google ScholarGoogle ScholarDigital LibraryDigital Library
  21. Claudia Leacock, George A. Miller, and Martin Chodorow. 1998. Using corpus statistics and WordNet relations for sense identification. Computational Linguistics 24, 1, 147--165. http://dl.acm.org/citation.cfm?id=972719.972726 Google ScholarGoogle ScholarDigital LibraryDigital Library
  22. Beth Logan, Jennifer Healey, Matthai Philipose, Emmanuel Munguia Tapia, and Stephen Intille. 2007. A long-term evaluation of sensing modalities for activity recognition. In Proceedings of the 9th International Conference on Ubiquitous Computing (UbiComp’07). Springer-Verlag, Berlin, Heidelberg, 483--500. http://dl.acm.org/citation.cfm?id=1771592.1771620 Google ScholarGoogle ScholarDigital LibraryDigital Library
  23. Susan McKeever, Juan Ye, Lorcan Coyle, and Simon Dobson. 2010. Activity reasoning using temporal evidence theory. Ambient Intelligence and Smart Environment 2, 3, 253--269. Google ScholarGoogle ScholarDigital LibraryDigital Library
  24. George A. Miller. 1995. WordNet: A lexical database for English. Communications of the ACM 38, 11, 39--41. DOI: http://dx.doi.org/10.1145/219717.219748 Google ScholarGoogle ScholarDigital LibraryDigital Library
  25. George Okeyo, Liming Chen, Hui Wang, and Roy Sterritt. 2011. Ontology-based learning framework for activity assistance in an adaptive smart home. In Activity Recognition in Pervasive Intelligent Environments. Atlantis Ambient and Pervasive Intelligence, Vol. 4. Springer, 237--263. DOI: http://dx.doi.org/10.2991/978-94-91216-05-3_11Google ScholarGoogle Scholar
  26. George Okeyo, Liming Chen, Hui Wang, and Roy Sterritt. 2014. Dynamic sensor data segmentation for real-time knowledge-driven activity recognition. Pervasive and Mobile Computing 10, 155--172. DOI: http://dx.doi.org/10.1016/j.pmcj.2012.11.004 Google ScholarGoogle ScholarDigital LibraryDigital Library
  27. Paulito Palmes, Hung Keng Pung, Tao Gu, Wenwei Xue, and Shaxun Chen. 2010. Object relevance weight pattern mining for activity recognition and segmentation. Pervasive and Mobile Computing 6, 1, 43--57. DOI: http://dx.doi.org/10.1016/j.pmcj.2009.10.004 Google ScholarGoogle ScholarDigital LibraryDigital Library
  28. Donald J. Patterson, Dieter Fox, Henry Kautz, and Matthai Philipose. 2005. Fine-grained activity recognition by aggregating abstract object usage. In Proceedings of the 9th IEEE International Symposium on Wearable Computers (ISWC’05). IEEE, Los Alamitos, CA, 44--51. DOI: http://dx.doi.org/10.1109/ISWC.2005.22 Google ScholarGoogle ScholarDigital LibraryDigital Library
  29. Martin Petzold, Francesco Barbabella, Jan Bobeth, Dagmar Kern, Christopher Mayer, and Martin Morandell. 2013. Towards an ambient assisted living user interaction taxonomy. In CHI’13 Extended Abstracts on Human Factors in Computing Systems (CHI EA’13). ACM, New York, NY, 49--54. DOI: http://dx.doi.org/10.1145/2468356.2468366 Google ScholarGoogle ScholarDigital LibraryDigital Library
  30. Parisa Rashidi and Diane J. Cook. 2010. Mining sensor streams for discovering human activity patterns over time. In Proceedings of the 10th IEEE International Conference on Data Mining (ICDM’10). IEEE, Los Alamitos, CA, 431--440. DOI: http://dx.doi.org/10.1109/ICDM.2010.40 Google ScholarGoogle ScholarDigital LibraryDigital Library
  31. Parisa Rashidi, Diane J. Cook, Lawrence B. Holder, and Maureen Schmitter-Edgecombe. 2011. Discovering activities to recognize and track in a smart environment. IEEE Transactions on Knowledge and Data Engineering 23, 4, 527--539. Google ScholarGoogle ScholarDigital LibraryDigital Library
  32. Philip Resnik. 1995. Using information content to evaluate semantic similarity in a taxonomy. In Proceedings of the 14th International Joint Conference on Artificial Intelligence, Vol. 1. Morgan Kaufmann, San Francisco, CA, 448--453. Google ScholarGoogle ScholarDigital LibraryDigital Library
  33. Daniele Riboni and Claudio Bettini. 2009. Context-aware activity recognition through a combination of ontological and statistical reasoning. In Proceedings of the 6th International Conference on Ubiquitous Intelligence and Computing (UIC’09). Springer-Verlag, Berlin, Heidelberg, 39--53. Google ScholarGoogle ScholarDigital LibraryDigital Library
  34. Daniel Roggen, Alberto Calatroni, Mirco Rossi, Thomas Holleczek, Killian Forster, Gerhard Troster, Paul Lukowicz, David Bannach, Gerald Pirkl, Alois Ferscha, Jakob Doppler, Clemens Holzmann, Marc Kurz, Gerald Holl, Ricardo Chavarriaga, Hesam Sagha, Hamidreza Bayati, Marco Creatura, and Jose del R. Millan. 2010. Collecting complex activity datasets in highly rich networked sensor environments. In Proceedings of the 7th International Conference on Networked Sensing Systems (INSS’10). 233--240.Google ScholarGoogle ScholarCross RefCross Ref
  35. Geetika Singla and Diane J. Cook. 2010. Recognizing independent and joint activities among multiple residents in smart environments. Ambient Intelligent and Humanized Computing 1, 1, 57--83.Google ScholarGoogle ScholarCross RefCross Ref
  36. Emmanuel Munguia Tapia, Tanzeem Choudhury, and Matthai Philipose. 2006. Building reliable activity models using hierarchical shrinkage and mined ontology. In Pervasive Computing. Lecture Notes in Computer Science, Vol. 2968, Springer, 17--32. Google ScholarGoogle ScholarDigital LibraryDigital Library
  37. Emmanuel Munguia Tapia, Stephen S. Intille, and Kent Larson. 2004a. Activity recognition in the home using simple and ubiquitous sensors. In Pervasive Computing. Lecture Notes in Computer Science, Vol. 3001. Springer, 158--175. DOI: http://dx.doi.org/10.1007/978-3-540-24646-6_10Google ScholarGoogle Scholar
  38. Emmanuel Munguia Tapia, Stephen S. Intille, and Kent Larson. 2004b. Activity recognition in the home using simple and ubiquitous sensors. In Pervasive Computing. Lecture Notes in Computer Science, Vol. 3001. Springer, 158--175. DOI: http://dx.doi.org/10.1007/978-3-540-24646-6_10Google ScholarGoogle Scholar
  39. Tim Van Kasteren, Athanasios Noulas, Gwenn Englebienne, and Ben Kröse. 2008. Accurate activity recognition in a home setting. In Proceedings of the 10th International Conference on Ubiquitous Computing (UbiComp’08). ACM, New York, NY, 1--9. DOI: http://dx.doi.org/10.1145/1409635.1409637 Google ScholarGoogle ScholarDigital LibraryDigital Library
  40. Ulrike Von Luxburg. 2010. Clustering stability: An overview. Foundations and Trends in Machine Learning 3, 2, 235--274. Google ScholarGoogle ScholarDigital LibraryDigital Library
  41. Shiaokai Wang, William Pentney, Ana-Maria Popescu, Tanzeem Choudhury, and Matthai Philipose. 2007. Common sense based joint training of human activity recognizers. In Proceedings of the 20th International Joint Conference on Artificial Intelligence (IJCAI’07). Morgan Kaufmann, San Francisco, CA, 2237--2242. http://dl.acm.org/citation.cfm?id=1625275.1625636 Google ScholarGoogle ScholarDigital LibraryDigital Library
  42. Zhibiao Wu and Martha Palmer. 1994. Verbs semantics and lexical selection. In Proceedings of the 32nd Annual Meeting of the Association for Computational Linguistics (ACL’94). Association for Computational Linguistics, Stroudsburg, PA, 133--138. DOI: http://dx.doi.org/10.3115/981732.981751 Google ScholarGoogle ScholarDigital LibraryDigital Library
  43. Juan Ye, Simon Dobson, and Susan McKeever. 2012. Situation identification techniques in pervasive computing: A review. Pervasive and Mobile Computing 8, 1, 36--66. Google ScholarGoogle ScholarDigital LibraryDigital Library
  44. Shumin Zhai and Victoria Bellotti. 2005. Introduction to sensing-based interaction. ACM Transactions on Computer-Human Interaction 12, 1, 1--2. DOI: http://dx.doi.org/10.1145/1057237.1057238 Google ScholarGoogle ScholarDigital LibraryDigital Library
  45. Vincent Wenchen Zheng, Derek Hao Hu, and Qiang Yang. 2009. Cross-domain activity recognition. In Proceedings of the 11th International Conference on Ubiquitous Computing (Ubicomp’09). ACM, New York, NY, 61--70. DOI: http://dx.doi.org/10.1145/1620545.1620554 Google ScholarGoogle ScholarDigital LibraryDigital Library

Index Terms

  1. USMART: An Unsupervised Semantic Mining Activity Recognition Technique

    Recommendations

    Comments

    Login options

    Check if you have access through your login credentials or your institution to get full access on this article.

    Sign in

    Full Access

    • Published in

      cover image ACM Transactions on Interactive Intelligent Systems
      ACM Transactions on Interactive Intelligent Systems  Volume 4, Issue 4
      Special Issue on Activity Recognition for Interaction and Regular Article
      January 2015
      190 pages
      ISSN:2160-6455
      EISSN:2160-6463
      DOI:10.1145/2688469
      Issue’s Table of Contents

      Copyright © 2014 ACM

      Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

      Publisher

      Association for Computing Machinery

      New York, NY, United States

      Publication History

      • Published: 13 November 2014
      • Revised: 1 June 2014
      • Accepted: 1 June 2014
      • Received: 1 September 2013
      Published in tiis Volume 4, Issue 4

      Permissions

      Request permissions about this article.

      Request Permissions

      Check for updates

      Qualifiers

      • research-article
      • Research
      • Refereed

    PDF Format

    View or Download as a PDF file.

    PDF

    eReader

    View online with eReader.

    eReader