Kristina Yordanova
Thomas Kirste
Skip Abstract Section
Abstract
Several emerging approaches to activity recognition (AR) combine symbolic representation of user actions with probabilistic elements for reasoning under uncertainty. These approaches provide promising results in terms of recognition performance, coping with the uncertainty of observations, and model size explosion when complex problems are modelled. But experience has shown that it is not always intuitive to model even seemingly simple problems. To date, there are no guidelines for developing such models. To address this problem, in this work we present a development process for building symbolic models that is based on experience acquired so far as well as on existing engineering and data analysis workflows. The proposed process is a first attempt at providing structured guidelines and practices for designing, modelling, and evaluating human behaviour in the form of symbolic models for AR. As an illustration of the process, a simple example from the office domain was developed. The process was evaluated in a comparative study of an intuitive process and the proposed process. The results showed a significant improvement over the intuitive process. Furthermore, the study participants reported greater ease of use and perceived effectiveness when following the proposed process. To evaluate the applicability of the process to more complex AR problems, it was applied to a problem from the kitchen domain. The results showed that following the proposed process yielded an average accuracy of 78%. The developed model outperformed state-of-the-art methods applied to the same dataset in previous work, and it performed comparably to a symbolic model developed by a model expert without following the proposed development process.
- John. R. Anderson. 1983. The Architecture of Cognition. Lawrence Erlbaum Associates, Mahwah, NJ. Google Scholar
Digital Library
- Sebastian Bader, Gernot Ruscher, and Thomas Kirste. 2010. A middleware for rapid prototyping smart environments. In Proceedings of the 12th ACM International Conference Adjunct Papers on Ubiquitous Computing. ACM, New York, NY, 355--356. DOI:http://dx.doi.org/10.1145/1864431.1864433 Google ScholarDigital Library
- Chris L. Baker, Rebecca Saxe, and Joshua B. Tenenbaum. 2009. Action understanding as inverse planning. Cognition 113, 3, 329--349.Google ScholarCross Ref
- Osman Balci. 2012. A life cycle for modeling and simulation. Simulation: Transactions of the Society for Modeling and Simulation International 88, 4, 1--14. Google ScholarDigital Library
- Jerry Banks, John S. Carlson II, Barry L. Nelson, and David M. Nicol. 2010. Discrete-Event-System Simulation. Pearson, Upper Saddle River, NJ.Google Scholar
- Barry W. Boehm. 1988. A spiral model of software development and enhancement. Computer 21, 5, 61--72. DOI:http://dx.doi.org/10.1109/2.59 Google ScholarDigital Library
- L. Chen, J. Hoey, C. D. Nugent, D. J. Cook, and Z. Yu. 2012. Sensor-based activity recognition. IEEE Transactions on Systems, Man, and Cybernetics, Part C: Applications and Reviews 42, 6, 790--808. DOI:http://dx.doi.org/10.1109/TSMCC.2012.2198883 Google ScholarDigital Library
- L. Chen, C. Nugent, and G. Okeyo. 2014. An ontology-based hybrid approach to activity modeling for smart homes. IEEE Transactions on Human-Machine Systems 44, 1, 92--105. DOI:http://dx.doi.org/10.1109/THMS.2013.2293714Google ScholarCross Ref
- Jongmyung Choi, RosaI. Arriaga, Hyun-Joo Moon, and Eun-Ser Lee. 2011. A context-driven development methodology for context-aware systems. In Convergence and Hybrid Information Technology. Lecture Notes in Computer Science, Vol. 6935. Springer, 429--436. DOI:http://dx.doi.org/10.1007/978-3-642-24082-9_53 Google ScholarDigital Library
- Paul R. Cohen. 1995. Empirical Methods for Artificial Intelligence. MIT Press, Cambridge, MA. Google ScholarDigital Library
- Mark Donnelly, Tommaso Magherini, Chris Nugent, Federico Cruciani, and Cristiano Paggetti. 2011. Annotating sensor data to identify activities of daily living. In Toward Useful Services for Elderly and People with Disabilities. Lecture Notes in Computer Science, Vol. 6719. Springer, 41--48. DOI:http://dx.doi.org/10.1007/978-3-642-21535-3_6 Google ScholarDigital Library
- Richard O. Duda, Peter E. Hart, and David G. Stork. 2001. Pattern Classification. Wiley & Sons, New York, NY. Google ScholarDigital Library
- Tore Dyba. 2000. An instrument for measuring the key factors of success in software process improvement. Empirical Software Engineering 5, 4, 357--390. DOI:http://dx.doi.org/10.1023/A:1009800404137 Google ScholarDigital Library
- Gerhard Fischer. 2012. Context-aware systems: The ‘right’ information, at the ‘right’ time, in the ‘right’ place, in the ‘right’ way, to the ‘right’ person. In Proceedings of the International Working Conference on Advanced Visual Interfaces (AVI’12). ACM, New York, NY, 287--294. DOI:http://dx.doi.org/10.1145/2254556.2254611 Google ScholarDigital Library
- Frederic Fondement and Raul Silaghi. 2004. Defining model driven engineering processes. In Proceedings of the 3rd Workshop in Software Model Engineering Satellite Workshop at the 7th International Conference on the UML.Google Scholar
- Mariano Frenandez-Lopez, Asuncion Gomez-Perez, and Natalia Juristo. 1997. METHONTOLOGY: From ontological art towards ontological engineering. In Proceedings of the Spring Symposium on Ontological Engineering of AAAI. 33--40.Google Scholar
- Milton Friedman. 1937. The use of ranks to avoid the assumption of normality implicit in the analysis of variance. Journal of the American Statistical Association 32, 200, 675--701.Google ScholarCross Ref
- Asuncion Gomez-Perez, Mariano Frenandez-Lopez, and Oscar Corcho. 2004. Ontological Engineering. Springer-Verlag, London. UK. Google ScholarDigital Library
- Avelino J. Gonzalez and Douglas D. Dankel. 1993. The Engineering Book of Knowledge-Based Systems: Theory and Practice. Prentice Hall, Hemel Hempstead, UK. Google ScholarDigital Library
- Bastian Hartmann. 2011. Human Worker Activity Recognition in Industrial Environments. Ph.D. Dissertation. Karlsruher Institut für Technologie, Karlsruhe, Germany.Google Scholar
- Laura M. Hiatt, Anthony M. Harrison, and J. Gregory Trafton. 2011. Accommodating human variability in human-robot teams through theory of mind. In Proceedings of the 22nd International Joint Conference on Artificial Intelligence (IJCAI’11). 2066--2071. DOI:http://dx.doi.org/10.5591/978-1-57735-516-8/IJCAI11-345 Google ScholarDigital Library
- Jesse Hoey, Thomas Plötz, Dan Jackson, Andrew Monk, Cuong Pham, and Patrick Olivier. 2011. Rapid specification and automated generation of prompting systems to assist people with dementia. Journal of Pervasive Mobile Computing 7, 3, 299--318. DOI:http://dx.doi.org/10.1016/j.pmcj.2010.11.007 Google ScholarDigital Library
- Jesse Hoey, Pascal Poupart, Axel von Bertoldi, Tammy Craig, Craig Boutilier, and Alex Mihailidis. 2010. Automated handwashing assistance for persons with dementia using video and a partially observable Markov decision process. Computer Vision and Image Understanding 114, 5, 503--519. DOI:http://dx.doi.org/10.1016/j.cviu.2009.06.008 Google ScholarDigital Library
- Watts S. Humphrey. 1989. Managing the Software Process. Addison Wesley Longman, Boston, MA. Google ScholarDigital Library
- Watts S. Humphrey. 2000. The Personal Software Process (PSP). Technical Report CMU/SEI-2000-TR-022. Software Engineering Institute, Carnegie Mellon University, Pittsburgh, PA.Google ScholarCross Ref
- Jadwiga Indulska and Karen Henricksen. 2008. Context awareness. In The Engineering Handbook of Smart Technology for Aging, Disability, and Independence, A. Helal, M. Mokhtari, and B. Abdulrazak (Eds.). John Wiley & Sons, Hoboken, NJ, 585--606. http://dx.doi.org/10.1002/9780470379424.ch31Google Scholar
- Jaykaran. 2010. How to select appropriate statistical test? Journal of Pharmaceutical Negative Results 1, 2, 61--63. DOI:http://dx.doi.org/10.4103/0976-9234.75708Google ScholarCross Ref
- David A. Kenny. 1987. Statistics for the Social and Behavioral Sciences. Little, Brown, Boston, MA.Google Scholar
- Thomas Kirste and Frank Krüger. 2012. CCBM—A Tool for Activity Recognition Using Computational Causal Behavior Models. Technical Report CS-01-12. Institut für Informatik, Universität Rostock, Rostock, Germany.Google Scholar
- Frank Krüger, Alexander Steiniger, Sebastian Bader, and Thomas Kirste. 2012a. Evaluating the robustness of activity recognition using computational causal behavior models. In Proceedings of the 2012 ACM Conference on Ubiquitous Computing (UbiComp’12). ACM, New York, NY, 1066--1074. DOI:http://dx.doi.org/10.1145/2370216.2370443 Google ScholarDigital Library
- Frank Krüger, Kristina Yordanova, Albert Hein, and Thomas Kirste. 2013. Plan synthesis for probabilistic activity recognition. In Proceedings of the 5th International Conference on Agents and Artificial Intelligence (ICAART’13). 283--288.Google Scholar
- Frank Krüger, Kristina Yordanova, Veit Köppen, and Thomas Kirste. 2012b. Towards tool support for computational causal behavior models for activity recognition. In Proceedings of the 1st Workshop on Situation-Aware Assistant Systems Engineering: Requirements, Methods, and Challenges (SeASE’12). 561--572.Google Scholar
- Frank Krüger, Martin Nyolt, Kristina Yordanova, Albert Hein, and Thomas Kirste. 2014. Computational state space models for activity and intention recognition. A feasibility study. PLoS ONE 9, 11, e109381. DOI:http://dx.doi.org/10.1371/journal.pone.0109381Google ScholarCross Ref
- Frank Krüger, Kristina Yordanova, Christoph Burghardt, and Thomas Kirste. 2012c. Towards creating assistive software by employing human behavior models. Journal of Ambient Intelligence and Smart Environments 4, 3, 209--226. DOI:http://dx.doi.org/10.3233/AIS-2012-0148 Google ScholarDigital Library
- Rensis Likert. 1932. A technique for the measurement of attitudes. Archives of Psychology 22, 140, 5--55.Google Scholar
- Kent Lyons, Helene Brashear, Tracy Westeyn, JungSoo Kim, and Thad Starner. 2007. GART: The gesture and activity recognition toolkit. In Human-Computer Interaction: HCI Intelligent Multimodal Interaction Environments. Lecture Notes in Computer Science, Vol. 4552. Springer, 718--727. DOI:http://dx.doi.org/10.1007/978-3-540-73110-8_78 Google ScholarDigital Library
- Paul Maier, Dominik Jain, and Martin Sachenbacher. 2011. Compiling AI engineering models for probabilistic inference. In KI 2011: Advances in Artificial Intelligence. Lecture Notes in Computer Science, Vol. 7006. Springer, 191--203. DOI:http://dx.doi.org/10.1007/978-3-642-24455-1_18 Google ScholarDigital Library
- Deborah J. Mayhew. 1999. The Usability Engineering Lifecycle: A Practitioner’s Handbook for User Interface Design. Morgan Kaufmann, San Francisco, CA. Google ScholarDigital Library
- Nolberto Munier. 2011. A Strategy for Using Multicriteria Analysis in Decision-Making. Springer, Netherlands.Google Scholar
- Jakob Nielsen and Thomas K. Landauer. 1993. A mathematical model of the finding of usability problems. In Proceedings of the International Conference on Human-Computer Interaction’93 and the Conference on Human Aspects in Computing Systems’93. ACM, New York, NY, 206--213. Google ScholarDigital Library
- Robert Nisbet, John Elder, and Gary Miner. 2009. Handbook of Statistical Analysis and Data Mining Applications. Elsevier, Toronto, Canada. Google ScholarDigital Library
- Martin Nyolt, Frank Krüger, Kristina Yordanova, Albert Hein, and Thomas Kirste. 2015. Marginal filtering in large state spaces. International Journal of Approximate Reasoning 61, 16--32. DOI:http://dx.doi. org/10.1016/j.ijar.2015.04.003 Google ScholarDigital Library
- Yoosoo Oh, A. Schmidt, and Woontack Woo. 2007. Designing, developing, and evaluating context-aware systems. In Proceedings of the International Conference on Multimedia and Ubiquitous Engineering, (MUE’07). IEEE, Los Alamitos, CA, 1158--1163. DOI:http://dx.doi.org/10.1109/MUE.2007.118 Google ScholarDigital Library
- George Okeyo, Liming Chen, Hui Wang, and Roy Sterritt. 2011. Ontology-based learning framework for activity assistance in an adaptive smart home. In Activity Recognition in Pervasive Intelligent Environments. Atlantis Ambient and Pervasive Intelligence, Vol. 4. Atlantis Press, Amsterdam, Netherlands, 237--263. DOI:http://dx.doi.org/ 10.2991/978-94-91216-05-3_11Google Scholar
- Ant Ozok. 2012. Survey Design and Implementation in HCI. CRC Press, New York, NY, 1259--1278.Google Scholar
- David Lorge Parnas and Paul C. Clements. 1986. A rational design process: How and why to fake it. IEEE Transactions on Software Engineering 12, 2, 251--257. http://dl.acm.org/citation.cfm?id=9794.9800 Google ScholarDigital Library
- Fabio Paternò, Cristiano Mancini, and Silvia Meniconi. 1997. ConcurTaskTrees: A diagrammatic notation for specifying task models. In Human-Computer Interaction: International Conference on Human-Computer Interaction’97, S. Howard, J. Hammond, and G. Lindgaard (Eds.). Springer, London, UK, 362--369. DOI:http://dx.doi.org/10.1007/978-0-387-35175-9_58 Google ScholarDigital Library
- Donald J. Patterson, Lin Liao, Dieter Fox, and Henry Kautz. 2003. Inferring high-level behavior from low-level sensors. In UbiComp 2003: Ubiquitous Computing. Lecture Notes in Computer Science, Vol. 2864. Springer, 73--89. DOI:http://dx.doi.org/10.1007/978-3-540-39653-6_6Google ScholarCross Ref
- David M. W. Powers. 2011. Evaluation: From precision, recall, and F-measure to ROC, informedness, markedness, and correlation. Journal of Machine Learning Technologies 2, 1, 37--63.Google ScholarCross Ref
- Miquel Ramírez and Hector Geffner. 2010. Probabilistic plan recognition using off-the-shelf classical planners. In Proceedings of the 24th National Conference of Artificial Intelligence (AAAI’10). 1211--1217.Google Scholar
- Miquel Ramírez and Hector Geffner. 2011. Goal recognition over POMDPs: Inferring the intention of a POMDP agent. In Proceedings of the 22nd International Joint Conference on Artificial Intelligence (IJCAI’11), Vol. 3. 2009--2014. DOI:http://dx.doi.org/10.5591/978-1-57735-516-8/IJCAI11-335Google Scholar
- Silvia Richter. 2011. Landmark-Based Heuristics and Search Control for Automated Planning. Ph.D. Dissertation. Institute of Intelligent and Integrated Systems, Griffith University, Brisbane, Australia.Google Scholar
- Silvia Richter and Matthias Westphal. 2010. The LAMA planner: Guiding cost-based anytime planning with landmarks. Journal of Artificial Intelligence Research 39, 1, 127--177. http://dl.acm.org/citation.cfm?id=1946417.1946420. Google ScholarCross Ref
- Patrice C. Roy, Sylvain Giroux, Bruno Bouchard, Abdenour Bouzouane, Clifton Phua, Andrei Tolstikov, and Jit Biswas. 2011. A possibilistic approach for activity recognition in smart homes for cognitive assistance to Alzheimer’s patients. In Activity Recognition in Pervasive Intelligent Environments. Atlantis Ambient and Pervasive Intelligence, Vol. 4. Atlantis Press, Amsterdam, 33--58. http://dx.doi.org/10.2991/978-94-91216-05-3\_2.Google Scholar
- Walker W. Royce. 1987. Managing the development of large software systems: Concepts and techniques. In Proceedings of the 9th International Conference on Software Engineering (ICSE’87). IEEE, Los Alamitos, CA, 328--338. http://dl.acm.org/citation.cfm?id=41765.41801. Google ScholarDigital Library
- Ioana Rus, Holger Neu, and Jürgen Münch. 2003. A systematic methodology for developing discrete event simulation models of software development processes. In Proceedings of the 4th International Workshop on Software Process Simulation Modeling (ProSim) at the International Conference on Software Engineering.Google Scholar
- Colin Shearer. 2000. The CRISP-DM model: The new blueprint for data mining. Journal of Data Warehousing 5, 4, 13--22.Google Scholar
- Ian Sommerville. 1996. Software Engineering. Addison-Wesley, Boston, MA. Google ScholarDigital Library
- Steffen Staab, Rudi Studer, Hans-Peter Schnurr, and York Sure. 2001. Knowledge processes and ontologies. IEEE Intelligent Systems 16, 1, 26--34. DOI:http://dx.doi.org/10.1109/5254.912382 Google ScholarDigital Library
- H. Storf, M. Becker, and M. Riedl. 2009. Rule-based activity recognition framework: Challenges, technique and learning. In Proceedings of the 3rd International Conference on Pervasive Computing Technologies for Healthcare (PervasiveHealth’09). 1--7. DOI:http://dx.doi.org/10.4108/ICST.PERVASIVEHEALTH2009.6108Google ScholarCross Ref
- Fernando de la Torre, Jessica Hodgins, Javier Montano, and Sergio Valcarcel. 2009. Guide to the Carnegie Mellon University Multimodal Activity (CMU-MMAC) Database. Technical Report CMU-RI-TR-08-22. Robotics Institute, Carnegie Mellon University, Pittsburgh, PA.Google Scholar
- Dorra Trabelsi, Sabah Mohammed, Faicel Chamroukhi, Latifa Oukhellou, and Yacine Amirat. 2013. An unsupervised approach for automatic activity recognition based on hidden Markov model regression. IEEE Transactions on Automation Science and Engineering 10, 3, 829--835.Google ScholarCross Ref
- J. Grogory Trafton, Laura M. Hiatt, Anthony M. Harrison, Franklin P. Tamborello, Sangeet S. Khemlani, and Alan C. Schultz. 2013. ACT-R/E: An embodied cognitive architecture for human-robot interaction. Journal of Human-Robot Interaction 2, 1, 30--55.Google ScholarDigital Library
- Michael Unterkalmsteiner, Tony Gorschek, A. K. M. Moinul Islam, Chow Kian Cheng, Rahadian B. Permadi, and Robert Feldt. 2012. Evaluation and measurement of software process improvement—a systematic literature review. IEEE Transactions on Software Engineering 38, 2, 398--424. DOI:http://dx.doi.org/10.1109/TSE.2011.26 Google ScholarDigital Library
- Mike Uschold and Martin King. 1995. Towards a methodology for building ontologies. In Proceedings of the Workshop on Basic Ontological Issues in Knowledge Sharing in Conjunction with the International Joint Conference in Artificial Intelligence (IJCAI-95). University of Edinburgh, Montreal, Canada, 6.1--6.10.Google Scholar
- Douglas L. Vail, Manuela M. Veloso, and John D. Lafferty. 2007. Conditional random fields for activity recognition. In Proceedings of the 6th International Joint Conference on Autonomous Agents and Multiagent Systems (AAMAS’07). ACM, New York, NY, Article No. 235. DOI:http://dx.doi.org/10.1145/1329125.1329409 Google ScholarDigital Library
- T. L. M. van Kasteren and B. J. A. Kröse. 2009. A sensing and annotation system for recording datasets in multiple homes. In Proceedings of the 27th Annual Conference on Human Factors and Computing Systems. ACM, New York, NY, 4763--4766.Google Scholar
- Tim L. M. van Kasteren, Athanasios Noulas, Gwenn Englebienne, and Ben Kröse. 2008. Accurate activity recognition in a home setting. In Proceedings of the 10th International Conference on Ubiquitous Computing. ACM, New York, NY, 1--9. DOI:http://dx.doi.org/10.1145/1409635.1409637 Google ScholarDigital Library
- Jean Vanderdonckt. 2008. Model-driven engineering of user interfaces: Promises, successes, failures, and challenges. In Proceedings of the Annual Romanian Conference on Human-Computer Interaction, Vol. 8. 1--10.Google Scholar
- Jamie A. Ward, Paul Lukowicz, and Hans W. Gellersen. 2011. Performance metrics for activity recognition. ACM Transactions on Intelligent Systems and Technology 2, 1, Article No. 6. DOI:http://dx.doi.org/10.1145/1889681.1889687 Google ScholarDigital Library
- Juan Ye, Graeme Stevenson, and Simon Dobson. 2014. USMART: An unsupervised semantic mining activity recognition technique. ACM Transactions on Interactive Intelligent Systems 4, 4, Article No. 16. DOI:http://dx.doi.org/10.1145/2662870 Google ScholarDigital Library
- Kristina Yordanova. 2011a. Human behaviour modelling approach for intention recognition in ambient assisted living. In Ambient Intelligence—Software and Applications. Advances in Intelligent and Soft Computing, Vol. 92. Springer, 247--251. DOI:http://dx.doi.org/10.1007/978-3-642-19937-0_32Google Scholar
- Kristina Yordanova. 2011b. Modelling human behaviour using partial order planning based on atomic action templates. In Proceedings of the 7th International Conference on Intelligent Environments. IEEE, Los Alamitos, CA, 338--341. DOI:http://dx.doi.org/10.1109/IE.2011.27 Google ScholarDigital Library
- Kristina Yordanova. 2014. Methods for Engineering Symbolic Human Behaviour Models for Activity Recognition. Ph.D. Dissertation. Institute of Computer Science, Rostock, Germany.Google Scholar
- Kristina Yordanova, Frank Krüger, and Thomas Kirste. 2012a. Strategies for modelling human behaviour for activity recognition with precondition-effect rules. In KI 2012: Advances in Artificial Intelligence. Lecture Notes in Computer Science, Vol. 7526. Springer, 257--261. DOI:http://dx.doi.org/10.1007/978-3-642-33347-7_27 Google ScholarDigital Library
- Kristina Yordanova, Frank Krüger, and Thomas Kirste. 2012b. Context aware approach for activity recognition based on precondition-effect rules. In Proceedings of the 2012 IEEE International Conference on Pervasive Computing and Communications Workshops (PERCOM Workshops’12). IEEE, Los Alamitos, CA, 602--607. DOI:http://dx.doi.org/10.1109/PerComW.2012.6197586Google ScholarCross Ref
- Kristina Yordanova, Martin Nyolt, and Thomas Kirste. 2014. Strategies for reducing the complexity of symbolic models for activity recognition. In Artificial Intelligence: Methodology, Systems, and Applications. Lecture Notes in Computer Science, Vol. 8722. Springer, 295--300. DOI:http://dx.doi.org/10.1007/978-3-319-10554-3_31Google Scholar
Index Terms
- A Process for Systematic Development of Symbolic Models for Activity Recognition
Recommendations
Ten steps applied to development and evaluation of process-based biogeochemical models of estuaries
The procedures involved in model development may be set out as a ten step process, beginning with defining the purpose of the model and ending with evaluation of the appropriateness and utility of the completed model. This process, recently outlined by ...
Approximately bisimilar symbolic models for nonlinear control systems
Control systems are usually modeled by differential equations describing how physical phenomena can be influenced by certain control parameters or inputs. Although these models are very powerful when dealing with physical phenomena, they are less suited ...
Data Generation Process Modeling for Activity Recognition
Machine Learning and Knowledge Discovery in Databases: Applied Data Science TrackAbstractThe dynamics of body movements are often driven by large and intricate low-level interactions involving various body parts. These dynamics are part of an underlying data generation process. Incorporating the data generation process into data-...
Comments