Activity recognition using ST-GCN with 3D motion data

ABSTRACT

For the Nurse Care Activity Recognition Challenge, an activity recognition algorithm was developed by Team TDU-DSML. A spatial-temporal graph convolutional network (ST-GCN) was applied to process 3D motion capture data included in the challenge dataset. Time-series data was divided into 20-second segments with a 10-second overlap. The recognition model with a tree-structure graph was then created. The prediction result was set to one-minute segments on the basis of a majority decision from each segment output. Our model was evaluated by using leave-one-subject-out cross-validation methods. An average accuracy of 57% for all six subjects was achieved.

References

1. S. Inoue, P. Lago, S. Takeda, A. Shamma, F. Faiz, N. Mairittha, and T. Mairittha, "Nurse Care Activity Recognition Challenge," 2019.
2. M. Ye, Q. Zhang, L. Wang, J. Zhu, R. Yang, and J. Gall, "A Survey on Human Motion Analysis from Depth Data," In M. Grzegorzek, C. Theobal, R. Koch, and A. Kolb (eds.) "Time-of-Flight and Depth Imaging. Sensors, Algorithms, and Applications," Lecture Notes in Computer Science, vol. 8200, pp. 7--45, Springer, Berlin, Heidelberg, 2013.
3. L. L. Presti and M. L. Cascia, "3D Skeleton-Based Human Action Classification," Pattern Recognition, vol. 53, pp. 130--147, 2016.
4. S.-E. Wei, V. Ramakrishna, T. Kanade, and Y. Sheikh, "Convolutional Pose Machines," arXiv preprint, arXiv:1602.00134, 2016.
5. W. Zhu, C. Lan, J. Xing, W. Zeng, Y. Li, L. Shen, and X. Xie, "Co-Occurrence Feature Learning for Skeleton Based Action Recognition Using Regularized Deep LSTM Networks," arXiv preprint, arXiv:1603.07772, 2016.
6. S. Song, C. Lan, J. Xing, W. Zeng, and J. Liu, "An End-to-End Spatio-Temporal Attention Model for Human Action Recognition from Skeleton Data," arXiv preprint, arXiv:1611.06067, 2016.
7. S. Yan, Y. Xiong, D. Lin "Spatial Temporal Graph Convolutional Networks for Skeleton-Based Action Recognition," arXiv preprint, arXiv:1801.07455, 2018.
8. L. Shi, Y. Zhang, J. Cheng, and H. Lu, "Two-Stream Adaptive Graph Convolutional Networks for Skeleton-Based Action Recognition," Proc. IEEE CVPR, pp. 12026--12035, 2019.
9. Y. Wang, Y. Sun, Z. Liu, S. E. Sarma, M. M. Bronstein, and J. M. Solomon, "Dynamic Graph CNN for Learning on Point Clouds," arXiv preprint, arXiv:1801.07829, 2018.
10. P. Diederik and J, Lei. "Adam: A Method for Stochastic Optimization," arXiv preprint, arXiv:1412.6980, 2014.
11. S. S. Alia, P. Lago, S. Takeda, T. Mairittha, N. Mairittha, F. Faiz, Y. Nishimura, K. Adachi, T. Okita, S. Inoue, and F. Charppillet, "Nurse Care Activity Recognition Challenge: Summary and Results," Proc. HASCA 2019.

Index Terms

• Computing methodologies

  • Artificial intelligence

    • Computer vision

      • Computer vision tasks

        • Activity recognition and understanding

  • Machine learning

• Human-centered computing

  • Human computer interaction (HCI)

    • HCI design and evaluation methods

      • User models