Nonlinear synchronization for automatic learning of 3D pose variability in human motion sequences

Abstract

A dense matching algorithm that solves the problem of synchronizing prerecorded human motion sequences, which show different speeds and accelerations, is proposed. The approach is based on minimization of MRF energy and solves the problem by using Dynamic Programming. Additionally, an optimal sequence is automatically selected from the input dataset to be a time-scale pattern for all other sequences. The paper utilizes an action specific model which automatically learns the variability of 3D human postures observed in a set of training sequences. The model is trained using the public CMU motion capture dataset for the walking action, and a mean walking performance is automatically learnt. Additionally, statistics about the observed variability of the postures and motion direction are also computed at each time step. The synchronized motion sequences are used to learn a model of human motion for action recognition and full-body tracking purposes.

References

1. T. B. Moeslund and E. Granum, "A survey of computer vision-based human motion capture," Computer Vision and Image Understanding, vol. 81, no. 3, pp. 231-268, 2001.
2. T. B. Moeslund, A. Hilton, and V. Krüger, "A survey of advances in vision-based human motion capture and analysis," Computer Vision and Image Understanding, vol. 104, no. 2-3, pp. 90-126, 2006.
3. L. Wang, W. Hu, and T. Tan, "Recent developments in human motion analysis," Pattern Recognition, vol. 36, no. 3, pp. 585- 601, 2003.
4. I. Rius, J. Varona, J. Gonzalez, and J. J. Villanueva, "Action spaces for efficient Bayesian tracking of human motion," in Proceedings of the 18th International Conference on Pattern Recognition (ICPR '06), vol. 1, pp. 472-475, 2006.
5. I. Rius, J. Varona, X. Roca, and J. Gonzàlez, "Posture constraints for bayesian human motion tracking," in Proceedings of the 4th International Conference on Articulated Motion and Deformable Objects (AMDO '06), vol. 4069, pp. 414-423, Port d'Andratx, Spain, July 2006.
6. L. Sigal and M. J. Black, "Measure locally, reason globally: occlusion-sensitive articulated pose estimation," in Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR '06), vol. 2, pp. 2041- 2048, 2006.
7. J. Gonzàlez, J. Varona, X. Roca, and J. J. Villanueva, "Analysis of human walking based on aSpaces," in Articulated Motion and Deformable Objects, vol. 3179 of Lecture Notes in Computer Science, pp. 177-188, Springer, Berlin, Germany, 2004.
8. T. J. Roberts, S. J. McKenna, and I. W. Ricketts, "Adaptive learning of statistical appearance models for 3D human tracking," in Proceedings of the British Machine Vision Conference (BMVC '02), pp. 121-165, Cardiff, UK, September 2002.
9. H. Sidenbladh, M. J. Black, and L. Sigal, "Implicit probabilistic models of human motion for synthesis and tracking," in Proceedings of the 7th European Conference on Computer Vision Copenhagen (ECCV '02), vol. 2350 of Lecture Notes in Computer Science, pp. 784-800, Springer, Copenhagen, Denmark, May 2002.
10. R. Urtasun, D. J. Fleet, A. Hertzmann, and P. Fua, "Priors for people tracking from small training sets," in Proceedings of the 10th IEEE International Conference on Computer Vision (ICCV '05), vol. 1, pp. 403-410, October 2005.
11. A. Nakazawa, S. Nakaoka, and K. Ikeuchi, "Matching and blending human motions using temporal scaleable dynamic programming," in Proceedings of the IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS '04), vol. 1, pp. 287-294, Sendai, Japan, September-October 2004.
12. Y. Bilu, P. K. Agarwal, and R. Kolodny, "Faster algorithms for optimal multiple sequence alignment based on pairwise comparisons," in Proceedings of the IEEE/ACM Transactions on Computational Biology and Bioinformatics (TCBB '06), vol. 3, pp. 408-422, October-December 2006.
13. M. Gong and Y.-H. Yang, "Real-time stereo matching using orthogonal reliability-based dynamic programming," IEEE Transactions on Image Processing, vol. 16, no. 3, pp. 879-884, 2007.
14. J. L. Williams, J. W. Fisher III, and A. S. Willsky, "Approximate dynamic programming for communication-constrained sensor network management," IEEE Transactions on Signal Processing, vol. 55, no. 8, pp. 4300-4311, 2007.
15. J. Kruskall and M. Liberman, "The symmetric time warping problem: from continuous to discrete," in Time Warps, String Edits, and Macromolecules: The Theory and Practice of Sequence Comparison, pp. 125-161, Madison-Wesley, Reading, Mass, USA, 1983.
16. E. Keogh and M. Pazzani, "Derivative dynamic time warping," in Proceedings of the 1st SIAM International Conference on Data Mining, pp. 1-12, Chicago, Ill, USA, 2001.
17. I. Rius, D. Rowe, J. Gonzalez, and F. Xavier Roca, "3D action modeling and reconstruction for 2D human body tracking," in Proceedings of the 3rd International Conference on Advances in Patten Recognition (ICAPR '05), vol. 3687, pp. 146-154, Bath, UK, August 2005.
18. V. M. Zatsiorsky, Kinematics of Human Motion, chapter 1, Human Kinematics, 1998.

Index Terms

• Computing methodologies

  • Artificial intelligence

    • Computer vision

      • Computer vision problems

      • Computer vision tasks

        • Scene understanding

  • Machine learning