Real-time classification of dance gestures from skeleton animation

ABSTRACT

We present a real-time gesture classification system for skeletal wireframe motion. Its key components include an angular representation of the skeleton designed for recognition robustness under noisy input, a cascaded correlation-based classifier for multivariate time-series data, and a distance metric based on dynamic time-warping to evaluate the difference in motion between an acquired gesture and an oracle for the matching gesture. While the first and last tools are generic in nature and could be applied to any gesture-matching scenario, the classifier is conceived based on the assumption that the input motion adheres to a known, canonical time-base: a musical beat. On a benchmark comprising 28 gesture classes, hundreds of gesture instances recorded using the XBOX Kinect platform and performed by dozens of subjects for each gesture class, our classifier has an average accuracy of 96:9%, for approximately 4-second skeletal motion recordings. This accuracy is remarkable given the input noise from the real-time depth sensor.

References

1. {AF02} Arikan O., Forsyth D. A.: Interactive motion generation from examples. ACM Trans. Graph. (2002). 2
2. {BD01} Bobick A., Davis J.: The recognition of human movement using temporal templates. IEEE Trans. on Pattern Anal. and Machine Intell (2001). 2
3. {BH00} Brand M., Hertzmann A.: Style machines. In Proc. of the conf. on Computer graphics and interactive techniques (2000), SIGGRAPH '00. 2
4. {Bis06} Bishop C.: Pattern recognition and machine learning, vol. 4. Springer New York, 2006. 6, 7
5. {BSP*04} Barbič J., Safonova A., Pan J.-Y., Faloutsos C., Hodgins J. K., Pollard N. S.: Segmenting motion capture data into distinct behaviors. In Proc. of Graphics Interface (2004). 2
6. {CB95} Campbell L., Bobick A.: Recognition of human body motion using phase space constraints. In Proc. of Intl. Conf. of Computer Vision (1995). 2
7. {FF05} Forbes K., Fiume E.: An efficient search algorithm for motion data using weighted pca. In Proc. of the ACM SIGGRAPH/Eurographics Symposium on Computer animation (2005). 3
8. {FRM94} Faloutsos C., Ranganathan M., Manolopoulos Y.: Fast subsequence matching in time-series databases. In Proc. of the ACM SIGMOD (1994). 3
9. {Har} Harmonix Music Systems: www.dancecentral.com. 1
10. {HGP04} Hsu E., Gentry S., Popović J.: Example-based control of human motion. In Proc. of the ACM SIGGRAPH/Eurographics symposium on Computer animation (2004). 1
11. {IWZL09} Ishigaki S., White T., Zordan V. B., Liu C. K.: Performance-based control interface for character animation. ACM Trans. Graph. (2009). 2
12. {Joh73} Johansson G.: Visual perception of biological motion and a model for its analysis. Perceiving events and objects (1973). 2
13. {Keo02} Keogh E.: Exact indexing of dynamic time warping. In Proc of the Intl. conf. on VLDB (2002). 3
14. {KGP02} Kovar L., Gleicher M., Pighin F.: Motion graphs. ACM Trans. Graph. (2002). 2
15. {KOF05} Kirk A., O'Brien J., Forsyth D.: Skeletal parameter estimation from optical motion capture data. In Proc. of Conf. Computer Vision and Pattern Recognition (2005). 3
16. {KPS03} Kim T., Park S., Shin S.: Rhythmic-motion synthesis based on motion-beat analysis. ACM Trans. Graph. (2003). 1
17. {KPZ*04} Keogh E., Palpanas T., Zordan V., Gunopulos D., Cardle M.: Indexing large human-motion databases. In Proc. of the Intl. Conf. on VLDB (2004). 2, 3
18. {LCR*02} Lee J., Chai J., Reitsma P., Hodgins J., Pollard N.: Interactive control of avatars animated with human motion data. ACM Trans. Graph. (2002). 2
19. {LNL05} Lv F., Nevatia R., Lee M. W.: 3D human action recognition using spatio-temporal motion templates. In Computer Vision in Human-Computer Interaction (2005). 2
20. {LZWM05} Liu G., Zhang J., Wang W., McMillan L.: A system for analyzing and indexing human-motion databases. In Proc. of the ACM SIGMOD (2005). 3
21. {MR06} Müller M., Röder T.: Motion templates for automatic classification and retrieval of motion capture data. In Proc. of the ACM SIGGRAPH/Eurographics Symposium on Computer animation (2006). 2, 3
22. {MRC05} Müller M., Röder T., Clausen M.: Efficient content-based retrieval of motion capture data. In ACM SIGGRAPH (2005). 3
23. {OFH08} Onuma K., Faloutsos C., Hodgins J.: FMDistance: A fast and effective distance function for motion capture data. Short Papers Proc. of EUROGRAPHICS (2008). 3, 9
24. {RCB02} Rose C., Cohen M. F., Bodenheimer B.: Verbs and adverbs: Multidimensional motion interpolation. IEEE Computer Graphics and Applications (2002). 10
25. {SFC*11} Shotton J., Fitzgibbon A., Cook M., Sharp T., Finocchio M., Moore R., Kipman A., Blake A.: Real-time human pose recognition in parts from single depth images. In Proc. Conf. Computer Vision and Pattern Recognition (2011). 3
26. {SH08} Slyper R., Hodgins J.: Action capture with accelerometers. In Proc. of the ACM SIGGRAPH/Eurographics Symposium on Computer Animation (2008). 9
27. {SLC04} Schuldt C., Laptev I., Caputo B.: Recognizing human actions: A local SVM approach. In Proc. Intl. Conf. on Pattern Recognition (2004). 2
28. {TC02} Tzanetakis G., Cook P.: Musical genre classification of audio signals. IEEE Trans. on Speech and Audio Processing (2002). 5
29. {WTK87} Witkin A., Terzopoulos D., Kass M.: Signal matching through scale space. Intl. Journal of Computer Vision (1987). 6

Index Terms

• Computing methodologies

  • Computer graphics

    • Animation