KinÊtre: animating the world with the human body

ABSTRACT

KinÊtre allows novice users to scan arbitrary physical objects and bring them to life in seconds. The fully interactive system allows diverse static meshes to be animated using the entire human body. Traditionally, the process of mesh animation is laborious and requires domain expertise, with rigging specified manually by an artist when designing the character. KinÊtre makes creating animations a more playful activity, conducted by novice users interactively "at runtime". This paper describes the KinÊtre system in full, highlighting key technical contributions and demonstrating many examples of users animating meshes of varying shapes and sizes. These include non-humanoid meshes and incomplete surfaces produced by 3D scanning - two challenging scenarios for existing mesh animation systems. Rather than targeting professional CG animators, KinÊtre is intended to bring mesh animation to a new audience of novice users. We demonstrate potential uses of our system for interactive storytelling and new forms of physical gaming.

References

1. Ahmed, N. et al. Automatic generation of personalized human avatars from multi-view video. In Proc. VRST, ACM (2005).
2. Allen, B., B. Curless, and Z. Popović. Articulated body deformation from range scan data. In Proc. SIGGRAPH (2002).
3. Anguelov, D. et al. Scape: shape completion and animation of people. ACM SIGGRAPH (2005).
4. Baran, I., and J. Popović. Automatic rigging and animation of 3d characters. ACM SIGGRAPH (2007).
5. Barnes, C. et al. Video puppetry. ACM SIGGRAPH Asia (2008).
6. Ben-Chen, M., O. Weber, and C. Gotsman. Variational harmonic maps for space deformation. In Proc. SIGGRAPH, ACM (2009).
7. Chen, Y., and G. Medioni. Object modelling by registration of multiple range images. Image Vision Computing (1992).
8. Curless, B., and M. Levoy. A volumetric method for building complex models from range images. In Proc. SIGGRAPH, ACM (1996).
9. Davis, J. et al. A sketching interface for articulated figure animation. In Proc. Symposium on Computer Animation, ACM SIGGRAPH/Eurographics (2003).
10. Dontcheva, M., G. Yngve, and Z. Popović. Layered acting for character animation. In Proc. SIGGRAPH, ACM (2003).
11. Garau, M. et al. The impact of avatar realism and eye gaze control on perceived quality of communication in a shared immersive virtual environment. In Proc. CHI, ACM (2003).
12. Igarashi, T., S. Matsuoka, and H. Tanaka. Teddy: A sketching interface for 3d freeform design. In Proc. SIGGRAPH, ACM (1999).
13. Igarashi, T., T. Moscovich, and J. F. Hughes. As-rigid-as-possible shape manipulation. ACM SIGGRAPH (2005).
14. Igarashi, T., T. Moscovich, and J. F. Hughes. Spatial keyframing for performance-driven animation. In Proc. Symposium on Computer Animation, ACM SIGGRAPH/Eurographics (2005).
15. Izadi, S. et al. Kinectfusion: real-time 3d reconstruction and interaction using a moving depth camera. In Proc. UIST, ACM (2011).
16. Jacobson, A. et al. Bounded biharmonic weights for real-time deformation. In Proc. SIGGRAPH, ACM (2011).
17. Jacobson, A., and O. Sorkine. Stretchable and twistable bones for skeletal shape deformation. In Proc. SIGGRAPH Asia, ACM (2011).
18. Johnson, M. P. et al. Sympathetic interfaces. In Proc. CHI, ACM (1999).
19. Joshi, P. et al. Harmonic coordinates for character articulation. In Proc. SIGGRAPH, ACM (2007).
20. Ju, T., S. Schaefer, and J. Warren. Mean value coordinates for closed triangular meshes. In Proc. SIGGRAPH, ACM (2005).
21. Kipp, M., and Q. Nguyen. Multitouch puppetry. In Proc. ITS, ACM (2010).
22. Knep, B. et al. Dinosaur input device. In Proc. CHI, ACM (1995).
23. Komatsu, K. Human skin model capable of natural shape variation. The Visual Computer 3, 5 (1988).
24. Lehtinen, J. et al. A meshless hierarchical representation for light transport. ACM SIGGRAPH (2008).
25. Lipman, Y., D. Levin, and D. Cohen-Or. Green coordinates. ACM SIGGRAPH (2008).
26. Magnenat-Thalmann, N., R. Laperrire, and D. Thalmann. Joint-dependent local deformations for hand animation and object grasping. In Proc. Graphics Interface, Canadian Info. Proc. Society (1988).
27. Schaefer, S., T. McPhail, and J. Warren. Image deformation using moving least squares. ACM SIGGRAPH (2006).
28. Shi, X. et al. Mesh puppetry: cascading optimization of mesh deformation with inverse kinematics. ACM SIGGRAPH (2007).
29. Shin, H. J. et al. Computer puppetry: An importance-based approach. ACM Trans. on Graphics 20, 2 (2001).
30. Shotton, J. et al. Real-time human pose recognition in parts from single depth images. In Proc. CVPR, IEEE (2011).
31. Sturman, D. J. Computer puppetry. IEEE Computer Graphics & Applications 18, 1 (1998).
32. Sumner, R. W., J. Schmid, and M. Pauly. Embedded deformation for shape manipulation. ACM SIGGRAPH (2007).
33. Thorne, M., D. Burke, and M. van de Panne. Motion doodles: an interface for sketching character motion. ACM SIGGRAPH (Aug. 2004).
34. Vlasic, D. et al. Articulated mesh animation from multi-view silhouettes. ACM SIGGRAPH (2008).
35. Yamane, K., Y. Ariki, and J. Hodgins. Animating non-humanoid characters with human motion data. In Proc. SCA, ACM SIGGRAPH/Eurographics (2010).

Index Terms

• Human-centered computing

  • Human computer interaction (HCI)