Region-of-interest volumetric visual hull refinement

ABSTRACT

This paper introduces a region-of-interest visual hull refinement technique, based on flexible voxel grids for volumetric visual hull reconstructions. Region-of-interest refinement is based on a multi-pass process, beginning with a focussed visual hull reconstruction, resulting in a first 3D approximation of the target, followed by a region-of-interest estimation, tasked with identifying features of interest, which in turn are used to locally refine the voxel grid and extract a higher-resolution surface representation for those regions. This approach is illustrated for the reconstruction of avatars for use in tele-immersion environments, where head and hand regions are of higher interest. To allow reproducability and direct comparison a publicly available data set for human visual hull reconstruction is used. This paper shows that region-of-interest reconstruction of the target is faster and visually comparable to higher resolution focused visual nhull reconstructions. This approach reduces the amount of data generated through the reconstruction, allowing faster post processing, as rendering or networking of the surface voxels. Reconstruction speeds support smooth interactions between the avatar and the virtual environment, while the improved resolution of its facial region and hands creates a higher-degree of immersion and potentially impacts the perception of body language, facial expressions and eye-to-eye contact.

References

1. Gross, M., Wurmlin, S., Naef, M., Lamboray, E., Spagno, C., Kunz, A., Koller-Meier, E., Svoboda, T., Gool, L. V., Lang, S., Strehlke, K., Moere, A. V., and Staadt, O. 2003. blue-c: A spatially immersive display and 3d video portal for telepresence. 819--827.
2. Kho, Y., and Garland, M. 2007. Sketching mesh deformations. In SIGGRAPH '07: ACM SIGGRAPH 2007 courses, ACM, New York, NY, USA, 41.
3. Knoblauch, D., and Kuester, F. 2009. Focused volumetric visual hull with color extraction. In ISVC '09: Proceedings of the 5th International Symposium on Advances in Visual Computing, Springer-Verlag, Berlin, Heidelberg, 208--217.
4. Knoblauch, D., Font, P. M., and Kuester, F. 2010. Virtualizeme: Real-time avatar creation for tele-immersion environments. In Proceeding of Virtual Reality Conference (VR), 2010 IEEE, IEEE, 279--280.
5. LaMar, E., Hamann, B., and Joy, K. I. 1999. Multiresolution techniques for interactive texture-based volume visualization. In VIS '99: Proceedings of the conference on Visualization '99, IEEE Computer Society Press, Los Alamitos, CA, USA, 355--361.
6. Laurentini, A. 1994. The visual hull concept for silhouette-based image understanding. IEEE Transactions on Pattern Analysis and Machine Intelligence 16, 2, 150--162.
7. Li, M., Magnor, M., and Seidel, H. 2003. Hardware-accelerated visual hull reconstruction and rendering. In Graphics Interface Proceedings 2003: Canadian Human-Computer Communications Society, AK Peters, Ltd., 65.
8. Matusik, W., Buehler, C., and McMillan, L. 2001. Polyhedral visual hulls for real-time rendering. In Rendering Techniques 2001: Proceedings of the Eurographics Workshop in London, United Kingdom, June 25--27, 2001, Springer, 115.
9. Nieto-Castanon, A., Ghosh, S., Tourville, J., and Guenther, F. 2003. Region of interest based analysis of functional imaging data. Neuroimage 19, 4, 1303--1316.
10. Potmesil, M. 1987. Generating octree models of 3D objects from their silhouettes in a sequence of images. Computer Vision, Graphics, and Image Processing 40, 1, 1--29.
11. Sadagic, A., H., T., Lanier, J., Fuchs, H., Van Dam, A., Daniilidis, K., Mulligan, J., Holden, L., and Zelenik, 2001. National tele-immersion initiative: Towards compelling tele-immersive collaborative environments. Presentation given at Medicine meets Virtual Reality 2001 Conference.
12. Vlasic, D., Baran, I., Matusik, W., and Popović, J. 2008. Articulated mesh animation from multi-view silhouettes. ACM Trans. Graph. 27, 3, 1--9.
13. Wu, W., Yang, Z., Nahrstedt, K., Kurillo, G., and Bajcsy, R. 2007. Towards multi-site collaboration in tele-immersive environments. In MULTIMEDIA '07: Proceedings of the 15th international conference on Multimedia, ACM, New York, NY, USA, 767--770.

Index Terms

• Computing methodologies

  • Artificial intelligence

    • Computer vision

      • Computer vision problems

        • Shape inference

  • Computer graphics

    • Graphics systems and interfaces

      • Virtual reality

    • Shape modeling

• Human-centered computing

  • Human computer interaction (HCI)

    • Interaction paradigms

      • Mixed / augmented reality

      • Virtual reality