Tinghuai Wang
ABSTRACT
Falling hardware costs have prompted an explosion in casual video capture by domestic users. Yet, this video is infrequently accessed post-capture and often lies dormant on users' PCs. We present a system to breathe life into home video repositories, drawing upon artistic stylization to create a "Digital Ambient Display" that automatically selects, stylizes and transitions between videos in a semantically meaningful sequence. We present a novel algorithm based on multi-label graph cut for segmenting video into temporally coherent region maps. These maps are used to both stylize video into cartoons and paintings, and measure visual similarity between frames for smooth sequence transitions. We demonstrate coherent segmentation and stylization over a variety of home videos.
- Agarwala, A., Hertzmann, A., Salesin, D., and Seitz, S. 2004. Keyframe-based tracking for rotoscoping and animation. In Proc. ACM SIGGRAPH, 294--302. Google Scholar
Digital Library
- Alahari, K., Kohli, P., and Torr. 2008. Reduce, reuse & recycle: Efficiently solving multi-label MRFs. In CVPR, 1--8.Google Scholar
- Arksey, N. 2007. Exploring the Design Space for Concurrent Use of Personal and Large Displays for In-Home Collaboration. Master's thesis, University of British Columbia.Google Scholar
- Bai, X., Wang, J., Simons, D., and Saprio, G. 2009. Video snapcut: Robust video object cutout using localized classifiers. In Proc. ACM SIGGRAPH. Google ScholarDigital Library
- Belongie, S., Malik, J., and Puzicha, J. 2002. Shape matching and object recognition using shape contexts. IEEE Trans. Pattern Anal. Machine Intel. (PAMI) 24, 509--521. Google ScholarDigital Library
- Bizzocchi, J. 2008. Winterscape and ambient video -- an intermedia border zone. In Proc. ACM Multimedia. Google ScholarDigital Library
- Black, M., and Anandan, P. 1993. A framework for the robust estimation of optical flow. In ICCV, 231--236.Google Scholar
- Blake, A., Rother, C., Brown, M., Prez, P., and Torr, P. 2004. Interactive image segmentation using an adaptive gmmrf model. In ECCV, 428--441.Google Scholar
- Bousseau, A., Neyret, F., Thollot, J., and Salesin, D. 2007. Video watercolorization using bidirectional texture advection. In Proc. ACM SIGGRAPH, 1--7. Google ScholarDigital Library
- Boykov, Y., and Funka-Lea, G. 2006. Graph cuts and efficient n-d image segmentation. IJCV 2, 70, 109--131. Google ScholarDigital Library
- Boykov, Y., and Kolmogorov, V. 2004. An experimental comparison of min-cut/max-flow algorithms for energy minimization in vision. IEEE Trans. Pattern Anal. Machine Intel. (PAMI) 26, 1124--1137. Google ScholarDigital Library
- Boykov, Y., Veksler, O., and Zabih, R. 2001. Fast approximate energy minimization via graph cuts. IEEE Trans. Pattern Anal. Machine Intel. (PAMI) 23, 1222--1239. Google ScholarDigital Library
- Collomosse, J., and Hall, P. 2003. Cubist style rendering from photographs. IEEE Trans. Visualization and Comp. Graphics (TVCG) 9, 4, 443--453. Google ScholarDigital Library
- Collomosse, J., Rowntree, D., and Hall, P. 2005. Stroke surfaces: Temporally coherent artistic animations from video. IEEE Trans. Visualization and Comp. Graphics (TVCG) 11, 540--549. Google ScholarDigital Library
- Collomosse, J. 2004. Higher Level Techniques for the Artistic Rendering of Images and Video. PhD thesis, University of Bath.Google Scholar
- Comaniciu, D., and Meer, P. 2002. Mean shift: A robust approach toward feature analysis. IEEE Trans. PAMI 24, 603--619. Google ScholarDigital Library
- DeCarlo, D., and Santella, A. 2002. Abstracted painterly renderings using eye-tracking data. In Proc. ACM SIGGRAPH, 769--776.Google Scholar
- Hall, P. M., and Hicks, Y. 2004. CSBU-2004-03: A method to add gaussian mixture models. Tech. rep., Univ. Bath.Google Scholar
- Hays, J., and Essa, I. A. 2004. Image and video based painterly animation. In Proc. ACM NPAR, 113--120. Google ScholarDigital Library
- Hertzmann, A., and Perlin, K. 2000. Painterly rendering for video and interaction. In Proc. ACM NPAR, 7--12. Google ScholarDigital Library
- Hertzmann, A. 1998. Painterly rendering with curved brush strokes of multiple sizes. In Proc. ACM SIGGRAPH, 453--460. Google ScholarDigital Library
- Jacobs, C., Finkelstein, A., and Salesin, S. 1995. Fast multiresolution image querying. In Proc. ACM SIGGRAPH, 277--286. Google ScholarDigital Library
- Kovar, L., Gleicher, M., and Pighin, F. 2002. Motion graphs. In Proc. ACM SIGGRAPH, 473--482. Google ScholarDigital Library
- Kyprianidis, J.-E., Kang, H., and Doellner, J. 2009. Image and video abstraction by anisotropic kuwahara filtering. In Proc. Pacific Graphics, vol. 28.Google Scholar
- Litwinowicz, P. 1997. Processing images and video for an impressionist effect. In Proc. ACM SIGGRAPH, 407--414. Google ScholarDigital Library
- Lowe, D. 2004. Distinctive image features from scale-invariant keypoints. IJCV 60, 91--110. Google ScholarDigital Library
- Meier, B. J. 1996. Painterly rendering for animation. In Proc. ACM SIGGRAPH, 477--484. Google ScholarDigital Library
- Paris, S. 2008. Edge-preserving smoothing and mean-shift segmentation of video streams. In ECCV, 460--473. Google ScholarDigital Library
- Perez, P., Gangnet, M., and Blake, A. 2003. Poisson image editing. In Proc. ACM SIGGRAPH, 313--318. Google ScholarDigital Library
- Price, B., Morse, B., and Cohen, S. 2009. Livecut: Learning-based interactive video segmentation by evaluation of multiple propogated cues. In ICCV.Google Scholar
- Ruiz, D., Takahashi, H., and Nakajima, M. 2003. Image categorization using color blobs in a mobile environment. In Proc. Eurographics, 427--432.Google Scholar
- Schodl, A., Skeliski, R., Salesin, D., and Essa, H. 2000. Video textures. In Proc. ACM SIGGRAPH, 489--498. Google ScholarDigital Library
- Slatter, D., Cheatle, P., and Greig, D. 2010. Faces from the web: Automatic selection and composition of media for casual screen consumption and printed art-work. In Proc. SPIE.Google Scholar
- Viola, P., and Jones, M. 2004. Robust real-time object detection. Intl. Journal. Computer vision (IJCV) 57, 2, 137--154. Google ScholarDigital Library
- Wang, J., Xu, Y., Shum, H., and Cohen, M. 2004. Video tooning. In Proc. ACM SIGGRAPH, vol. 23, 574--583. Google ScholarDigital Library
- Wang, T., Mansfield, A., Hu, R., and Collomosse, J. 2009. An evolutionary approach to automatic video editing. In Proc. 6th European Conf. on Visual Media Production (CVMP). Google ScholarDigital Library
- Winnemoller, H., Olsen, S., and Gooch, B. 2006. Real-time video abstraction. In Proc. ACM SIGGRAPH, 1221--1226. Google ScholarDigital Library
- Xiao, J., Zhang, X., Cheatle, P., Gao, Y., and Atkins, C. 2008. Mixed-initiative photo collage authoring. In Proc. ACM Multimedia, 509--518. Google ScholarDigital Library
- You, W., Feis, S., and Lea, R. 2008. Studying vision-based multiple-user interaction with in-home large displays. In Proc. 3rd ACM workshop on Human-Centred Computing (HCC), 19--26. Google ScholarDigital Library
Index Terms
- Video stylization for digital ambient displays of home movies
Recommendations
Extended papers from NPAR 2010: Stylized ambient displays of digital media collections
The falling cost of digital cameras and camcorders has encouraged the creation of massive collections of personal digital media. However, once captured, this media is infrequently accessed and often lies dormant on users' PCs. We present a system to ...
Pointillist video stylization based on particle tracing
We present an algorithm that stylizes an input video into a painterly animation without user intervention. In particular, we focus on pointillist animation with stable temporal coherence. Temporal coherence is an important problem in non-photorealistic ...
Promoting Social Connectedness through Human Activity-Based Ambient Displays
ITAP '16: Proceedings of the International Symposium on Interactive Technology and Ageing PopulationsExisting research in ambient assisted living (AAL) confirms the success of context aware applications in conveying feelings of connectedness and creating a sense of co-presence. Awareness of an elder's activities could trigger emotional responses and ...
Comments