ABSTRACT
Screen size and display resolution limit the experience of watching videos on mobile devices. The viewing experience can be improved by determining important or interesting regions within the video (called regions of interest, or ROIs) and displaying only the ROIs to the viewer. Previous work focuses on analyzing the video content using visual attention model to infer the ROIs. Such content-based technique, however, has limitations. In this paper, we propose an alternative paradigm to infer ROIs from a video. We crowdsource from a large number of users through their implicit viewing behavior using a zoom and pan interface, and infer the ROIs from their collective wisdom. A retargeted video, consisting of relevant shots determined from historical users behavior, can be automatically generated and replayed to subsequent users who would prefer a less interactive viewing experience. This paper presents how we collect the user traces, infer the ROIs and their dynamics, group the ROIs into shots, and automatically reframe those shots to improve the aesthetics of the video. A user study with 48 participants shows that our automatically retargeted video is of comparable quality to one handcrafted by an expert user
- D. Arijon. Grammar of the Film Language. Silman-James Press, 1991.Google Scholar
- D. Comaniciu and P. Meer. Mean shift: A robust approach toward feature space analysis. IEEE Trans. Pattern Anal. Mach. Intell., 24(5):603--619, 2002. Google ScholarDigital Library
- P. Doubek, I. Geys, T. Svoboda, and L. V. Gool. Cinematographic rules applied to a camera network. In Proc. of the 5th Workshop on Omnidirectional Vision, pages 17--30, 2004.Google Scholar
- H. El-Alfy, D. Jacobs, and L. Davis. Multi-scale video cropping. In Proc. of MULTIMEDIA '07, pages 97?106, Augsburg, Germany, 2007. Google ScholarDigital Library
- X. Fan, X. Xie, H.-Q. Zhou, and W.-Y. Ma. Looking into video frames on small displays. In Proc. of MULTIMEDIA '03, pages 247--250, Berkeley, CA, 2003. Google ScholarDigital Library
- M. L. Gleicher and F. Liu. Re-cinematography: Improving the camerawork of casual video. ACM Trans. Multimedia Comput. Commun. Appl., 5(1):1--28, 2008. Google ScholarDigital Library
- J. Han, K. N. Ngan, M. Li, and H. Zhang. Unsupervised extraction of visual attention objects in color images. IEEE Trans. Circuits Syst. Video Techn., 16(1):141--145, 2006. Google ScholarDigital Library
- L.-w. He, M. F. Cohen, and D. H. Salesin. The virtual cinematographer: a paradigm for automatic real-time camera control and directing. In Proc. of SIGGRAPH '96, pages 217--224, 1996. Google ScholarDigital Library
- T.-H. Huang, K.-Y. Cheng, and Y.-Y. Chuang. A collaborative benchmark for region of interest detection algorithms. In Proc. of CVPR '09, Miami, FL, June 2009.Google ScholarCross Ref
- L. Itti, C. Koch, and E. Niebur. A model of saliency-based visual attention for rapid scene analysis. IEEE Trans. Pattern Anal. Mach. Intell., 20(11):1254--1259, 1998. Google ScholarDigital Library
- T.-Y. Li and X.-Y. Xiao. An interactive camera planning system for automatic cinematographer. In Proc. of Multimedia Modeling, pages 310--315, Los Alamitos, 2005. Google ScholarDigital Library
- F. Liu and M. Gleicher. Video retargeting: automating pan and scan. In Proc. of MULTIMEDIA '06, pages 241--250, Santa Barbara, CA, 2006. Google ScholarDigital Library
- Y. Pritch, A. Rav-Acha, and S. Peleg. Nonchronological video synopsis and indexing. IEEE Trans. Pattern Anal. Mach. Intell., 30(11):1971--1984, 2008. Google ScholarDigital Library
- N. Quang Minh Khiem, G. Ravindra, A. Carlier, and W. T. Ooi. Supporting zoomable video streams with dynamic region-of-interest cropping. In Proc. of ACM MMSYS '10, pages 259--270, Phoenix, AZ, 2010. Google ScholarDigital Library
- M. Rubinstein, A. Shamir, and S. Avidan. Improved seam carving for video retargeting. ACM Trans. Graph., 27(3):1--9, 2008. Google ScholarDigital Library
- D. Shamma, R. Shaw, P. Shafton, and Y. Liu. Watch What I Watch. In Proc. ACM MIR '07, Augsburg, Germany, 2007. Google ScholarDigital Library
- N. Ukita, T. Ono, and M. Kidode. Region extraction of a gaze object using the gaze point and view image sequences. In Proc. of the 7th International Conference on Multimodal Interfaces, pages 129--136, Torento, Italy, 2005. Google ScholarDigital Library
- D. Walther and C. Koch. Modeling attention to salient proto-objects. Neural Networks, 19:1395--1407, 2006. Google ScholarDigital Library
- X. Xie, H. Liu, S. Goumaz, and W.-Y. Ma. Learning user interest for image browsing on small-form-factor devices. In Proc. of CHI '05, pages 671--680, Portland, OR, 2005. Google ScholarDigital Library
Index Terms
- Crowdsourced automatic zoom and scroll for video retargeting
Recommendations
Video retargeting: automating pan and scan
MM '06: Proceedings of the 14th ACM international conference on MultimediaWhen a video is displayed on a smaller display than originally intended, some of the information in the video is necessarily lost. In this paper, we introduce Video Retargeting that adapts video to better suit the target display, minimizing the ...
Combining content-based analysis and crowdsourcing to improve user interaction with zoomable video
MM '11: Proceedings of the 19th ACM international conference on MultimediaThis paper introduces a new paradigm for interacting with zoomable video. Our interaction technique reduces the number of zooms and pans required by providing recommended viewports to the users, and replaces multiple zoom and pan actions with a simple ...
Towards characterizing users' interaction with zoomable video
SAPMIA '10: Proceedings of the 2010 ACM workshop on Social, adaptive and personalized multimedia interaction and accessWe conducted a user study with 4 video clips and 37 viewing sessions on how users interact with a web-based zoomable video system, where users can zoom and pan within the video to view selected regions-of-interest with more detail. The study shows that ...
Comments