Rapid serial visual presentation techniques for consumer digital video devices

ABSTRACT

In this paper we propose a new model for a class of rapid serial visual presentation (RSVP) interfaces [16] in the context of consumer video devices. The basic spatial layout "explodes" a sequence of image frames into a 3D trail in order to provide more context for a spatial/temporal presentation. As the user plays forward or back, the trail advances or recedes while the image in the foreground focus position is replaced. The design is able to incorporate a variety of methods for analyzing or highlighting images in the trail. Our hypotheses are that users can navigate more quickly and precisely to points of interest when compared to conventional consumer-based browsing, channel flipping, or fast-forwarding techniques. We report on an experiment testing our hypotheses in which we found that subjects were more accurate but not faster in browsing to a target of interest in recorded television content with a TV remote.

References

1. 1. Aoki., H., Shimotsuji, S., & Hori, O. (1996) A shot classification method of selecting effective key-frames for video browsing. In Proceedings of ACM Multimedia '96 (November, Boston MA,USA), ACM, pp. 1-10.]]
2. 2. Boreczky, J., Girgensohn, A., Golovchinsky, G., & Uchihashi, S. (2000) An interactive comic book presentation for exploring video. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (The Hague, Netherlands), ACM, pp. 185-192.]]
3. 3. Christel, M. G., Winkler, D. B., & Taylor, C. R. (1997) Multimedia abstractions for a digital video library, Proceedings of the second ACM international conference on Digital Libraries (July 23-26, Philadelphia PA, USA), ACM, pp.21-29.]]
4. 4. Christel, M. G., Hauptmann, A. G., Wactlar, H. D., & Tobun, D. N. (2002) Collages as dynamic summaries for news video. In Proceedings of the ACM Multimedia '02 (December, Juan-les-Pins, FRANCE), ACM, pp. 561-569.]]
5. 5. Coltheart, V. (Ed.) (1999) Fleeting Memories: Cognition of Brief Visual Stimuli. MIT Press, Cambridge MA, USA.]]
6. 6. De Bruijn, O., & Spence, R. (2000) Rapid serial visual presentation: a space-time trade-off in information presentation. In Proceedings of the Working Conference on Advanced Visual Interfaces (AVI 2000) (May, Palermo, ITALY), ACM, pp. 189-192.]]
7. 7. De Bruijn, O., & Spence, R. (2002) Patterns of eye gaze during Rapid Serial Visual Presentation. In Proceedings of Advanced Visual Interfaces (AVI 2002) (May, Trento, Italy), ACM, pp. 209-217.]]
8. 8. Ding, W., Marchionini, G., & Soergel, D. (1999) Multimodal surrogates for video browsing. In Proceedings of the fourth ACM Conference on Digital Libraries (August, Berkeley CA, USA), ACM, pp. 85-93.]]
9. 9. Drucker, S. M., Glatzer, A., De Mar, S., & Wong, C. (2002) SmartSkip: Consumer level browsing and skipping of digital video content. In Proceedings of CHI 2002 (April, Minneapolis MN, USA), ACM, pp. 219-226.]]
10. 10. Elliot, E. (1993) Watch, grab, arrange, see: thinking with motion images via streams and collages. MSVS thesis, Massachusetts Institute of Technology, Cambridge MA, USA.]]
11. 11. Freeman, E., & Gelernter, D. (1996) Lifestreams: a storage model for personal data. ACM SIGMOD Record, 25, 1, 80-86.]]
12. 12. Healey, C. G., Booth, K. S., & Enns, J. T. (1996) High-speed visual estimation using preattentive processing. ACM Transactions on Computer Human Interaction (TOCHI), 3, 2, 107-135.]]
13. 13. Lee, H. Smeaton, A., Berrut, C., Murphy, N. Marlow, S, & O'Connor, N.E. (2000) Implementation and analysis of several keyframe-based browsing interfaces to digital video. In Proceedings of Research and Advanced Technology for Digital Libraries, 4th European Conference, ECDL 2000 (September, Lisbon, PORTUGAL), LNCS 1923, Springer, pp. 206-218.]]
14. 14. Li, F. C., Gupta, A., Sanocki, E., He, L., & Rui, Y. (2000) Browsing digital video. In Proceedings of CHI 2000 (April, The Hague, Netherlands), ACM, pp. 169-176.]]
15. 15. Spence, R. (2001) Information Visualization. ACM Press & Addison-Wesley, Harlow, England.]]
16. 16. Spence R. (2002) Rapid, serial and visual: a presentation technique with potential. Information Visualization, 1, 1, 13-19.]]
17. 17. Tse, T., Marchionini, G., Ding, W., Slaughter, W., & Komlodi, A. (1998) Dynamic Key Frame Presentation Techniques for Augmenting Video Browsing. In Proceedings Advanced Visual Interfaces (May, L'Aquila, ITALY), ACM, pp. 185-194.]]
18. 18. Wittenburg, K., Ali-Ahmad, W., LaLiberte, D., & Lanning, T. (1998) Rapid-Fire image previews for information navigation. In Proceedings of Advanced Visual Interfaces (May, L'Aquila, ITALY), ACM, pp. 76-82.]]
19. 19. Wittenburg, K., Nicol, J., Paschetto, J., & Martin, C. (1999) Browsing with dynamic key frame collages in web-based entertainment video services. In Proceedings of IEEE International Conference on Multimedia Computing and Systems (June, Florence, ITALY), IEEE, Vol. 2, pp. 913-918.]]
20. 20. Wittenburg, K., Chiyoda, C., Heinrichs, M., & Lanning, T. (2000) Browsing through rapid-fire imaging: requirements and industry initiatives. In Proceedings of Electronic Imaging '2000: Internet Imaging (January, San Jose CA, USA), SPIE, pp. 48-56.]]
21. 21. Wittenburg, K., Lanning, T., Forlines, C., & Esenther, A. (2003) Rapid serial visual presentation techniques for visualizing a third data dimension. In Proceedings of HCI International 2003 (June, Crete, GREECE), Lawrence Erlbaum, Vol. 4, pp. 810-814.]]
22. 22. Xiong, Z., Radhakrishnan, R., & Divakaran, A. (2003) Generation of sports highlights using motion activity in combination with a common audio feature extraction framework. In Proceedings of IEEE International Conference on Image Processing (ICIP) (September, Barcelona SPAIN), IEEE, to appear.]]
23. 23. Zhang, H., Kankanhalli, A., & Smoliar, S. (1993) Automatic partitioning of full-motion video. Multimedia Systems 1, 1, 10-28.]]

Index Terms

• Computing methodologies

  • Artificial intelligence

    • Computer vision

      • Computer vision problems

        • Video segmentation

      • Computer vision tasks

        • Video summarization

      • Image and video acquisition

        • 3D imaging

  • Modeling and simulation

    • Simulation types and techniques

      • Visual analytics

• Human-centered computing

  • Human computer interaction (HCI)

    • Interaction paradigms

      • Graphical user interfaces