Maria Eskevich
Gareth J.F. Jones
Robin Aly
ABSTRACT
Searching for relevant webpages and following hyperlinks to related content is a widely accepted and effective approach to information seeking on the textual web. Existing work on multimedia information retrieval has focused on search for individual relevant items or on content linking without specific attention to search results. We describe our research exploring integrated multimodal search and hyperlinking for multimedia data. Our investigation is based on the MediaEval 2012 Search and Hyperlinking task. This includes a known-item search task using the Blip10000 internet video collection, where automatically created hyperlinks link each relevant item to related items within the collection. The search test queries and link assessment for this task was generated using the Amazon Mechanical Turk crowdsourcing platform. Our investigation examines a range of alternative methods which seek to address the challenges of search and hyperlinking using multimodal approaches. The results of our experiments are used to propose a research agenda for developing effective techniques for search and hyperlinking of multimedia content.
- M Bron, B Huurnink, and M de Rijke. Linking archives using document enrichment and term selection. In Proceedings of TPDL 2011, pages 2357--2360, 2011. Google Scholar
Digital Library
- K. Chatfield, V. Lempitsky, A. Vedaldi, and A. Zisserman. The devil is in the details: an evaluation of recent feature encoding method. In Proceedings of BMVC 2011, 2011.Google ScholarCross Ref
- R.G. Cinbis, Jakob Verbeek, and Cordelia Schmid. Unsupervised Metric Learning for Face Identification in TV Video. In Proceedings of ICCV 2011, Barcelona, Spain, 2011. Google ScholarDigital Library
- M. Eskevich, G.J. F. Jones, S. Chen, R. Aly, R.J.F. Ordelman, and M. Larson. Search and Hyperlinking Task at Mediaeval 2012. In MediaEval, volume 927 of CEUR Workshop Proceedings. CEUR-WS.org, 2012.Google Scholar
- M. Eskevich, G.J.F. Jones, M. Larson, and R.J.F. Ordelman. Creating a data collection for evaluating rich speech retrieval. In Proceedings of LREC 2012, Istanbul, Turkey, 2012.Google Scholar
- M. Eskevich, G.J.F. Jones, M. Larson, C. Wartena, R. Aly, T. Verschoor, and R.J.F. Ordelman. Comparing retrieval effectiveness of alternative content segmentation methods for internet video search. In Proeedings of CBMI 2012, 2012.Google ScholarCross Ref
- M. Eskevich, W. Magdy, and G.J.F. Jones. New metrics for meaningful evaluation of informally structured speech retrieval. In Proceedings of ECIR 2012, pages 170--181, 2012. Google ScholarDigital Library
- J.S. Garofolo, C.G.P. Auzanne, and E.M. Voorhees. The TREC spoken document retrieval track: A success story. In Proceedings of RIAO 2000, pages 1--8, 2000.Google Scholar
- A. Girgensohn, L. Wilcox, F. Shipman, and S. Bly. Designing affordances for the navigation of detail-on-demand hypervideo. In Proceedings of AVI 2004, pages 290--297. ACM, 2004. Google ScholarDigital Library
- L. Hardman. Modelling and authoring hypermedia documents. PhD thesis, Universiteit Amsterdam, 1998.Google Scholar
- D. Hiemstra. Using language models for information retrieval. PhD thesis, University of Twente, 2001.Google Scholar
- P. Hoffmann, T. Kochems, and M. Herczeg. HyLive: Hypervideo-Authoring for Live Television. In Changing Television Environments, pages 51--60. Springer, 2008. Google ScholarDigital Library
- T. Kaneko, T. Takigami, and T. Akiba. STD based on hough transform and SDR using STD results: Experiments at NTCIR-9 SpokenDoc. In Proceedings of Ninth NTCIR Workshop Meeting, 2011.Google Scholar
- P. Kelm, S. Schmiedeke, and T. Sikora. Feature-based Video Key Frame Extraction for low Quality Video Sequences. In Proceedings of WIAMIS 2009.Google ScholarCross Ref
- Lori Lamel and Jean-Luc Gauvain. Speech processing for audio indexing. In Advances in Natural Language Processing, volume 5221 of LNCS, pages 4--15. 2008. Google ScholarDigital Library
- M. Larson, M. Eskevich, R. Ordelman, C. Kofler, S. Schmiedeke, and G. J. F. Jones. Overview of MediaEval 2011 Rich Speech Retrieval Task and Genre Tagging Task. In MediaEval 2011 Workshop, Pisa, Italy, 2011.Google Scholar
- M. Larson, C. Kofler, and A. Hanjalic. Reading between the tags to predict real-world size-class for visually depicted objects in images. In Proceedings of ACM MM, 2011. Google ScholarDigital Library
- M. Larson, M. Soleymani, M. Eskevich, P. Serdyukov, R.J.F. Ordelman, and G. J. F. Jones. The community and the crowd: Multimedia benchmark dataset development. IEEE MultiMedia, 19(3):15, 2012. Google ScholarDigital Library
- M.A. Larson, S. Schmiedeke, P. Kelm, A. Rae, V. Mezaris, T. Piatrik, M. Soleymani, F. Metze, and G.J.F. Jones, editors. Working Notes Proceedings of the MediaEval 2012 Workshop, Santa Croce in Fossabanda, Pisa, Italy, October 4--5, 2012, volume 927 of CEUR Workshop Proceedings. CEUR-WS.org, 2012.Google Scholar
- B. Meixner, K. Matusik, C. Grill, and H. Kosch. Towards an easy to use authoring tool for interactive non-linear video. Multimedia Tools and Applications, pages 1--26, 2012.Google Scholar
- P. N. Mendes, M. Jakob, A. García-Silva, and C. Bizer. DBpedia spotlight: Shedding light on the web of documents. In Proceedings of the 7th International Conference on Semantic Systems (I-Semantics), 2011. Google ScholarDigital Library
- D. Milne and I.H. Witten. Learning to link with wikipedia. In Proceeding of CIKM 2008, pages 509--518. ACM, 2008. Google ScholarDigital Library
- J. Morang, R.J.F. Ordelman, F.M.G. de Jong, and A.J. van Hessen. InfoLink: analysis of Dutch broadcast news and cross-media browsing. In Proceedings of ICME 2005, Los Alamitos, 2005.Google ScholarCross Ref
- P. Pecina, P. Hoffmannova, G. J. F. Jones, Y. Zhang, and D. W. Oard. Overview of the CLEF 2007 cross-language speech retrieval track. In Proceedings of CLEF 2007, pages 674--686, 2007. Google ScholarDigital Library
- S. Robertson, H. Zaragoza, and M. Taylor. Simple BM25 extension to multiple weighted fields. In Proceedings of ACM CIKM 2004, 2004. Google ScholarDigital Library
- A. Rousseau, F. Bougares, P. Deléglise, H. Schwenk, and Y. Estèv. Lium's systems for the iwslt 2011 speech translation tasks. In Proceedings of IWSLT 2011, 2011.Google Scholar
- I. Sawhney, N. and Balcom, D. and Smith. Authoring and navigating video in space and time. MultiMedia, IEEE, 4(4):30--39, 1997. Google ScholarDigital Library
- F. Shipman, A. Girgensohn, and L. Wilcox. Authoring, viewing, and generating hypervideo: An overview of Hyper-Hitchcock. ACM Trans. Multimedia Comput. Commun. Appl., (2):15:1---15:19, 2008. Google ScholarDigital Library
- J. Sivic and A. Zisserman. Video google: a text retrieval approach to object matching in videos. In Proceedings of ICCV 2003, pages 1470 --1477 vol.2, 2003. Google ScholarDigital Library
- A. F. Smeaton, P. Over, and W. Kraaij. Evaluation campaigns and trecvid. In Proceedings of MIR 2006, Santa Barbara, California, USA, 2006. Google ScholarDigital Library
- A. W. M. Smeulders, M. Worring, S. Santini, A. Gupta, and R. Jain. Content-based image retrieval at the end of the early years. IEEE Trans. Pattern Anal. Mach. Intell., 22(12):1349--1380, 2000. Google ScholarDigital Library
- M. Utiyama and H. Isahara. A statistical model for domain-independent text segmentation. In Proceedings of ACL 2001. Google ScholarDigital Library
- E. Voorhees, D.K. Harman, National Institute of Standards, and Technology (US). TREC: Experiment and evaluation in information retrieval. MIT press USA, 2005. Google ScholarDigital Library
- E.M. Voorhees. The TREC-8 Question Answering Track Report. In Proceedings of TREC-8, pages 77--82, 1999.Google Scholar
- R. Yan. Probabilistic Models for Combining Diverse Knowledge Sources in Multimedia Retrieval. PhD thesis, Carnegie Mellon University, 2006. Google ScholarDigital Library
Index Terms
- Multimedia information seeking through search and hyperlinking
Recommendations
Content-based multimedia information retrieval: State of the art and challenges
Extending beyond the boundaries of science, art, and culture, content-based multimedia information retrieval provides new paradigms and methods for searching through the myriad variety of media all over the world. This survey reviews 100+ recent ...
Visual Search of Web Multimedia Information Supported by the XHMG System
GRC '07: Proceedings of the 2007 IEEE International Conference on Granular Computingfacilities in the XHMG system. The XHMG system allows modeling, integration, search, and retrieval of Web multimedia data (hypermedia) from heterogeneous data sources based on its content and semantics. The paper shows the basic XHMG structural ...
Conversational Search for Multimedia Archives
Advances in Information RetrievalAbstractThe growth of media archives (including text, speech, video and audio) has led to significant interest in developing search methods for multimedia content. An ongoing challenge of multimedia search is user interaction during the search process, ...
Comments