Stevan Rudinac
ABSTRACT
In this paper we present a novel approach to semantic-theme-based video retrieval that considers entire videos as retrieval units and exploits automatically detected visual concepts to improve the results of retrieval based on spoken content. We deploy a query prediction method that makes use of a coherence indicator calculated on top returned documents and taking into account the information about visual concepts presence in videos to make a choice between query expansion methods. The main contribution of our approach is in its ability to exploit noisy shot-level concept detection to improve semantic-theme-based video retrieval. Strikingly, improvement is possible using an extremely limited set of concepts. In the experiments performed on TRECVID 2007 and 2008 datasets our approach shows an interesting performance improvement compared to the best performing baseline.
- Aly, R., Doherty, A., Hiemstra, D., and Smeaton, A. 2010. Beyond shot retrieval: searching for broadcast news items using language models of concepts. In ECIR, Milton Keynes, UK, 2010. Google Scholar
Digital Library
- Hsu, W. H., Kennedy, L. S., and Chang, S. 2006. Video search reranking via information bottleneck principle. In ACM MM, Santa Barbara, CA, USA, 2006. Google ScholarDigital Library
- Jiang, Y-G., Yanagawa, A., Chang, S-F., and Ngo, C-W. 2008. CU-VIREO374: Fusing Columbia374 and VIREO374 for Large Scale Semantic Concept Detection. Columbia University ADVENT Technical Report #223-2008-1.Google Scholar
- Rudinac, S., Larson, M., and Hanjalic, A. 2010. Exploiting result consistency to select query expansions for spoken content retrieval. In ECIR, Milton Keynes, UK, 2010. Google ScholarDigital Library
- Snoek, C. G. M., van de Sande, K. E. A., de Rooij, O., et al. 2009. The MediaMill TRECVID 2009 semantic video search engine. In TRECVID Workshop, Gaithersburg, USA, 2009.Google Scholar
Index Terms
- Visual concept-based selection of query expansions for spoken content retrieval
Recommendations
Exploiting noisy visual concept detection to improve spoken content based video retrieval
MM '10: Proceedings of the 18th ACM international conference on MultimediaIn this paper, we present a technique for unsupervised construction of concept vectors, concept-based representations of complete video units, from the noisy shot-level output of a set of visual concept detectors. We deploy these vectors to improve ...
Spoken content retrieval: beyond cascading speech recognition with text retrieval
Spoken content retrieval refers to directly indexing and retrieving spoken content based on the audio rather than text descriptions. This potentially eliminates the requirement of producing text descriptions for multimedia content for indexing and ...
Cross-media retrieval using query dependent search methods
The content-based cross-media retrieval is a new type of multimedia retrieval in which the media types of query examples and the returned results can be different. In order to learn the semantic correlations among multimedia objects of different ...
Comments