Discovery of Environmental Web Resources Based on the Combination of Multimedia Evidence

ABSTRACT

This work proposes a framework for the discovery of environmental Web resources providing air quality measurements and forecasts. Motivated by the frequent occurrence of heatmaps in such Web resources, it exploits multimedia evidence at different stages of the discovery process. Domain-specific queries generated using empirical information and machine learning driven query expansion are submitted both to the Web and Image search services of a general-purpose search engine. Post-retrieval filtering is performed by combining textual and visual (heatmap-related) evidence in a supervised machine learning framework. Our experimental results indicate improvements in the effectiveness when performing heatmap recognition based on SURF and SIFT descriptors using VLAD encoding and when combining multimedia evidence in the discovery process.

References

1. H. Bay, T. Tuytelaars, and L. Van Gool. SURF: Speeded Up Robust Features. In Proc. of the 9th European Conference on Computer Vision (ECCV), pages 404--417. 2006.
2. R. Cao and C. Tan. Text/graphics separation in maps. In Proc. of 4th IAPR International Workshop on Graphics Recognition (GREC), pages 167--177. 2002.
3. C. C. Chang and C. J. Lin. LIBSVM: a library for support vector machines. ACM Transactions on Intelligent Systems and Technology, 2(3):27, 2011.
4. S. F. Chang, T. Sikora, and A. Puri. Overview of the MPEG-7 standard. IEEE Transactions on Circuits and Systems for Video Technology, 11(6):688--695, 2001.
5. K. Chatfield, K. Simonyan, A. Vedaldi, and A. Zisserman. Return of the devil in the details: Delving deep into convolutional nets. In Proc. of the British Machine Vision Conference (BMVC), 2014.
6. J. Deng, W. Dong, R. Socher, L.-J. Li, K. Li, and L. Fei-Fei. Imagenet: A large-scale hierarchical image database. In Proc. of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pages 248--255, 2009.
7. V. Epitropou, K. Karatzas, and A. Bassoukos. A method for the inverse reconstruction of environmental data applicable at the chemical weather portal. In Proc. of the GI-Forum Symposium and Exhibit on Applied Geoinformatics, pages 58--68, 2010.
8. E. Fox and J. Shaw. Combination of multiple searches. In Proc. of TREC-2, pages 243--252, 1994.
9. T. C. Henderson and T. Linton. Raster map image analysis. In Proc. of the 10th International Conference on Document Analysis and Recognition (ICDAR), pages 376--380, 2009.
10. H. Jégou, M. Douze, C. Schmid, and P. Pérez. Aggregating local descriptors into a compact image representation. In Proc. of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pages 3304--3311, 2010.
11. K. Karatzas and N. Moussiopoulos. Urban air quality management and information systems in Europe: legal framework and information access. Journal of Environmental Assessment Policy and Management, 2(02):263--272, 2000.
12. A. Krizhevsky, I. Sutskever, and G. E. Hinton. ImageNet classification with deep Convolutional Neural Networks. In Advances in Neural Information Processing Systems 25 (NIPS), pages 1097--1105. 2012.
13. D. G. Lowe. Distinctive image features from scale-invariant keypoints. International Journal of Computer Vision, 60(2):91--110, 2004.
14. H. P. Luong, S. Gauch, and Q. Wang. Ontology-based focused crawling. In Proc. of the International Conference on Information, Process, and Knowledge Management, pages 123--128, 2009.
15. M. Lupu, M. Salampasis, and A. Hanbury. Domain specific search. In Professional Search in the Modern World, pages 96--117. 2014.
16. F. Markatopoulou, N. Pittaras, O. Papadopoulou, V. Mezaris, and I. Patras. A study on the use of a binary local descriptor and color extensions of local descriptors for video concept detection. In Proc. of the 21st International Conference on Multimedia Modeling (MMM), pages 282--293, 2015.
17. A. K. McCallum, K. Nigam, J. Rennie, and K. Seymore. Automating the construction of internet portals with machine learning. Information Retrieval, 3(2):127--163, 2000.
18. A. Moumtzidou, S. Vrochidis, E. Chatzilari, and I. Kompatsiaris. Discovery of environmental nodes based on heatmap recognition. In Proc. of the 20th IEEE International Conference on Image Processing (ICIP), 2013.
19. A. Moumtzidou, S. Vrochidis, S. Tonelli, I. Kompatsiaris, and E. Pianta. Discovery of environmental nodes in the web. In Proc. of the 5th International Retrieval Facility Conference (IRFC), volume 7356, pages 58--72, 2012.
20. H. Nabeshima, R. Miyagawa, Y. Suzuki, and K. Iwanuma. Rapid synthesis of domain-specific web search engines based on semi-automatic training-example generation. In Proc. of the 2006 IEEE/WIC/ACM International Conference on Web Intelligence, pages 769--772, 2006.
21. P. Over, G. Awad, W. Kraaij, and A. F. Smeaton. TRECVID 2007 - overview. In TRECVID 2007 workshop participants notebook papers, 2007.
22. S. Oyama, T. Kokubo, and T. Ishida. Domain-specific web search with keyword spices. IEEE Transactions on Knowledge and Data Engineering, 16(1):17--27, 2004.
23. J. R. Quinlan. C4.5: Programs for Machine Learning. Morgan Kaufmann Publishers, 1993.
24. T. T. Tang, D. Hawking, N. Craswell, and K. Griffiths. Focused crawling for both topical relevance and quality of medical information. In Proc. of the 14th ACM International Conference on Information and Knowledge Management, pages 147--154, 2005.
25. T. Tsikrika, A. Moumtzidou, S. Vrochidis, and I. Kompatsiaris. Focussed crawling of environmental web resources: A pilot study on the combination of multimedia evidence. In Proc. of the 1st International Workshop on Environmental Multimedia Retrieval (EMR), pages 61--68, 2014.
26. J. Yuan and etal. THU and ICRC at TRECVID 2007. In TRECVID 2007 workshop participants notebook papers, 2007.

Index Terms

• Information systems

  • Information retrieval