Cross-media Topic Detection with Refined CNN based Image-Dominant Topic Model

ABSTRACT

Online heterogenous data is springing up while the data has the rich auxiliary information (e.g. pictures and videos) around the text. However, traditional topic models are suffering from the limitations to discover the topics effectively from the cross-media data. Incorporating with the convolutional neural network (CNN) feature, we propose a novel image dominant topic model, which projects both the text modality and the visual modality into a semantic simplex. Further, an improved CNN feature is introduced to capture more visual details by fusing the convolutional layer and fully-connected layer. Experimental comparisons with state-of-the-art methods in the cross-media topic detection task show the effectiveness of our model.

References

1. D. M. Blei and M. I. Jordan. Modeling annotated data. In ACM SIGIR, pages 127--134, 2003.
2. D. M. Blei, A. Y. Ng, and M. I. Jordan. Latent dirichlet allocation. JMLR, 3:993--1022, 2003.
3. J. Cao, Y. D. Zhang, Y. C. Song, Z. N. Chen, X. Zhang, and J. T. Li. Mcg-webv: A benchmark dataset for web video analysis. Beijing: Institute of Computing Technology, 10:324--334, 2009.
4. J. Chang and D. M. Blei. Relational topic models for document networks. In AISTATS, pages 81--88, 2009.
5. T. L. Griffiths and M. Steyvers. Finding scientific topics. NAS, 101(suppl 1):5228--5235, 2004.
6. T. Hofmann. Probabilistic latent semantic indexing. In ACM SIGIR, pages 50--57, 1999.
7. Y. Jia, E. Shelhamer, J. Donahue, S. Karayev, J. Long, R. Girshick, S. Guadarrama, and T. Darrell. Caffe: Convolutional architecture for fast feature embedding. In ACM Multimedia, pages 675--678, 2014.
8. A. Krizhevsky, I. Sutskever, and G. E. Hinton. Imagenet classification with deep convolutional neural networks. In NIPS, pages 1097--1105, 2012.
9. T. P. Minka. Expectation propagation for approximate bayesian inference. In UAI, pages 362--369, 2001.
10. Z. X. Niu, G. Hua, X. B. Gao, and Q. Tian. Semi-supervised relational topic model for weakly annotated image recognition in social media. In IEEE CVPR, pages 4233--4240, 2014.
11. M. Rosen-Zvi, T. Griffiths, M. Steyvers, and P. Smyth. The author-topic model for authors and documents. In UAI, pages 487--494, 2004.
12. C. Wang, D. M. Blei, and F. F. Li. Simultaneous image classification and annotation. In IEEE CVPR, pages 1903--1910, 2009.
13. Y. Wang, J. Liu, J. S. Qu, Y. L. Huang, J. M. Chen, and X. Feng. Hashtag graph based topic model for tweet mining. In IEEE ICDM, pages 1025--1030, 2014.
14. M. D. Zeiler and R. Fergus. Visualizing and understanding convolutional networks. In ECCV, pages 818--833. 2014.
15. Y. Zheng, Y. J. Zhang, and L. Hugo. A deep and autoregressive approach for topic modeling of multimodal data. arXiv preprint:1409.3970, 2014.

Index Terms

• Information systems

  • Information retrieval

    • Document representation