Placing Broadcast News Videos in their Social Media Context Using Hashtags

ABSTRACT

With the growth of social media platforms in recent years, social media is now a major source of information and news for many people around the world. In particular the rise of hashtags have helped to build communities of discussion around particular news, topics, opinions, and ideologies. However, television news programs still provide value and are used by a vast majority of the population to obtain their news, but these videos are not easily linked to broader discussion on social media. We have built a novel pipeline that allows television news to be placed in its relevant social media context, by leveraging hashtags. In this paper, we present a method for automatically collecting television news and social media content (Twitter) and discovering the hashtags that are relevant for a TV news video. Our algorithms incorporate both the visual and text information within social media and television content, and we show that by leveraging both modalities we can improve performance over single modality approaches.

References

1. J. Allan, J. G. Carbonell, G. Doddington, J. Yamron, and Y. Yang. Topic detection and tracking pilot study final report. 1998.
2. S. Amer-Yahia, S. Anjum, A. Ghenai, A. Siddique, S. Abbar, S. Madden, A. Marcus, and M. El-Haddad. Maqsa: a system for social analytics on news.
3. L. Ballan, M. Bertini, T. Uricchio, and A. Del Bimbo. Data-driven approaches for social image and video tagging. Multimedia Tools and Applications, 74(4):1443--1468, 2015.
4. H. Becker, M. Naaman, and L. Gravano. Beyond trending topics: Real-world event identification on twitter. 2011.
5. H.-C. Chang. A new perspective on twitter hashtag use: Diffusion of innovation theory. Proc. of the American Society for Information Science and Technology, 47(1):1--4, 2010.
6. E. Denton, J. Weston, M. Paluri, L. Bourdev, and R. Fergus. User conditional hashtag prediction for images.
7. S. Doan, B.-K. H. Vo, and N. Collier. An analysis of twitter messages in the 2011 tohoku earthquake. In Electronic Healthcare, pages 58--66. Springer, 2011.
8. F. Godin, V. Slavkovikj, W. De Neve, B. Schrauwen, and R. Van de Walle. Using topic models for twitter hashtag recommendation. In Proc. of the International Conference on World Wide Web, pages 593--596, 2013.
9. J. Hurlock and M. L. Wilson. Searching twitter: Separating the tweet from the chaff. In ICWSM, pages 161--168, 2011.
10. L. Jiang, M. Yu, M. Zhou, X. Liu, and T. Zhao. Target-dependent twitter sentiment classification.
11. B. Jou, H. Li, J. G. Ellis, D. Morozoff-Abegauz, and S.-F. Chang. Structured exploration of who, what, when, and where in heterogeneous multimedia news sources. In Proc. of the 21st ACM international conference on Multimedia, pages 357--360. ACM, 2013.
12. H. Li, J. G. Ellis, H. Ji, and S.-F. Chang. Event specific multimodal pattern mining for knowledge base construction. In Proceedings of the 24th ACM international conference on Multimedia. ACM, 2016.
13. H. Li, B. Jou, J. G. Ellis, D. Morozoff, and S.-F. Chang. News rover: Exploring topical structures and serendipity in heterogeneous multimedia news. In Proc. of the 21st ACM international conference on Multimedia, pages 449--450. ACM, 2013.
14. S. Liao, A. Jain, and S. Li. A fast and accurate unconstrained face detector. Transactions on Pattern Analysis and Machine Intelligence, 2014.
15. K. Massoudi, M. Tsagkias, M. De Rijke, and W. Weerkamp. Incorporating query expansion and quality indicators in searching microblog posts. In Advances in information retrieval, pages 362--367. Springer, 2011.
16. A. Pak and P. Paroubek. Twitter as a corpus for sentiment analysis and opinion mining. In LREc, volume 10, pages 1320--1326, 2010.
17. S. Petrović, M. Osborne, and V. Lavrenko. Streaming first story detection with application to twitter. In Human Language Technologies: The 2010 Annual Conference of the North American Chapter of the Association for Computational Linguistics, pages 181--189, 2010.
18. S. Phuvipadawat and T. Murata. Breaking news detection and tracking in twitter. In Web Intelligence and Intelligent Agent Technology (WI-IAT), 2010 IEEE/WIC/ACM International Conference on, volume 3, pages 120--123. IEEE, 2010.
19. T. Sakaki, M. Okazaki, and Y. Matsuo. Earthquake shakes twitter users: real-time event detection by social sensors. In Proc. of the 19th international conference on World wide web, pages 851--860. ACM, 2010.
20. J. Sankaranarayanan, H. Samet, B. E. Teitler, M. D. Lieberman, and J. Sperling. Twitterstand: news in tweets. In Proc. of the 17th acm sigspatial international conference on advances in geographic information systems, pages 42--51. ACM, 2009.
21. T. A. Small. What the hashtag? a content analysis of canadian politics on twitter. Information, Communication & Society, 14(6):872--895, 2011.
22. O. Tsur and A. Rappoport. What's in a hashtag?: content based prediction of the spread of ideas in microblogging communities. In Proc. of the fifth ACM international conference on Web search and data mining, pages 643--652. ACM, 2012.
23. A. Tumasjan, T. O. Sprenger, P. G. Sandner, and I. M. Welpe. Predicting elections with twitter: What 140 characters reveal about political sentiment. ICWSM, 10:178--185, 2010.
24. X. Wang, M. S. Gerber, and D. E. Brown. Automatic crime prediction using events extracted from twitter posts. In Social Computing, Behavioral-Cultural Modeling and Prediction, pages 231--238. Springer, 2012.
25. Y. Yang, J. G. Carbonell, R. D. Brown, T. Pierce, B. T. Archibald, and X. Liu. Learning approaches for detecting and tracking news events. IEEE Intelligent Systems, (4):32--43, 1999.
26. Y. Yang, T. Pierce, and J. Carbonell. A study of retrospective and on-line event detection. In Proc. of the 21st annual international ACM SIGIR conference on Research and development in information retrieval, pages 28--36. ACM, 1998.
27. D. Yi, Z. Lei, S. Liao, and S. Z. Li. Learning face representation from scratch. arXiv preprint arXiv:1411.7923, 2014.
28. J. Yuan, Y. Wu, and M. Yang. Discovery of collocation patterns: from visual words to visual phrases. In Computer Vision and Pattern Recognition, 2007. CVPR'07. IEEE Conference on, pages 1--8. IEEE, 2007.
29. J. Yuan, M. Yang, and Y. Wu. Mining discriminative co-occurrence patterns for visual recognition. In Computer Vision and Pattern Recognition (CVPR), 2011 IEEE Conference on, pages 2777--2784. IEEE, 2011.
30. B. Zhou, A. Lapedriza, J. Xiao, A. Torralba, and A. Oliva. Learning deep features for scene recognition using places database. In Advances in neural information processing systems, pages 487--495, 2014.

Index Terms

• Computing methodologies

  • Artificial intelligence

    • Computer vision

      • Computer vision tasks

        • Visual content-based indexing and retrieval

• Information systems

  • Information retrieval

    • Specialized information retrieval

      • Multimedia and multimodal retrieval