Joint sentiment/topic model for sentiment analysis

ABSTRACT

Sentiment analysis or opinion mining aims to use automated tools to detect subjective information such as opinions, attitudes, and feelings expressed in text. This paper proposes a novel probabilistic modeling framework based on Latent Dirichlet Allocation (LDA), called joint sentiment/topic model (JST), which detects sentiment and topic simultaneously from text. Unlike other machine learning approaches to sentiment classification which often require labeled corpora for classifier training, the proposed JST model is fully unsupervised. The model has been evaluated on the movie review dataset to classify the review sentiment polarity and minimum prior information have also been explored to further improve the sentiment classification accuracy. Preliminary experiments have shown promising results achieved by JST.

References

1. A. Abbasi, H. Chen, and A. Salem. Sentiment analysis in multiple languages: Feature selection for opinion classification in web forums. ACM Trans. Inf. Syst., 26(3):1--34, 2008.
2. D. M. Blei, A. Y. Ng, and M. I. Jordan. Latent dirichlet allocation. J. Mach. Learn. Res., 3:993--1022, 2003.
3. J. Blitzer, M. Dredze, and F. Pereira. Biographies, bollywood, boom-boxes and blenders: Domain adaptation for sentiment classification. In Proceedings of the 45th Annual Meeting of the Association of Computational Linguistics, pages 440--447, Prague, Czech Republic, June 2007. Association for Computational Linguistics.
4. Y. Choi, C. Cardie, E. Riloff, and S. Patwardhan. Identifying sources of opinions with conditional random fields and extraction patterns. In Proceedings of Human Language Technology Conference and Conference on Empirical Methods in Natural Language Processing, pages 355--362, Vancouver, British Columbia, Canada, October 2005. Association for Computational Linguistics.
5. K. Eguchi and V. Lavrenko. Sentiment retrieval using generative models. In Proceedings of the 2006 Conference on Empirical Methods in Natural Language Processing, pages 345--354, Sydney, Australia, July 2006. Association for Computational Linguistics.
6. T. Griffiths and M. Steyvers. Finding scientific topics. Proceedings of the National Academy of Sciences, 101(90001):5228--5235, 2004.
7. S. Hayakawa and E. Ehrlich. Choose the right word: A contemporary guide to selecting the precise word for every situation. Collins, 1994.
8. T. Hofmann. Probabilistic latent semantic indexing. In SIGIR '99: Proceedings of the 22nd annual international ACM SIGIR conference on Research and development in information retrieval, pages 50--57, New York, NY, USA, 1999. ACM.
9. N. Kaji and M. Kitsuregawa. Automatic construction of polarity-tagged corpus from html documents. In Proceedings of the COLING/ACL on Main conference poster sessions, pages 452--459, Morristown, NJ, USA, 2006. Association for Computational Linguistics.
10. A. Kennedy and D. Inkpen. Sentiment classification of movie reviews using contextual valence shifters. Computational Intelligence, 22(2):110--125, 2006.
11. S.-M. Kim and E. Hovy. Determining the sentiment of opinions. In COLING '04: Proceedings of the 20th international conference on Computational Linguistics, page 1367, Morristown, NJ, USA, 2004. Association for Computational Linguistics.
12. W. Li and A. McCallum. Pachinko allocation: Dag-structured mixture models of topic correlations. In ICML '06: Proceedings of the 23rd international conference on Machine learning, pages 577--584, 2006.
13. R. McDonald, K. Hannan, T. Neylon, M. Wells, and J. Reynar. Structured models for fine-to-coarse sentiment analysis. In Proceedings of the 45th Annual Meeting of the Association of Computational Linguistics, pages 432--439, Prague, Czech Republic, June 2007. Association for Computational Linguistics.
14. Q. Mei, X. Ling, M. Wondra, H. Su, and C. Zhai. Topic sentiment mixture: modeling facets and opinions in weblogs. In WWW '07: Proceedings of the 16th international conference on World Wide Web, pages 171--180, New York, NY, USA, 2007. ACM.
15. B. Pang and L. Lee. A sentimental education: sentiment analysis using subjectivity summarization based on minimum cuts. In ACL '04: Proceedings of the 42nd Annual Meeting on Association for Computational Linguistics, page 271, Morristown, NJ, USA, 2004. Association for Computational Linguistics.
16. B. Pang, L. Lee, and S. Vaithyanathan. Thumbs up?: sentiment classification using machine learning techniques. In EMNLP '02: Proceedings of the ACL-02 conference on Empirical methods in natural language processing, pages 79--86, Morristown, NJ, USA, 2002. Association for Computational Linguistics.
17. M. F. Porter. An algorithm for suffix stripping. pages 313--316, 1997.
18. M. Steyvers and T. Griffiths. Probabilistic Topic Models. Handbook of Latent Semantic Analysis, page 427, 2007.
19. I. Titov and R. McDonald. A joint model of text and aspect ratings for sentiment summarization. In Proceedings of ACL-08: HLT, pages 308--316, Columbus, Ohio, June 2008. Association for Computational Linguistics.
20. I. Titov and R. McDonald. Modeling online reviews with multi-grain topic models. In WWW '08: Proceeding of the 17th international conference on World Wide Web, pages 111--120, New York, NY, USA, 2008. ACM.
21. P. D. Turney. Thumbs up or thumbs down?: semantic orientation applied to unsupervised classification of reviews. In ACL '02: Proceedings of the 40th Annual Meeting on Association for Computational Linguistics, pages 417--424, Morristown, NJ, USA, 2001. Association for Computational Linguistics.
22. P. D. Turney and M. L. Littman. Unsupervised learning of semantic orientation from a hundred-billion-word corpus. CoRR, cs.LG/0212012, 2002.
23. H. M. Wallach. Topic modeling: beyond bag-of-words. In ICML '06: Proceedings of the 23rd international conference on Machine learning, pages 977--984, 2006.
24. C. Whitelaw, N. Garg, and S. Argamon. Using appraisal groups for sentiment analysis. In CIKM '05: Proceedings of the 14th ACM international conference on Information and knowledge management, pages 625--631, New York, NY, USA, 2005. ACM.
25. J. Zhao, K. Liu, and G. Wang. Adding redundant features for CRFs-based sentence sentiment classification. In Proceedings of the 2008 Conference on Empirical Methods in Natural Language Processing, pages 117--126, Honolulu, Hawaii, October 2008. Association for Computational Linguistics.

Index Terms

• Computing methodologies

  • Artificial intelligence

    • Natural language processing

      • Language resources