Chaitanya Chemudugunta
Padhraic Smyth
ABSTRACT
Statistical topic models provide a general data-driven framework for automated discovery of high-level knowledge from large collections of text documents. While topic models can potentially discover a broad range of themes in a data set, the interpretability of the learned topics is not always ideal. Human-defined concepts, on the other hand, tend to be semantically richer due to careful selection of words to define concepts but they tend not to cover the themes in a data set exhaustively. In this paper, we propose a probabilistic framework to combine a hierarchy of human-defined semantic concepts with statistical topic models to seek the best of both worlds. Experimental results using two different sources of concept hierarchies and two collections of text documents indicate that this combination leads to systematic improvements in the quality of the associated language models as well as enabling new techniques for inferring and visualizing the semantics of a document.
- D. M. Blei and J. D. Lafferty. Correlated topic models. In NIPS, 2005.Google Scholar
Digital Library
- D. M. Blei, A. Y. Ng, and M. I. Jordan. Latent Dirichlet allocation. J. Mach Learn. Res., 3:993--1022, 2003. Google ScholarDigital Library
- D. Boyd-Graber, D. Blei, and X. Zhu. A topic model for word sense disambiguation. In Proc. 2007 Joint Conf. Empirical Methods in Nat'l. Lang. Processing and Compt'l. Nat'l. Lang. Learning, pages 1024--1033, 2007.Google Scholar
- C. Chemudugunta, A. Holloway, P. Smyth, and M. Steyvers. Modeling documents by combining semantic concepts with unsupervised statistical learning. In International Semantic Web Conference, 2008. Google ScholarDigital Library
- C. Chemudugunta, P. Smyth, and M. Steyvers. Modeling general and specific aspects of documents with a probabilistic topic model. In NIPS 19, 2007.Google Scholar
- C. Chemudugunta, P. Smyth, and M. Steyvers. Text modeling using unsupervised topic models and concept hierarchies. url: http://arxiv.org/abs/0808.0973. August 2008.Google Scholar
- T. L. Griffiths and M. Steyvers. Finding scientific topics. In Proc. of Nat'l. Academy of Science, volume 101, pages 5228--5235, 2004.Google ScholarCross Ref
- G. Ifrim, M. Theobald, and G. Weikum. Learning word-to-concept mappingsfor automatic text classification. In 22nd ICML-LWS, pages 18--26, 2005.Google Scholar
- D. M. Mimno, W. Li, and A. McCallum. Mixtures of hierarchical topics with pachinko allocation. In ICML, pages 633--640, 2007. Google ScholarDigital Library
- M. Steyvers, P. Smyth, M. Rosen-Zvi, and T. L. Griffiths. Probabilistic author-topic models for information discovery. In KDD, pages 306--315, 2004. Google ScholarDigital Library
Index Terms
- Combining concept hierarchies and statistical topic models
Recommendations
Automatic labeling of multinomial topic models
KDD '07: Proceedings of the 13th ACM SIGKDD international conference on Knowledge discovery and data miningMultinomial distributions over words are frequently used to model topics in text collections. A common, major challenge in applying all such topic models to any text mining problem is to label a multinomial topic model accurately so that a user can ...
Modeling Documents by Combining Semantic Concepts with Unsupervised Statistical Learning
ISWC '08: Proceedings of the 7th International Conference on The Semantic WebHuman-defined concepts are fundamental building-blocks in constructing knowledge bases such as ontologies. Statistical learning techniques provide an alternative automated approach to concept definition, driven by data rather than prior knowledge. In ...
Topic modeling with network regularization
WWW '08: Proceedings of the 17th international conference on World Wide WebIn this paper, we formally define the problem of topic modeling with network structure (TMN). We propose a novel solution to this problem, which regularizes a statistical topic model with a harmonic regularizer based on a graph structure in the data. ...
Comments