Probabilistic topic models

Abstract

Surveying a suite of algorithms that offer a solution to managing large document archives.

References

1. Asuncion, A., Welling, M., Smyth, P., Teh, Y. On smoothing and inference for topic models. In Uncertainty in Artificial Intelligence (2009).
2. Bart, E., Welling, M., Perona, P. Unsupervised organization of image collections: Taxonomies and beyond. Trans. Pattern Recognit. Mach. Intell. 33, 11 (2010) (2301--2315).
3. Blei, D., Griffiths, T., Jordan, M. The nested Chinese restaurant process and Bayesian nonparametric inference of topic hierarchies. J. ACM 57, 2 (2010), 1--30.
4. Blei, D., Jordan, M. Modeling annotated data. In Proceedings of the 26th Annual International ACM SIGIR Conference on Research and Development in Information Retrieval (2003), ACM Press, 127--134.
5. Blei, D., Lafferty, J. Dynamic topic models. In International Conference on Machine Learning (2006), ACM, New York, NY, USA, 113--120.
6. Blei, D., Lafferty, J. A correlated topic model of Science. Ann. Appl. Stat., 1, 1 (2007), 17--35.
7. Blei, D., McAuliffe, J. Supervised topic models. In Neural Information Processing Systems (2007).
8. Blei, D., Ng, A., Jordan, M. Latent Dirichlet allocation. J. Mach. Learn. Res. 3 (January 2003), 993--1022.
9. Box, G. Sampling and Bayes' inference in scientific modeling and robustness. J. Roy. Stat. Soc. 143, 4 (1980), 383--430.
10. Boyd-Graber, J., Blei, D. Syntactic topic models. In Neural Information Processing Systems (2009).
11. Buntine, W. Variational extensions to EM and multinomial PCA. In European Conference on Machine Learning (2002).
12. Buntine, W., Jakulin, A. Discrete component analysis. Subspace, Latent Structure and Feature Selection. C. Saunders, M. Grobelink, S. Gunn, and J. Shawe-Taylor, Eds. Springer, 2006.
13. Chang, J., Blei, D. Hierarchical relational models for document networks. Ann. Appl. Stat. 4, 1 (2010).
14. Deerwester, S., Dumais, S., Landauer, T., Furnas, G., Harshman, R. Indexing by latent semantic analysis. J. Am. Soc. Inform. Sci. 41, 6 (1990), 391--407.
15. Doyle, G., Elkan, C., Accounting for burstiness in topic models. In International Conference on Machine Learning (2009), ACM, 281--288..
16. Fei-Fei, L., Perona, P. A Bayesian hierarchical model for learning natural scene categories. In IEEE Computer Vision and Pattern Recognition (2005), 524--531.
17. Gerrish, S., Blei, D. A language-based approach to measuring scholarly impact. In International Conference on Machine Learning (2010).
18. Griffiths, T., Steyvers, M., Blei, D., Tenenbaum, J. Integrating topics and syntax. Advances in Neural Information Processing Systems 17. L. K. Saul, Y. Weiss, and L. Bottou, eds. MIT Press, Cambridge, MA, 2005, 537--544.
19. Grimmer, J. A Bayesian hierarchical topic model for political texts: Measuring expressed agendas in senate press releases. Polit. Anal. 18, 1 (2010), 1.
20. Hoffman, M., Blei, D., Bach, F. On-line learning for latent Dirichlet allocation. In Neural Information Processing Systems (2010).
21. Hofmann, T. Probabilistic latent semantic analysis. In Uncertainty in Artificial Intelligence (UAI) (1999).
22. Jordan, M., Ghahramani, Z., Jaakkola, T., Saul, L. Introduction to variational methods for graphical models. Mach. Learn. 37 (1999), 183--233.
23. Li, J., Wang, C., Lim, Y., Blei, D., Fei-Fei, L., Building and using a semantivisual image hierarchy. In Computer Vision and Pattern Recognition (2010).
24. Li, W., McCallum, A. Pachinko allocation: DAG-structured mixture models of topic correlations. In International Conference on Machine Learning (2006), 577--584.
25. Mimno, D., McCallum, A. Topic models conditioned on arbitrary features with Dirichlet-multinomial regression. In Uncertainty in Artificial Intelligence (2008).
26. Newman, D., Chemudugunta, C., Smyth, P. Statistical entity-topic models. In Knowledge Discovery and Data Mining (2006).
27. Pritchard, J., Stephens, M., Donnelly, P. Inference of population structure using multilocus genotype data. Genetics 155 (June 2000), 945--959.
28. Reisinger, J., Waters, A., Silverthorn, B., Mooney, R. Spherical topic models. In International Conference on Machine Learning (2010).
29. Rosen-Zvi, M., Griffiths, T., Steyvers, M., Smith, P., The author-topic model for authors and documents. In Proceedings of the 20th Conference on Uncertainty in Artificial Intelligence (2004), AUAI Press, 487--494.
30. Rubin, D. Bayesianly justifiable and relevant frequency calculations for the applied statistician. Ann. Stat. 12, 4 (1984), 1151--1172.
31. Sivic, J., Russell, B., Zisserman, A., Freeman, W., Efros, A., Unsupervised discovery of visual object class hierarchies. In Conference on Computer Vision and Pattern Recognition (2008).
32. Socher, R., Gershman, S., Perotte, A., Sederberg, P., Blei, D., Norman, K. A Bayesian analysis of dynamics in free recall. In Advances in Neural Information Processing Systems 22. Y. Bengio, D. Schuurmans, J. Lafferty, C. K. I. Williams, and A. Culotta, Eds, 2009.
33. Steyvers, M., Griffiths, T. Probabilistic topic models. Latent Semantic Analysis: A Road to Meaning. T. Landauer, D. McNamara, S. Dennis, and W. Kintsch, eds. Lawrence Erlbaum, 2006.
34. Teh, Y., Jordan, M., Beal, M., Blei, D. Hierarchical Dirichlet processes. J. Am. Stat. Assoc. 101, 476 (2006), 1566--1581.
35. Wainwright, M., Jordan, M. Graphical models, exponential families, and variational inference. Found. Trends Mach. Learn. 1(1--2) (2008), 1--305.
36. Wallach, H. Topic modeling: Beyond bag of words. In Proceedings of the 23rd International Conference on Machine Learning (2006).
37. Wang, C., Blei, D. Decoupling sparsity and smoothness in the discrete hierarchical Dirichlet process. Advances in Neural Information Processing Systems 22. Y. Bengio, D. Schuurmans, J. Lafferty, C. K. I. Williams, and A. Culotta, Eds. 2009, 1982--1989.
38. Wang, C., Thiesson, B., Meek, C., Blei, D. Markov topic models. In Artificial Intelligence and Statistics (2009).

Index Terms

• Information systems

  • Information retrieval

  • Information systems applications