Abstract: This paper introduces a new topic model based
on an admixture of Poisson Markov Random
Fields (APM), which can model dependencies
between words as opposed to previous independent topic models such as PLSA (Hofmann, 1999), LDA (Blei et al., 2003) or SAM
(Reisinger et al., 2010). We propose a class of
admixture models that generalizes previous topic
models and show an equivalence between the
conditional distribution of LDA and independent
Poissons—suggesting that APM subsumes the
modeling power of LDA. We present a tractable
method for estimating the parameters of an APM
based on the pseudo log-likelihood and demonstrate the benefits of APM over previous models
by preliminary qualitative and quantitative experiments.
Download: pdf, slides, poster, code
Citation
- Admixture of Poisson MRFs: A Topic Model with Word Dependencies (pdf, slides, poster, software, code)
D. Inouye, P. Ravikumar, I. Dhillon.
In International Conference on Machine Learning (ICML), pp. 683-691, June 2014.
Bibtex: