Research‎ > ‎

Reliable and Scalable Variational Inference for the HDP

AUTHORS
Michael Hughes
Dae Il Kim
Erik Sudderth

FULL TEXT: PDF

SUPPLEMENT: PDF

ABSTRACT

We introduce a new variational inference objective for hierarchical Dirichlet process admixture models. Our approach provides novel and scalable algorithms for learning nonparametric topic models of text documents and Gaussian admixture models of image patches. Improving on the point estimates of topic probabilities used in previous work, we define full variational posteriors for all latent variables and optimize parameters via a novel surrogate likelihood bound. We show that this approach has crucial advantages for data-driven learning of the num- ber of topics. Via merge and delete moves that remove redundant or irrelevant topics, we learn compact and interpretable models with less computation. Scaling to millions of documents is possible using stochastic or memoized variational updates. 

CODE

Available online: bitbucket.org/michaelchughes/bnpy-dev/
Ċ
Mike Hughes,
Feb 16, 2015, 6:46 AM
Ċ
Mike Hughes,
Feb 16, 2015, 6:46 AM
Comments