The nested Chinese restaurant process and Bayesian nonparametric inference of topic hierarchies

Details The nested Chinese restaurant process and Bayesian nonparametric inference of topic hierarchies The nested Chinese restaurant process and Bayesian nonparametric inference of topic hierarchies

2007-10-03

arxiv.org/abs/0710.0845v3

Statistics

Machine Learning

Scientific paper

We present the nested Chinese restaurant process (nCRP), a stochastic process which assigns probability distributions to infinitely-deep, infinitely-branching trees. We show how this stochastic process can be used as a prior distribution in a Bayesian nonparametric model of document collections. Specifically, we present an application to information retrieval in which documents are modeled as paths down a random tree, and the preferential attachment dynamics of the nCRP leads to clustering of documents according to sharing of topics at multiple levels of abstraction. Given a corpus of documents, a posterior inference algorithm finds an approximation to a posterior distribution over trees, topics and allocations of words to levels of the tree. We demonstrate this algorithm on collections of scientific abstracts from several journals. This model exemplifies a recent trend in statistical machine learning--the use of Bayesian nonparametric methods to infer distributions on flexible data structures.

Affiliated with

Also associated with

No associations

Rating

The nested Chinese restaurant process and Bayesian nonparametric inference of topic hierarchies does not yet have a rating. At this time, there are no reviews or comments for this scientific paper.

If you have personal experience with The nested Chinese restaurant process and Bayesian nonparametric inference of topic hierarchies, we encourage you to share that experience with our LandOfFree.com community. Your opinion is very important and The nested Chinese restaurant process and Bayesian nonparametric inference of topic hierarchies will most certainly appreciate the feedback.

Rate now

Comments { 0 }

Profile ID: LFWR-SCP-O-286109