$k$-MLE: A fast algorithm for learning statistical mixture models

Details $k$-MLE: A fast algorithm for learning statistical mixture models $k$-MLE: A fast algorithm for learning statistical mixture models

2012-03-23

arxiv.org/abs/1203.5181v1

Computer Science

Learning

31 pages, Extend preliminary paper presented at IEEE ICASSP 2012

Scientific paper

We describe $k$-MLE, a fast and efficient local search algorithm for learning finite statistical mixtures of exponential families such as Gaussian mixture models. Mixture models are traditionally learned using the expectation-maximization (EM) soft clustering technique that monotonically increases the incomplete (expected complete) likelihood. Given prescribed mixture weights, the hard clustering $k$-MLE algorithm iteratively assigns data to the most likely weighted component and update the component models using Maximum Likelihood Estimators (MLEs). Using the duality between exponential families and Bregman divergences, we prove that the local convergence of the complete likelihood of $k$-MLE follows directly from the convergence of a dual additively weighted Bregman hard clustering. The inner loop of $k$-MLE can be implemented using any $k$-means heuristic like the celebrated Lloyd's batched or Hartigan's greedy swap updates. We then show how to update the mixture weights by minimizing a cross-entropy criterion that implies to update weights by taking the relative proportion of cluster points, and reiterate the mixture parameter update and mixture weight update processes until convergence. Hard EM is interpreted as a special case of $k$-MLE when both the component update and the weight update are performed successively in the inner loop. To initialize $k$-MLE, we propose $k$-MLE++, a careful initialization of $k$-MLE guaranteeing probabilistically a global bound on the best possible complete likelihood.

Affiliated with

Also associated with

No associations

Rating

$k$-MLE: A fast algorithm for learning statistical mixture models does not yet have a rating. At this time, there are no reviews or comments for this scientific paper.

If you have personal experience with $k$-MLE: A fast algorithm for learning statistical mixture models, we encourage you to share that experience with our LandOfFree.com community. Your opinion is very important and $k$-MLE: A fast algorithm for learning statistical mixture models will most certainly appreciate the feedback.

Rate now

Comments { 0 }

Profile ID: LFWR-SCP-O-471768