Variational approximation for mixtures of linear mixed models

Details Variational approximation for mixtures of linear mixed models Variational approximation for mixtures of linear mixed models

2011-12-20

arxiv.org/abs/1112.4675v1

Statistics

Applications

36 pages, 5 figures, 2 tables, submitted to JCGS

Scientific paper

Mixtures of linear mixed models (MLMMs) are useful for clustering grouped data in applications such as gene expression time course experiments. These models can be estimated by likelihood maximization through the EM algorithm and the optimal number of components determined by comparing different mixture models using penalized log-likelihood criteria such as BIC. In this paper, we propose fitting MLMMs with variational methods which can perform parameter estimation and model selection simultaneously. We consider MLMMs where the response distribution is a normal mixture and describe a variational approximation where the variational lower bound and parameter updates are in closed form, allowing for fast evaluation. A new variational greedy algorithm (VGA) is developed for model selection and learning of the mixture components. Starting with one component, this algorithm adds new components to the mixture model after searching for the optimal way to split each mixture component. This approach allows an automatic initialization of the algorithm and also automatically returns a plausible number of mixture components. In cases of weak identifiability of certain model parameters, we use hierarchical centering to reparametrize the model and show empirically that there is a gain in efficiency by variational algorithms through the use of hierarchical centering similar to that in MCMC algorithms. Related to this we prove that the approximate rate of convergence of the variational algorithm by Gaussian approximation is equal to that of the corresponding Gibbs sampler. This gives insight into the effects of different parametrizations on convergence rates in a linear mixed model, insight which is also relevant to mixture models.

Affiliated with

Also associated with

No associations

Rating

Variational approximation for mixtures of linear mixed models does not yet have a rating. At this time, there are no reviews or comments for this scientific paper.

If you have personal experience with Variational approximation for mixtures of linear mixed models, we encourage you to share that experience with our LandOfFree.com community. Your opinion is very important and Variational approximation for mixtures of linear mixed models will most certainly appreciate the feedback.

Rate now

Comments { 0 }

Profile ID: LFWR-SCP-O-56288