A Family of Computationally Efficient and Simple Estimators for Unnormalized Statistical Models

Details A Family of Computationally Efficient and Simple Estimators for Unnormalized Statistical Models A Family of Computationally Efficient and Simple Estimators for Unnormalized Statistical Models

2012-03-15

arxiv.org/abs/1203.3506v1

Computer Science

Learning

Appears in Proceedings of the Twenty-Sixth Conference on Uncertainty in Artificial Intelligence (UAI2010)

Scientific paper

We introduce a new family of estimators for unnormalized statistical models. Our family of estimators is parameterized by two nonlinear functions and uses a single sample from an auxiliary distribution, generalizing Maximum Likelihood Monte Carlo estimation of Geyer and Thompson (1992). The family is such that we can estimate the partition function like any other parameter in the model. The estimation is done by optimizing an algebraically simple, well defined objective function, which allows for the use of dedicated optimization methods. We establish consistency of the estimator family and give an expression for the asymptotic covariance matrix, which enables us to further analyze the influence of the nonlinearities and the auxiliary density on estimation performance. Some estimators in our family are particularly stable for a wide range of auxiliary densities. Interestingly, a specific choice of the nonlinearity establishes a connection between density estimation and classification by nonlinear logistic regression. Finally, the optimal amount of auxiliary samples relative to the given amount of the data is considered from the perspective of computational efficiency.

Affiliated with

Also associated with

No associations

Rating

A Family of Computationally Efficient and Simple Estimators for Unnormalized Statistical Models does not yet have a rating. At this time, there are no reviews or comments for this scientific paper.

If you have personal experience with A Family of Computationally Efficient and Simple Estimators for Unnormalized Statistical Models, we encourage you to share that experience with our LandOfFree.com community. Your opinion is very important and A Family of Computationally Efficient and Simple Estimators for Unnormalized Statistical Models will most certainly appreciate the feedback.

Rate now

Comments { 0 }

Profile ID: LFWR-SCP-O-32306