Estimating divergence functionals and the likelihood ratio by convex risk minimization

Details Estimating divergence functionals and the likelihood ratio by convex risk minimization Estimating divergence functionals and the likelihood ratio by convex risk minimization

2008-09-04

arxiv.org/abs/0809.0853v2

IEEE Transactions on Information Theory, 56(11), 5847--5861, 2010

Mathematics

Statistics Theory

28 pages

Scientific paper

We develop and analyze $M$-estimation methods for divergence functionals and the likelihood ratios of two probability distributions. Our method is based on a non-asymptotic variational characterization of $f$-divergences, which allows the problem of estimating divergences to be tackled via convex empirical risk optimization. The resulting estimators are simple to implement, requiring only the solution of standard convex programs. We present an analysis of consistency and convergence for these estimators. Given conditions only on the ratios of densities, we show that our estimators can achieve optimal minimax rates for the likelihood ratio and the divergence functionals in certain regimes. We derive an efficient optimization algorithm for computing our estimates, and illustrate their convergence behavior and practical viability by simulations.

Affiliated with

Also associated with

No associations

Rating

Estimating divergence functionals and the likelihood ratio by convex risk minimization does not yet have a rating. At this time, there are no reviews or comments for this scientific paper.

If you have personal experience with Estimating divergence functionals and the likelihood ratio by convex risk minimization, we encourage you to share that experience with our LandOfFree.com community. Your opinion is very important and Estimating divergence functionals and the likelihood ratio by convex risk minimization will most certainly appreciate the feedback.

Rate now

Comments { 0 }

Profile ID: LFWR-SCP-O-142204