A Bernstein-Von Mises Theorem for discrete probability distributions

Details A Bernstein-Von Mises Theorem for discrete probability distributions A Bernstein-Von Mises Theorem for discrete probability distributions

2008-07-14

arxiv.org/abs/0807.2096v2

Electronic Journal of Statistics 2009, Vol. 3, 114-148

Mathematics

Statistics Theory

Published in at http://dx.doi.org/10.1214/08-EJS262 the Electronic Journal of Statistics (http://www.i-journals.org/ejs/) by t

Scientific paper

10.1214/08-EJS262

We investigate the asymptotic normality of the posterior distribution in the discrete setting, when model dimension increases with sample size. We consider a probability mass function $\theta_0$ on $\mathbbm{N}\setminus \{0\}$ and a sequence of truncation levels $(k_n)_n$ satisfying $k_n^3\leq n\inf_{i\leq k_n}\theta_0(i).$ Let $\hat{\theta}$ denote the maximum likelihood estimate of $(\theta_0(i))_{i\leq k_n}$ and let $\Delta_n(\theta_0)$ denote the $k_n$-dimensional vector which $i$-th coordinate is defined by \sqrt{n} (\hat{\theta}_n(i)-\theta_0(i)) for $1\leq i\leq k_n.$ We check that under mild conditions on $\theta_0$ and on the sequence of prior probabilities on the $k_n$-dimensional simplices, after centering and rescaling, the variation distance between the posterior distribution recentered around $\hat{\theta}_n$ and rescaled by $\sqrt{n}$ and the $k_n$-dimensional Gaussian distribution $\mathcal{N}(\Delta_n(\theta_0),I^{-1}(\theta_0))$ converges in probability to $0.$ This theorem can be used to prove the asymptotic normality of Bayesian estimators of Shannon and R\'{e}nyi entropies. The proofs are based on concentration inequalities for centered and non-centered Chi-square (Pearson) statistics. The latter allow to establish posterior concentration rates with respect to Fisher distance rather than with respect to the Hellinger distance as it is commonplace in non-parametric Bayesian statistics.

Affiliated with

Also associated with

No associations

Rating

A Bernstein-Von Mises Theorem for discrete probability distributions does not yet have a rating. At this time, there are no reviews or comments for this scientific paper.

If you have personal experience with A Bernstein-Von Mises Theorem for discrete probability distributions, we encourage you to share that experience with our LandOfFree.com community. Your opinion is very important and A Bernstein-Von Mises Theorem for discrete probability distributions will most certainly appreciate the feedback.

Rate now

Comments { 0 }

Profile ID: LFWR-SCP-O-442861