Mathematics – Statistics Theory
Scientific paper
2009-01-11
_Electronic Journal of Statistics_, vol. 3 (2009): 1039--1074
Mathematics
Statistics Theory
36 pages, 1 figure. v2: typo fixes, minor formatting changes. v3: Improved notation, added references, new theorem on converge
Scientific paper
10.1214/09-EJS485
Much is now known about the consistency of Bayesian updating on infinite-dimensional parameter spaces with independent or Markovian data. Necessary conditions for consistency include the prior putting enough weight on the correct neighborhoods of the data-generating distribution; various sufficient conditions further restrict the prior in ways analogous to capacity control in frequentist nonparametrics. The asymptotics of Bayesian updating with mis-specified models or priors, or non-Markovian data, are far less well explored. Here I establish sufficient conditions for posterior convergence when all hypotheses are wrong, and the data have complex dependencies. The main dynamical assumption is the asymptotic equipartition (Shannon-McMillan-Breiman) property of information theory. This, along with Egorov's Theorem on uniform convergence, lets me build a sieve-like structure for the prior. The main statistical assumption, also a form of capacity control, concerns the compatibility of the prior and the data-generating process, controlling the fluctuations in the log-likelihood when averaged over the sieve-like sets. In addition to posterior convergence, I derive a kind of large deviations principle for the posterior measure, extending in some cases to rates of convergence, and discuss the advantages of predicting using a combination of models known to be wrong. An appendix sketches connections between these results and the replicator dynamics of evolutionary theory.
No associations
LandOfFree
Dynamics of Bayesian Updating with Dependent Data and Misspecified Models does not yet have a rating. At this time, there are no reviews or comments for this scientific paper.
If you have personal experience with Dynamics of Bayesian Updating with Dependent Data and Misspecified Models, we encourage you to share that experience with our LandOfFree.com community. Your opinion is very important and Dynamics of Bayesian Updating with Dependent Data and Misspecified Models will most certainly appreciate the feedback.
Profile ID: LFWR-SCP-O-531880