Computer Science – Learning
Scientific paper
2003-01-16
IEEE Transactions on Information Theory, 49:8 (2003) 2061--2067
Computer Science
Learning
8 twocolumn pages
Scientific paper
The probability of observing $x_t$ at time $t$, given past observations $x_1...x_{t-1}$ can be computed with Bayes' rule if the true generating distribution $\mu$ of the sequences $x_1x_2x_3...$ is known. If $\mu$ is unknown, but known to belong to a class $M$ one can base ones prediction on the Bayes mix $\xi$ defined as a weighted sum of distributions $\nu\in M$. Various convergence results of the mixture posterior $\xi_t$ to the true posterior $\mu_t$ are presented. In particular a new (elementary) derivation of the convergence $\xi_t/\mu_t\to 1$ is provided, which additionally gives the rate of convergence. A general sequence predictor is allowed to choose an action $y_t$ based on $x_1...x_{t-1}$ and receives loss $\ell_{x_t y_t}$ if $x_t$ is the next symbol of the sequence. No assumptions are made on the structure of $\ell$ (apart from being bounded) and $M$. The Bayes-optimal prediction scheme $\Lambda_\xi$ based on mixture $\xi$ and the Bayes-optimal informed prediction scheme $\Lambda_\mu$ are defined and the total loss $L_\xi$ of $\Lambda_\xi$ is bounded in terms of the total loss $L_\mu$ of $\Lambda_\mu$. It is shown that $L_\xi$ is bounded for bounded $L_\mu$ and $L_\xi/L_\mu\to 1$ for $L_\mu\to \infty$. Convergence of the instantaneous losses are also proven.
No associations
LandOfFree
Convergence and Loss Bounds for Bayesian Sequence Prediction does not yet have a rating. At this time, there are no reviews or comments for this scientific paper.
If you have personal experience with Convergence and Loss Bounds for Bayesian Sequence Prediction, we encourage you to share that experience with our LandOfFree.com community. Your opinion is very important and Convergence and Loss Bounds for Bayesian Sequence Prediction will most certainly appreciate the feedback.
Profile ID: LFWR-SCP-O-535088