Mathematics – Statistics Theory
Scientific paper
2006-05-31
Mathematics
Statistics Theory
Scientific paper
We present here a PAC-Bayesian point of view on adaptive supervised classification. Using convex analysis, we show how to get local measures of the complexity of the classification model involving the relative entropy of posterior distributions with respect to Gibbs posterior measures. We discuss relative bounds, comparing two classification rules, to show how the margin assumption of Mammen and Tsybakov can be replaced with some empirical measure of the covariance structure of the classification model. We also show how to associate to any posterior distribution an {\em effective temperature} relating it to the Gibbs prior distribution with the same level of expected error rate, and how to estimate this effective temperature from data, resulting in an estimator whose expected error rate adaptively converges according to the best possible power of the sample size. Then we introduce a PAC-Bayesian point of view on transductive learning and use it to improve on known Vapnik's generalization bounds, extending them to the case when the sample is independent but not identically distributed. Eventually we review briefly the construction of Support Vector Machines and show how to derive generalization bounds for them, measuring the complexity either through the number of support vectors or through transductive or inductive margin estimates.
No associations
LandOfFree
PAC-Bayesian inductive and transductive learning does not yet have a rating. At this time, there are no reviews or comments for this scientific paper.
If you have personal experience with PAC-Bayesian inductive and transductive learning, we encourage you to share that experience with our LandOfFree.com community. Your opinion is very important and PAC-Bayesian inductive and transductive learning will most certainly appreciate the feedback.
Profile ID: LFWR-SCP-O-711424