Kernel Partial Least Squares is Universally Consistent

Details Kernel Partial Least Squares is Universally Consistent Kernel Partial Least Squares is Universally Consistent

2009-02-25

arxiv.org/abs/0902.4380v2

JMLR Workshop and Conference Proceedings 9 (AISTATS 2010) 57-64, 2010

Statistics

Methodology

18 pages, no figures

Scientific paper

We prove the statistical consistency of kernel Partial Least Squares Regression applied to a bounded regression learning problem on a reproducing kernel Hilbert space. Partial Least Squares stands out of well-known classical approaches as e.g. Ridge Regression or Principal Components Regression, as it is not defined as the solution of a global cost minimization procedure over a fixed model nor is it a linear estimator. Instead, approximate solutions are constructed by projections onto a nested set of data-dependent subspaces. To prove consistency, we exploit the known fact that Partial Least Squares is equivalent to the conjugate gradient algorithm in combination with early stopping. The choice of the stopping rule (number of iterations) is a crucial point. We study two empirical stopping rules. The first one monitors the estimation error in each iteration step of Partial Least Squares, and the second one estimates the empirical complexity in terms of a condition number. Both stopping rules lead to universally consistent estimators provided the kernel is universal.

Affiliated with

Also associated with

No associations

Rating

Kernel Partial Least Squares is Universally Consistent does not yet have a rating. At this time, there are no reviews or comments for this scientific paper.

If you have personal experience with Kernel Partial Least Squares is Universally Consistent, we encourage you to share that experience with our LandOfFree.com community. Your opinion is very important and Kernel Partial Least Squares is Universally Consistent will most certainly appreciate the feedback.

Rate now

Comments { 0 }

Profile ID: LFWR-SCP-O-158085