Pac-bayesian bounds for sparse regression estimation with exponential weights

Details Pac-bayesian bounds for sparse regression estimation with exponential weights Pac-bayesian bounds for sparse regression estimation with exponential weights

2010-09-14

arxiv.org/abs/1009.2707v2

Electronic Journal of Statistics, Vol 5(2011), 127-145

Mathematics

Statistics Theory

19 pages

Scientific paper

10.1214/11-EJS601

We consider the sparse regression model where the number of parameters $p$ is larger than the sample size $n$. The difficulty when considering high-dimensional problems is to propose estimators achieving a good compromise between statistical and computational performances. The BIC estimator for instance performs well from the statistical point of view \cite{BTW07} but can only be computed for values of $p$ of at most a few tens. The Lasso estimator is solution of a convex minimization problem, hence computable for large value of $p$. However stringent conditions on the design are required to establish fast rates of convergence for this estimator. Dalalyan and Tsybakov \cite{arnak} propose a method achieving a good compromise between the statistical and computational aspects of the problem. Their estimator can be computed for reasonably large $p$ and satisfies nice statistical properties under weak assumptions on the design. However, \cite{arnak} proposes sparsity oracle inequalities in expectation for the empirical excess risk only. In this paper, we propose an aggregation procedure similar to that of \cite{arnak} but with improved statistical performances. Our main theoretical result is a sparsity oracle inequality in probability for the true excess risk for a version of exponential weight estimator. We also propose a MCMC method to compute our estimator for reasonably large values of $p$.

Affiliated with

Also associated with

No associations

Rating

Pac-bayesian bounds for sparse regression estimation with exponential weights does not yet have a rating. At this time, there are no reviews or comments for this scientific paper.

If you have personal experience with Pac-bayesian bounds for sparse regression estimation with exponential weights, we encourage you to share that experience with our LandOfFree.com community. Your opinion is very important and Pac-bayesian bounds for sparse regression estimation with exponential weights will most certainly appreciate the feedback.

Rate now

Comments { 0 }

Profile ID: LFWR-SCP-O-396415