Honest variable selection in linear and logistic regression models via $\ell_1$ and $\ell_1+\ell_2$ penalization

Details Honest variable selection in linear and logistic regression models via $\ell_1$ and $\ell_1+\ell_2$ penalization Honest variable selection in linear and logistic regression models via $\ell_1$ and $\ell_1+\ell_2$ penalization

2008-08-29

arxiv.org/abs/0808.4051v2

Electronic Journal of Statistics 2008, Vol. 2, 1153-1194

Mathematics

Statistics Theory

Published in at http://dx.doi.org/10.1214/08-EJS287 the Electronic Journal of Statistics (http://www.i-journals.org/ejs/) by t

Scientific paper

10.1214/08-EJS287

This paper investigates correct variable selection in finite samples via $\ell_1$ and $\ell_1+\ell_2$ type penalization schemes. The asymptotic consistency of variable selection immediately follows from this analysis. We focus on logistic and linear regression models. The following questions are central to our paper: given a level of confidence $1-\delta$, under which assumptions on the design matrix, for which strength of the signal and for what values of the tuning parameters can we identify the true model at the given level of confidence? Formally, if $\widehat{I}$ is an estimate of the true variable set $I^*$, we study conditions under which $\mathbb{P}(\widehat{I}=I^*)\geq 1-\delta$, for a given sample size $n$, number of parameters $M$ and confidence $1-\delta$. We show that in identifiable models, both methods can recover coefficients of size $\frac{1}{\sqrt{n}}$, up to small multiplicative constants and logarithmic factors in $M$ and $\frac{1}{\delta}$. The advantage of the $\ell_1+\ell_2$ penalization over the $\ell_1$ is minor for the variable selection problem, for the models we consider here. Whereas the former estimates are unique, and become more stable for highly correlated data matrices as one increases the tuning parameter of the $\ell_2$ part, too large an increase in this parameter value may preclude variable selection.

Affiliated with

Also associated with

No associations

Rating

Honest variable selection in linear and logistic regression models via $\ell_1$ and $\ell_1+\ell_2$ penalization does not yet have a rating. At this time, there are no reviews or comments for this scientific paper.

If you have personal experience with Honest variable selection in linear and logistic regression models via $\ell_1$ and $\ell_1+\ell_2$ penalization, we encourage you to share that experience with our LandOfFree.com community. Your opinion is very important and Honest variable selection in linear and logistic regression models via $\ell_1$ and $\ell_1+\ell_2$ penalization will most certainly appreciate the feedback.

Rate now

Comments { 0 }

Profile ID: LFWR-SCP-O-542612