Fast global convergence of gradient methods for high-dimensional statistical recovery

Details Fast global convergence of gradient methods for high-dimensional statistical recovery Fast global convergence of gradient methods for high-dimensional statistical recovery

2011-04-25

arxiv.org/abs/1104.4824v2

Statistics

Machine Learning

Scientific paper

Many statistical $M$-estimators are based on convex optimization problems formed by the combination of a data-dependent loss function with a norm-based regularizer. We analyze the convergence rates of projected gradient and composite gradient methods for solving such problems, working within a high-dimensional framework that allows the data dimension $\pdim$ to grow with (and possibly exceed) the sample size $\numobs$. This high-dimensional structure precludes the usual global assumptions---namely, strong convexity and smoothness conditions---that underlie much of classical optimization analysis. We define appropriately restricted versions of these conditions, and show that they are satisfied with high probability for various statistical models. Under these conditions, our theory guarantees that projected gradient descent has a globally geometric rate of convergence up to the \emph{statistical precision} of the model, meaning the typical distance between the true unknown parameter $\theta^*$ and an optimal solution $\hat{\theta}$. This result is substantially sharper than previous convergence results, which yielded sublinear convergence, or linear convergence only up to the noise level. Our analysis applies to a wide range of $M$-estimators and statistical models, including sparse linear regression using Lasso ($\ell_1$-regularized regression); group Lasso for block sparsity; log-linear models with regularization; low-rank matrix recovery using nuclear norm regularization; and matrix decomposition. Overall, our analysis reveals interesting connections between statistical precision and computational efficiency in high-dimensional estimation.

Affiliated with

Also associated with

No associations

Rating

Fast global convergence of gradient methods for high-dimensional statistical recovery does not yet have a rating. At this time, there are no reviews or comments for this scientific paper.

If you have personal experience with Fast global convergence of gradient methods for high-dimensional statistical recovery, we encourage you to share that experience with our LandOfFree.com community. Your opinion is very important and Fast global convergence of gradient methods for high-dimensional statistical recovery will most certainly appreciate the feedback.

Rate now

Comments { 0 }

Profile ID: LFWR-SCP-O-295173