High-dimensional regression with noisy and missing data: Provable guarantees with non-convexity

Details High-dimensional regression with noisy and missing data: Provable guarantees with non-convexity High-dimensional regression with noisy and missing data: Provable guarantees with non-convexity

2011-09-16

arxiv.org/abs/1109.3714v1

Mathematics

Statistics Theory

39 pages, 12 figures

Scientific paper

Although the standard formulations of prediction problems involve fully-observed and noiseless data drawn in an i.i.d. manner, many applications involve noisy and/or missing data, possibly involving dependence as well. We study these issues in the context of high-dimensional sparse linear regression, and propose novel estimators for the cases of noisy, missing, and/or dependent data. Many standard approaches to noisy or missing data, such as those using the EM algorithm, lead to optimization problems that are inherently non-convex, and it is difficult to establish theoretical guarantees on practical algorithms. While our approach also involves optimizing non-convex programs, we are able to both analyze the statistical error associated with any global optimum, and more surprisingly, to prove that a simple algorithm based on projected gradient descent will converge in polynomial time to a small neighborhood of the set of all global minimizers. On the statistical side, we provide non-asymptotic bounds that hold with high probability for the cases of noisy, missing, and/or dependent data. On the computational side, we prove that under the same types of conditions required for statistical consistency, the projected gradient descent algorithm is guaranteed to converge at a geometric rate to a near-global minimizer. We illustrate these theoretical predictions with simulations, showing close agreement with the predicted scalings.

Affiliated with

Also associated with

No associations

Rating

High-dimensional regression with noisy and missing data: Provable guarantees with non-convexity does not yet have a rating. At this time, there are no reviews or comments for this scientific paper.

If you have personal experience with High-dimensional regression with noisy and missing data: Provable guarantees with non-convexity, we encourage you to share that experience with our LandOfFree.com community. Your opinion is very important and High-dimensional regression with noisy and missing data: Provable guarantees with non-convexity will most certainly appreciate the feedback.

Rate now

Comments { 0 }

Profile ID: LFWR-SCP-O-142039