Statistics – Machine Learning
Scientific paper
2012-03-25
Statistics
Machine Learning
Scientific paper
Sparsity-constrained optimization has wide applicability in machine learning, statistics, and signal processing problems such as feature selection and Compressive Sensing. A vast body of work has studied the sparsity-constrained optimization from theoretical, algorithmic, and application aspects in the context of sparse estimation in linear models where the fidelity of the estimate is measured by the squared error. In contrast, relatively less effort has been made in the study of sparsity-constrained optimization in cases where nonlinear models are involved or the cost function is not quadratic. In this paper we propose a greedy algorithm, Gradient Support Pursuit (GraSP), to approximate sparse minima of cost functions of arbitrary form. Should a cost function have a Stable Restricted Hessian (SRH) or a Stable Restricted Linearization (SRL), both of which are introduced in this paper, our algorithm is guaranteed to produce a sparse vector within a bounded distance from the true sparse optimum. Our approach generalizes known results for quadratic cost functions that arise in sparse linear regression and Compressive Sensing. We also evaluate the performance of GraSP through numerical simulations on synthetic data, where the algorithm is employed for sparse logistic regression with and without $\ell_2$-regularization.
Bahmani Sohail
Boufounos Petros
Raj Bhiksha
No associations
LandOfFree
Greedy Sparsity-Constrained Optimization does not yet have a rating. At this time, there are no reviews or comments for this scientific paper.
If you have personal experience with Greedy Sparsity-Constrained Optimization, we encourage you to share that experience with our LandOfFree.com community. Your opinion is very important and Greedy Sparsity-Constrained Optimization will most certainly appreciate the feedback.
Profile ID: LFWR-SCP-O-74960