Statistics – Machine Learning
Scientific paper
2009-11-20
Journal of Machine Learning Research, 12(May):1537-1586, 2011
Statistics
Machine Learning
51 pages, 9 figures
Scientific paper
We analyze the convergence behaviour of a recently proposed algorithm for regularized estimation called Dual Augmented Lagrangian (DAL). Our analysis is based on a new interpretation of DAL as a proximal minimization algorithm. We theoretically show under some conditions that DAL converges super-linearly in a non-asymptotic and global sense. Due to a special modelling of sparse estimation problems in the context of machine learning, the assumptions we make are milder and more natural than those made in conventional analysis of augmented Lagrangian algorithms. In addition, the new interpretation enables us to generalize DAL to wide varieties of sparse estimation problems. We experimentally confirm our analysis in a large scale $\ell_1$-regularized logistic regression problem and extensively compare the efficiency of DAL algorithm to previously proposed algorithms on both synthetic and benchmark datasets.
Sugiyama Masashi
Suzuki Taiji
Tomioka Ryota
No associations
LandOfFree
Super-Linear Convergence of Dual Augmented-Lagrangian Algorithm for Sparsity Regularized Estimation does not yet have a rating. At this time, there are no reviews or comments for this scientific paper.
If you have personal experience with Super-Linear Convergence of Dual Augmented-Lagrangian Algorithm for Sparsity Regularized Estimation, we encourage you to share that experience with our LandOfFree.com community. Your opinion is very important and Super-Linear Convergence of Dual Augmented-Lagrangian Algorithm for Sparsity Regularized Estimation will most certainly appreciate the feedback.
Profile ID: LFWR-SCP-O-432541