A Bregman Extension of quasi-Newton updates II: Convergence and Robustness Properties

Details A Bregman Extension of quasi-Newton updates II: Convergence and Robustness Properties A Bregman Extension of quasi-Newton updates II: Convergence and Robustness Properties

2010-10-14

arxiv.org/abs/1010.2846v1

Statistics

Computation

39 pages, 1 figures, 3 tables

Scientific paper

We propose an extension of quasi-Newton methods, and investigate the convergence and the robustness properties of the proposed update formulae for the approximate Hessian matrix. Fletcher has studied a variational problem which derives the approximate Hessian update formula of the quasi-Newton methods. We point out that the variational problem is identical to optimization of the Kullback-Leibler divergence, which is a discrepancy measure between two probability distributions. Then, we introduce the Bregman divergence as an extension of the Kullback-Leibler divergence, and derive extended quasi-Newton update formulae based on the variational problem with the Bregman divergence. The proposed update formulae belong to a class of self-scaling quasi-Newton methods. We study the convergence property of the proposed quasi-Newton method, and moreover, we apply the tools in the robust statistics to analyze the robustness property of the Hessian update formulae against the numerical rounding errors included in the line search for the step length. As the result, we found that the influence of the inexact line search is bounded only for the standard BFGS formula for the Hessian approximation. Numerical studies are conducted to verify the usefulness of the tools borrowed from robust statistics.

Affiliated with

Also associated with

No associations

Rating

A Bregman Extension of quasi-Newton updates II: Convergence and Robustness Properties does not yet have a rating. At this time, there are no reviews or comments for this scientific paper.

If you have personal experience with A Bregman Extension of quasi-Newton updates II: Convergence and Robustness Properties, we encourage you to share that experience with our LandOfFree.com community. Your opinion is very important and A Bregman Extension of quasi-Newton updates II: Convergence and Robustness Properties will most certainly appreciate the feedback.

Rate now

Comments { 0 }

Profile ID: LFWR-SCP-O-285164