Mathematics – Optimization and Control
Scientific paper
2011-06-19
Mathematics
Optimization and Control
Scientific paper
We present the information-geometric optimization (IGO) method, which turns any smooth parametric family of probability distributions on an arbitrary search space $X$ into a continuous-time black-box optimization method on $X$. Invariance as a design principle keeps the number of arbitrary choices to a minimum. IGO conducts a natural gradient ascent using an adaptive, time-dependent transformation of the objective function. The cross-entropy method is recovered in a particular case with a large time step. From specific families of distributions on discrete or continuous spaces, IGO naturally recovers versions of known algorithms: CMA-ES for Gaussian distributions, and PBIL for Bernoulli distributions. IGO is invariant under reparametrization of the search space $X$, under a change of parameters of the probability distribution, and under increasing transformation of the function to be optimized. Theoretical considerations suggest that IGO achives minimal diversity loss through optimization. First experiments using restricted Boltzmann machines show that IGO may be able to spontaneously perform multimodal optimization.
Arnold Ludovic
Auger Anne
Hansen Nikolaus
Ollivier Yann
No associations
LandOfFree
Information-Geometric Optimization Algorithms: A Unifying Picture via Invariance Principles does not yet have a rating. At this time, there are no reviews or comments for this scientific paper.
If you have personal experience with Information-Geometric Optimization Algorithms: A Unifying Picture via Invariance Principles, we encourage you to share that experience with our LandOfFree.com community. Your opinion is very important and Information-Geometric Optimization Algorithms: A Unifying Picture via Invariance Principles will most certainly appreciate the feedback.
Profile ID: LFWR-SCP-O-723065