Replica theory for learning curves for Gaussian processes on random graphs

Details Replica theory for learning curves for Gaussian processes on random graphs Replica theory for learning curves for Gaussian processes on random graphs

2012-02-27

arxiv.org/abs/1202.5918v1

Statistics

Machine Learning

Scientific paper

Statistical physics approaches can be used to derive accurate predictions for the performance of inference methods learning from potentially noisy data, as quantified by the learning curve giving average error versus number of training examples. We analyse a challenging problem in the area of non-parametric inference where an effectively infinite number of parameters has to be learned, specifically Gaussian process regression. When the inputs are vertices on a random graph and the outputs the noisy function values being fitted, we show that replica techniques can be used to obtain exact predictions in the limit of large graphs. The covariance of the Gaussian process prior is defined by a random walk kernel, the discrete analogue of squared exponential kernels on continuous spaces. Conventionally this kernel is normalised only globally, so that the prior variance can differ between vertices; as a more principled alternative we consider local normalisation, where the prior variance is uniform. We compare our predictions with numerically simulated learning curves for two paradigmatic graph ensembles: Erdos-Renyi graphs with a Poisson degree distribution, and an ensemble with a power law degree distribution. We find excellent agreement between our predictions and numerical simulations. We also compare our predictions to existing learning curve approximations. These show significant deviations; we analyse briefly where these arise.

Affiliated with

Also associated with

No associations

Rating

Replica theory for learning curves for Gaussian processes on random graphs does not yet have a rating. At this time, there are no reviews or comments for this scientific paper.

If you have personal experience with Replica theory for learning curves for Gaussian processes on random graphs, we encourage you to share that experience with our LandOfFree.com community. Your opinion is very important and Replica theory for learning curves for Gaussian processes on random graphs will most certainly appreciate the feedback.

Rate now

Comments { 0 }

Profile ID: LFWR-SCP-O-263492