Locally Regularized Readouts for Echo-state Networks

Details Locally Regularized Readouts for Echo-state Networks Locally Regularized Readouts for Echo-state Networks

2011-10-19

arxiv.org/abs/1110.4304v2

Statistics

Machine Learning

10 pages, pre-print

Scientific paper

Echo state network (ESN) is viewed as a temporal non-orthogonal expansion with pseudo-random parameters. Such expansions naturally give rise to regressors of various relevance to a teacher output. This study shows using the orthogonal forward regression (OFR) algorithm that often only certain amount of the generated echo-regressors effectively explain variance of the teacher output. Remaining echo-regressors increase the explained variance of the teacher signal only marginally and often causes numerical ill-conditioning. It is therefore desirable to identify and drop echo-regressors which do not improve the model quality and prevent ill-conditioning. In this study we present locally regularized linear readout which together with OFR analysis is capable of identifying significant echo-regressors and penalizing (or removing) the undesired non-significant echo-regressors. Locally regularized linear readout presented here is studied not only as a technique that improves robustness and accuracy of an ESN but also as a technique that enables better evaluation of ESN pseudo-random parameters and ESN dimensionality. Moreover, linear readouts have limitations in terms of their flexibility which might be sometimes insufficient for a task at hand. For such cases, we present locally regularized radial basis function (RBF) readout. It is a flexible and parsimonious readout with excellent generalization abilities and is viable alternative to a relevance vector machine (RVM) or a feed-forward neural network based readouts.

Affiliated with

Also associated with

No associations

Rating

Locally Regularized Readouts for Echo-state Networks does not yet have a rating. At this time, there are no reviews or comments for this scientific paper.

If you have personal experience with Locally Regularized Readouts for Echo-state Networks, we encourage you to share that experience with our LandOfFree.com community. Your opinion is very important and Locally Regularized Readouts for Echo-state Networks will most certainly appreciate the feedback.

Rate now

Comments { 0 }

Profile ID: LFWR-SCP-O-597463