Equations of States in Singular Statistical Estimation

Details Equations of States in Singular Statistical Estimation Equations of States in Singular Statistical Estimation

2007-12-05

arxiv.org/abs/0712.0653v2

Computer Science

Learning

Scientific paper

Learning machines which have hierarchical structures or hidden variables are singular statistical models because they are nonidentifiable and their Fisher information matrices are singular. In singular statistical models, neither the Bayes a posteriori distribution converges to the normal distribution nor the maximum likelihood estimator satisfies asymptotic normality. This is the main reason why it has been difficult to predict their generalization performances from trained states. In this paper, we study four errors, (1) Bayes generalization error, (2) Bayes training error, (3) Gibbs generalization error, and (4) Gibbs training error, and prove that there are mathematical relations among these errors. The formulas proved in this paper are equations of states in statistical estimation because they hold for any true distribution, any parametric model, and any a priori distribution. Also we show that Bayes and Gibbs generalization errors are estimated by Bayes and Gibbs training errors, and propose widely applicable information criteria which can be applied to both regular and singular statistical models.

Affiliated with

Also associated with

No associations

Rating

Equations of States in Singular Statistical Estimation does not yet have a rating. At this time, there are no reviews or comments for this scientific paper.

If you have personal experience with Equations of States in Singular Statistical Estimation, we encourage you to share that experience with our LandOfFree.com community. Your opinion is very important and Equations of States in Singular Statistical Estimation will most certainly appreciate the feedback.

Rate now

Comments { 0 }

Profile ID: LFWR-SCP-O-310250