A frequentist two-sample test based on Bayesian model selection

Details A frequentist two-sample test based on Bayesian model selection A frequentist two-sample test based on Bayesian model selection

2011-04-14

arxiv.org/abs/1104.2826v1

Statistics

Methodology

Paper and supplementary material

Scientific paper

Despite their importance in supporting experimental conclusions, standard statistical tests are often inadequate for research areas, like the life sciences, where the typical sample size is small and the test assumptions difficult to verify. In such conditions, standard tests tend to be overly conservative, and fail thus to detect significant effects in the data. Here we define a novel statistical test for the two-sample problem. Several characteristics make it an attractive alternative to classical two-sample tests: 1) It is based on Bayesian model selection, and thus takes into account uncertainty about the model's parameters, mitigating the problem of small samples size; 2) The null hypothesis is compared with several alternative hypotheses, making the test suitable in different experimental scenarios; 3) The test is constructed as a frequentist test, and defines significance with the conventional bound on Type I errors. We analyze the power of the test and find that it is higher than the power of other standard options, like the t-test (up to 25% higher) for a wide range of sample and effect sizes, and is at most 1% lower when the assumptions of the t-test are perfectly matched. We discuss and evaluate two variants of the test, that define different prior distributions over the parameters of the hypotheses.

Affiliated with

Also associated with

No associations

Rating

A frequentist two-sample test based on Bayesian model selection does not yet have a rating. At this time, there are no reviews or comments for this scientific paper.

If you have personal experience with A frequentist two-sample test based on Bayesian model selection, we encourage you to share that experience with our LandOfFree.com community. Your opinion is very important and A frequentist two-sample test based on Bayesian model selection will most certainly appreciate the feedback.

Rate now

Comments { 0 }

Profile ID: LFWR-SCP-O-167112