Error probabilities in default Bayesian hypothesis testing

Xin Gu*, Herbert Hoijtink, J, Mulder

*Corresponding author for this work

Research output: Contribution to journalArticleScientificpeer-review

Abstract

This paper investigates the classical type I and type II error probabilities of default Bayes factors for a Bayesian t test. Default Bayes factors quantify the relative evidence between the null hypothesis and the unrestricted alternative hypothesis without needing to specify prior distributions for the unknown parameters based on one's prior beliefs. It is shown that in most typical situations in psychological research (when either observing no, small, medium or large effects) default Bayes factors are asymmetric in information, i.e., they result in unequal error probabilities. The tendency to either prefer the null hypothesis or the alternative hypothesis varies for different default Bayes factors. Although this asymmetry in information is a natural property of a Bayes factor, severe cases of asymmetry may be undesirable in a default setting because the underlying default priors are not a translation of one's prior beliefs. A calibration scheme is presented to make a default Bayes factor symmetric in information under certain conditions. (C) 2015 Elsevier Inc. All rights reserved.

Original languageEnglish
Pages (from-to)130-143
JournalJournal of Mathematical Psychology
Volume72
DOIs
Publication statusPublished - 2016

Keywords

  • Default Bayes factor
  • Effect size
  • Error probabilities
  • One sample t test
  • FREQUENTIST
  • SELECTION

Cite this

@article{8b9810a272f24703a1cc1fb1413a730a,
title = "Error probabilities in default Bayesian hypothesis testing",
abstract = "This paper investigates the classical type I and type II error probabilities of default Bayes factors for a Bayesian t test. Default Bayes factors quantify the relative evidence between the null hypothesis and the unrestricted alternative hypothesis without needing to specify prior distributions for the unknown parameters based on one's prior beliefs. It is shown that in most typical situations in psychological research (when either observing no, small, medium or large effects) default Bayes factors are asymmetric in information, i.e., they result in unequal error probabilities. The tendency to either prefer the null hypothesis or the alternative hypothesis varies for different default Bayes factors. Although this asymmetry in information is a natural property of a Bayes factor, severe cases of asymmetry may be undesirable in a default setting because the underlying default priors are not a translation of one's prior beliefs. A calibration scheme is presented to make a default Bayes factor symmetric in information under certain conditions. (C) 2015 Elsevier Inc. All rights reserved.",
keywords = "Default Bayes factor, Effect size, Error probabilities, One sample t test, FREQUENTIST, SELECTION",
author = "Xin Gu and Herbert Hoijtink and J, Mulder",
year = "2016",
doi = "10.1016/j.jmp.2015.09.001",
language = "English",
volume = "72",
pages = "130--143",
journal = "Journal of Mathematical Psychology",
issn = "0022-2496",
publisher = "ACADEMIC PRESS INC ELSEVIER SCIENCE",

}

Error probabilities in default Bayesian hypothesis testing. / Gu, Xin; Hoijtink, Herbert; Mulder, J,.

In: Journal of Mathematical Psychology, Vol. 72, 2016, p. 130-143.

Research output: Contribution to journalArticleScientificpeer-review

TY - JOUR

T1 - Error probabilities in default Bayesian hypothesis testing

AU - Gu, Xin

AU - Hoijtink, Herbert

AU - Mulder, J,

PY - 2016

Y1 - 2016

N2 - This paper investigates the classical type I and type II error probabilities of default Bayes factors for a Bayesian t test. Default Bayes factors quantify the relative evidence between the null hypothesis and the unrestricted alternative hypothesis without needing to specify prior distributions for the unknown parameters based on one's prior beliefs. It is shown that in most typical situations in psychological research (when either observing no, small, medium or large effects) default Bayes factors are asymmetric in information, i.e., they result in unequal error probabilities. The tendency to either prefer the null hypothesis or the alternative hypothesis varies for different default Bayes factors. Although this asymmetry in information is a natural property of a Bayes factor, severe cases of asymmetry may be undesirable in a default setting because the underlying default priors are not a translation of one's prior beliefs. A calibration scheme is presented to make a default Bayes factor symmetric in information under certain conditions. (C) 2015 Elsevier Inc. All rights reserved.

AB - This paper investigates the classical type I and type II error probabilities of default Bayes factors for a Bayesian t test. Default Bayes factors quantify the relative evidence between the null hypothesis and the unrestricted alternative hypothesis without needing to specify prior distributions for the unknown parameters based on one's prior beliefs. It is shown that in most typical situations in psychological research (when either observing no, small, medium or large effects) default Bayes factors are asymmetric in information, i.e., they result in unequal error probabilities. The tendency to either prefer the null hypothesis or the alternative hypothesis varies for different default Bayes factors. Although this asymmetry in information is a natural property of a Bayes factor, severe cases of asymmetry may be undesirable in a default setting because the underlying default priors are not a translation of one's prior beliefs. A calibration scheme is presented to make a default Bayes factor symmetric in information under certain conditions. (C) 2015 Elsevier Inc. All rights reserved.

KW - Default Bayes factor

KW - Effect size

KW - Error probabilities

KW - One sample t test

KW - FREQUENTIST

KW - SELECTION

U2 - 10.1016/j.jmp.2015.09.001

DO - 10.1016/j.jmp.2015.09.001

M3 - Article

VL - 72

SP - 130

EP - 143

JO - Journal of Mathematical Psychology

JF - Journal of Mathematical Psychology

SN - 0022-2496

ER -