Why checking model assumptions using null hypothesis significance tests does not suffice

A plea for plausibility

Research output: Contribution to journalArticleScientificpeer-review

7 Downloads (Pure)

Abstract

This article explores whether the null hypothesis significance testing (NHST) framework provides a sufficient basis for the evaluation of statistical model assumptions. It is argued that while NHST-based tests can provide some degree of confirmation for the model assumption that is evaluated-formulated as the null hypothesis-these tests do not inform us of the degree of support that the data provide for the null hypothesis and to what extent the null hypothesis should be considered to be plausible after having taken the data into account. Addressing the prior plausibility of the model assumption is unavoidable if the goal is to determine how plausible it is that the model assumption holds. Without assessing the prior plausibility of the model assumptions, it remains fully uncertain whether the model of interest gives an adequate description of the data and thus whether it can be considered valid for the application at hand. Although addressing the prior plausibility is difficult, ignoring the prior plausibility is not an option if we want to claim that the inferences of our statistical model can be relied upon.

Original languageEnglish
Pages (from-to)548-559
JournalPsychonomic Bulletin & Review
Volume25
Issue number2
DOIs
Publication statusPublished - 2018

Keywords

  • Statistical inference
  • Bayesian statistics
  • Belief updating
  • Statistics
  • BAYESIAN STATISTICAL-INFERENCE
  • P-VALUES
  • PSYCHOLOGY

Cite this

@article{9354aaf67720476b854ff316cd884b56,
title = "Why checking model assumptions using null hypothesis significance tests does not suffice: A plea for plausibility",
abstract = "This article explores whether the null hypothesis significance testing (NHST) framework provides a sufficient basis for the evaluation of statistical model assumptions. It is argued that while NHST-based tests can provide some degree of confirmation for the model assumption that is evaluated-formulated as the null hypothesis-these tests do not inform us of the degree of support that the data provide for the null hypothesis and to what extent the null hypothesis should be considered to be plausible after having taken the data into account. Addressing the prior plausibility of the model assumption is unavoidable if the goal is to determine how plausible it is that the model assumption holds. Without assessing the prior plausibility of the model assumptions, it remains fully uncertain whether the model of interest gives an adequate description of the data and thus whether it can be considered valid for the application at hand. Although addressing the prior plausibility is difficult, ignoring the prior plausibility is not an option if we want to claim that the inferences of our statistical model can be relied upon.",
keywords = "Statistical inference, Bayesian statistics, Belief updating, Statistics, BAYESIAN STATISTICAL-INFERENCE, P-VALUES, PSYCHOLOGY",
author = "J. Tijmstra",
year = "2018",
doi = "10.3758/s13423-018-1447-4",
language = "English",
volume = "25",
pages = "548--559",
journal = "Psychonomic Bulletin & Review",
issn = "1069-9384",
publisher = "Springer",
number = "2",

}

Why checking model assumptions using null hypothesis significance tests does not suffice : A plea for plausibility. / Tijmstra, J.

In: Psychonomic Bulletin & Review, Vol. 25, No. 2, 2018, p. 548-559.

Research output: Contribution to journalArticleScientificpeer-review

TY - JOUR

T1 - Why checking model assumptions using null hypothesis significance tests does not suffice

T2 - A plea for plausibility

AU - Tijmstra, J.

PY - 2018

Y1 - 2018

N2 - This article explores whether the null hypothesis significance testing (NHST) framework provides a sufficient basis for the evaluation of statistical model assumptions. It is argued that while NHST-based tests can provide some degree of confirmation for the model assumption that is evaluated-formulated as the null hypothesis-these tests do not inform us of the degree of support that the data provide for the null hypothesis and to what extent the null hypothesis should be considered to be plausible after having taken the data into account. Addressing the prior plausibility of the model assumption is unavoidable if the goal is to determine how plausible it is that the model assumption holds. Without assessing the prior plausibility of the model assumptions, it remains fully uncertain whether the model of interest gives an adequate description of the data and thus whether it can be considered valid for the application at hand. Although addressing the prior plausibility is difficult, ignoring the prior plausibility is not an option if we want to claim that the inferences of our statistical model can be relied upon.

AB - This article explores whether the null hypothesis significance testing (NHST) framework provides a sufficient basis for the evaluation of statistical model assumptions. It is argued that while NHST-based tests can provide some degree of confirmation for the model assumption that is evaluated-formulated as the null hypothesis-these tests do not inform us of the degree of support that the data provide for the null hypothesis and to what extent the null hypothesis should be considered to be plausible after having taken the data into account. Addressing the prior plausibility of the model assumption is unavoidable if the goal is to determine how plausible it is that the model assumption holds. Without assessing the prior plausibility of the model assumptions, it remains fully uncertain whether the model of interest gives an adequate description of the data and thus whether it can be considered valid for the application at hand. Although addressing the prior plausibility is difficult, ignoring the prior plausibility is not an option if we want to claim that the inferences of our statistical model can be relied upon.

KW - Statistical inference

KW - Bayesian statistics

KW - Belief updating

KW - Statistics

KW - BAYESIAN STATISTICAL-INFERENCE

KW - P-VALUES

KW - PSYCHOLOGY

U2 - 10.3758/s13423-018-1447-4

DO - 10.3758/s13423-018-1447-4

M3 - Article

VL - 25

SP - 548

EP - 559

JO - Psychonomic Bulletin & Review

JF - Psychonomic Bulletin & Review

SN - 1069-9384

IS - 2

ER -