Sequential Probability Ration Tests: Conservative and Robust

Research output: Working paperDiscussion paperOther research output

459 Downloads (Pure)

Abstract

In practice, most computers generate simulation outputs sequentially, so it is
attractive to analyze these outputs through sequential statistical methods such as sequential probability ratio tests (SPRTs). We investigate several SPRTs for choosing between two hypothesized values for the mean output (response). One SPRT is published in Wald (1945), and allows general distribution types. For a normal (Gaussian) distribution this SPRT assumes a known variance, but in our modified SPRT we estimate the variance. Another SPRT is published in Hall (1962), and assumes a normal distribution with an unknown variance estimated from a pilot sample. We also investigate a modification, replacing this pilot-sample estimator by a fully sequential estimator. We present a sequence of Monte
Carlo experiments for quantifying the performance of these SPRTs. In experiment #1 the simulation outputs are normal. This experiment suggests that Wald (1945)’s SPRT with estimated variance gives significantly high error rates. Hall (1962)’s original and modified SPRTs are conservative; i.e., the actual error rates are much smaller than the prespecified (nominal) rates. The most efficient SPRT is our modified Hall (1962) SPRT. In experiment #2 we examine the robustness of the various SPRTs in case of nonnormal output. If we know that the output has a specific nonnormal distribution such as the exponential distribution, then we may also apply Wald (1945)’s original SPRT. Throughout our investigation we pay special attention to the design and analysis of these experiments.
Original languageEnglish
Place of PublicationTilburg
PublisherCentER, Center for Economic Research
Number of pages39
Volume2017-001
Publication statusPublished - 10 Jan 2017

Publication series

NameCentER Discussion Paper
Volume2017-001

    Fingerprint

Keywords

  • sequential test
  • Wald
  • Hall
  • robustness
  • lognormal
  • gamma distribution
  • Monte Carlo

Cite this

Kleijnen, J. P. C., & Shi, W. (2017). Sequential Probability Ration Tests: Conservative and Robust. (CentER Discussion Paper; Vol. 2017-001). Tilburg: CentER, Center for Economic Research.