Sequential Probability Ration Tests: Conservative and Robust

Research output: Working paperDiscussion paperOther research output

637 Downloads (Pure)


In practice, most computers generate simulation outputs sequentially, so it is
attractive to analyze these outputs through sequential statistical methods such as sequential probability ratio tests (SPRTs). We investigate several SPRTs for choosing between two hypothesized values for the mean output (response). One SPRT is published in Wald (1945), and allows general distribution types. For a normal (Gaussian) distribution this SPRT assumes a known variance, but in our modified SPRT we estimate the variance. Another SPRT is published in Hall (1962), and assumes a normal distribution with an unknown variance estimated from a pilot sample. We also investigate a modification, replacing this pilot-sample estimator by a fully sequential estimator. We present a sequence of Monte
Carlo experiments for quantifying the performance of these SPRTs. In experiment #1 the simulation outputs are normal. This experiment suggests that Wald (1945)’s SPRT with estimated variance gives significantly high error rates. Hall (1962)’s original and modified SPRTs are conservative; i.e., the actual error rates are much smaller than the prespecified (nominal) rates. The most efficient SPRT is our modified Hall (1962) SPRT. In experiment #2 we examine the robustness of the various SPRTs in case of nonnormal output. If we know that the output has a specific nonnormal distribution such as the exponential distribution, then we may also apply Wald (1945)’s original SPRT. Throughout our investigation we pay special attention to the design and analysis of these experiments.
Original languageEnglish
Place of PublicationTilburg
PublisherCentER, Center for Economic Research
Number of pages39
Publication statusPublished - 10 Jan 2017

Publication series

NameCentER Discussion Paper


  • sequential test
  • Wald
  • Hall
  • robustness
  • lognormal
  • gamma distribution
  • Monte Carlo


Dive into the research topics of 'Sequential Probability Ration Tests: Conservative and Robust'. Together they form a unique fingerprint.

Cite this