Validation of Simulation, With and Without Real Data

Research output: Working paperDiscussion paperOther research output

284 Downloads (Pure)

Abstract

This paper gives a survey on how to validate simulation models through the application of mathematical statistics. The type of statistical test actually applied, depends on the availability of data on the real system: (i) no data, (ii) only output data, and (iii) both input and output data. In case (i), the system analysts can still experiment with the simulation model to obtain simulated data. Those experiments should be guided by the statistical theory on design of experiments (DOE); an inferior - but popular - approach is to change only one factor at a time. In case (ii), real and simulated output data may be compared through the well-known Student t statistic. In case (iii), trace-driven simulation becomes possible. Then validation, however, should not proceed as follows: make a scatter plot with real and simulated outputs, fit a line, and test whether that line has unit slope and passes through the origin. Instead, better tests are presented. Several case studies are summarized, to illustrate the three types of situations.
Original languageEnglish
Place of PublicationTilburg
PublisherOperations research
Number of pages21
Volume1998-22
Publication statusPublished - 1998

Publication series

NameCentER Discussion Paper
Volume1998-22

Fingerprint

Statistical tests
Design of experiments
Statistical methods
Experiments
Statistics
Availability
Students

Keywords

  • verification
  • credibility
  • assessment
  • sensitivity
  • robustness
  • regression

Cite this

Kleijnen, J. P. C. (1998). Validation of Simulation, With and Without Real Data. (CentER Discussion Paper; Vol. 1998-22). Tilburg: Operations research.
Kleijnen, J.P.C. / Validation of Simulation, With and Without Real Data. Tilburg : Operations research, 1998. (CentER Discussion Paper).
@techreport{40e44c6d3f654f8f8e30cf9761b60eb8,
title = "Validation of Simulation, With and Without Real Data",
abstract = "This paper gives a survey on how to validate simulation models through the application of mathematical statistics. The type of statistical test actually applied, depends on the availability of data on the real system: (i) no data, (ii) only output data, and (iii) both input and output data. In case (i), the system analysts can still experiment with the simulation model to obtain simulated data. Those experiments should be guided by the statistical theory on design of experiments (DOE); an inferior - but popular - approach is to change only one factor at a time. In case (ii), real and simulated output data may be compared through the well-known Student t statistic. In case (iii), trace-driven simulation becomes possible. Then validation, however, should not proceed as follows: make a scatter plot with real and simulated outputs, fit a line, and test whether that line has unit slope and passes through the origin. Instead, better tests are presented. Several case studies are summarized, to illustrate the three types of situations.",
keywords = "verification, credibility, assessment, sensitivity, robustness, regression",
author = "J.P.C. Kleijnen",
note = "Pagination: 21",
year = "1998",
language = "English",
volume = "1998-22",
series = "CentER Discussion Paper",
publisher = "Operations research",
type = "WorkingPaper",
institution = "Operations research",

}

Kleijnen, JPC 1998 'Validation of Simulation, With and Without Real Data' CentER Discussion Paper, vol. 1998-22, Operations research, Tilburg.

Validation of Simulation, With and Without Real Data. / Kleijnen, J.P.C.

Tilburg : Operations research, 1998. (CentER Discussion Paper; Vol. 1998-22).

Research output: Working paperDiscussion paperOther research output

TY - UNPB

T1 - Validation of Simulation, With and Without Real Data

AU - Kleijnen, J.P.C.

N1 - Pagination: 21

PY - 1998

Y1 - 1998

N2 - This paper gives a survey on how to validate simulation models through the application of mathematical statistics. The type of statistical test actually applied, depends on the availability of data on the real system: (i) no data, (ii) only output data, and (iii) both input and output data. In case (i), the system analysts can still experiment with the simulation model to obtain simulated data. Those experiments should be guided by the statistical theory on design of experiments (DOE); an inferior - but popular - approach is to change only one factor at a time. In case (ii), real and simulated output data may be compared through the well-known Student t statistic. In case (iii), trace-driven simulation becomes possible. Then validation, however, should not proceed as follows: make a scatter plot with real and simulated outputs, fit a line, and test whether that line has unit slope and passes through the origin. Instead, better tests are presented. Several case studies are summarized, to illustrate the three types of situations.

AB - This paper gives a survey on how to validate simulation models through the application of mathematical statistics. The type of statistical test actually applied, depends on the availability of data on the real system: (i) no data, (ii) only output data, and (iii) both input and output data. In case (i), the system analysts can still experiment with the simulation model to obtain simulated data. Those experiments should be guided by the statistical theory on design of experiments (DOE); an inferior - but popular - approach is to change only one factor at a time. In case (ii), real and simulated output data may be compared through the well-known Student t statistic. In case (iii), trace-driven simulation becomes possible. Then validation, however, should not proceed as follows: make a scatter plot with real and simulated outputs, fit a line, and test whether that line has unit slope and passes through the origin. Instead, better tests are presented. Several case studies are summarized, to illustrate the three types of situations.

KW - verification

KW - credibility

KW - assessment

KW - sensitivity

KW - robustness

KW - regression

M3 - Discussion paper

VL - 1998-22

T3 - CentER Discussion Paper

BT - Validation of Simulation, With and Without Real Data

PB - Operations research

CY - Tilburg

ER -

Kleijnen JPC. Validation of Simulation, With and Without Real Data. Tilburg: Operations research. 1998. (CentER Discussion Paper).