A User's Guide to the Brave New World of Designing Simulation Experiments

J.P.C. Kleijnen, S.M. Sanchez, T.W. Lucas, T.M. Cioppa

Research output: Working paperDiscussion paperOther research output

480 Downloads (Pure)

Abstract

Many simulation practitioners can get more from their analyses by using the statistical theory on design of experiments (DOE) developed specifically for exploring computer models.In this paper, we discuss a toolkit of designs for simulationists with limited DOE expertise who want to select a design and an appropriate analysis for their computational experiments.Furthermore, we provide a research agenda listing problems in the design of simulation experiments -as opposed to real world experiments- that require more investigation.We consider three types of practical problems: (1) developing a basic understanding of a particular simulation model or system; (2) finding robust decisions or policies; and (3) comparing the merits of various decisions or policies.Our discussion emphasizes aspects that are typical for simulation, such as sequential data collection.Because the same problem type may be addressed through different design types, we discuss quality attributes of designs.Furthermore, the selection of the design type depends on the metamodel (response surface) that the analysts tentatively assume; for example, more complicated metamodels require more simulation runs.For the validation of the metamodel estimated from a specific design, we present several procedures.
Original languageEnglish
Place of PublicationTilburg
PublisherOperations research
Number of pages40
Volume2003-1
Publication statusPublished - 2003

Publication series

NameCentER Discussion Paper
Volume2003-1

Fingerprint

Experiments
Design of experiments

Keywords

  • simulation
  • bifurcation
  • experimental desgin
  • policy
  • sampling

Cite this

Kleijnen, J. P. C., Sanchez, S. M., Lucas, T. W., & Cioppa, T. M. (2003). A User's Guide to the Brave New World of Designing Simulation Experiments. (CentER Discussion Paper; Vol. 2003-1). Tilburg: Operations research.
Kleijnen, J.P.C. ; Sanchez, S.M. ; Lucas, T.W. ; Cioppa, T.M. / A User's Guide to the Brave New World of Designing Simulation Experiments. Tilburg : Operations research, 2003. (CentER Discussion Paper).
@techreport{a6910d11f9bc4246b1a72acf330acba8,
title = "A User's Guide to the Brave New World of Designing Simulation Experiments",
abstract = "Many simulation practitioners can get more from their analyses by using the statistical theory on design of experiments (DOE) developed specifically for exploring computer models.In this paper, we discuss a toolkit of designs for simulationists with limited DOE expertise who want to select a design and an appropriate analysis for their computational experiments.Furthermore, we provide a research agenda listing problems in the design of simulation experiments -as opposed to real world experiments- that require more investigation.We consider three types of practical problems: (1) developing a basic understanding of a particular simulation model or system; (2) finding robust decisions or policies; and (3) comparing the merits of various decisions or policies.Our discussion emphasizes aspects that are typical for simulation, such as sequential data collection.Because the same problem type may be addressed through different design types, we discuss quality attributes of designs.Furthermore, the selection of the design type depends on the metamodel (response surface) that the analysts tentatively assume; for example, more complicated metamodels require more simulation runs.For the validation of the metamodel estimated from a specific design, we present several procedures.",
keywords = "simulation, bifurcation, experimental desgin, policy, sampling",
author = "J.P.C. Kleijnen and S.M. Sanchez and T.W. Lucas and T.M. Cioppa",
note = "Pagination: 40",
year = "2003",
language = "English",
volume = "2003-1",
series = "CentER Discussion Paper",
publisher = "Operations research",
type = "WorkingPaper",
institution = "Operations research",

}

Kleijnen, JPC, Sanchez, SM, Lucas, TW & Cioppa, TM 2003 'A User's Guide to the Brave New World of Designing Simulation Experiments' CentER Discussion Paper, vol. 2003-1, Operations research, Tilburg.

A User's Guide to the Brave New World of Designing Simulation Experiments. / Kleijnen, J.P.C.; Sanchez, S.M.; Lucas, T.W.; Cioppa, T.M.

Tilburg : Operations research, 2003. (CentER Discussion Paper; Vol. 2003-1).

Research output: Working paperDiscussion paperOther research output

TY - UNPB

T1 - A User's Guide to the Brave New World of Designing Simulation Experiments

AU - Kleijnen, J.P.C.

AU - Sanchez, S.M.

AU - Lucas, T.W.

AU - Cioppa, T.M.

N1 - Pagination: 40

PY - 2003

Y1 - 2003

N2 - Many simulation practitioners can get more from their analyses by using the statistical theory on design of experiments (DOE) developed specifically for exploring computer models.In this paper, we discuss a toolkit of designs for simulationists with limited DOE expertise who want to select a design and an appropriate analysis for their computational experiments.Furthermore, we provide a research agenda listing problems in the design of simulation experiments -as opposed to real world experiments- that require more investigation.We consider three types of practical problems: (1) developing a basic understanding of a particular simulation model or system; (2) finding robust decisions or policies; and (3) comparing the merits of various decisions or policies.Our discussion emphasizes aspects that are typical for simulation, such as sequential data collection.Because the same problem type may be addressed through different design types, we discuss quality attributes of designs.Furthermore, the selection of the design type depends on the metamodel (response surface) that the analysts tentatively assume; for example, more complicated metamodels require more simulation runs.For the validation of the metamodel estimated from a specific design, we present several procedures.

AB - Many simulation practitioners can get more from their analyses by using the statistical theory on design of experiments (DOE) developed specifically for exploring computer models.In this paper, we discuss a toolkit of designs for simulationists with limited DOE expertise who want to select a design and an appropriate analysis for their computational experiments.Furthermore, we provide a research agenda listing problems in the design of simulation experiments -as opposed to real world experiments- that require more investigation.We consider three types of practical problems: (1) developing a basic understanding of a particular simulation model or system; (2) finding robust decisions or policies; and (3) comparing the merits of various decisions or policies.Our discussion emphasizes aspects that are typical for simulation, such as sequential data collection.Because the same problem type may be addressed through different design types, we discuss quality attributes of designs.Furthermore, the selection of the design type depends on the metamodel (response surface) that the analysts tentatively assume; for example, more complicated metamodels require more simulation runs.For the validation of the metamodel estimated from a specific design, we present several procedures.

KW - simulation

KW - bifurcation

KW - experimental desgin

KW - policy

KW - sampling

M3 - Discussion paper

VL - 2003-1

T3 - CentER Discussion Paper

BT - A User's Guide to the Brave New World of Designing Simulation Experiments

PB - Operations research

CY - Tilburg

ER -

Kleijnen JPC, Sanchez SM, Lucas TW, Cioppa TM. A User's Guide to the Brave New World of Designing Simulation Experiments. Tilburg: Operations research. 2003. (CentER Discussion Paper).