A Methodology for Fitting and Validating Metamodels in Simulation

J.P.C. Kleijnen, R. Sargent

Research output: Working paperDiscussion paperOther research output

345 Downloads (Pure)

Abstract

This expository paper discusses the relationships among metamodels, simulation models, and problem entities. A metamodel or response surface is an approximation of the input/output function implied by the underlying simulation model. There are several types of metamodel: linear regression, splines, neural networks, etc. This paper distinguishes between fitting and validating a metamodel. Metamodels may have different goals: (i) understanding, (ii) prediction, (iii) optimization, and (iv) verification and validation. For this metamodeling, a process with thirteen steps is proposed. Classic design of experiments (DOE) is summarized, including standard measures of fit such as the R-square coefficient and cross-validation measures. This DOE is extended to sequential or stagewise DOE. Several validation criteria, measures, and estimators are discussed. Metamodels in general are covered, along with a procedure for developing linear regression (including polynomial) metamodels.
Original languageEnglish
Place of PublicationTilburg
PublisherOperations research
Number of pages36
Volume1997-116
Publication statusPublished - 1997

Publication series

NameCentER Discussion Paper
Volume1997-116

Fingerprint

Design of experiments
Linear regression
Splines
Polynomials
Neural networks

Keywords

  • Simulation
  • approximation
  • response surface
  • modelling
  • regression

Cite this

Kleijnen, J. P. C., & Sargent, R. (1997). A Methodology for Fitting and Validating Metamodels in Simulation. (CentER Discussion Paper; Vol. 1997-116). Tilburg: Operations research.
Kleijnen, J.P.C. ; Sargent, R. / A Methodology for Fitting and Validating Metamodels in Simulation. Tilburg : Operations research, 1997. (CentER Discussion Paper).
@techreport{b8c2217cf02c4786b70477587495aec8,
title = "A Methodology for Fitting and Validating Metamodels in Simulation",
abstract = "This expository paper discusses the relationships among metamodels, simulation models, and problem entities. A metamodel or response surface is an approximation of the input/output function implied by the underlying simulation model. There are several types of metamodel: linear regression, splines, neural networks, etc. This paper distinguishes between fitting and validating a metamodel. Metamodels may have different goals: (i) understanding, (ii) prediction, (iii) optimization, and (iv) verification and validation. For this metamodeling, a process with thirteen steps is proposed. Classic design of experiments (DOE) is summarized, including standard measures of fit such as the R-square coefficient and cross-validation measures. This DOE is extended to sequential or stagewise DOE. Several validation criteria, measures, and estimators are discussed. Metamodels in general are covered, along with a procedure for developing linear regression (including polynomial) metamodels.",
keywords = "Simulation, approximation, response surface, modelling, regression",
author = "J.P.C. Kleijnen and R. Sargent",
note = "Pagination: 36",
year = "1997",
language = "English",
volume = "1997-116",
series = "CentER Discussion Paper",
publisher = "Operations research",
type = "WorkingPaper",
institution = "Operations research",

}

Kleijnen, JPC & Sargent, R 1997 'A Methodology for Fitting and Validating Metamodels in Simulation' CentER Discussion Paper, vol. 1997-116, Operations research, Tilburg.

A Methodology for Fitting and Validating Metamodels in Simulation. / Kleijnen, J.P.C.; Sargent, R.

Tilburg : Operations research, 1997. (CentER Discussion Paper; Vol. 1997-116).

Research output: Working paperDiscussion paperOther research output

TY - UNPB

T1 - A Methodology for Fitting and Validating Metamodels in Simulation

AU - Kleijnen, J.P.C.

AU - Sargent, R.

N1 - Pagination: 36

PY - 1997

Y1 - 1997

N2 - This expository paper discusses the relationships among metamodels, simulation models, and problem entities. A metamodel or response surface is an approximation of the input/output function implied by the underlying simulation model. There are several types of metamodel: linear regression, splines, neural networks, etc. This paper distinguishes between fitting and validating a metamodel. Metamodels may have different goals: (i) understanding, (ii) prediction, (iii) optimization, and (iv) verification and validation. For this metamodeling, a process with thirteen steps is proposed. Classic design of experiments (DOE) is summarized, including standard measures of fit such as the R-square coefficient and cross-validation measures. This DOE is extended to sequential or stagewise DOE. Several validation criteria, measures, and estimators are discussed. Metamodels in general are covered, along with a procedure for developing linear regression (including polynomial) metamodels.

AB - This expository paper discusses the relationships among metamodels, simulation models, and problem entities. A metamodel or response surface is an approximation of the input/output function implied by the underlying simulation model. There are several types of metamodel: linear regression, splines, neural networks, etc. This paper distinguishes between fitting and validating a metamodel. Metamodels may have different goals: (i) understanding, (ii) prediction, (iii) optimization, and (iv) verification and validation. For this metamodeling, a process with thirteen steps is proposed. Classic design of experiments (DOE) is summarized, including standard measures of fit such as the R-square coefficient and cross-validation measures. This DOE is extended to sequential or stagewise DOE. Several validation criteria, measures, and estimators are discussed. Metamodels in general are covered, along with a procedure for developing linear regression (including polynomial) metamodels.

KW - Simulation

KW - approximation

KW - response surface

KW - modelling

KW - regression

M3 - Discussion paper

VL - 1997-116

T3 - CentER Discussion Paper

BT - A Methodology for Fitting and Validating Metamodels in Simulation

PB - Operations research

CY - Tilburg

ER -

Kleijnen JPC, Sargent R. A Methodology for Fitting and Validating Metamodels in Simulation. Tilburg: Operations research. 1997. (CentER Discussion Paper).