Belief Elicitation: A Horse Race among Truth Serums

Research output: Working paperDiscussion paperOther research output

334 Downloads (Pure)

Abstract

In survey studies, probabilistic expectations about uncertain events are typically elicited by asking respondents for their introspective beliefs. If more complex procedures are feasible, beliefs can be elicited by incentive compatible revealed preference mechanisms (“truth serums”). Various mechanisms have been proposed in the literature, which differ in the degree to which they account for respondents’ deviations from expected value maximization. In this paper, we pit non-incentivized introspection against five truth serums, to elicit beliefs in a simple two-player game. We test the internal validity (additivity and predictive power for own behavior), and the external validity (predictive power for other players’ behavior, or accuracy) of each method. We find no differences among the truth serums. Beliefs from incentivized methods are better predictors of subjects’ own behavior compared to introspection. However, introspection performs equally well as the truth serums in terms of accuracy and additivity.
Original languageEnglish
Place of PublicationTilburg
PublisherEconomics
Number of pages38
Volume2011-117
Publication statusPublished - 2011

Publication series

NameCentER Discussion Paper
Volume2011-117

Fingerprint

Serum
Additivity
Predictive power
Incentive compatible
Deviation
External validity
Expected value
Revealed preference
Internal validity
Predictors

Keywords

  • belief measurement
  • subjective probability
  • scoring rules
  • outcome matching
  • probability matching
  • internal validity
  • external validity

Cite this

Trautmann, S. T., & van de Kuilen, G. (2011). Belief Elicitation: A Horse Race among Truth Serums. (CentER Discussion Paper; Vol. 2011-117). Tilburg: Economics.
Trautmann, S.T. ; van de Kuilen, G. / Belief Elicitation : A Horse Race among Truth Serums. Tilburg : Economics, 2011. (CentER Discussion Paper).
@techreport{c62319d0674f4def8e9fd19faff3ca6f,
title = "Belief Elicitation: A Horse Race among Truth Serums",
abstract = "In survey studies, probabilistic expectations about uncertain events are typically elicited by asking respondents for their introspective beliefs. If more complex procedures are feasible, beliefs can be elicited by incentive compatible revealed preference mechanisms (“truth serums”). Various mechanisms have been proposed in the literature, which differ in the degree to which they account for respondents’ deviations from expected value maximization. In this paper, we pit non-incentivized introspection against five truth serums, to elicit beliefs in a simple two-player game. We test the internal validity (additivity and predictive power for own behavior), and the external validity (predictive power for other players’ behavior, or accuracy) of each method. We find no differences among the truth serums. Beliefs from incentivized methods are better predictors of subjects’ own behavior compared to introspection. However, introspection performs equally well as the truth serums in terms of accuracy and additivity.",
keywords = "belief measurement, subjective probability, scoring rules, outcome matching, probability matching, internal validity, external validity",
author = "S.T. Trautmann and {van de Kuilen}, G.",
note = "Pagination: 38",
year = "2011",
language = "English",
volume = "2011-117",
series = "CentER Discussion Paper",
publisher = "Economics",
type = "WorkingPaper",
institution = "Economics",

}

Trautmann, ST & van de Kuilen, G 2011 'Belief Elicitation: A Horse Race among Truth Serums' CentER Discussion Paper, vol. 2011-117, Economics, Tilburg.

Belief Elicitation : A Horse Race among Truth Serums. / Trautmann, S.T.; van de Kuilen, G.

Tilburg : Economics, 2011. (CentER Discussion Paper; Vol. 2011-117).

Research output: Working paperDiscussion paperOther research output

TY - UNPB

T1 - Belief Elicitation

T2 - A Horse Race among Truth Serums

AU - Trautmann, S.T.

AU - van de Kuilen, G.

N1 - Pagination: 38

PY - 2011

Y1 - 2011

N2 - In survey studies, probabilistic expectations about uncertain events are typically elicited by asking respondents for their introspective beliefs. If more complex procedures are feasible, beliefs can be elicited by incentive compatible revealed preference mechanisms (“truth serums”). Various mechanisms have been proposed in the literature, which differ in the degree to which they account for respondents’ deviations from expected value maximization. In this paper, we pit non-incentivized introspection against five truth serums, to elicit beliefs in a simple two-player game. We test the internal validity (additivity and predictive power for own behavior), and the external validity (predictive power for other players’ behavior, or accuracy) of each method. We find no differences among the truth serums. Beliefs from incentivized methods are better predictors of subjects’ own behavior compared to introspection. However, introspection performs equally well as the truth serums in terms of accuracy and additivity.

AB - In survey studies, probabilistic expectations about uncertain events are typically elicited by asking respondents for their introspective beliefs. If more complex procedures are feasible, beliefs can be elicited by incentive compatible revealed preference mechanisms (“truth serums”). Various mechanisms have been proposed in the literature, which differ in the degree to which they account for respondents’ deviations from expected value maximization. In this paper, we pit non-incentivized introspection against five truth serums, to elicit beliefs in a simple two-player game. We test the internal validity (additivity and predictive power for own behavior), and the external validity (predictive power for other players’ behavior, or accuracy) of each method. We find no differences among the truth serums. Beliefs from incentivized methods are better predictors of subjects’ own behavior compared to introspection. However, introspection performs equally well as the truth serums in terms of accuracy and additivity.

KW - belief measurement

KW - subjective probability

KW - scoring rules

KW - outcome matching

KW - probability matching

KW - internal validity

KW - external validity

M3 - Discussion paper

VL - 2011-117

T3 - CentER Discussion Paper

BT - Belief Elicitation

PB - Economics

CY - Tilburg

ER -

Trautmann ST, van de Kuilen G. Belief Elicitation: A Horse Race among Truth Serums. Tilburg: Economics. 2011. (CentER Discussion Paper).