Estimating public performance through an MTMM model

The case of police performance in 26 European countries

M. Barlage, J.A. van den Born, A. van Witteloostuijn

Research output: Contribution to journalArticleScientificpeer-review

Abstract

Organisational performance is notoriously difficult to measure in the public sector. More often than not, objective performance measures are difficult to construct. Subjective performance is a popular alternative to, as well as, complement of objective performance measures. However, such subjective perception measures are likely to be biased. The bias tends to depend upon the specific stakeholder's position vis-à-vis the focal organisation. We show how a multi-trait–multi-method (MTMM) model cannot only determine the validity of performance measures, but is also valuable in generating estimates the potential biases in both method (e.g. respondent type) and trait (e.g. performance measure) of these subjective performance measures. To demonstrate the benefits of this methodology in public management, we apply this method to the subjective performance of Police Forces in 26 European countries. Our policing example demonstrates that single-handedly the available subjective performance measures are not reliable estimates of overall police performance. Moreover, the analysis shows that all three rater groups have significant bias, with police employees most positively biased about their own performance. Interestingly enough this bias depends on the performance measure; corporate managers are most biased about the police catching burglars, while health care professionals are more biased about policing arrival times.
Original languageEnglish
Pages (from-to)377-396
JournalPolicy Studies
Volume35
Issue number4
Early online date13 Jan 2014
DOIs
Publication statusPublished - 2014

Fingerprint

police
performance
trend
public management
public sector
stakeholder
employee
manager
health care
methodology

Cite this

@article{a68669dc954447a49a749a55fca8c9bc,
title = "Estimating public performance through an MTMM model: The case of police performance in 26 European countries",
abstract = "Organisational performance is notoriously difficult to measure in the public sector. More often than not, objective performance measures are difficult to construct. Subjective performance is a popular alternative to, as well as, complement of objective performance measures. However, such subjective perception measures are likely to be biased. The bias tends to depend upon the specific stakeholder's position vis-{\`a}-vis the focal organisation. We show how a multi-trait–multi-method (MTMM) model cannot only determine the validity of performance measures, but is also valuable in generating estimates the potential biases in both method (e.g. respondent type) and trait (e.g. performance measure) of these subjective performance measures. To demonstrate the benefits of this methodology in public management, we apply this method to the subjective performance of Police Forces in 26 European countries. Our policing example demonstrates that single-handedly the available subjective performance measures are not reliable estimates of overall police performance. Moreover, the analysis shows that all three rater groups have significant bias, with police employees most positively biased about their own performance. Interestingly enough this bias depends on the performance measure; corporate managers are most biased about the police catching burglars, while health care professionals are more biased about policing arrival times.",
author = "M. Barlage and {van den Born}, J.A. and {van Witteloostuijn}, A.",
year = "2014",
doi = "10.1080/01442872.2013.875154#.Uz0FLvmSyiw",
language = "English",
volume = "35",
pages = "377--396",
journal = "Policy Studies",
issn = "0144-2872",
publisher = "Routledge",
number = "4",

}

Estimating public performance through an MTMM model : The case of police performance in 26 European countries. / Barlage, M.; van den Born, J.A.; van Witteloostuijn, A.

In: Policy Studies, Vol. 35, No. 4, 2014, p. 377-396.

Research output: Contribution to journalArticleScientificpeer-review

TY - JOUR

T1 - Estimating public performance through an MTMM model

T2 - The case of police performance in 26 European countries

AU - Barlage, M.

AU - van den Born, J.A.

AU - van Witteloostuijn, A.

PY - 2014

Y1 - 2014

N2 - Organisational performance is notoriously difficult to measure in the public sector. More often than not, objective performance measures are difficult to construct. Subjective performance is a popular alternative to, as well as, complement of objective performance measures. However, such subjective perception measures are likely to be biased. The bias tends to depend upon the specific stakeholder's position vis-à-vis the focal organisation. We show how a multi-trait–multi-method (MTMM) model cannot only determine the validity of performance measures, but is also valuable in generating estimates the potential biases in both method (e.g. respondent type) and trait (e.g. performance measure) of these subjective performance measures. To demonstrate the benefits of this methodology in public management, we apply this method to the subjective performance of Police Forces in 26 European countries. Our policing example demonstrates that single-handedly the available subjective performance measures are not reliable estimates of overall police performance. Moreover, the analysis shows that all three rater groups have significant bias, with police employees most positively biased about their own performance. Interestingly enough this bias depends on the performance measure; corporate managers are most biased about the police catching burglars, while health care professionals are more biased about policing arrival times.

AB - Organisational performance is notoriously difficult to measure in the public sector. More often than not, objective performance measures are difficult to construct. Subjective performance is a popular alternative to, as well as, complement of objective performance measures. However, such subjective perception measures are likely to be biased. The bias tends to depend upon the specific stakeholder's position vis-à-vis the focal organisation. We show how a multi-trait–multi-method (MTMM) model cannot only determine the validity of performance measures, but is also valuable in generating estimates the potential biases in both method (e.g. respondent type) and trait (e.g. performance measure) of these subjective performance measures. To demonstrate the benefits of this methodology in public management, we apply this method to the subjective performance of Police Forces in 26 European countries. Our policing example demonstrates that single-handedly the available subjective performance measures are not reliable estimates of overall police performance. Moreover, the analysis shows that all three rater groups have significant bias, with police employees most positively biased about their own performance. Interestingly enough this bias depends on the performance measure; corporate managers are most biased about the police catching burglars, while health care professionals are more biased about policing arrival times.

U2 - 10.1080/01442872.2013.875154#.Uz0FLvmSyiw

DO - 10.1080/01442872.2013.875154#.Uz0FLvmSyiw

M3 - Article

VL - 35

SP - 377

EP - 396

JO - Policy Studies

JF - Policy Studies

SN - 0144-2872

IS - 4

ER -