Harvesting the wisdom of the crowd

Using online ratings to explore care experiences in regions

R.J.P. Hendrikx, M.D. Spreeuwenberg, H.W. Drewes, J.N. Struijs, D. Ruwaard, C.A. Baan

Research output: Contribution to journalArticleScientificpeer-review

2 Downloads (Pure)

Abstract

Background: 
Regional population health management (PHM) initiatives need an understanding of regional patient experiences to improve their services. Websites that gather patient ratings have become common and could be a helpful tool in this effort. Therefore, this study explores whether unsolicited online ratings can provide insight into (differences in) patient’s experiences at a (regional) population level.
Methods: 
Unsolicited online ratings from the Dutch website Zorgkaart Nederland (year = 2008–2017) were used. Patients rated their care providers on six dimensions from 1 to 10 and these ratings were geographically aggregated based on nine PHM regions. Distributions were explored between regions. Multilevel analyses per provider category, which produced Intraclass Correlation Coefficients (ICC), were performed to determine clustering of ratings of providers located within regions. If ratings were clustered, then this would indicate that differences found between regions could be attributed to regional characteristics (e.g. demographics or regional policy).
Results: 
In the nine regions, 70,889 ratings covering 4100 care providers were available. Overall, average regional scores (range = 8.3–8.6) showed significant albeit small differences. Multilevel analyses indicated little clustering between unsolicited provider ratings within regions, as the regional level ICCs were low (ICC pioneer site < 0.01). At the provider level, all ICCs were above 0.11, which showed that ratings were clustered.
Conclusions: 
Unsolicited online provider-based ratings are able to discern (small) differences between regions, similar to solicited data. However, these differences could not be attributed to the regional level, making unsolicited ratings not useful for overall regional policy evaluations. At the provider level, ratings can be used by regions to identify under-performing providers within their regions.
Original languageEnglish
Article number801
Number of pages9
JournalBMC Health Services Research
Volume18
Issue number1
DOIs
Publication statusPublished - 2018

Fingerprint

Cluster Analysis

Cite this

Hendrikx, R.J.P. ; Spreeuwenberg, M.D. ; Drewes, H.W. ; Struijs, J.N. ; Ruwaard, D. ; Baan, C.A. / Harvesting the wisdom of the crowd : Using online ratings to explore care experiences in regions. In: BMC Health Services Research. 2018 ; Vol. 18, No. 1.
@article{29d30b33ab05415ab0e8d47642a25aa9,
title = "Harvesting the wisdom of the crowd: Using online ratings to explore care experiences in regions",
abstract = "Background: Regional population health management (PHM) initiatives need an understanding of regional patient experiences to improve their services. Websites that gather patient ratings have become common and could be a helpful tool in this effort. Therefore, this study explores whether unsolicited online ratings can provide insight into (differences in) patient’s experiences at a (regional) population level.Methods: Unsolicited online ratings from the Dutch website Zorgkaart Nederland (year = 2008–2017) were used. Patients rated their care providers on six dimensions from 1 to 10 and these ratings were geographically aggregated based on nine PHM regions. Distributions were explored between regions. Multilevel analyses per provider category, which produced Intraclass Correlation Coefficients (ICC), were performed to determine clustering of ratings of providers located within regions. If ratings were clustered, then this would indicate that differences found between regions could be attributed to regional characteristics (e.g. demographics or regional policy).Results: In the nine regions, 70,889 ratings covering 4100 care providers were available. Overall, average regional scores (range = 8.3–8.6) showed significant albeit small differences. Multilevel analyses indicated little clustering between unsolicited provider ratings within regions, as the regional level ICCs were low (ICC pioneer site < 0.01). At the provider level, all ICCs were above 0.11, which showed that ratings were clustered.Conclusions: Unsolicited online provider-based ratings are able to discern (small) differences between regions, similar to solicited data. However, these differences could not be attributed to the regional level, making unsolicited ratings not useful for overall regional policy evaluations. At the provider level, ratings can be used by regions to identify under-performing providers within their regions.",
author = "R.J.P. Hendrikx and M.D. Spreeuwenberg and H.W. Drewes and J.N. Struijs and D. Ruwaard and C.A. Baan",
year = "2018",
doi = "10.1186/s12913-018-3566-z",
language = "English",
volume = "18",
journal = "BMC Health Services Research",
issn = "1472-6963",
publisher = "BMC",
number = "1",

}

Harvesting the wisdom of the crowd : Using online ratings to explore care experiences in regions. / Hendrikx, R.J.P.; Spreeuwenberg, M.D.; Drewes, H.W.; Struijs, J.N.; Ruwaard, D.; Baan, C.A.

In: BMC Health Services Research, Vol. 18, No. 1, 801, 2018.

Research output: Contribution to journalArticleScientificpeer-review

TY - JOUR

T1 - Harvesting the wisdom of the crowd

T2 - Using online ratings to explore care experiences in regions

AU - Hendrikx, R.J.P.

AU - Spreeuwenberg, M.D.

AU - Drewes, H.W.

AU - Struijs, J.N.

AU - Ruwaard, D.

AU - Baan, C.A.

PY - 2018

Y1 - 2018

N2 - Background: Regional population health management (PHM) initiatives need an understanding of regional patient experiences to improve their services. Websites that gather patient ratings have become common and could be a helpful tool in this effort. Therefore, this study explores whether unsolicited online ratings can provide insight into (differences in) patient’s experiences at a (regional) population level.Methods: Unsolicited online ratings from the Dutch website Zorgkaart Nederland (year = 2008–2017) were used. Patients rated their care providers on six dimensions from 1 to 10 and these ratings were geographically aggregated based on nine PHM regions. Distributions were explored between regions. Multilevel analyses per provider category, which produced Intraclass Correlation Coefficients (ICC), were performed to determine clustering of ratings of providers located within regions. If ratings were clustered, then this would indicate that differences found between regions could be attributed to regional characteristics (e.g. demographics or regional policy).Results: In the nine regions, 70,889 ratings covering 4100 care providers were available. Overall, average regional scores (range = 8.3–8.6) showed significant albeit small differences. Multilevel analyses indicated little clustering between unsolicited provider ratings within regions, as the regional level ICCs were low (ICC pioneer site < 0.01). At the provider level, all ICCs were above 0.11, which showed that ratings were clustered.Conclusions: Unsolicited online provider-based ratings are able to discern (small) differences between regions, similar to solicited data. However, these differences could not be attributed to the regional level, making unsolicited ratings not useful for overall regional policy evaluations. At the provider level, ratings can be used by regions to identify under-performing providers within their regions.

AB - Background: Regional population health management (PHM) initiatives need an understanding of regional patient experiences to improve their services. Websites that gather patient ratings have become common and could be a helpful tool in this effort. Therefore, this study explores whether unsolicited online ratings can provide insight into (differences in) patient’s experiences at a (regional) population level.Methods: Unsolicited online ratings from the Dutch website Zorgkaart Nederland (year = 2008–2017) were used. Patients rated their care providers on six dimensions from 1 to 10 and these ratings were geographically aggregated based on nine PHM regions. Distributions were explored between regions. Multilevel analyses per provider category, which produced Intraclass Correlation Coefficients (ICC), were performed to determine clustering of ratings of providers located within regions. If ratings were clustered, then this would indicate that differences found between regions could be attributed to regional characteristics (e.g. demographics or regional policy).Results: In the nine regions, 70,889 ratings covering 4100 care providers were available. Overall, average regional scores (range = 8.3–8.6) showed significant albeit small differences. Multilevel analyses indicated little clustering between unsolicited provider ratings within regions, as the regional level ICCs were low (ICC pioneer site < 0.01). At the provider level, all ICCs were above 0.11, which showed that ratings were clustered.Conclusions: Unsolicited online provider-based ratings are able to discern (small) differences between regions, similar to solicited data. However, these differences could not be attributed to the regional level, making unsolicited ratings not useful for overall regional policy evaluations. At the provider level, ratings can be used by regions to identify under-performing providers within their regions.

U2 - 10.1186/s12913-018-3566-z

DO - 10.1186/s12913-018-3566-z

M3 - Article

VL - 18

JO - BMC Health Services Research

JF - BMC Health Services Research

SN - 1472-6963

IS - 1

M1 - 801

ER -