Effectiveness of a digital platform-based implementation strategy to prevent work stress in a healthcare organization

A digital platform-based implementation strategy showed potential for preventing stress among healthcare workers. The experimental group showed a slight decline in stress

Working in the healthcare sector has internationally been qualified as stressful (1)(2)(3).According to a large survey among Dutch employees, 37.6% of healthcare workers who reported health complaints leading to sickness absence indicated that these health complaints were caused by work stress (4).Work stress is a risk factor for mental and physical health problems (5,6) and contributes considerably to sickness absence (7).This puts a financial burden on organizations and society.
Interventions that are effective at reducing stress in workers are available (3,8,9), but they are often not used by organizations.Westgaard & Winkel (10) argue that failure of organizations to create more sustainable psychosocial work environments using interventions can be due to insufficient resources, and the fact that (work stress) challenges constantly change over time.They also argue that information about evidence-based work stress prevention does not reach stakeholders.Additional 2 Scand J Work Environ Health -online first Effectiveness of a stress prevention implementation strategy potential complications of the use of work stress prevention are the notions that there is insufficient awareness of work stress and that perceptions of stakeholders with regard to psychosocial challenges often differ (11,12).If awareness about work stress could be raised, stakeholders could work together to pinpoint the psychosocial challenges relevant to their specific organizational context (10).Then, an appropriate, preferably evidencebased, organizational stress management intervention could be selected and used.
Once a stress management intervention is selected, its implementation is confronted with several barriers related to the context in which the intervention is used, the organization, the intervention user, or the intervention itself (13).For example, in their process evaluation of a stress risk assessment tool, Biron and colleagues (14) found that high turnover and changes in team composition made use of the intervention very difficult.Hasson and colleagues (15) found that during implementation, managers and HR professionals seldom managed to perform the roles that they deemed necessary for intervention success.Finally, Ipsen and colleagues (16) reported that a lack of resources slowed the implementation process.Offering organizations guidance in overcoming these barriers may facilitate the implementation of appropriate stress management interventions and reduce work stress among employees.
Within the project Stress Prevention@Work (SP@W) a multifaceted, integral implementation strategy (henceforth: the strategy) was developed (17).The strategy aimed to raise awareness of work stress among stakeholders and direct organizations to a proper (psychosocial) risk analysis, conducted in a participative manner and identifying organizational risk factors for work stress.It consecutively helped to identify and select appropriate interventions and overcome implementation barriers.The strategy consisted of a digital platform that contained a stepwise approach and a collaborative learning network.It was developed together with organizations of different sizes and from various sectors, including the healthcare sector.To the best of the authors' knowledge, no similar strategies have been evaluated before, which suggests that a research gap exists.
The objective of this study was to investigate the effectiveness of a digital platform-based implementation strategy -compared to a control group -on stress, determinants of work stress (ie, psychosocial work factors) and the level of implementation among healthcare workers.

Study design, participants and procedure
Using a cluster-controlled study design, the effectiveness of the strategy was assessed within a large Dutch healthcare organization (>4500 employees).A total of 30 teams indicated that they were willing to participate in the trial.By way of matching (based on work setting and team size), teams were assigned to either the experimental or control group by an independent researcher, who had no information about the perceived stress levels in the teams.The experimental group (15 teams; N=252) received the strategy, and the control group (15 teams; N=221) was put on a waiting-list.The protocol stated that the experimental group should implement the strategy in the first 6 months of the trial, but they had access to the strategy for the complete duration of the trial (12 months).Measurements were performed at baseline (May, 2016; T0), after 6 months (T1) and after 12 months (T2).Participants were approached by email to complete an online questionnaire that took 16 minutes on average.Detailed information about the context of the study, the strategy, the study population, and data collection can be found elsewhere (17).The Medical Ethical Committee of the VU University Medical Center, Amsterdam, the Netherlands, approved the trial (reference number: 2015.480).

Strategy
The strategy provided to the teams in the experimental group consisted of a stepwise approach embedded in a digital platform.The main goal of the strategy was to promote the use of interventions aimed at prevention of work stress at the primary (prevention of stress incidence) and secondary (reduction of the prevalence of work stress) level.Initially, a collaborative learning network, in which organizations could share and develop knowledge about stress prevention, was also part of the strategy.Several meetings of the collaborative learning network were organized before the start of the trial, for a wide range of organizations.After initial success, attendance rates deteriorated quickly to a point where the collaborative learning network was no longer considered feasible due to sustainability issues.Therefore, it was omitted from the strategy before the start of the trial.

Digital platform
The digital platform provided information, screening and planning tools.It also contained a search engine with a broad selection of interventions relevant to work stress prevention.Even though the strategy focused on organizational stress prevention, it did not dictate which intervention should be selected.Therefore, interventions aimed at the individual or at (part of) the organization could be selected.Interventions ranged from fairly simple and free measures, such as a printable guideline that helps employees and supervisors to start a conversation about work stress, to more elaborate and costly measures, such as tailor-made prevention projects provided by external consultants.Responsibility for implementation of the intervention depended on the intervention selected and choices made by the team itself.
The pages on the digital platform corresponded with the stepwise approach, which consisted of five steps: (i) awareness raising (acknowledgment of the fact that management of work stress is important); (ii) assessment (screening for relevant determinants of work stress to target for prevention); (iii) planning (setting intervention goals and selecting interventions); (iv) implementation (implementing interventions in the organization); and (v) evaluation (assessing whether goals set in the planning step were achieved) (18,19).Using these sequential steps, a continuous, cyclical work stress prevention process can be performed.The platform was developed in cooperation with organizations from different sectors, such as healthcare, education, transport, and ICT (17).
One member of each of the teams in the experimental group received a training in the use of the digital platform.Additionally, shortly after the 6-month followup measurement, a meeting was held during which these team members could share their experiences with the use of the digital platform.

Primary outcome: stress
Stress was measured using the stress sub-scale of the short version of the Depression Anxiety and Stress Scale (DASS-21) [for items, see (20), p340].In the stress sub-scale of the DASS-21, seven statements are presented, such as "I found it hard to wind down", and "I found myself getting agitated".Participants indicated to what extent these statements applied to them in the past week, on a Likert-scale ranging from 0 ("never") to 3 ("almost always").The result was a 0-21 scale score, with a higher score representing more stress.Different scale scores represent different levels of stress severity (0-7=normal; 8-9=mild; 10-12=moderate; 13-16=severe; 17-21=extremely severe) (21).DASS-21 has been validated in non-clinical settings (22,23).

Secondary outcomes
Determinants of work stress.All psychosocial work factors (psychological demands, social support, and autonomy) were measured using sub-scales from the Job Content Questionnaire (JCQ) (24) [for items, see (25)].The psychological demands subscale consisted of five state-ments (eg, "My job requires working very fast").Social support was measured using a sub-scale for coworker support including four statements (eg, "People I work with are helpful in getting the job done"), and a subscale for supervisor support also including four statements (eg, "My supervisor pays attention to what I am saying").Finally, autonomy was measured using a subscale including three statements (eg, "My job allows me to make a lot of decisions on my own").Participants indicated to what extent they agreed with all statements, using a Likert-scale ranging from 1 ("Strongly disagree") to 4 ("Strongly agree").Consequently, scale scores for psychological demands ranged from 5-20, scale scores for both coworker and supervisor support ranged from 4-16, and scale scores for autonomy ranged from 3-12.
Level of Implementation.The goal of the strategy was to increase the use of interventions aimed at work stress prevention.This was measured in the 12-month follow-up questionnaire at the worker level, using two questions [ie, "To what extent have work-related stress management interventions been implemented in your team since May 2016?", and "How often did you discuss work-related stress in your team (for example during a team meeting of your team)?"]Both questions could be answered using a Likert-scale ranging from 1 ("not at all/barely") to 10 ("very much").

Covariates
Covariates taken into account in the present study were age, marital status (partner: yes/no), hours worked per week, and working night shifts (yes/no).Educational level was taken into account as well.The levels of education were low (lower general secondary education, preparatory secondary vocational education), medium (intermediate vocational training, higher general secondary education, pre-university education), and high (higher vocational education, university education).Gender was not taken into account because the final study sample consisted almost exclusively of women.

Analyses
Analyses were performed in accordance with the intention-to-treat principle.Descriptive statistics were used to report on baseline study sample characteristics (ie, age, educational level, marital status, hours worked per week, working night shifts, psychological demands, coworker support, supervisor support, autonomy, and stress).Baseline differences between the experimental and control group in study sample characteristics were assessed using Chi square tests for educational level, having a partner or not, and working night shifts, and an independent samples T-test for age, hours worked per week, psychological demands, coworker support, supervisor support, autonomy, and stress.Linear mixed models analyses were performed with the two follow-up measurement as outcome and strategy as independent variable, adjusted for baseline value of the outcome to assess the overall effect of the strategy.The same analysis was performed but including time and the interaction between strategy and time in order to estimate the effect of the strategy at the two follow-up measurements.A random intercept for individual was included to account for the correlation between the repeated measures within the individual.A random intercept for team was included in the model to account for correlated observation within the teams if this resulted in an improvement of the model, which was determined by a likelihood ratio test.For all categorical confounders, the distribution over the categories was assessed for suitability as potential confounder.Then, all potential confounders were added to the model separately to check for confounding.If the regression coefficient of strategy changed more than 10% by adding any of these possible confounders, then the confounder was added to the model.An interaction term for strategy with age was tested to check for effect modification based on multiplicativity.If the interaction term was statistically significant, effect modification was assumed.Finally, time, and the interaction between strategy and time were included to differentiate between the two follow-up measurements.An overall effect was reported separately for all outcomes, and the effects for the two follow-ups were also reported.Since the implementation indicators were measured once, we performed linear mixed models analysis to assess differences between the experimental group and the control group.A random intercept for team was included to account for differences between teams.
Loss-to-follow-up analyses were performed for the baseline measures, comparing participants who participated in the baseline measurement and at least one follow-up to those who participated in the baseline measurement only, using Chi square tests for categorical variables and independent samples T-test for continuous variables.The main reason for non-response was turnover.P<0.05 were considered statistically significant.IBM statistical package for the social sciences (SPSS, version 22, IBM Corp, Armonk, NY, USA) was used to analyze the data.

Results
Of the 473 employees invited to complete the baseline questionnaire, 304 participants did so (response rate: 64%).Of the 304 participants who participated in the baseline measurement, 210 participants (69%) also participated in at least one of the follow-up measurements (see figure 1).At baseline, there were no differences between the experimental and control group with regard to age, educational level, marital status, hours worked per week, psychological demands, coworker support, supervisor support, autonomy, or stress.Participants in the experimental group reported working night shifts more often (54.4%)than those in the control group (30.3%).Those lost to follow-up did not differ from sustained participants with regard to age, educational level, marital status, hours worked per week, working night shifts, psychological demands, supervisor support, autonomy, or stress, but did have a slightly lower score on coworker support (score difference = 0.53, P<0.05).In table 1, descriptive results are presented for baseline values of gender, age, educational level, hours worked per week, working night shifts, psychological demands, coworker support, supervisor support, autonomy, and stress.
Note: Respondents who participated in baseline and both follow-ups were: n(control group)=58, and n(experimental group)=61  No significant effects were found for the determinants of work stress.Age was an effect modifier for the association between the strategy and psychological demands.Adding a random intercept for team only resulted in significant improvement of the model for coworker support.There were no statistically significant differences between the experimental and control group with regard to the extent to which stress management interventions had been implemented within the teams or how often work stress had been discussed within the teams.

Discussion
An overall effect of the strategy was found for stress, with participants in the control group reporting higher stress levels over time compared to those in the experimental group.No differences between the experimental and control group were found for determinants of work stress or the level of implementation.The effect of the strategy was mainly caused by the control group showing an increase in work stress, while this increase did not occur in the experimental group.The effect found for stress (in favor of the experimental group) is in line with existing stress management intervention literature.In their review, Ruotsalainen and colleagues (3) concluded that different types of stress management interventions, such as person-directed and organizational interventions, reduced stress among healthcare workers.However, not all interventions render favorable results.Examples exist of failure to impact exposure to stressors (eg, workflow interruptions for physicians) (26), failure to impact stress-related outcomes (eg, emotional exhaustion, coping, and stress in nurses) (27,28), or mixed impact on psychosocial work factors (eg, improved autonomy, but decreased job satisfaction in caregivers) (29).Our strategy improved neither psychosocial work factors, talking about work stress nor intervention use.Hence, it should be considered that the strategy affected stress through a mechanism not measured in the current study (eg, by affecting work stress determinants other than psychological demands, social support and autonomy) or that factors unrelated to the strategy affected stress.A possible explanation for the absence of effects on implementation indicators might be that our operationalization of the implementation indicators was measured using two single items.A more detailed measurement of implementation indicators could have provided more insight into the working mechanisms of the strategy.In addition, reports show that psychosocial Effectiveness of a stress prevention implementation strategy work factors can be impacted by organizational work stress prevention initiatives (30) but that relevant changes in these determinants of work stress may unfold over a period that is longer than one year (31).
A factor important to the interpretation of the present findings is that implementation of the strategy was only moderately successful.During the trial, some contextual factors such as personnel shortage, turnover, and organizational restructuring hindered the use of the strategy considerably (32).Due to turnover, a total of nine new team members had to be trained in the use of the digital platform during the trial.This stagnated use of the strategy considerably.If the original digital platform participant would have stayed on, more steps of the strategy could have been completed, which could have led to more favorable findings.Organizational restructuring and turnover have been reported to hinder implementation (33,34).Suboptimal implementation increased the possibility that factors other than the content of the strategy could have impacted stress (35,36).The evaluation of an intervention focused on team development showed that effects were driven more by management behavior and characteristics of the teams than by the intervention itself, reporting a strong and unique relationship between line manager behavior and attitudes and outcome measures (37).Alternative explanations of the effect on stress may exist.For instance, being allocated to the control condition could have instigated frustration among control group team members, causing them to experience more stress.At the same time, team members in the experimental group may have experienced relief because they were allocated to the group that would receive the strategy.This may have prevented them from experiencing the increase in stress that occurred in the control group.This nuanced perspective on effect interpretation does not warrant dismissal of the notion that the strategy reduced or prevented stress.Rather, it calls for future research to disambiguate the strategy's potential further.
The different levels of implementation enabled us to compare the effect of the strategy in experimental teams that had successfully implemented the strategy to teams that had not reached the lower level for successful implementation.In a post-hoc analysis, a linear mixed models analysis was performed, comparing teams that finished at least four steps on the digital platform to the teams in the control group.The effect for stress in the teams that completed four steps on the digital platform was slightly larger (B=-1.16,95% CI -2.31--0.16)than the effect found for the experimental teams that completed three steps or fewer compared to the control group (B=-0.79,95% CI -1.80-0.22),although CI overlap.Hence, teams that received the strategy to a greater extent reported a larger reduction in stress than those who received the strategy to a lesser extent.

Strengths and limitations
In interpreting the findings of the current study, some strengths and limitations should be taken into account.A strength of this study was the use of multi-level analysis.It provided the opportunity to account for differences at team level and also reduced selective drop-out because it also uses "incomplete" cases (ie, participants who did not fill out the questionnaire either at 6-or 12-month follow-up).The use of a well-established, validated scale to measure stress was also a strength of this study.Average stress scores found in the current study were largely comparable to other samples in the healthcare sector (38) and in a larger, more general sample of adults (22).The majority of the sample of the current study was female.Those included in the analyses were all female.This limits the generalization of the findings beyond women in the healthcare sector.At the same time, most samples in stress prevention evaluation studies conducted in the healthcare sector consist of mainly women, making comparisons with other studies possible (28,37).A limitation in the design of the study was the fact that randomization at the individual level was not possible because the strategy was directed at teams.Because randomization at team level was not feasible due to the small number of teams, teams were matched.This increased risk of bias.However, there were no relevant baseline differences between the experimental and control group with regard to outcomes and covariates, suggesting that the groups were comparable for at least those parameters.The objective of the current study was to assess the effectiveness of the implementation strategy, which does not put emphasis on the content of the interventions used as a result of the strategy.More information about which specific interventions were used, and when they were used, could have provided additional insights.Another limitation was the considerable loss-to-follow-up, partly due to high turnover.This could bias the findings because those lost to follow-up might have ceased to participate due to work stress.However, analyses showed almost no baseline differences between those lost to follow-up and sustained participants, making it less plausible that large differences between these groups existed.Still, there is a chance that teams that were too busy or experienced high work stress did not participate, which could have led to selection bias.This might have led to a group of complete cases with relatively lower stress levels, less room for improvement, and underestimation of the effects.It could also explain the fact that the average stress scores found in the current study could be categorized as "normal" (ie, non-problematic), even though working in healthcare has been described as stressful.As the strategy aimed at primary and secondary prevention of work stress, the non-problematic levels of stress did not pose a problem for evaluation of the effectiveness of the strategy.

Implications
Future use of the strategy would be supported by making more team members responsible for using the digital platform instead of just one team member.This way, turnover would have less influence on the use of the strategy.Management should also make more time available to team members for use of the strategy, especially in teams that are very busy.A way to do this would be to schedule additional personnel for the hours spent on using the strategy.That way, being busy would not be a barrier to addressing work stress.Furthermore, it should be considered if there is a fit between the strategy and its user.Participants working with the digital platform had limited computer skills because their daily work did not require them to work with computers.Perhaps, more training should be provided to workers like this, or sufficient computer skills should be a prerequisite for working with the digital platform.Finally, it is possible that the content of the strategy did not sufficiently reflect the local (implementation) needs of the organization, hindering implementation.A more elaborate needs assessment could support development of a more tailor-made strategy.More psychosocial work factors could be taken into account, such as the psychosocial safety climate (perceived readiness of management to prevent stressful working conditions).This might clarify which psychosocial factors are impacted by the strategy.In future studies, a longer follow-up period is necessary to provide insight into changes in implementation indicators, psychosocial work factors and their (possibly lagged) impact on work stress (39).Additionally, studying a larger number of teams would make randomization at team level more feasible, increasing opportunities for making causal inferences.

Concluding remarks
The present implementation strategy appeared to be effective in preventing stress among healthcare workers.No differences were found between the experimental and control group with regard to determinants of work stress or the extent to which teams talk about work stress or use stress management interventions.Overcoming implementation barriers of stress management interventions can contribute to prevention of work stress, but more research is necessary to explore working mechanisms related to the effectiveness of the strategy.

Figure 1 .
Figure 1.Flow chart of participants during baseline, and at 6-and 12-month follow-up.Note: respondent who participated in baseline and both followups were N (control group) =58 and N (experimental group) =61.

Table 1 .
Baseline characteristics of the study population (N=304) [SD=standard deviation]In table 2, group means are presented for stress and determinants of stress at baseline, and at 6 and 12 months.In table 3, the overall effect of the strategy, and effects at 6 and 12 months are presented for stress, the determinants of stress, and level of implementation.
a Lower general secondary education, preparatory secondary vocational education.b Intermediate vocational training, higher general secondary education, pre-university education.c Higher vocational education, university education d P<0.01 Stress, determinants of stress, and implementation indicators

Table 2 .
Stress, psychological demands, coworker support, supervisor support, autonomy at baseline, and implementation indicators at 6 and 12 months follow-up.[SD=standard deviation.]

Table 3 .
Results of a multilevel analysis of the effect of the strategy on stress, psychological demands, coworker support, supervisor support, autonomy, and implementation indicators [B=regression coefficient; CI=confidence interval; mth=month] "To what extent have work-related stress management interventions been implemented in your team since May 2016?" e "How often did you discuss work-related stress in your team (for example during a team meeting of your team)?" cRandom intercept at team level.d