Research Article | Open Access
Iselin Reknes, Guy Notelaers, Nils Magerøy, Ståle Pallesen, Bjørn Bjorvatn, Bente Elisabeth Moen, Ståle Einarsen, "Aggression from Patients or Next of Kin and Exposure to Bullying Behaviors: A Conglomerate Experience?", Nursing Research and Practice, vol. 2017, Article ID 1502854, 12 pages, 2017. https://doi.org/10.1155/2017/1502854
Aggression from Patients or Next of Kin and Exposure to Bullying Behaviors: A Conglomerate Experience?
Although workplace violence and aggression have been identified as important stressors in the nursing profession, studies simultaneously comparing patient-initiated aggression and exposure to bullying behaviors at work are rather scarce. The aim of this study was to compare aggression from patients or next of kin and exposure to bullying behaviors in terms of prevalence, health-related quality of life outcomes, and potential overlap in those targeted. In the period of 2008-2009, data were collected among 2059 members of the Norwegian Nurses Organization. Latent class (LC) analysis and a multivariate analysis of variance (MANOVA) were used to investigate the proposed relationships. The results showed that aggression from patients or next of kin and exposure to bullying behaviors were perceived as separate and independent stressors. Although aggression from patients or next of kin was more frequent than workplace bullying, the latter was the only significant stressor related to health-related quality of life in terms of reduced mental health functioning. Although being a rather infrequent experience, exposure to bullying behaviors seems to have more severe health-related outcomes for nurses than aggression from patients or next of kin. Hence, the results of the study strengthen previous findings and suggest that managers must aim to maintain a positive psychosocial work environment with zero-tolerance for bullying.
Nurses are part of a professional group known to be at risk of workplace violence and aggression , which includes risks of being exposed to physical assaults, threat of assaults, and verbal abuse . The negative behavior may include various perpetrators and may take many forms , yet mainly being either consumer-related violence (i.e., violence and aggression by clients and/or patients against staff) or relationship violence (i.e., exposure to harassment and bullying at work by one’s colleagues or superiors) . Although nurses may have close working relationships with both patients and colleagues, violence from these groups may happen in isolation and therefore be evaluated by nurses as different and separate phenomena when experienced. Yet, these types of violence may also occur simultaneously or be sequentially linked to each other and thus be experienced as more or less conglomerate stressors. Some nurses may also be vulnerable to both. In line with this, previous research has stated a need for more research including the outcomes of both insider and outsider initiated violence (e.g., [5, 6]). Even though some studies have addressed this call [6–8], studies simultaneously comparing health-related outcomes of aggression from patients or next of kin and exposure to bullying behaviors are still warranted (for an exception see ). Hence, the aim of the present study is to investigate the occurrence and overlap between aggression from patients or next of kin and exposure to bullying behaviors and to determine their relative relationship with health-related outcomes and well-being.
2.1. Aggression and Bullying at Work
Aggression has been defined as goal-directed behavior with intent to harm or injure another person or object . Aggressive behaviors can be of a sporadic character, also describing one-time occurrences, carried out by various types of perpetrators [3, 11–13]. Yet, aggression may also become a constant threat even if the actual sources tend to vary. Workplace bullying on the other hand is characterized as a process of negative behaviors including harassing, offending, or socially excluding someone or negatively affecting someone’s work. The behaviors occur repeatedly and regularly over a period of time, often involving a power imbalance between the target and the perpetrator, where the target tends to end up in an inferior position (see ). Hence, workplace bullying is a process where the target is exposed to negative behaviors over time often by the same perpetrator(s) who are among one’s peers and/or superiors .
A review of the prevalence of patient-initiated violence showed that the reported proportion of verbal abuse varied from 22% to 90%, threat of violence varied from 12% to 64%, and physical violence varied from 2% to 32% . On the other hand, approximately 3% to 15% of the European workforce may have experienced serious or occasional bullying at work , hence being a less prevalent problem. Some studies indicate, however, that employees in the social and health sector are at a higher risk of being subjected to bullying behaviors from colleagues or superiors than employees in general (see ). Illustrative of this, the prevalence of bullying at work among nurses varies from 4% , via 21% , 34% , 44% , to 86% .
In general, however, studies tend to show that aggression from patients or consumers is more prevalent than aggression from colleagues or superiors [2, 5, 16, 22]. Previous studies comparing the prevalence of aggression from patients and bullying from colleagues in the nursing profession have, however, shown quite similar prevalence of the two stressors: 36% versus 32% , 29% versus 26% , and 30% versus 34% . Hence, although previous research shows that aggression from patients is being more prevalent than aggression from colleagues among nurses (e.g., ), the picture is blurred when it comes to patient- or next of kin-initiated aggression versus workplace bullying.
2.2. Health Outcomes of Aggression and Bullying at Work
Aggression and bullying may affect the nurses in several ways. Differences in how nurses perceive aggressive and bullying behaviors may differ according to several factors: how they perceive the situation, who the perpetrators are, how they define what they are exposed to, and whether they believe it to be part of their job (see [26, 27]). When perceived as a stressor, however, any exposure to violent or aggressive episodes from patients or colleagues may likely result in negative outcomes . Being exposed to outsider violence (i.e., from customers, clients, patients, or others) has been associated with both physical and psychological health problems for those targeted . For instance, in a systematic review of nonsomatic effects of patient assaults , the predominant responses were anger, fear or anxiety, posttraumatic stress disorder symptoms, guilt, self-blame, and shame. Also, nurses seem to react similarly across different cultures and settings . Furthermore, strong relationships between bullying from colleagues and health outcomes, like burnout , psychological distress , and depression , are documented among nurses.
Being exposed to stressful events at work is likely to increase the target’s level of cognitive activation, which according to Harvey  can be described as worrying or repetitive thoughts, triggering autonomic arousal and emotional stress. Previous research has shown that worrying is related to somatic health problems . This corresponds with the Cognitive Activation Theory of Stress (CATS; ), which states that prolonged cognitive and physiological activation, as a result of being exposed to a stressor evaluated as threatening over time, will increase the risk of impaired health when the target feels unable to cope with the situation. The duration of exposure seems, however, to affect the target’s reactions to the stressor. Hence, as aggression typically is of a more sporadic character than workplace bullying [11, 12], which is defined as repeated and sustained exposure, it is likely that the latter will have a more severe impact on the targets’ health due to the assumptions in CATS. In line with this, worrying has been shown to act as a mediator on the relationship between bullying and sleep quality .
An additional explanation of the proposed difference in health-related outcomes of aggression from patients or next of kin as compared to exposure to bullying behaviors may be the tendency to put people in the in-group according to the level of identification . People similar to oneself can be labeled as members of the in-group while people different from oneself are labeled as members of the out-group . Nurses are therefore likely to see their colleagues as a part of their in-group of whom they compare themselves to. Being excluded from social events or from the fellowship at work (i.e., from the in-group) could make the targets nervous and anxious, due to the lack of a sense of belonging . In a hospital setting, patients, on the other hand, usually stay for a short period of time, and the continuous replacement of patients may contribute to the evaluation of patients as members of the out-group with less impact on the nurses’ health and well-being. Nurses may also perceive bullying behaviors from colleagues as something different and more harmful than aggression from patients or next of kin, just because the latter problem is more prevalent and in many cases may be considered as “part of the job” by nurses (e.g., ).
To our knowledge, a comparison of health-related outcomes of aggression from patients or next of kin and workplace bullying has only been addressed in one previous study, conducted among Australian nurses and midwives . In Demir and Rodwell’s  study, the results showed that both bullying from colleagues and aggression, in the form of verbal sexual harassment from patients and their family members, were linked to higher levels of psychological distress. This result was explained according to self-attribution processes (see also ), where nurses blame themselves for the occurrence of these stressors, leading to psychological stress. However, in a study by Farrell and Shafiei , the results showed that nurses were less worried by patient-initiated aggression compared to bullying from colleagues, because the latter made them feel less safe and confident at work. Hence, it may be that bullying from colleagues generates other consequences for those affected compared to aggression from patients or next of kin.
2.3. Aims of the Present Study
Previous studies have shown that outsider aggression has less impact on the employees’ work experience, because the perpetrators are not members of the organization, and thereby the employees may be less likely to blame the organization for these negative encounters (see [6, 7]). Hence, exposure to bullying behaviors may have more negative outcomes for those targeted despite the presumably lower prevalence compared to outsider aggression. Following this, we want to investigate the relationship between aggression from patients or next of kin and exposure to bullying behaviors at work in terms of whether these stressors appear separately or overlap in terms of targets, hence being perceived as a conglomerate experience. In this study, we are also interested in how prevalent these two stressors are. Based on previous research, we would expect aggression from patients or next of kin to be more commonly experienced than exposure to bullying behaviors. Furthermore, based on previous research and the stressors’ difference in nature of sustained exposure (see also CATS: ), we would expect exposure to bullying behaviors to have more severe health-related outcomes for those targeted compared to aggression from patients or next of kin. The following hypotheses will be investigated.
Hypothesis 1. Among nurses, aggression from patients or next of kin is more commonly experienced than workplace bullying.
Hypothesis 2. Exposure to bullying behaviors is more strongly related to negative health-related quality of life outcomes than aggression from patients or next of kin.
The study is based on cross-sectional data from the Survey of Shift work, Sleep and Health (SUSSH), collected in 2008-2009 among members of the Norwegian Nurses Organization (NNO). A recommendation letter of the study from NNO, an information letter explaining the purpose of the study, a questionnaire, and a prepaid envelope for return were sent to all participants’ home address.
An invitation to participate in the study was initially sent to 6000 Norwegian nurses. However, due to a number of wrong addresses, the invitation was received by 5400 nurses, of whom 2059 responded (38.1% response rate). The sample consisted of 1857 (90.2%) women and the mean age was 33.1 years (SD = 8.17), ranging from 21 to 63 years.
Nine items from the Negative Acts Questionnaire (NAQ; ) were used to measure exposure to bullying behaviors at work. This short NAQ instrument  measures negative behaviors which may be perceived as bullying if systematically repeated over time, that is, () “someone withholding information which affects your performance,” () “repeated reminders of your errors or mistakes,” () “persistent criticism of your work or work-effort,” (4) “spreading of gossip and rumors about you,” () “having insulting or offensive remarks made about your person, attitudes or your private life,” () “being ignored or excluded,” () “being ignored or facing a hostile reaction when you approach,” () “practical jokes carried out by people you do not get along with,” and () “being shouted at or being the target of spontaneous anger.” Respondents were asked how often, during the last six months, they had been exposed to such negative behaviors at their workplace with responses scored on a 5-point scale from 1 = “Never” to 5 = “Daily.” Cronbach’s alpha for this scale was .75.
Aggression from patients or next of kin was measured with three items asking how often, during the last 6 months, the respondents had been exposed to (1) “difficult patients or next of kin,” (2) “verbal assaults or spontaneous tantrums from patients or next of kin,” and (3) “physical assaults or threats of such assaults from patients or next of kin.” Responses were given on a 5-point scale from 1 = “Never” to 5 = “Daily.” Cronbach’s alpha for this scale was .81.
Health-related quality of life outcomes were measured with the SF-12v2. Two summary scores were used, one Mental Component Summary Score (MCS) and one Physical Component Summary Score (PCS), which have been calculated on an algorithm from the user’s manual for the SF-12v2® health survey (range: 0–100) . High scores on these component summary scores are positive in terms of good health-related quality of life.
Age and gender were included as control variables as these factors have been related to differences in exposure to violence at work .
3.4. Ethical Approval
The study was approved by the Regional Committee for Medical and Health Research Ethics, Health Region West (REK-Vest). Moreover, participation was voluntary, and the respondents were allowed to resign from the study at any time. Also, they had to provide informed consent in written form before being included in the study.
3.5. Analytic Strategy
Following recent trends in the measurement of workplace bullying, and in particular the identification of targets, we used latent class (LC) analysis in the present study to determine the relationship between aggression from patients or next of kin and exposure to bullying behaviors. LC analysis is very suited because it is not dependent upon distributional assumptions [47, 48]. LC analysis [49–53] is a statistical method that systematically classifies respondents into mutually exclusive groups with respect to a given trait (e.g., exposure to workplace bullying) that is not directly observed (manifest). The classes are not directly observable; they are latent . Furthermore, one empirically investigates whether the assumption about the relationship between the latent variable (e.g., experiencing bullying/aggression) and the frequencies of reported behaviors (e.g., negative social behaviors/aggressive behaviors) is acceptable . LC analysis enables the researcher to identify mutually exclusive groups that adequately describe the dispersion of observations in the -way contingency table of discrete variables (i.e., workplace bullying/aggression). The goal of LC analysis is to determine the smallest number of latent classes, sufficiently explaining (or accounting for) the associations observed between the manifest variables (the twelve items in our study) .
The starting point for a latent class model is homogeneity; that is, everybody resides in the same group. This model is a 1-latent class cluster (LCC) model. Instead of increasing the number of latent class clusters, however, the number of latent variables (factors) may be increased as well. Magidson and Vermunt  label this type of LC models as latent class factor (LCF) models because of the natural analogy to standard factor analysis. This would follow the assumption that aggression from patients or next of kin and exposure to bullying behaviors are different constructs for the nurses included in this study. Like with traditional confirmatory factor models, a priori knowledge about the relationship between items and latent variables is needed . As in traditional reflexive measurement models, the discrete latent variable must adequately explain the initial relationship between the indicators.
For the examination of the relationships between aggression from patients or next of kin and exposure to bullying behaviors, the software package Latent Gold 5 was used . Evaluating the fit of LC models is not straightforward. There are many indicators of fit and rules of thumb that should be taken into account. The Bayesian Information Criterion (BIC) is most often used, and among others McCutcheon  and Hagenaars  suggest accepting the model with the lowest BIC. However, for very sparse tables, it is likely that the squared log-likelihood () will not follow a chi-squared () distribution. Therefore, Langeheine et al.  suggested a bootstrapping procedure that was implemented in Latent Gold. This procedure is used here because bullying and aggression items are highly skewed and many combinations of specific negative acts are uncommon, which results in very sparse tables. This bootstrapping procedure is used in two ways. Firstly, it is used to assess the overall fit of a table where values equal to or higher than 0.05 indicate overall fit. Secondly, it is used to determine whether adding latent classes leads to an improvement of fit. Hence, the difference in −2LL with a certain difference in degrees of freedom is bootstrapped. A nonsignificant value indicates that the model does not improve with an additional class, whereas a significant value (<0.05) conversely indicates an improvement of fit. In addition to the BIC and the bootstrapping procedure, it is also important to inspect the bivariate residuals (BVR). The BVR show how much association between each pair of indicators remains, using the 1-cluster model as a reference. Ideally, the value should be lower than 3.84, being a value which corresponds to significant with 1 degree of freedom . However, as follows distribution, the BVR is also quite sensitive for large sizes. Therefore, we suggest using a more relative threshold, where the reduction of the BVR should be at least 90% . Finally there are also more descriptive measures to assess fit. A much used approach is to inspect the proportion reduction in . Being a global fit measure, the proportion reduction of error (PRE) expresses how much has been reduced, compared to the 1-cluster model, which serves as a baseline model. Furthermore, we also assessed how well classes were separated and inspected , entropy , and the total rate of classification errors due to adjacent erroneous classification.
After the measurement model was established, the latent class classifications were exported to a SPSS file . A multivariate analysis of variance (MANOVA) was conducted in order to simultaneously compare health-related quality of life outcomes, measured by one physical health component summary score and one mental health component summary score (SF-12v2), of aggression from patients or next of kin and exposure to bullying behaviors. This was done in order to avoid a type I error inflation because of a multiple testing procedure that would arise from repeating independent analyses of variance (ANOVAs).
4.1. Selection of Appropriate Measurement Model
Table 1 gives an overview of the different measurement models estimated with Latent Gold 5 and their respective fit measures. In particular, we evaluated fit with multiple measures that are the Bayesian Information Criterion (BIC) based upon the log-likelihood, the number of parameters in the statistical model ( par), the squared log-likelihood (), the proportion reduction of error, which approximates the explained variance of the model, the bootstrapped value of the squared log-likelihood, and, finally, the percentage of adjacent classification errors.
|Note. BIC: Bayesian Information Criterion; par: number of parameters in the statistical model; : squared log-likelihood; PRE: proportion reduction of error; Class. err.: classification errors. Models are divided into a one-factor model, bifactorial models with different numbers of classes, and a final model with two factors including five classes each, where the error correlation between items 1 and 10 is .20. The model with the best fit is presented in bold.|
As indicated above, establishing fit of a latent class model is complicated, and different measures must be assessed. Among the factor models with no error correlation, the factor model which distinguished 6 latent classes for workplace bullying and 5 latent classes for aggression from patients or next of kin had the lowest BIC. However, the bootstrap of indicated that the model was significant. The bootstrapped difference test, which establishes whether it is statistically sound to add classes, yielded clearly that a 2-factor model, modelling 5 levels for each factor, was fitting less well. Indeed, distinguishing more latent classes like 6 classes for workplace bullying was associated with a nonsignificant bootstrapped −2LL difference test, which thus yielded deterioration of fit. The inspection of the bivariate residuals of the model with 5 levels for each factor showed that within the factors themselves the initial bivariate residuals were sufficiently explained. However, several bivariate relationships between aggression and bullying indicators were not explained for more than 90%, indicating a potential need for allowing cross-loadings instead of adding latent classes . Hence, we allowed the first item of the bullying measure (i.e., someone withholding information which affects your performance) to correlate () with the first item of the aggression measure (i.e., difficult patients or next of kin). This model is depicted in the last row of Table 1.
Altogether the fit indices yielded that this 2-factor model, where each of the factors has 5 different (ordered) classes, respectively, was the most suited model. More specifically, the BIC was the lowest and, importantly, the value of the bootstrap procedure was nonsignificant (), which indicates that this model fitted the data well. We note that a single latent class factor model did not fit the data better than a 2-latent-class-factor model. Furthermore, the correlation between the factors was only 0.28, which indicates that aggression from patients or next of kin and exposure to bullying behaviors were only weakly related and thereby not reflecting the same concept. Hence, these stressors are seen as different construct in the eyes of the beholder, in our case nurses. All conditions mentioned above support the notion that these stressors can be regarded as two separate factors.
4.2. Meaning of the Latent Classes of Aggression and Workplace Bullying
Conditional probabilities allowed us to depict the precise meaning of the latent classes. Given that we used five response categories for each indicator, the table should have been at least sixty rows (twelve items with 5 response alternatives each) by 10 columns long (2 factors with 5 different classes each). We chose to simplify this table by portraying the conditional means in Table 2. These means are the average score to an item given the latent class membership (for a more detailed view, please consult the first author). Additionally, the size of the different classes for each factor is presented in percentages on the first row.
|Note. Exposure to bullying behaviors is measured by the Negative Acts Questionnaire (NAQ).|
The exposure to bullying behaviors factor consisted of 5 latent classes. In the first class, hardly any exposure to bullying behaviors was reported, leaving us with the label of “never bullied” for this class. In the second class, there was a slightly higher frequency of bullying behaviors reported, although it was still low, and we decided to label this class “hardly any bullying behaviors.” In the third class, the average of the nine items increased further but was still below 2. Therefore, we suggest labeling this class “rarely bullied.” In the fourth class, average of exposure points to occasional reports of bullying behaviors, indicating that “occasionally bullied” is a suitable label for this class. Finally, in the fifth class, all averages were above 2, except for the sixth item (i.e., being ignored or excluded), indicating that the frequency of bullying behavior was highest in this class. Thus we suggest labeling this class “frequently bullied.” Meanwhile 50% were not exposed at all to any bullying behaviors at work, 6% were hardly exposed to bullying behaviors, 29% were rarely exposed to bullying behaviors, 13% were occasionally exposed, and, finally, 2% were more frequently exposed to bullying in the current sample (Table 2, first row).
With respect to aggressive behaviors from patients or next of kin, the latent class factor also consisted of 5 latent classes. In the first class, hardly any exposure was reported. This class was therefore labeled “never exposed to aggression.” In the second class, aggressive behavior was rarely reported expect for the first item where respondents on average claimed to have experienced it occasionally; hence, we labeled this class “rarely exposed to aggression.” In the third class, the average of both the first item (i.e., difficult patients or next of kin) and the second item (i.e., verbal assaults or spontaneous tantrums from patients or next of kin) was approximately 2, being occasionally experienced. Therefore, we suggest labeling this class “occasionally exposed to aggression.” In the fourth class, the average of these two items increased above 3. Hence, the respondent reported monthly aggression from patients or next of kin, leaving us with the label “monthly exposed to aggression” for this class. In the last class, labeled “weekly exposed to aggression,” the conditional average of aggression points to weekly reports of aggression from patients or next of kin. It is clear that the frequency of aggression was higher in this class. Almost 6% of the respondents reported weekly exposure to aggressive behaviors. In addition, 13% reported experiencing aggressive behaviors monthly. Almost 48% were occasionally exposed to aggression. Furthermore, 12% reported rare exposure to aggression, while only 21% reported no exposure to aggression at all (Table 2, first row). Hence, the first hypothesis was supported, as aggression from patients or next of kin was more commonly experienced by nurses than exposure to bullying behaviors.
4.3. Health-Related Outcomes of Aggression and Workplace Bullying
To compare the potential outcomes of aggression from patients or next of kin and exposure to bullying behaviors on physical and mental health-related quality of life, a MANOVA was used, adjusting for age and gender. Age was the only control variable with significant relations to the health-related outcome measures. There was a statistically significant difference between the different bullying classes on the combined dependent variables: (6, 3784) = 11.85; = 0.000; Wilk’s Lambda = .96; partial eta squared = .02. When considered separately with ANOVAs, the only significant difference was found for the Mental Component Summary Score (MCS) (see Table 3). An inspection of the estimated marginal mean scores for MCS among the bullying classes, adjusted for age and gender, indicated that respondents in the “frequently bullied” class reported the lowest level of mental health-related quality of life (M = 39.47; SE = 1.70), as compared to the other classes.
|Note. PCS: Physical Component Summary Score; MCS: Mental Component Summary Score.|
Regarding aggression from patients or next of kin, there was no significant difference on the combined dependent variables: F(8, 3784) = 1.43; p = 0.18; Wilk’s Lambda = .99; partial eta squared = .000, indicating no difference between the different aggression classes in relation to reported physical and mental health-related quality of life. Hence, the second hypothesis was supported, as exposure to bullying behaviors seems to have more severe health-related outcomes for those targeted than aggression from patients or next of kin.
Furthermore, the interaction effect between aggression from patients or next of kin and exposure to bullying behaviors on the combined dependent variables was not significant: F(24, 3784) = 1.02; p = 0.44; Wilk’s Lambda = .99; partial eta squared = .01, indicating that the relationship between exposure to bullying behaviors and health-related quality of life was not dependent on the presence of aggression from patients or next of kin, or vice versa.
The aim of the present study was to compare exposure to aggression from patients or next of kin and exposure to bullying behaviors in terms of prevalence and health-related quality of life outcomes. Furthermore, we wanted to determine if these stressors are part of a conglomerate experience or if the nurses’ tendencies to report aggression and bullying differ and thereby being unrelated. Firstly, a two-latent-class-factor solution fitted better the data than a latent class cluster solution. Secondly, the correlation between the two factors was rather small. This indicates that these stressors were not intertwined phenomena constituting a conglomerate experience by the nurses. Furthermore, both aggression from patients or next of kin and exposure to bullying behaviors came in many levels of frequency with only a small minority of nurses experiencing systematic and frequent exposure. With respect to prevalence, aggression was by far the most common stressor and its reported frequency was also higher, supporting H1. However, only exposure to bullying behaviors was related to negative health-related outcomes in terms of reduced mental health-related quality of life, supporting H2. The latter finding strengthens previous research findings indicating that insider violence, in our case workplace bullying, is a major workplace distress factor which is more negatively related to health-related outcomes than is outsider aggression [7, 25].
5.1. Aggression Versus Workplace Bullying
In the present study, a two-factor model with five levels for each factor had the best fit for the data, with aggression from patients or next of kin being far more prevalent than exposure to bullying behaviors from colleagues. Nurses spend most of their working time with patients which may contribute to higher prevalence of aggression from patients in this occupation compared to other health professional groups . Illustrative of this, the result in the present study corresponds with previous research comparing the prevalence of insider and outsider violence among nurses. In Farrell and colleagues’  study of Australian nurses, 60% reported being exposed to verbal abuse. Within this group, 74% reported verbal abuse from patients/clients, while 29% reported verbal abuse from colleagues. On the other hand, 24% of the total sample reported being exposed to physical abuse. Within this group, 97% had experienced physical abuse from patients/clients, while only 4% reported physical abuse from colleagues. Hence, outsider abuse was by far more prevalent than was insider abuse.
With regard to aggression from patients or next of kin, the results of the present study showed that 21% had a probability of reporting no aggression, while 6% were likely to report weekly exposure. Previous studies have shown that exposure to violence among nurses seems to vary between different European countries. In a study by Estryn-Behar and colleagues , nurses in Belgium (23%), Germany (28%), the UK (29%), and France (39%) reported the highest rates of frequent exposure (monthly or more), while nurses in Norway (9%) reported the lowest rates of violent episodes from patients and relatives.
With regard to exposure to bullying behaviors, the results in the present study showed that the probability of being part of the target class was 2%. On the other hand, the probability of being a nontarget was 50%. Parallel research has been done on the latent class cluster structure of NAQ in Great Britain  and Spain . Despite differences in the numbers of clusters in these studies, the probability of not being a target of bullying was approximately the same (i.e., 30%), as well as the probability of being part of the respective severe bullying cluster (i.e., 5%). However, these studies showed slightly higher prevalence of bullying exposure than the present study did.
An explanation of the difference in prevalence between aggression and bullying may be the fact that patients more often behave in an aggressive manner against nurses as they are probably more likely to get away with it while venting anger and obtaining rewards in the process . The target may also have less problems confronting and reporting these kinds of attacks as aggression from patients in many cases is seen as part of the profession  and therefore is not related to the nurse personally. Being bullied, on the other hand, is shown to be related to self-blame and shame responses in the nurses , and one can question whether these negative behaviors have been underreported in the present study. However, the lower prevalence of aggression and bullying at work in Norway, compared to other European countries, may also be explained by a more egalitarian and feminist culture which may indicate less tolerance for aggressive behaviors and power abuse in these workplaces (e.g., ). Also, a combination of cold climatic conditions and high national wealth may create a culture of cooperation with generally low levels of harassment . Norwegian employees are furthermore protected by the Norwegian Work Environment Act, which bans harassment to occur in the workplace and protects the employees from inappropriate behaviors and stress . Moreover, the Labor Inspectorate serves as a controlling authority, by giving advice, requiring investigation and the development of policies and procedures, and engaging in appropriate activity to solve the problem . Hence, Norwegian employees may be in a position which protects them against high levels of violence at work, including both aggression and bullying, although it must be mentioned that the legal frameworks do not necessarily work in practice as intended.
5.2. Exposure to Bullying Behaviors and Health
Neither aggression from patients or next of kin nor exposure to bullying behaviors at work was related to nurses’ perceived physical health-related quality of life. However, the results showed that being exposed to bullying behaviors was related to reduced mental health-related quality of life. These findings correspond with earlier research in a wide array of samples showing that, generally, bullying is a predictor of mental health problems over and above other work-related stressors (e.g., [71–75]).
Albeit the literature comparing health-related outcomes of bullying from colleagues and aggression from patients specifically is scarce, there are some studies comparing aggression from coworkers with aggression from patients, which to some degree parallels the result in the present study. In an Italian study among nursing students and nurses, the results showed that both physical violence and nonphysical violence from colleagues had a stronger relationship with psychological problems than patient- and relatives-initiated violence . In LeBlanc and Kelloway’s study , the results showed that public-initiated violence and aggression were differently associated with personal and organizational outcomes as compared to coworker-initiated aggression. The former was mainly associated with intent to leave, while the latter was negatively related to emotional well-being, psychosomatic well-being, and affective commitment. Illustrative of this, Merecz et al.  reported that aggression from colleagues was a stronger predictor of negative changes in health among nurses, despite verbal aggression from patients being the most common type of aggression. Also, in the meta-analysis by Hershcovis and Barling  on workplace aggression and various work and health outcomes, coworker aggression had a stronger adverse relationship with physical well-being than outsider aggression. There were, however, no significant differences between the groups in terms of depression, emotional exhaustion, and general health. Hence, even though patient-initiated violence increases the level of psychological stress and anxiety in nurses , it seems like systematic nurse-to-nurse aggression is perceived as the most distressing of the two , as indicated in the present study regarding workplace bullying.
An essential distinction between the two stressors included in this study concerns the duration of the problem related to a given perpetrator or perpetrators, where bullying is a process carried out over time by the same perpetrator(s), while incivility or aggression by patients typically is of a more sporadic character. A relevant theoretical explanation for this difference may be found in the CATS theory , where prolonged activation, first cognitive and then physiological, increases the risk of reduced health. Hence, one-time occurrences or short-time exposure may not cause ill-health to the same degree, especially when the target is able to cope with the situation, even if it tends to happen again with other perpetrators. Furthermore, in a hospital setting, patients usually stay for a short period of time, and the continuous replacement of patients may contribute to the evaluation of patients as part of an out-group. In a study by Riva and Andrighetto , social punishment (e.g., being humiliated in front of colleagues or being excluded from a group of friends) was perceived as less painful when the target was an out-group member as compared to an in-group member. Hence, it may be that nurses do distance themselves from the patients and that the aggressive behaviors are perceived as less harmful than bullying from colleagues of whom the nurses more highly identify themselves with and whom they may depend on at a daily basis. Also, the patients’ illness may make their actions being perceived as less intentional. Illustrative of this, the results in a systematic review by Hahn and colleagues , among employees in general health care settings, indicated that the patients’ health status, like dementia, recovering from unconsciousness, drug withdrawal, and mental illness, inclined them to be aggressive towards the staff. This illustrates the fact that nurses may believe that patients do not intend to hurt them or that the assault is part of the nature of nursing work . Being exposed to bullying behaviors may, on the other hand, be evaluated as more insulting, because the nurses are not always in a position to withdraw from on-going interactions with colleagues and superiors  and thereby affect health negatively. When being targeted by members of the in-group, it may thus be harder to seek help, with impaired health as a possible outcome of prolonged worrying.
5.3. Methodological Considerations
This study has several strengths. First of all it uses latent class (LC) analysis to determine the relationship between aggression and workplace bullying, a method being very suited because it does not depend upon distributional assumptions [47, 48], which is often a problem in studies of bullying and aggression. Also, a MANOVA was conducted to determine health-related quality of life outcomes of aggression from patients or next of kin and exposure to bullying behaviors. Hence, we have likely avoided a type I error inflation because of a multiple testing procedure that would arise from repeating several ANOVAs. A strength is also the large sample size ().
However, there are some limitations that should be mentioned. As most respondents are females (90.2%), it may weaken our ability to generalize the results to other more gender balanced occupations. Furthermore, use of cross-sectional data hinders the possibility to draw causal explanations for the findings. Also, the response rate was somewhat low (38.1%), indicating that there is a possibility for selection bias to occur. Hence, those with health problems and experiences with exposure to abuse at work could possibly have chosen not to participate in the study, and thereby the results may have been affected. This may also be related to the “healthy worker effect,” where healthy workers have a higher possibility of attending work than those who are sick . When the data are based on self-reports, there is also an increased risk of common method variance . However, the behavioral nature of the items measuring exposure to bullying behaviors and aggression from patients or next of kin should reduce this problem somewhat through their more “objective” nature. One other issue concerns the assessment of aggression from patients or next of kin, where the item “difficult patients or next of kin” may not correspond with aggressive patients or next of kin per se. The reliability of the scale was, however, acceptable (α = .81), and the scale is assumed to be a measure of patient aggression, although we acknowledge that better framing of this particular item could have been more appropriate.
To our knowledge, this study is one of the first to simultaneously compare the nature and health-related outcomes of patient aggression and workplace bullying in the nursing profession. The results suggest that aggressive behaviors from patients or next of kin and exposure to bullying behaviors are independent experiences and are seen as different constructs in the eyes of the beholder. Although aggression from patients or next of kin turned out to be more prevalent than exposure to bullying behaviors, the latter was the only one related to negative health-related outcomes among those targeted, in terms of reduced mental health-related quality of life. Hence, workplace bullying, although few are highly exposed, has a more severe impact on nurses’ mental health than patient or next of kin aggression, at least in the present sample of Norwegian nurses. Information about the difference in how these stressors affect nurses’ health functioning may increase managers’ and nurses’ understanding of the importance of reducing these stressors at work and in particular exposure to workplace bullying in order to maintain the nurses’ good mental health functioning and well-being. Following this, the results in the present study suggest that intervention strategies should be implemented by health care management in order to generate and maintain a positive psychosocial work environment with zero-tolerance for bullying.
The authors declare that they have no competing interests.
The project generating the data used in the present study received a grant from the Western Norway Regional Health Authority (no personal payment/salary) and “NordForsk’s Nordic Program on Health and Welfare” to cover expenses related to practical administration and data collection. The Norwegian Nurses Organization also provided economic support to cover postal services related to mailing of questionnaires (no personal payment/salary). Also, the study is incorporated in the project “Workplace Bullying: From Mechanisms and Moderators to Problem Treatment” (250127), financed by the Research Council of Norway.
- S. Hahn, A. Zeller, I. Needham, G. Kok, T. Dassen, and R. J. G. Halfens, “Patient and visitor violence in general hospitals: a systematic review of the literature,” Aggression and Violent Behavior, vol. 13, no. 6, pp. 431–441, 2008.
- N. Magnavita and T. Heponiemi, “Workplace violence against nursing students and nurses: An Italian experience,” Journal of Nursing Scholarship, vol. 43, no. 2, pp. 203–210, 2011.
- M. Estryn-Behar, B. van der Heijden, D. Camerino et al., “Violence risk in nursing—results from the European ‘NEXT’ study,” Occupational Medicine, vol. 58, no. 2, pp. 107–114, 2008.
- V. Bowie, “Defining violence at work: a new typology,” in Violence at Work Causes, Patterns and Prevention, M. Gill, B. Fisher, and V. Bowie, Eds., pp. 1–20, Willian Publishing, Devon, UK, 2002.
- A. A. Grandey, J. H. Kern, and M. R. Frone, “Verbal abuse from outsiders versus insiders: comparing frequency, impact on emotional exhaustion, and the role of emotional labor,” Journal of Occupational Health Psychology, vol. 12, no. 1, pp. 63–79, 2007.
- M. S. Hershcovis and J. Barling, “Towards a multi-foci approach to workplace aggression: a meta-analytic review of outcomes from different perpetrators,” Journal of Organizational Behavior, vol. 31, no. 1, pp. 24–44, 2010.
- M. M. LeBlanc and E. K. Kelloway, “Predictors and outcomes of workplace violence and aggression,” Journal of Applied Psychology, vol. 87, no. 3, pp. 444–453, 2002.
- D. Merecz, M. Drabek, and A. Mościcka, “Aggression at the workplace - psychological consequences of abusive encounter with coworkers and clients,” International Journal of Occupational Medicine and Environmental Health, vol. 22, no. 3, pp. 243–260, 2009.
- D. Demir and J. Rodwell, “Psychosocial antecedents and consequences of workplace aggression for hospital nurses,” Journal of Nursing Scholarship, vol. 44, no. 4, pp. 376–384, 2012.
- L. Berkowitz, Aggression: Its Causes, Consequences, and Control, McGraw-Hill, New York, NY, USA, 1993.
- M. Hodgins, S. MacCurtain, and P. Mannix-McNamara, “Workplace bullying and incivility: a systematic review of interventions,” International Journal of Workplace Health Management, vol. 7, no. 1, pp. 54–72, 2014.
- H. Hoel, C. Rayner, and C. L. Cooper, “Workplace bullying,” in International Review of Industrial and Organizational Psychology, C. L. Cooper and I. T. Robertson, Eds., vol. 14, John Wiley & Sons, Chichester, UK, 1999.
- S. L. Johnson and R. E. Rea, “Workplace bullying: concerns for nurse leaders,” Journal of Nursing Administration, vol. 39, no. 2, pp. 84–90, 2009.
- S. Einarsen, H. Hoel, D. Zapf, and C. L. Cooper, “The concept of bullying and harassment at work: the European tradition,” in Bullying and Harassment in the Workplace Developments in Theory, Research and Practice, S. Einarsen, H. Hoel, D. Zapf, and C. L. Cooper, Eds., pp. 3–39, CRC Press, Boca Raton, Fla, USA, 2nd edition, 2011.
- H. Leymann, “The content and development of mobbing at work,” European Journal of Work and Organizational Psychology, vol. 5, no. 2, pp. 165–184, 1996.
- L. Pompeii, J. Dement, A. Schoenfisch et al., “Perpetrator, worker and workplace characteristics associated with patient and visitor perpetrated violence (Type II) on hospital workers: a review of the literature and existing occupational injury data,” Journal of Safety Research, vol. 44, no. 1, pp. 57–64, 2013.
- D. Zapf, J. Escartín, S. Einarsen, H. Hoel, and M. Vartia, “Empirical findings on prevalence and risk groups of bullying in the workplace,” in Bullying and Harassment in the Workplace Developments in Theory, Research and Practice, S. Einarsen, H. Hoel, D. Zapf, and C. L. Cooper, Eds., pp. 75–105, CRC Press, Boca Raton, Fla, USA, 2nd edition, 2011.
- W. Eriksen and S. Einarsen, “Gender minority as a risk factor of exposure to bullying at work: the case of male assistant nurses,” European Journal of Work and Organizational Psychology, vol. 13, no. 4, pp. 473–492, 2004.
- P. A. Berry, G. L. Gillespie, D. Gates, and J. Schafer, “Novice nurse productivity following workplace bullying,” Journal of Nursing Scholarship, vol. 44, no. 1, pp. 80–87, 2012.
- L. Quine, “Workplace bullying in nurses,” Journal of Health Psychology, vol. 6, no. 1, pp. 73–84, 2001.
- A. Yildirim and D. Yildirim, “Mobbing in the workplace by peers and managers: mobbing experienced by nurses working in healthcare facilities in Turkey and its effect on nurses,” Journal of Clinical Nursing, vol. 16, no. 8, pp. 1444–1453, 2007.
- K.-L. Edward, K. Ousey, P. Warelow, and S. Lui, “Nursing and aggression in the workplace: a systematic review,” British Journal of Nursing, vol. 23, no. 12, pp. 653–659, 2014.
- G. A. Farrell and T. Shafiei, “Workplace aggression, including bullying in nursing and midwifery: a descriptive survey (the SWAB study),” International Journal of Nursing Studies, vol. 49, no. 11, pp. 1423–1431, 2012.
- J. Rodwell, D. Demir, and R. L. Flower, “The oppressive nature of work in healthcare: predictors of aggression against nurses and administrative staff,” Journal of Nursing Management, vol. 21, no. 6, pp. 888–897, 2013.
- G. A. Farrell, “Aggression in clinical settings: nurses' views—a follow-up study,” Journal of Advanced Nursing, vol. 29, no. 3, pp. 532–541, 1999.
- M. L. Lanza, “Nurses as patient assault victims: an update, synthesis, and recommendations,” Archives of Psychiatric Nursing, vol. 6, no. 3, pp. 163–171, 1992.
- M. L. Lanza, “Reactions of nurses to a patient assault vignette,” Western Journal of Nursing Research, vol. 10, no. 1, pp. 45–54, 1988.
- T. J. Rippon, “Aggression and violence in health care professions,” Journal of Advanced Nursing, vol. 31, no. 2, pp. 452–460, 2000.
- N. Lanctôt and S. Guay, “The aftermath of workplace violence among healthcare workers: a systematic literature review of the consequences,” Aggression and Violent Behavior, vol. 19, no. 5, pp. 492–501, 2014.
- I. Needham, C. Abderhalden, R. J. G. Halfens, J. E. Fischer, and T. Dassen, “Non-somatic effects of patient aggression on nurses: a systematic review,” Journal of Advanced Nursing, vol. 49, no. 3, pp. 283–296, 2005.
- B. C. Allen, P. Holland, and R. Reynolds, “The effect of bullying on burnout in nurses: the moderating role of psychological detachment,” Journal of Advanced Nursing, vol. 71, no. 2, pp. 381–390, 2015.
- J. Rodwell and D. Demir, “Psychological consequences of bullying for hospital and aged care nurses,” International Nursing Review, vol. 59, no. 4, pp. 539–546, 2012.
- D. Yildirim, “Bullying among nurses and its effects,” International Nursing Review, vol. 56, no. 4, pp. 504–511, 2009.
- A. G. Harvey, “A cognitive model of insomnia,” Behaviour Research and Therapy, vol. 40, no. 8, pp. 869–893, 2002.
- J. F. Brosschot, W. Gerin, and J. F. Thayer, “The perseverative cognition hypothesis: a review of worry, prolonged stress-related physiological activation, and health,” Journal of Psychosomatic Research, vol. 60, no. 2, pp. 113–124, 2006.
- H. Ursin and H. R. Eriksen, “The cognitive activation theory of stress,” Psychoneuroendocrinology, vol. 29, no. 5, pp. 567–592, 2004.
- A. Rodríguez-Muñoz, G. Notelaers, and B. Moreno-Jiménez, “Workplace bullying and sleep quality: the mediating role of worry and need for recovery,” Behavioral Psychology, vol. 19, no. 2, pp. 453–468, 2011.
- J. C. Turner, “Towards a cognitive redefinition of the social group,” in Social Identity and Intergroup Relations, H. Tajfel, Ed., Cambrigde University Press, New York, NY, USA, 1982.
- J. E. Stets, “Identity theory,” in Contemporary Social Psychological Theories, P. J. Burke, Ed., pp. 88–110, Stanford University Press, Stanford, Calif, USA, 2006.
- R. F. Baumeister and D. M. Tice, “Point-counterpoints: anxiety and social exclusion,” Journal of Social and Clinical Psychology, vol. 9, no. 2, pp. 165–195, 1990.
- W. Zernike and P. Sharpe, “Patient aggression in a general hospital setting: do nurses perceive it to be a problem?” International Journal of Nursing Practice, vol. 4, no. 2, pp. 126–133, 1998.
- N. A. Bowling and T. A. Beehr, “Workplace harassment from the Victim's perspective: a theoretical model and meta-analysis,” Journal of Applied Psychology, vol. 91, no. 5, pp. 998–1012, 2006.
- S. Einarsen, H. Hoel, and G. Notelaers, “Measuring exposure to bullying and harassment at work: validity, factor structure and psychometric properties of the negative acts questionnaire-revised,” Work and Stress, vol. 23, no. 1, pp. 24–44, 2009.
- “The construction and validity of the Short-Negative Acts Questionnaire,” in Proceedings of the 6th International Conference on Workplace Bullying: Sharing our Knowledge, G. Notelaers and S. Einarsen, Eds., Ecole des Sciences de la Gestion. Université du Québec à Montréal, Montreal, Canadá, June 2008.
- J. E. Ware, M. A. Kosinski, D. M. Turner-Bowker, M. Sundaram, B. Gandek, and M. E. Maruish, User's Manual for the SF-12v2 (Health Survey), Quality Metric Incorporated, Lincoln, RI, USA, 2002.
- S. Lawoko, J. J. F. Soares, and P. Nolan, “Violence towards psychiatric staff: a comparison of gender, job and environmental characteristics in England and Sweden,” Work and Stress, vol. 18, no. 1, pp. 39–55, 2004.
- G. Notelaers, S. Einarsen, H. De Witte, and J. K. Vermunt, “Measuring exposure to bullying at work: the validity and advantages of the latent class cluster approach,” Work and Stress, vol. 20, no. 4, pp. 289–302, 2006.
- G. Notelaers, J. K. Vermunt, E. Baillien, S. Einarsen, and H. de Witte, “Exploring risk groups workplace bullying with categorical data,” Industrial Health, vol. 49, no. 1, pp. 73–88, 2011.
- J. K. Vermunt, “The use of restricted latent class models for defining and testing nonparametric and parametric item response theory models,” Applied Psychological Measurement, vol. 25, no. 3, pp. 283–294, 2001.
- J. K. Vermunt and J. Magidson, “Latent class cluster analysis,” in Applied latent class analysis, pp. 89–106, Cambridge Univ. Press, Cambridge, UK, 2002.
- J. Magidson and J. K. Vermunt, “Latent class factor and cluster models, bi-plots, and related graphical displays,” Sociological Methodology, vol. 31, no. 1, pp. 223–264, 2001.
- J. Magidson and J. K. Vermunt, “Latent class models for clustering: a comparison with K-means,” Canadian Journal of Marketing Research, vol. 20, pp. 37–44, 2002.
- J. Magidson and J. K. Vermunt, “Latent class models,” in The Sage Handbook for Quantitative Methodology, D. Kaplan, Ed., pp. 175–198, Sage, Thousand Oakes, Calif, USA, 2004.
- J. K. Vermunt, “Toepassingen van latent klasse analyse in sociaal wetenschappelijk onderzoek,” Sociale Wetenschappen, vol. 47, pp. 2–14, 2004.
- J. K. Vermunt and J. Magidson, “Factor analysis with categorical indicators: a comparison between traditional and latent class approaches,” in New Developments in Categorical Data Analysis for the Social and Behavioral Sciences, A. v. D. Ark, M. A. Croon, and K. Sijtsma, Eds., pp. 41–62, Erlbaum, Mahwah, NJ, USA, 2005.
- J. K. Vermunt and J. Magidson, Latent Gold, Statistical Innovations, Belmont, Mass, USA, 5th edition, 2013.
- A. C. McCutcheon, Latent Class Analysis, Sage, Beverly Hills, Calif, USA, 1987.
- J. A. Hagenaars, Categorical Longitudinal Data—Loglinear Analysis of Panel, Trend and Cohort Data, Sage, Newbury Park, Calif, USA, 1990.
- R. Langeheine, J. Pannekoek, and F. Van De Pol, “Bootstrapping goodness-of-fit measures in categorical data analysis,” Sociological Methods and Research, vol. 24, no. 4, pp. 492–516, 1996.
- Statistical Innovatations, Markov Tutorial #2: Latent GOLD Longitudinal Analysis of Life Satisfaction, Statistical Innovatations, 2013.
- N. De Cuyper, T. Rigotti, H. De Witte, and G. Mohr, “Balancing psychological contracts: validation of a typology,” The International Journal of Human Resource Management, vol. 19, no. 4, pp. 543–561, 2008.
- J. S. Uebersax, “A practical guide to conditional dependence in latent class models,” 2009, http://www.john-uebersax.com/stat/condep.htm.
- J. Vanderslott, “A study of incidents of violence towards staff by patients in an NHS Trust hospital,” Journal of Psychiatric and Mental Health Nursing, vol. 5, no. 4, pp. 291–298, 1998.
- G. A. Farrell, C. Bobrowski, and P. Bobrowski, “Scoping workplace aggression in nursing: findings from an Australian study,” Journal of Advanced Nursing, vol. 55, no. 6, pp. 778–787, 2006.
- J. M. Leon-Perez, G. Notelaers, A. Arenas, L. Munduate, and F. J. Medina, “Identifying victims of workplace bullying by integrating traditional estimation approaches into a latent class cluster model,” Journal of Interpersonal Violence, vol. 29, no. 7, pp. 1155–1177, 2014.
- D. M. Felblinger, “Incivility and bullying in the workplace and nurses' shame responses,” JOGNN—Journal of Obstetric, Gynecologic & Neonatal Nursing, vol. 37, no. 2, pp. 234–242, 2008.
- S. Einarsen, “Harassment and bullying at work: a review of the Scandinavian approach,” Aggression and Violent Behavior, vol. 5, no. 4, pp. 379–401, 2000.
- E. van de Vliert, S. Einarsen, and M. B. Nielsen, “Are national levels of employee harassment cultural covariations of climato-economic conditions?” Work and Stress, vol. 27, no. 1, pp. 106–122, 2013.
- Directorate of Labour Inspection, Act Relating to Working Environment, Working Hours and Employment Protection, etc. (Working Environment Act), Directorate of Labour Inspection, Trondheim, Norway, 2016.
- Nordic Bullying Network Group, State of the Art Report on Bullying at the Workplace in the Nordic Countries, Nordic Council of Ministers, 2011.
- L. B. Finne, S. Knardahl, and B. Lau, “Workplace bullying and mental distress—a prospective study of Norwegian employees,” Scandinavian Journal of Work, Environment and Health, vol. 37, no. 4, pp. 276–287, 2011.
- M. B. Nielsen, J. Hetland, S. B. Matthiesen, and S. Einarsen, “Longitudinal relationships between workplace bullying and psychological distress,” Scandinavian Journal of Work, Environment and Health, vol. 38, no. 1, pp. 38–46, 2012.
- M. Kivimäki, M. Virtanen, M. Vartia, M. Elovainio, J. Vahtera, and L. Keltikangas-Järvinen, “Workplace bullying and the risk of cardiovascular disease and depression,” Occupational and Environmental Medicine, vol. 60, no. 10, pp. 779–783, 2003.
- M. B. Nielsen and S. Einarsen, “Outcomes of exposure to workplace bullying: a meta-analytic review,” Work and Stress, vol. 26, no. 4, pp. 309–332, 2012.
- E. Lahelma, T. Lallukka, M. Laaksonen, P. Saastamoinen, and O. Rahkonen, “Workplace bullying and common mental disorders: a follow-up study,” Journal of Epidemiology and Community Health, vol. 66, no. 6, article e3, 2012.
- M. Bernaldo-De-Quirós, A. T. Piccini, M. M. Gómez, and J. C. Cerdeira, “Psychological consequences of aggression in pre-hospital emergency care: cross sectional survey,” International Journal of Nursing Studies, vol. 52, no. 1, pp. 260–270, 2015.
- P. Riva and L. Andrighetto, “‘Everybody feels a broken bone, but only we can feel a broken heart’: group membership influences the perception of targets' suffering,” European Journal of Social Psychology, vol. 42, no. 7, pp. 801–806, 2012.
- A. J. McMichael, “Standardized mortality ratios and the ‘healthy worker effect’: scratching beneath the surface,” Journal of Occupational Medicine, vol. 18, no. 3, pp. 165–168, 1976.
- P. M. Podsakoff, S. B. MacKenzie, J.-Y. Lee, and N. P. Podsakoff, “Common method biases in behavioral research: a critical review of the literature and recommended remedies,” Journal of Applied Psychology, vol. 88, no. 5, pp. 879–903, 2003.
Copyright © 2017 Iselin Reknes et al. This is an open access article distributed under the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.