Abstract
Cardiac arrest is a leading cause of mortality among firefighters. We sought to develop a valid method for determining the costs of a workplace prevention program for firefighters. In 2012, we developed a draft framework using human resource accounting and in-depth interviews with experts in the firefighting and insurance industries. The interviews produced a draft cost model with 6 components and 26 subcomponents. In 2013, we randomly sampled 100 fire chiefs out of >7,400 affiliated with the International Association of Fire Chiefs. We used the Content Validity Index (CVI) to identify the content valid components of the draft cost model. This was accomplished by having fire chiefs rate the relevancy of cost components using a 4-point Likert scale (highly relevant to not relevant). We received complete survey data from 65 fire chiefs (65% response rate). We retained 5 components and 21 subcomponents based on CVI scores ≥0.70. The five main components include, (1) investment costs, (2) orientation and training costs, (3) medical and pharmaceutical costs, (4) education and continuing education costs, and (5) maintenance costs. Data from a diverse sample of fire chiefs has produced a content valid method for calculating the cost of a prevention program among firefighters.
1. Introduction
Firefighters in the United States suffer cardiac-related line of duty death (LODD) at exceedingly high rates with events 10–100 times more likely to occur during, or in the hours immediately following, fire suppression [1]. The underlying mechanism for cardiovascular events among firefighters is unknown, but it is believed to be multifactorial including both occupational and lifestyle factors. Occupational factors such as shift work, lifestyle factors, and the exposures associated with fire suppression (e.g., smoke, chemicals) may predispose the firefighter to earlier onset of heart disease [2]. Additionally, rates of hypertension, overweight, and obesity within the fire service exceed those seen in the general population increasing the baseline risk for a cardiovascular event [3, 4].
Fire suppression activities result in significant cardiovascular strain and provide potential triggers for ischemic events (e.g., myocardial infarction, stroke). There is a rapid rise in heart rate following the activation of a fire department which may persist for as long as 20 minutes [5, 6]. Even in cases where heavy work is not being performed, the repetitive upper body exercise associated with tool use raises heart rate disproportionately to oxygen consumption [7]. Finally, the heat stress associated with fire suppression has been shown to enhance coagulation potential and activate platelets reducing platelet closure time [8–10].
Given the multiple risk factors and potential triggers for cardiovascular events associated with fire suppression, there is a tangible need for preventative measures among firefighters. Potential interventions include weight loss and fitness programming initiatives and pharmaceutical interventions. Two studies have been funded through the FEMA Fire Prevention & Safety mechanism to examine the role of daily or acute aspirin on platelet activation in firefighters. However, the cost and benefit for these types of programs have not been examined in great detail within the fire service. Lack of a valid method for quantifying costs with any type of intervention, fitness or pharmacological, may prevent program adoption amongst those most likely to benefit from a prevention program. In this study, we sought to create a valid method for quantifying the costs of programs that aim to reduce firefighter mortality related to on-duty cardiac events.
2. Methods
We used a mixed-method approach approved by the University of Pittsburgh Institutional Review Board that included qualitative in-depth interviews, a cross-sectional survey, and validity calculation. First, in 2012, we performed in-depth interviews with four experts. The experts included a convenience sample of fire department chiefs, mid-level department administrators, and a representative of insurance companies. We included a representative from the insurance industry in order to identify the principal components of cost that an insurance company might cover for a fire department. We recruited these experts from a convenience sample of experts listed in an internally maintained database of fire industry contacts who have participated in previous fire safety research.
Our in-depth interviews were guided by theories of costs estimation accepted in Human Resource Accounting (HRA), which includes direct and indirect costs [11]. Direct costs refer to materials, goods, and other tangibles commonly listed on balance sheets and budgets as reoccurring or one-time purchases with capital from the organization. Indirect costs refer to the time that organizational staff devotes to specific components of their occupation. We chose to follow a HRA framework based on evidence of “real world” utility and application of previously developed HRA-based cost models [12–16]. We used HRA to frame our interview questions, which were designed to identify common themes/components of costs considered important and relevant by industry experts. For example, we asked expert interviewees to consider and comment on the amount of time employees may devote to activities such as exercising and continuing education while on-duty and the costs that a fire department may incur as a result. This is an example of indirect costs [12–16]. We ended in-depth interviews once the interviewer declared saturation [17]. Saturation refers to the point when the interviewer determines that interviewees are no longer expanding the range of information on a cost component or offering new information beyond that already provided.
Next, in 2013, we randomly sampled 100 fire chiefs from the International Association of Fire Chiefs (IAFC) database of 7,461 fire chiefs located within the USA and internationally. We then contacted each fire chief via a mailed recruitment packet. The packet included (1) a description of the study, (2) letter of support from the National Fallen Firefighters Foundation (NFFF), and (3) a $5.00 gift card. The recruitment letter directed fire chiefs to a study web site where they rated the relevancy of each component of the draft method for calculating costs. We obtained ratings with a four-point Likert scale anchored from “Highly Relevant” to “Not Relevant.”
Finally, we calculated the content validity of fire chief ratings using the widely accepted Content Validity Index (CVI). The CVI is proportion of raters assigning a rating of 3 = quite relevant or 4 = highly relevant [18]. Two CVI scores may be calculated, one for components (the Scale-CVI or S-CVI) and one for subcomponents (the Item-CVI or I-CVI) [18]. We calculated S-CVI and I-CVI scores based on techniques developed by Lynn, Waltz, and Bausell with S-CVI and I-CVI scores as benchmarks for content validity in this study [18]. We then stratified subcomponents into direct and indirect costs.
3. Results
In-depth interviews with our four experts resulted in a draft method for cost calculation that included six components and 26 subcomponents. Components included (1) preparatory costs, (2) investment costs, (3) orientation and training costs, (4) medical care and pharmaceutical costs, (5) education and continuing education costs, and (6) maintenance costs. Figure 1 is a graphical illustration of this draft model for cost calculation.
We received 65 completed surveys from fire chiefs (65% response rate). The S-CVI score for preparatory costs (0.66) fell below the 0.70 benchmark suggesting that fire chiefs do not believe this component should be considered a valid part of cost calculation (Table 1). Content validity scores showed that two additional subcomponents fell below the 0.70 cut point for I-CVI. These include one subcomponent of direct costs for the Orientation & Training Costs component and one subcomponent of Education & Continuing Education component.
In total, greater than 70% of fire chiefs surveyed in this study found five of the original six components of costs and 21 of the original 26 subcomponents to be content valid. CVI scores were highest (quite relevant to highly relevant) for the medical and pharmaceutical Costs component, suggesting most fire chiefs were supportive of this component and its subcomponents. Table 2 presents a final draft of the method for calculating costs of a firefighter program to reduce on-duty related cardiac events.
4. Discussion
Risk of morbidity and mortality from cardiovascular events in the line of duty is high among firefighters [1]. Fire chiefs, fire department insurers, and others share a common concern for firefighter health, wellness, and safety. There are numerous potential solutions for reducing risk; however, the costs associated with these solutions or programs are unknown and likely vary by fire department. We have developed a model to aide fire chiefs that can support the calculation of cost associated with the implementation and maintenance of a program to prevent cardiovascular events or reduce the cardiovascular-related morbidity and mortality among firefighters. Our model represents a starting point from which fire chiefs and others can begin. Application of this model to a candidate program or intervention is required to fully test model efficacy. Lack of data from any actual model application precludes us from evaluating select measures of reliability, construct validity, and predictive validity. The principal benefit of this model is its validity. The components and subcomponents of this model were developed based on a widely accepted framework for cost estimation and calibrated to fit the fire industry with input from 65 randomly sampled fire chiefs.
Like those before us [12–16], we began by developing a framework for cost calculation based on the Human Resource Accounting (HRA) approach. We have demonstrated success with this approach in the past by developing a model for determining the cost associated with turnover in Emergency Medical Services (EMS) organizations [19]. Others have applied the HRA approach as well, which we believe is supportive evidence for HRA’s value in development of cost calculation modeling [12–16]. However, we believe that no one approach is optimal and acknowledge that additional research is needed to: (1) fully expose known and unknown factors associated with program cost using different frameworks, (2) to determine feasibility in application in organizations to which they were designed; and (3) to document the perceived utility among the administrators that will use these models.
The value of our work at this time is supported by the growing body of evidence that cardiovascular events are the most common cause of firefighter line-of-duty death and believed to be higher than other professions [1]. We have previously shown that, in one large community, non-fatal cardiac events occur in similar proportions for fire, police, and EMS making this tool widely applicable across public safety [20]. Improving firefighter fitness is a key step in reducing the number of job-related cardiovascular events [21]. However, there are exposures and stressors associated with fire suppression that cannot be mitigated by fitness alone [2]. Therefore, it is reasonable for the fire service to pursue both fitness related programs and other potential solutions such as technology (e.g., physiologic monitoring, enhanced respiratory protection) and pharmacotherapy to reduce the number of cardiovascular events in public safety providers.
Our work will also help expedite future work employing cost-benefit, cost-effectiveness, and related types of economic valuation and analysis techniques. These techniques require multiple steps, including the identification and gathering of data to value costs of diverse decision options [22]. Experts in methods for quantitative synthesis point out that the single most common thread among all techniques is cost valuation or the determination of costs [22]. Given its importance, the overarching goal of cost-effectiveness analysis—to compare the outcome of different decision options in terms of their costs—is not achievable without a comprehensive identification of costs, including direct and indirect costs [22]. In this study, we have demonstrated a focused effort to comprehensively identify the contributors to the cost of a prevention program for firefighters. Our study has addressed a gap not yet filled by the literature and completed a fundamental step to cost analyses that will accelerate future efforts to reduce firefighter morbidity and mortality.
We recognize that many recommendations exist for improving firefighter health and safety. Reports in the peer-reviewed literature have examined the utility of standard health surveillance [23], advanced imaging [24–26], and the potential need for enhanced respiratory protection following fire suppression [27, 28] to mitigate the risk of cardiovascular disease among firefighters. However, few studies have examined the costs associated with these programs [24]. Additionally, there are two trials of aspirin therapy in firefighters to reduce heat stress-induced platelet activation (http://www.clinicaltrials.gov). It is likely that a clinically viable program will be identified to improve the health and safety of the public safety provider. However, given limited resources in a public safety organization, the costs associated with new programs must be balanced against the existing operations’ needs of that organization.
5. Limitations
Our study was limited by the sample of key informants, experts, and stakeholders interviewed and the 65 fire chiefs that responded to our survey. There is reason to believe that additional elements costs may have surfaced with added interviews and responses from additional fire chiefs. However, our sample was selected at random and more than half of fire chiefs surveyed provided input on the relevancy (content validity) of the method for cost calculation.
6. Conclusions
The cost calculation model presented in this paper represents formative work towards calculating the costs associated with programs and interventions led by fire department administrators targeting firefighters. We believe this model is a starting point for prevention in a higher risk industry and has broad applicability for industries with low penetration of prevention programs. While formative, decision makers may use this model as a framework for determining cost of prevention programs.
Ethical Approval
This study was approved by the University of Pittsburgh Institutional Review Board (IRB).
Conflict of Interests
The authors declare that there is no conflict of interests regarding the publication of this paper.
Acknowledgments
This study was funded by FEMA Assistance to Firefighters Grant, Fire Prevention & Safety EMW-2008-FP-01638. Dr. Patterson is also supported by the National Center for Research Resources and the National Institutes of Health through Grant no. 2KL2 RR024154-06. Drs. Patterson, Hostler, Suyama, and Reis conceived the concept and idea for the paper/study. Dr. Patterson produced the first draft of the paper. Dr. Patterson and Mr. Weaver analyzed study data and synthesized findings and results. Drs. Patterson, Hostler, Suyama, Reis, and Mr. Weaver edited the first draft paper and edited multiple drafts before arriving to the final draft for consideration of publication. The authors would like to acknowledge assistance from the National Fallen Firefighters Foundation (NFFF) and International Association of Fire Chiefs (IAFC). This study would have not been possible without their support.