Abstract

Background. Several international groups develop asthma guidelines. Conflicting recommendations across guidelines have been described in several disease areas and may contribute to practice variability. Accordingly, we compared the latest Canadian Thoracic Society (CTS) asthma guideline with contemporaneous international asthma guidelines to evaluate conflicting recommendations and their causes. Methods. We identified the latest CTS asthma guideline update (2012) and the following societies which also updated their guidelines in 2012: the British Thoracic Society and Scottish Intercollegiate Guidelines Network and the Global Initiative for Asthma. We compared these three guidelines on (1) key methodological factors and (2) adult pharmacotherapy recommendations. Results. Methods used and documentation provided for literature search strategy and dates, evidence synthesis, outcomes considered, evidence appraisal, and recommendation formulation varied between guidelines. Criteria used to define suboptimal asthma control varied widely between guidelines. Inhaled corticosteroid dosing recommendations diverged, as did recommendations surrounding use of budesonide/formoterol as a reliever and controller and recommendations in the subsequent step. Conclusions. There are important differences between recommendations provided in contemporaneous asthma guidelines. Causes include differences in methods used for interpreting evidence and formulating recommendations. Adopting a common set of valid and explicit methods across international societies could harmonize recommendations and facilitate guideline implementation.

1. Introduction

Asthma is the third most common chronic disease in adults, affecting 8.1% of the population, or 2.4 million Canadians, and increasing in prevalence [1, 2]. Annually, 150 000 emergency room visits and 60 000 hospital admissions in Canada are attributable to asthma [2], with an economic burden of $1.8 billion in 2011 [3] and a disproportionate cost attributable to patients with poorly controlled symptoms [4].

Effective therapies for asthma exist and well-controlled asthma is achievable in most patients. Several international groups have developed and regularly update evidence-based guidelines aimed at improving asthma control and outcomes. The Canadian Thoracic Society (CTS) published the first Canadian Asthma Guideline in 1990, the US National Heart, Lung and Blood Institute (NHLBI) first issued an asthma guideline in 1991, the Global Initiative for Asthma (GINA) has produced guidelines since 1995, and the British Thoracic Society (BTS) and Scottish Intercollegiate Guidelines Network (SIGN) produced their first joint British guideline in 2001.

However, despite these high-quality guidelines, over half of the patients with asthma remain poorly controlled [58]. This has been found to be attributable both to poor patient adherence and to gaps between guideline-recommended care and actual care, across jurisdictions [7, 9, 10]. In fact, the existence of multiple guidelines may result in conflicts between guideline recommendations which may contribute to practice variation and possibly even to poor overall guideline uptake [1113]. Conflicting recommendations across guidelines have been described in areas as diverse as cancer care (e.g., non-small cell lung cancer) [14], cancer screening (e.g., prostate [15] and breast cancer screening [16]), periodic health exam (e.g., adolescent care [17]), and several chronic diseases, including diabetes mellitus [18] and chronic obstructive pulmonary disease (COPD) [19]. These conflicting recommendations have been shown to cause confusion and frustration among users [11, 12, 19] and to decrease clinician and public trust of guidelines [11, 20, 21]. Accordingly, interguideline conflict may be an incremental contributor to poor guideline adherence.

Given the important consequences of conflicting guideline recommendations, we sought to perform a comparative analysis of the latest Canadian Asthma Guideline with contemporaneous asthma guidelines from around the world.

2. Methods

We identified the latest CTS asthma guideline, published in April 2012 [22], and the following societies which also published/updated their guidelines in 2012: the BTS/SIGN (January 2012) [23] and GINA (December 2012) [24]. All three are well-respected international guideline groups which provide information on their guideline production methodologies [2426]. It should be noted that although the GINA group refers to its product as a “document” or “resource,” it is used as a guideline for all intents and purposes.

We compared these guidelines on the following key methodological factors: literature search strategy and dates; evidence synthesis; outcomes considered; evidence appraisal; recommendation formulation; and overlaps in guideline committee memberships.

Next, we analyzed and compared adult pharmacotherapy recommendations provided in each guideline, in the following three areas: (1) criteria for controller initiation and escalation (asthma control criteria); (2) recommendations for therapeutic escalation of baseline medications (for suboptimal control) (“stepping up” therapy); and (3) recommendations for therapeutic deescalation of baseline medications (after achieving good control) (“stepping down” therapy). We did not compare nonpharmacotherapy recommendations, as these were not reviewed in the 2012 CTS guideline. We also believed that the high quality of evidence for pharmacotherapy choices would minimize the chances of expert opinion-related differences between guidelines.

For comparison of controller escalation recommendations, to account for differences in naming conventions for therapeutic “steps” between guidelines, we designated “step 1” as first-line therapy and steps 2–4 as the subsequent sequential therapeutic intensification recommendations. We did not consider steps beyond “step 4” because they applied to only a small percentage of asthma patients with “difficult-to-treat” asthma, and because evidence is less well-established for these steps [27, 28]. Furthermore, where guidelines offered preferred and alternative therapeutic recommendations, we limited our main comparisons to each guideline’s preferred treatment recommendations (those explicitly designated as such and/or featured in tables and/or figures). When guidelines referred to a prior guideline version for specific recommendations which were not covered in the current version or for recommendation references, we consulted the relevant prior guideline version.

For each of the above-described content areas, we also documented the level of evidence attached to each recommendation and compared the references used to support each recommendation, between guidelines. When recommendations and supporting references differed between guidelines, we determined whether the observed differences in references could simply be explained by differences in literature search dates between guidelines.

3. Results

3.1. Methodological Factors (Table 1)

The 2012 CTS guideline was not presented as a comprehensive, independent guideline statement but rather addressed four very specific PICO questions and reiterated recommendations from the 2010 CTS guideline in other areas. However, this guideline did address therapeutic escalations directly, enabling appropriate comparisons to contemporaneous guidelines. Although it did not revisit asthma control definitions, it reiterated prior definitions, which was the case for the other two guidelines as well.

Literature search strategies varied slightly between guidelines. Search dates were similar for BTS/SIGN and CTS guidelines. The BTS/SIGN guideline had the least current literature review (December 2009), whereas the GINA guideline had the most current (June 2012). Methodological approaches, their documentation, and usage of validated tools varied greatly between guidelines.

With respect to guideline committee membership, there were several common members (including several in leadership roles) between both the GINA and BTS/SIGN guidelines and the GINA and CTS guidelines. The GINA Science Committee included one participant who was cochair of the BTS/SIGN Pharmacological Management Committee and on the BTS/SIGN executive and steering group. The GINA Board of Directors included two participants who were on the 2012 and 2010 CTS Guideline committees (including one who was the Chair of the GINA Board of Directors), and the GINA Science Committee included one participant who was on the 2012 and 2010 CTS Guideline committees and one participant who was on the 2010 CTS Guideline committee.

3.2. Adult Pharmacotherapy Recommendations

The highest age category to which recommendations were made applicable was similar between guidelines, as follows: BTS/SIGN > 12 years; GINA “adolescents and adults;” and CTS ≥ 12 years (unless otherwise specified for certain recommendations).

3.2.1. Criteria for Controller Initiation and Escalation (Table 2)

All three guidelines presented criteria for suboptimal control; however, these varied considerably. The GINA guideline recommended using a four-week look-back period to characterize “average” control according to criteria, whereas neither the BTS/SIGN nor the CTS guidelines provided a recommended duration for criterion evaluation. These criteria seemed to apply to both initiation and escalation of controller therapy, as none of the guidelines explicitly recommended distinct criteria for each of these situations.

3.2.2. Therapeutic Escalation by Step (Figure 1)

The CTS guideline was unique in that it presented an “Asthma Management Continuum” diagram, suggesting that management should be approached as more of a continuum than as a series of distinct steps. However, specific recommendations for stepwise therapeutic intensifications were also clearly presented in the text and formed the basis for this comparative analysis.

All three guidelines recommended that all patients start therapy with a short-acting inhaled beta-2-agonist used as needed in step 1. For comparative purposes, the most recent citation for this recommendation in each guideline was published prior to the least current literature review (BTS/SIGN, December 2009), as follows: CTS, 2009; BTS/SIGN, 1999; and GINA, 1997. Next, all guidelines recommended inhaled corticosteroids (ICSs) in step 2; however, recommended doses were higher in the BTS/SIGN guideline. The CTS and BTS/SIGN guidelines did not provide specific references for their initial ICS dose recommendation, and the most recent citation for this in the GINA guideline was from 2005. In step 3, all three guidelines recommended addition of a LABA; however, the GINA and BTS/SIGN guidelines also allowed for use of a single inhaler of budesonide/formoterol as a reliever and a controller (SMART), whereas the CTS guideline did not, concluding that there was insufficient evidence to make this recommendation. Most recent citations for recommendations at this step were as follows: CTS, 2011; BTS/SIGN, 2009; and GINA, 2006. Seven of the references used in the CTS guideline for this step were published after the end-date of the BTS/SIGN literature search. Finally, in step 4, the GINA and BTS/SIGN guidelines suggested increasing the ICS dose (with the BTS/SIGN uniquely suggesting to discontinue the LABA if ineffective), whereas the CTS guideline recommended adding a leukotriene receptor antagonist or switching to the SMART approach. Most recent citations for recommendations at this step were as follows: CTS, 2011; BTS/SIGN, 2007; GINA, 2004. Three of the references used in the CTS guideline for this step were published after the end-date of the BTS/SIGN literature search.

3.2.3. Therapeutic Deescalation by Step (Table 3)

Although all three guidelines recommended reducing to the lowest possible therapeutic dose and regimen after achieving good control, only the GINA guideline provided guidance on each sequential deescalation. However, two of these five recommendations provided in the GINA guideline were derived through panel consensus judgment.

4. Discussion

We compared the latest Canadian Asthma Guidelines with two other contemporaneous international guidelines and found major differences between guideline methodologies, asthma control criteria, and recommendations for deescalation of therapy and more minor differences (along with several similarities) between recommendations for escalation of therapy. To our knowledge, this is the first detailed comparative analysis of contemporaneous asthma guidelines presented in the literature.

Previous authors have described varying degrees of interguideline variability in COPD and asthma. Iqbal and colleagues [19] described important differences in recommendations for diagnosis, staging, and therapy of COPD between international guidelines. In 2008, a report by Myers [29] described structure and process differences between non-contemporaneous asthma guidelines, including which key areas were addressed, which evidence grading systems were used, and intended target audiences. It also noted disparities in basic content elements such as whether asthma control was defined, the number of levels of asthma severity, and the number of treatment steps in each guideline.However, this study did not assess methods in detail nor compare control criteria and therapeutic recommendations directly. Furthermore, objective guideline development tools and techniques to improve quality and to harmonize guideline production methodologies have been adopted since 2008. For example, both the CTS [25] and BTS/SIGN groups [26] used the AGREE instrument [30] in 2012 guideline production. Such tools would be expected to mitigate obvious causes of guideline divergence such as variability in literature search, evidence synthesis, and evidence appraisal methods.

4.1. Reasons for Differences between Asthma Guidelines (Figure 2)

Reasons for the differences that we have observed between asthma guidelines are likely multifactorial. One of the most important causes is likely the observed difference in the evidence used (cited) by each guideline group for each recommendation. In turn, there are several possible explanations for this. Search strategies did differ slightly between guidelines; howeve, each was reasonably comprehensive and would not be expected to miss any major studies (Table 1). Search dates differed significantly between guidelines, but as demonstrated in Tables 2 and 3, and in the description of most recent cited references for each therapeutic escalation step (in “Results”), the vast majority of cited studies were published prior to the least current literature review (BTS/SIGN, December 2009), whereby all studies should have been identified by each guideline group. The only exception to this is the handful of studies cited in escalation steps 3 and 4 of the CTS guideline which were published subsequent to the BTS/SIGN literature review date and thus would not have been available to the BTS/SIGN group. However, these would have been available to the GINA group. Accordingly, the differences in evidence used are unlikely to be attributable to differences in literature search strategies and/or dates and are more likely due to differences in evidence interpretation (i.e., synthesis, outcomes considered, and appraisal), methods for which did vary widely between guidelines (Table 1). The importance and impact of these differences in evidence interpretation are demonstrated by our finding that several comparable recommendations were ascribed a certain level of evidence in one guideline and either a different level or no level at all in another guideline (Table 2, Figure 1).

Differences in evidence interpretation might have occurred for both valid and nonvalid reasons. Perfectly valid differences in judgment may have occurred regarding which studies were relevant, the risk of bias in individual studies, and the relative balance of demonstrated benefits and adverse effects. Such differences are an inevitable product from the “human” aspect of evidence interpretation, whereby it may be unrealistic to expect two independently produced guidelines to provide identical recommendations in each area. Interestingly, these guideline committees were not completely independent, as there were overlapping guideline committee members and leaders between both GINA and BTS/SIGN and GINA and CTS guidelines. Although one would have expected overlapping committee members to bring common perspectives to the different guideline processes and/or to reconcile major differences in recommendations, this may have been hampered by fundamental differences in process between guidelines. Next, nonvalid factors such as inadequate critical appraisal and/or a failure to consider all relevant outcomes (particularly patient-relevant outcomes) may also have contributed [11]. In all 3 guidelines, evidence selection, synthesis, and appraisal were performed by only a small number (2–4) of committee members, and only the BTS/SIGN group used defined study inclusion and exclusion criteria. Furthermore, although the CTS and BTS/SIGN guidelines used (distinct) validated tools for evidence appraisal, the GINA guideline appears to have employed an informal process (Table 1). Lastly, only the CTS guideline specified which outcomes were considered (including several patient-relevant outcomes). Another concern is the possibility of subjectivity relating to conflicts of interest and the personal values and beliefs of guideline writers [12]. These factors have been shown to contribute to citation selection bias and evidence interpretation bias, even when using objective methods and tools [11, 31]. Financial conflicts of interest aside, “inherent” conflicts such as an author’s specialty or publications were recently shown to predict the nature of recommendations surrounding mammography screening, in a study of 12 guidelines and 178 authors [32]. Accordingly, methodological quality does not necessarily correlate with recommendation validity [14].

Differences between methods used for formulating recommendations may also have played a role in the observed variability. The GINA guideline sought full committee consensus, the BTS/SIGN guideline sought unanimity within a small working group, and the CTS guideline sought a majority in a full committee vote (Table 1). In a previous study, Shekelle et al. [33] noted that even with identical committee membership and literature search findings, differences in methods used to combine the literature with expert judgment resulted in important differences in recommendations. For example, in areas of controversy, the informal consensus method produced “lowest common denominator” statements that all panelists agreed upon, but were much less specific than recommendations formulated through a method in which consensus was not mandated.

A myriad of other factors may also have contributed to the variability in recommendations. The relative paucity of available evidence in certain areas, such as the definition of asthma control, required increased use of subjective expert judgments, which are inherently variable [13, 19]. Heterogeneous asthma disease definitions across guidelines and heterogeneous asthma phenotypes across research studies [34] may also have contributed to differences in evidence selection, prioritization, and interpretation between guidelines. Factors such as the availability of health care services and/or medications, and the costs of an intervention in the targeted jurisdiction can also influence how guideline writers view and apply published literature during guideline development [11, 19]. Indeed, the CTS and BTS/SIGN guidelines target audiences in Canada and the United Kingdom, respectively, whereas the GINA guideline targets a global audience. However, we believe that these were unlikely to have been contributors to the differences that we have noted, as relevant medications are widely available, and none of the guidelines explicitly considered costs in their recommendations.

4.2. The Consequences of Differences between Asthma Guidelines

The major consequence of disagreement between guidelines is variability in practitioner behavior [13], which may have important clinical implications for asthma evaluation and management. For example, asthma control criteria varied both in terms of recommended cutoffs for elements that were common across guidelines and in terms of which elements were included in each guideline (Table 2). There were also more subtle differences, such as whether daytime symptoms should be counted each time they occurred or on each day that they occurred (the latter allowing for a single day with several daytime symptom episodes to count only once), and whether rescue beta agonist use should be counted by number of doses or by number of times the rescue puffer was required (the latter allowing for several consecutive rescue doses at one time to count only once), and whether use prior to exercise was included. The practical impact of these differences was demonstrated in a review of primary care electronic medical records which compared asthma control according to CTS and GINA guidelines and noted interguideline discordance between asthma control ratings in more than half of the patients deemed “in control” by at least one guideline [35]. The criteria resulting in the highest degree of discordance were lung function, absenteeism from school or work, and the frequency of daytime symptoms [35].

There is also a theoretical concern that the observed guideline variability may contribute to the poor uptake of asthma guidelines, which has been noted particularly in the areas of asthma control determination [7] and escalation of therapy [8] (only 39% of Canadian physicians report basing asthma therapy on guideline recommendations) [7]. Asthma guidelines have been noted to be confusing [36] to begin with, and in other diseases, conflicts between major guidelines have been shown to compound confusion, reduce usability, and frustrate practitioners [11, 12, 19]. Qualitative studies suggest that these types of conflicts undermine clinician trust in guidelines, thereby acting as a potential barrier to guideline uptake [11, 20] and contributing to poor guideline adherence [21]. In one study, a positive attitude towards an asthma guideline was more strongly associated with self-reported adherence to the guideline than knowledge of guideline elements themselves [37]. It should be noted that knowledge of asthma guidelines likely remains the most fundamental barrier to uptake [38], and major guideline implementation activities appropriately focus on increasing practitioner knowledge of recommendations. However, our analysis suggests the possibility that attitudinal barriers to asthma guideline acceptance may also exist, particularly in primary care, and merit consideration as a potentially modifiable contributor to poor guideline adherence. However, given that barriers to asthma guideline uptake are diverse, occurring at the level of the patient, the health delivery system, and/or the practitioner [10], quantifying the relative importance, if any, of the observed guideline variability on asthma guideline uptake will require further research.

Finally, studies have found that conflicting recommendations can lead to more test ordering [13] and resulting patient harm [14]. Furthermore, studies suggest that many patients are also aware that conflicting guideline recommendations exist, and this leads to diminished confidence in the medical system and physician recommendations and reduced overall patient adherence [16]. Similarly, conflicting recommendations have been noted to diminish decision makers’ faith in guidelines [39], which may in turn limit guideline implementers’ ability to enact system-level changes to enable better care.

5. Conclusions

We have demonstrated important differences in the basic recommendations surrounding adult asthma care between contemporaneous international guidelines. This has important effects on the consistency of clinical asthma care and may possibly impact overall guideline uptake. Given that this variability appears to have been driven by methodological differences in areas such as evidence interpretation and recommendation formulation, universal acceptance and application of a single, transparent, and explicit set of standards for these processes would likely reduce the observed differences [14, 49]. A common methodological approach to ensure detailed peer review of asthma guidelines by primary care practitioners might also uncover areas where interguideline conflicts are particularly impactful to end-users. Such methodological changes would require a concerted effort for cooperation between the leaders of various international asthma guideline organizations. Given the shared goals and collegiality between international asthma experts, as demonstrated by the common membership and leadership across these guidelines, this goal is achievable and should be a priority. Furthermore, as respiratory organizations increasingly turn their attention to the difficult task of guideline implementation [50, 51], congruence across major international guidelines could enable harmonization and collaboration in these knowledge translation initiatives across jurisdictions, thereby increasing their chances of success.

Appendix

Recommendation Rating Systems

See Tables 4, 5, and 6.

Abbreviation List

CTS:Canadian Thoracic Society
NHLBI:National Heart, Lung and Blood Institute
GINA:Global Initiative on Asthma
BTS:British Thoracic Society
SIGN:Scottish Intercollegiate Guidelines Network
COPD:Chronic obstructive pulmonary disease
ICS:Inhaled corticosteroid
SMART:Budesonide/formoterol as a reliever and a controller
LABA:Long-acting beta agonist.

Competing Interests

No conflict of interests exists for the authors of this paper.

Authors’ Contributions

Samir Gupta conceived of the study; Samir Gupta and Emily Paolucci analyzed data and prepared the paper; Louis-Philippe Boulet and Alan Kaplan helped to design the study and critically appraised the paper.

Acknowledgments

The authors would like to thank Dr. Nadia Griller for her contributions to this analysis. Dr. Gupta is supported by the Li Ka Shing Knowledge Institute of St. Michael’s Hospital and the University of Toronto. This work was supported by the Canadian Institutes of Health Research and The Ontario Lung Association.