Abstract

Objective. To evaluate the quality of methodologies used in previous systematic reviews (SRs) and compare efficacy of different acupuncture therapies for allergic rhinitis. Methods. Seven electronic databases were searched for systematic reviews (SRs) performed on different acupuncture therapies for allergic rhinitis from inception to 15 November 2019. The AMSTAR2 instrument was employed to assess the methodological quality of included SRs. Eligible randomized controlled trials (RCTs) were selected from the included systematic reviews. We also included recent RCTs published by 15 November 2019. Cochrane risk of bias tool was utilized to determine risk of bias of the included RCTs. Pairwise meta-analyses were performed using the random-effects model. Network meta-analysis of the included RCTs was carried out using frequentist framework. Results. We identified 2 SRs with low quality and 18 SRs with very low quality, both of which contained 33 eligible RCTs (n = 3769). Most of these studies had unclear risk of bias. On the basis of ranking probability, NMA analysis showed that acupuncture at the sphenopalatine ganglion acupoint (OR: 1.31, 95% CI 1.07 to 1.61) had the highest probability of improving global allergic rhinitis symptoms, followed by San-Fu-Tie (OR: 1.17, 95% CI 1.08 to 1.27), manual acupuncture (OR:1.15, 95% CI 1.07 to 1.24) compared with conventional western medicine treatment. Moreover, direct comparison of the follow-up period showed that the clinical outcomes of acupuncture and related therapies at three-month (OR:1.34, 95% CI 1.17 to 1.55), six-month (OR: 1.31, 95% CI 1.10 to 1.57), and twelve-month (OR: 1.30, 95%CI 1.11 to 1.53) follow-up were better than those of traditional western medicine. Conclusion. These results indicate that for patients with allergic rhinitis who are unresponsive to conventional western medicine or cannot tolerate the side effects, acupuncture at the sphenopalatine ganglion acupoint is an effective alternative therapy. Further studies are advocated to deeply explore methodological quality of SRs by incorporating high-quality RCTs.

1. Introduction

Allergic rhinitis (AR) affects 10% to 20% of the of the world’s population [1]. AR negatively impacts patients’ social life by affecting sleep, school performance, and work productivity [24]. This condition also financially burdens affected individuals [5].

Acupuncture is widely used for the management of AR. The 2015 American Clinical Practice Guidelines for AR [6] and the 2018 Chinese Society of Allergy Guidelines for Diagnosis and Treatment of Allergic Rhinitis [7] recommend acupuncture for management of AR. Among the advantages cited by the American guidelines is that acupuncture offers an effective alternative therapeutic option, for the alleviation of AR symptoms, thereby improving quality of life while reducing drug prescriptions and associated side effects. However, there is limited credible evidence for its efficacy because available randomized controlled trials (RCTs) did not compare acupuncture with traditional medicinal therapy and have methodological flaws [6].

In recent years, more RCTs have compared the effectiveness of acupuncture to that of traditional medicinal therapy in allergic rhinitis. Multiple systematic reviews have documented that acupuncture therapy has higher efficacy relative to western medicine [810]. However, whether different acupuncture therapies are indeed superior to western medicine or even substitutes for western medicine has not been interrogated. The Chinese and American guidelines do not recommend specific types of acupuncture for AR management, and the choice of which type to prescribe is at the clinician’s discretion.

Here, we conducted an overview of systematic reviews to evaluate the quality of methodology used in their preparation. We then compared the efficacy of different acupuncture therapies alone with that of western medicines through network meta-analysis. This study aimed to provide some references for clinical decision making.

2. Methods

2.1. Inclusion Criteria

SR/meta-analysis based on RCTs on the benefits of acupuncture against AR was included in the study if they met the set criteria outlined below for participants, interventions, control groups, and outcome indicators. SRs had to be in either the English or Chinese language.

2.1.1. Participants

Inclusion of AR patients was not restricted by age, race, or gender. Participants were included if they had a clear AR diagnosis but were not limited to specific diagnostic criteria.

2.1.2. Intervention and Comparisons

Different acupuncture protocols, including manual acupuncture, appoint catgut embedding, acupoint herb application, San-Fu-Tie, acupuncturing the sphenopalatine ganglion acupoint alone (Table 1) were included in the study. The control group was constituted as recommended by guidelines for conventional drugs including nasal corticosteroid spray, antihistamines, and immunotherapy. And it is important to note that acupoint herb application can be classified into San-Fu acupoint herb application (only applied during Sanfu period) and non-San-Fu-Tie acupoint herb application (applied without a specific time frame). In this study, acupoint herb application referred to non-San-Fu-Tie acupoint herb application.

2.1.3. Outcome

To be included in an NMA, trials must include the outcome of nasal symptoms (sneezing, rhinorrhea, nasal itching, and nasal obstruction) [11], and the criteria for outcome evaluation must be clear. The outcome evaluation is also recommended for the standardization of clinical outcomes used in AR [11]. Outcomes on symptom improvements are reported on short 3-point Likert scales. On the basis of clinical observations, these short ordinal outcomes were dichotomized as “improvement” or “no improvement” in accordance with guidelines of the Cochrane handbook. In this study, the outcomes “markedly effective” and “effective” were grouped as “improvement,” and the outcome “no improvement” classified as “nonbeneficial.” Likewise, a binary assessment of global symptoms was used in the recently published systematic review of Fu et al. [10].

2.2. Search Strategy and Selection Criteria

Literature analyzed in this review was searched on Cochrane library, PubMed, EMBASE, as well as on four Chinese databases (Wan Fang Digital Journals, China National Knowledge Infrastructure, Chinese Biomedical Database, and VIP Database). The searches were made for systematic reviews of acupuncture-related treatments for AR for up to November 15, 2019. A grey literature search was also done. Only publications in English or Chinese were included in this study. A typical search, as done on PubMed, is shown on Figure 1.

2.3. Literature Selection and Data Extraction

Two researchers (Z. J. H and H. X. X) independently screened the literature results and extracted data. Disagreements between Z. J. H and H. X. X were resolved by consulting a third researcher (Z. Y. Y). Where clarification was needed, the paper’s author was contacted for more information. Literature screening involved first excluding any duplicated articles. Where articles were of high similarity, the most comprehensive one was selected. Next, the titles and abstracts were read, and any obviously irrelevant literature excluded from further analysis. Full texts of the remaining articles were then read to determine the final inclusion.

Eligible RCTs were selected from the included systematic reviews and updated RCTs that were available up to November 15, 2019. RCT screening for inclusion was done as described above, and the following information independently extracted from the RCTs by Z. J. H and H. X. X: author, year of publication, number of patients enrolled in the study, participant characteristics, duration of diseases, diagnostic criteria, duration of patient enrollment, types and uses of interventions, efficacy standard, and follow-up duration.

2.4. Methodological Quality and Risk of Bias Assessment

To assess the quality of methodology, we used the AMSTAR2 scale [12]. Compared with its predecessor, AMSTAR2 offered a more comprehensive assessment. AMSTAR2 takes into account 16 items that are answered “yes” and “no.” Items 2, 4, 7, 9, 11, 13, and 15 are critical fields. If the analysis reveals no item defect or only one noncritical item defect, the methodology was high quality and the SR considered accurate and comprehensive. In cases of more than one noncritical item defect, but no critical item defect, the methodology was of medium quality and the SR considered accurate. In cases of a key item defect, with or without a noncritical item defect, the methodology was considered of low quality and the SR considered not be accurate or comprehensive. In cases of more than one critical flaw, with or without noncritical item defect, the methodology was considered to be of extremely low quality and the SR considered inaccurate and incomplete [12].

The Cochrane risk of bias tool was used to estimate the risk of bias in each included RCT, which took the following into account: random sequence generation, allocation concealment, blinding of participants and personnel, blinding of outcome assessment, incomplete outcome data, selective outcome reporting, and other sources of bias. Risk of bias for each field was scored low, high, or unclear [13].

2.5. Data Synthesis

To execute network meta-analysis, we first performed pairwise meta-analysis, followed by network meta-analysis. Pairwise meta-analysis compared different acupuncture types relative to the control group since outcome indicators were binary. We used odd ratios (OR) and 95% confidence interval (CI) to assess the effects of different acupuncture therapies on nasal symptoms. Heterogeneity between pairwise comparisons was determined by I2 and values; I2 < 25% is considered low heterogeneity, 25–50% moderate heterogeneity, and >50% high heterogeneity. was considered indicative of significant heterogeneity [14].

Random-effects network meta-analysis was done using STATA version 14.0. Network meta-analyses were executed using a network meta-analysis package [15].

The network i command was used to evaluate inconsistency of all comparisons.

Potential therapeutic benefits of interventions were ranked using surface under the cumulative ranking curve (SUCRA) [16].

Potential small study effects within the network were detected using comparison-adjusted funnel plots [17]. Sensitivity analysis was done by excluding studies with a sample sizes of 60 or by excluding studies with significant levels of heterogeneity.

3. Results

3.1. Literature Search Results and Study Selection

Twenty SRs [810, 1834] were selected from 99 records. The 20 SRs comprised 194 RCTs, while 44 RCTs were updated simultaneously as described for eligibility inclusion. Eventually, 33 RCTs [3567] were included in the study. An outline of the screening procedure is shown in Figure 2.

3.2. Characteristics of Included RCTs

The 33 RCTs consisted of 3769 patients in studies published between 2007 and 2019. The patients included in the studies were aged between 3 and 78 years and exhibited disease course for 6 months to 36 years. The study sample size ranged from 53 to 339 patients. Treatment duration varied between 7 days and a year. Duration of follow-up ranged from 1 month to 3 years. Together, the 33 RCTs had 17 diagnostic and efficacy evaluation criteria, of which 16 were Chinese. The control groups for the 33 RCTs were all treated with western medicine, including nasal corticosteroids, antihistamines, immunotherapy alone, or their combination. Together, the 33 RCTs studied 5 types of acupuncture, including manual acupuncture (n = 11), acupoint herb application (n = 7), San-Fu-Tie (n = 8), acupoint catgut embedding (n = 5), and sphenopalatine ganglion acupoint (n = 2). Characteristics of the included studies are shown on Table 2.

3.3. Methodological Quality of Included SRs

AMSTAR 2 was used to assess the quality of methodology used in the 20 included SRs. This analysis revealed that in 3 SRs, only one critical item failed the quality test, and the quality of methodology was low. In the other SRs, more than one critical item failed the quality test and quality of methodology was very low. Among the critical items, the compliance rate on item 2 and item 4 was 10%; for item 7, it was 60%; for item 9, it was 85%; for item 11, it was 100%; for item 13, it was 95%; and for item 15, it was 70%. For the noncritical items, for items 1 and 3, it was 100%; for items 5 and 6, it was 70%; for items 10 and 16, it was 10%; for item 12, it was 95%; and for item 14, it was 80%. None of the SRs fully met the standards for item 8 (Table 3).

3.4. Risk of Bias among Included RCTs

To minimize the risk of bias, 14 of the 33 RCTs reported using the random number table method, 1 adopted card randomization, all of which were assessed as low risk, and 15 only described randomness but did not describe the method of randomization used (these were considered unclear). In terms of allocation of concealment, only 2 trials stated opaque envelopes that were considered low risk. In terms of incomplete report outcomes, 3 trials described appropriate methods that were used in handling incomplete data and were considered low risk. The rest were not described and were judged as unclear. None of the included trials described the methods used to blind participants, implementers, and outcome evaluators, which were considered unclear risks. Because of their failure to provide protocols, the selective outcome report biases, and other biases, they were considered unclear. Details of risk bias are shown in Figure 3.

3.5. Results of Pairwise Meta-Analyses

We did a meta-analysis of direct comparison between five acupuncture therapies and western medicine (Table 4). Results from this analysis show that manual acupuncture, San-Fu-Tie, acupuncturing the sphenopalatine ganglion acupoint and acupoint herbal application were superior to western medicine. The acupoint catgut embedding acupuncture method did not emerge as being superior to western medicines.

3.6. Results of Network Meta-Analysis
3.6.1. Network Plot

All 33 RCTs included in our study two-arm trials. In this analysis node, size represents sample size, whereas line thickness represents the number of comparisons between two studies. This study revealed that the number of comparisons was highest between acupuncture and western medicine (n = 11) (Figure 4).

3.6.2. Results of Comparative Effectiveness of the Six Interventions after the Treatment and Follow-Up Period

This analysis found that acupuncturing the sphenopalatine ganglion acupoint (OR: 1.31, 95% CI 1.07 to 1.61), San-Fu-Tie (OR: 1.17, 95% CI 1.08 to 1.27), and manual acupuncture (OR: 1.15, 95% CI 1.07 to 1.24) were more effective than western medicine at ameliorating the symptoms of AR (Table 5). Analysis of results from the follow-up period showed that manual acupuncture (OR: 1.63, 95% CI 1.09 to 2.44), acupoint herbal application (OR: 1.38, 95% CI 1.17 to 1.64), and acupoint catgut embedding (OR: 1.25, 95% CI 1.05 to 1.50) were more effective than western medicine at ameliorating AR symptoms. No statistically significant difference was observed between the five acupuncture treatments (Table 5).

3.6.3. Results of Interventions Rank Probabilities

The cumulative area of SUCRA represents the probability of each intervention ameliorating the symptoms of AR (Figure 5). This analysis revealed that acupuncturing at the sphenopalatine ganglion acupoint (92.4%) exhibits the highest probability of improving AR symptoms, followed by San-Fu-Tie (82.8%), acupuncture (62.2%), acupoint herb application (34.4%), acupoint catgut embedding (24.8%), and western medicine (3.3%).

3.6.4. Sensitivity Analysis and Small Sample Effect

We performed sensitivity analysis in two ways, one by excluding 2 studies [47, 60] whose sample size was smaller than 60. Results of this analysis indicated that exclusion of the 2 studies did not alter the ranking established earlier (Figure 6(a)); SUCRA—E (92.3%) > D (72.1%) > A (68.8%) > C (33.9%) > B (28.8%) > F (4.1%). The other sensitivity test was done by excluding study [64] that exhibited a significant level of heterogeneity, which too, did not alter the ranking (Figure 6(b)); SUCRA—E (92.6%) > D (72.4%) > A (68.4%) > C (33.8%) > B (28.7%) > F (4.0%).

An adjusted funnel plot analysis revealed a symmetrical funnel plot and no obvious small sample effects (Figure 7). However, the distribution scattered one point outside of the funnel plot. The point scattered out of the funnel plot represented a heterogeneous study [64]. Exclusion of this study resulted in scattering of all the points inside the funnel plot (Figure 8).

3.7. Follow-Up Efficiency

Analysis of the follow-up data revealed that 7 trials (20.6%) reported effective rates. Of these, 1 trial reported effective rates after 1 month of follow-up, 4 reported effective rates after 3 months of follow-up, 3 reported effective rates after 6 months of follow-up, and 3 trials reported effective rates after 12 months of follow-up. With the exception of the 1-month follow-up, the follow-up rates for 3, 6, and 12 months indicated that acupuncture therapy was superior to western medicine group (Table 6).

3.8. Adverse Effects and Loss to Follow-Up
3.8.1. Adverse Events

Four (4) trials, 1 on manual acupuncture, 1 on San-Fu-Tie, and 2 on acupoint catgut embedding, reported adverse outcome. The 4 reported symptoms of the adverse reactions alleviated after discontinuation of the treatment or after a few days’ rest. However, none reported serious adverse events occurred. Specifically, Huang[39] reported that 8 cases receiving western medicine complained of drowsiness and fatigue, 1 case complained of joint pain and fatigue, and 1 case complained of slight rash. Symptoms stopped on discontinuation of oral loratadine treatment. In the acupuncture group, 6 case patients complained of improper pressure during needle extraction, resulting in slight subcutaneous hematoma that cleared naturally within days.

Liu [46] reported 2 cases in the acupoint embedding treatment group who had mild adverse reactions, and one who had bruises after acupoint treatment. These effects were considered to have been caused by damaged subcutaneous capillaries, and the symptoms resolved after hot compress. On the day of treatment, a patient complained that their left arm was not flexible, and this was considered to have been caused by a strong sense in hegu acupoint. The symptoms resolved within 3 days. One patient treated with western medicine presented with a nasal cavity and a small amount of blood in the nose after one month of treatment, but during follow-up, it was self-reported that the symptoms had resolved.

Feng [47] reported 2 cases of adverse reactions in both acupoint embedding and western medicine groups. One case treated with acupoint embedding exhibited induration at the embedding acupoint while another patient reported fainting during the acupuncture procedure. In the western medicine group, one patient was allergic to medicine and another complained of drowsiness. This paper did not provide treatment or prognosis details.

Wang [54] reported one case that suffered skin irritation after San-Fu-Tie treatment and 11 cases of adverse reactions in the western medicine treated group, including 3 cases of mild nosebleeds, 2 cases of drug dependence, 4 cases of dry nasal cavity, and 2 cases of perforated nasal septum. The difference between the two groups was statistically significant (χ2 = 9.470, ).

3.8.2. Dropout Patients

Three trials reported patient dropout. Liang et al. [50] reported that 15 patients receiving acupuncture dropped out of the study. Of these, 1 dropped out due to dizziness during treatment, 4 dropped out for personal reasons, and 10 had not received treatment as prescribed. In the western medicine-treated group, 15 cases dropped from the study. Among them, 6 withdrew from the study by themselves, and 9 cases were not treated as prescribed.

Liu [46] reported that 2 patients receiving acupoint embedding acupuncture dropped out of the study, 1 for work-related reasons, and the other because of protocol violation. Among the western medicine-treated group, 2 cases dropped out (one because of protocol violation and the other one elected to stop treatment).

Wang et al. [62] reported that 1 case in the acupuncture-treated group and 2 cases in the western medicine-treated dropped out of the study due to incomplete compliance with the test requirements.

Overall, the dropout rate between the two groups is balanced and mainly caused by violation of the treatment plan or moving, rather than poor efficacy.

4. Discussion

In this study, we used AMSTAR2 to evaluate the quality of methodology used in 20 SRs. Our analysis shows that 85% of the SRs had very low quality of methodology, which lowers credibility of the conclusions by drawn in the systematic reviews that acupuncture is superior to western medicine. In terms of ameliorating the symptoms of AR, the 33 RCTs that we included in our direct comparison analysis indicated that the 4 types of acupuncture therapy have a superior performance relative to western medicine. However, in the direct comparison of San-Fu-Tie and compared western medicine, a heterogeneity of 60.9% was observed. After a sensitivity analysis, 1 study [64] was withdrawn, and the heterogeneity fell to 0. This manipulation did not change the outcome of the earlier analysis, indicating that the results were robust. Among the 8 trials in San-Fu-Tie was compared with western medicine, only 1 study [64] indicated a relatively good efficacy, although this was not statistically significant. San-Fu-Tie can therefore still be regarded as superior to western medicine. In NMA, ranking on the basis of improved AR symptoms identified acupuncturing the sphenopalatine ganglion acupoint as the best treatment and western medicine as the worst. Subsequent sensitivity analysis did not significantly change the ranking results.

This study assessed the comparison between the efficacy of the 6 interventions at improving AR during the follow-up period and found acupuncture to still be better than western medicine after 12 months of follow-up. In terms of adverse events, acupunctural therapies do not show overt adverse reactions. Therefore, patients who are intolerant to western medicine or interested in alternative therapy can consider acupunctural procedures.

Our results are partly consistent with those from previous studies. Fu [10] showed that acupuncture of the sphenopalatine ganglion points is better than other approached, including sham needles, western medicine, and conventional acupuncture points. However, there was no significant difference and obvious heterogeneity (RR = 1.18, 95% CI [0.88, 1.57], I2 = 93%) between acupuncturing the sphenopalatine ganglion acupoint and western medicine alone. The reason is that 2 studies evaluated by the authors did not report clear diagnostic criteria or their efficacy criteria and were therefore excluded from our study. A systematic review by Zhang et al. [9] showed that acupuncture alone (OR = 2.42, 95% CI 1.15 to 5.09) I2 = 2%) was better than western medicine therapy. A study by Zhou et al. [34] showed that there was no statistical difference between acupuncture alone and western medicine within less than 3 months (OR = 1.04, 95% CI 0.92 to 1.17), but acupoint application plus western medicine was better than western medicine alone (OR 1.22, 95% CI 1.12 to1.33). Interestingly, after treatment for beyond 3 months, acupoint application alone appears better than western medicine, and this might be related to the better prognosis of acupoint application. To the best of our knowledge, no systematic review has analyzed San-Fu-Tie and western medicine alone. A review by Xie et al. [8] shows that acupoint catgut embedding alone is better than western medicine. However, our review of this report revealed studies with ambiguous diagnosis, and incorrect random methods were included, all of which were excluded in our study.

Acupuncture is traditional medicine method that can balance Yin and Yang and improve physique at a high efficiency. The mechanisms of its effects are progressively becoming clearer. Research shows that acupuncture can reduce immunoglobulin [68], regulate Th1/Th2 levels [69], inhibit the release of inflammatory mediators [70], and reduce inflammatory neuropeptides [71].

Acupuncturing the sphenopalatine ganglion acupoint, combined with the theory of traditional Chinese and western medicine, was first reported in 1990 [72]. In this approach, acupuncture is used to stimulate the sphenopalatine ganglion with the aim of adjusting the balance between sympathetic and parasympathetic functions. The mechanism of acupuncture action the sphenopalatine ganglion in the treatment of AR includes 1) the stimulation of the sympathetic nerve fibers in the distribution area of this ganglion, which causes blood vessel constriction, reduced blood flow to the nasal mucosa and cavernous body, reduced glandular secretion, widening of the nasal passage, and reduced the turbinate, thereby improving ventilation and ameliorating associated symptoms, and (2) through stimulation of central nervous autonomic control. It is hypothesized that after the nerve ganglion is stimulated, the stimulation is reflected on the central system, which in turn stimulates bilateral nerves and therefore simultaneously regulates bilateral nasal mucosa [7374].

One limitation of our study is that it did not include indirect comparisons and placebos. These were excluded because our aim was to establish which acupunctural procedures are better alternatives to western medicine. In addition, the types of western medicine are not studied separately. This is mainly because there are many kinds of such treatments with various applications. These will be the focus of future studies.

Second, only two of the included SRs were rated of low quality of methodology while the methodologies of remainder were considered to be of extremely low quality. This is attributable to the fact that in the literature search of key items, most of the studies were not registered in advance, and grey literature was ignored in literature searches. Future SRs reporting standards should adhere to the requirements of PRISMA and make improvements to draw more rigorous conclusions [75].

Third, the RCT methodology of included studies was generally poor since most of the risk items are unclear, particularly allocation concealment, blindness, and selective outcome reporting. This limitation might lower credibility of our conclusions. We recommend that in future, researchers follow CONSORT reporting norms [76]. With regard to implementation of blind method, because the reported outcomes in this study are subjective symptom scores, third-party evaluators may implement blind methods so as to reduce bias and enhance reliability of the results.

Fourth, the included RCTs are from China, and there may be differences in diagnostic criteria and outcome evaluation criteria between studies. Because the report of acupuncture use is not sufficiently detailed, it may cause limitations in the recommendation of conclusions. In this study, we have attempted to describe acupoints and usage methods in as much detail as possible in the feature table. We recommend that future acupuncture clinical trials follow the STRICTA standard [77] and be done in as many countries and regions as feasible.

5. Conclusion

Our findings indicate that acupuncturing the sphenopalatine ganglion acupoint is a viable alternative for patients with allergic rhinitis and are unresponsive to conventional western medicine or cannot tolerate its side effects. We recommend that future acupuncture systematic reviews put more emphasis on the quality of methodologies used in the underlying studies and that high-quality RCTs be undertaken to enhance the reliability of the conclusions drawn.

Conflicts of Interest

The authors declare that there are no conflicts of interest concerning this work.

Authors’ Contributions

J. H. Zhang and Y. Y. Zhang were responsible for study design. J. H. Zhang and H. B. Yu involved in manuscript writing and revision. J. H. Zhang and X. X. Huang carried out literature retrieval. J. H. Zhang and L. Y. Hu performed discretion of eligibility. J. H. Zhang and Y. R. Chen were responsible for quality assessment. J. H. Zhang and K. Lan extracted data. J. H. Zhang and Y. Y. Zhang carried out statistical analysis. All authors reviewed and agreed to the contents in the final version of this manuscript.

Acknowledgments

This study was supported by Shenzhen’s Sanming Project (SZSM201612001).