Abstract

Background. Chronic urticaria (CU) can severely impair the quality of life. Acupuncture and related therapies have been widely used in the treatment of CU in China. This study aimed to summarize and critically evaluate the methodological and reporting quality of relevant systematic reviews (SRs) and present objective and comprehensive evidence on the effectiveness and safety of acupuncture and related therapies for CU. Methods. Eight electronic databases were searched from inception to October 2021 for SRs examining acupuncture and related therapies for CU, and gray literature was manually searched. Two authors independently identified SRs and extracted data. The methodological and reporting quality of these SRs were assessed by the Assessment of Multiple Systematic Reviews 2 (AMSTAR 2) tool and preferred reporting items for SRs and meta-analyses (PRISMA, 2020), respectively. In addition, the risk of bias in systematic reviews (ROBIS) was used to evaluate the risk of bias. The Grading of Recommendations Assessment, Development, and Evaluation (GRADE) framework was applied to evaluate the quality of evidence for outcome measures. Results. In total, 23 SRs, including a total of 11 outcome indicators, were published before October 2021. The AMSTAR-2 results showed that the methodological quality of all SRs was critically low; items 2, 3, 7, 9, 10, and 16 were found to have particularly low quality. For PRISMA, the reporting quality of the included SRs was unsatisfactory, and major reporting flaws were observed in the search strategy, synthesis method, certainly assessment, reporting biases, registrations, and financial support of the included SRs. For ROBIS, 22 SRs (95.65%) had a high risk of bias. Among the 55 outcomes assessed using the GRADE framework, there were 3 (5.45%) outcomes with moderate-quality evidence, 6 (10.91%) outcomes with low-quality evidence, and 46 (83.64%) outcomes with very low-quality evidence. We found the moderate quality of evidence indicating that the total effective rate and curing rate of the acupuncture group were higher than those of the western medicine group, and the recurrence rate was lower than that of the western medicine group. Conclusions. Acupuncture and related therapies for the treatment of CU are supported by low-quality evidence-based medicine. However, considering the poor quality of these SRs, we suggest that studies with more rigorous designs, larger sample sizes, and higher methodological and reporting quality are necessary to provide stronger evidence. Registration. The protocol for this study has been registered (PROSPERO registration number: CRD42021259131).

1. Introduction

Urticaria is a skin disease marked by the appearance of wheals (hives), angioedema, or both [1, 2]. When a person has transient wheals that last more than 6 weeks and occur virtually daily [2, 3], it is defined as chronic urticaria (CU). The CU can affect approximately 1% of the world’s population of all ages, mainly young and middle-aged women [4]. CU can cause anxiety, depression, sleep and sexuality disturbances, and severely impair quality of life [58]. According to the evidence, patients with CSU can experience a significant loss of productivity at work, school, or in daily activities [9, 10]. In addition, the decline in productivity also leads to high direct and indirect healthcare costs to treat CU, with large socioeconomic impacts [4, 11, 12].

The current international guidelines [1] and consensus statements [3, 13] highly suggest a stepwise approach to complete symptom control. However, many patients have an inadequate response to the majority of these drugs [14]. Furthermore, if patients continue to be symptomatic despite the use of H1-antihistamines, the guidelines do not provide guidance on the selection, dose, and duration of alternative treatment options. Although omalizumab and CsA have been shown to be effective [1517], the prices are high and can place a significant financial burden on patients. Their widespread adoption will be determined by legal and economic factors [18]. As a result, an increasing number of patients are seeking alternative treatment options.

In China, acupuncture and related therapies have been widely used to treat CU. At present, a series of systematic reviews (SRs) have proven the efficacy and safety of specific treatments [1941]. In the field of evidence-based medicine, SRs have long been regarded as one of the most important sources of high-quality, trustworthy information. However, these SRs examining acupuncture and related therapies for CU were published in different years, included various acupuncture methods, and examined different outcomes. The conclusions are not completely consistent among these SRs. Therefore, a rigorous methodological approach is used herein to summarize and critically assess the methodological and reporting quality of relevant SRs, as well as to provide objective and comprehensive findings on the efficacy and safety of acupuncture and related therapies for CU. We believe this work can help to provide a better reference and evidence support for acupuncture and related therapies in CU treatment.

2. Materials and Methods

2.1. Search Strategy

We searched the following electronic databases from inception to October 2021: PubMed, Embase, Web of Science, Cochrane Library, Chinese Biological Medicine Database (CBM), Chinese National Knowledge Infrastructure (CNKI), China Science and Technology Journal Database (VIP), and Wang Fang Data Database. The search strategy included keywords and medical subject headings for “acupuncture,” “urticaria,” “systematic review,” and other related terms. In addition, additional citations were discovered by manual searches of reference lists and personal collections. Supplemental 1 includes the detailed search strategy for the eight electronic databases mentioned above.

2.2. Study Selection

Two reviewers independently identified studies that met the following criteria: (i) randomized controlled trials (RCTs), quasi-randomized control trials, or controlled clinical trials (CCTs) were included in SRs; (ii) all interventional SRs that investigated the efficacy and safety of acupuncture and related therapies in CU were eligible. There were no restrictions based on age, gender, nationality, or geographic location; (iii) either acupuncture and related therapies alone or combined with other conventional Western medicine were recognized as a treatment intervention; the control types included conventional treatments that were different from the treatment intervention, placebo, or no treatment; (iv) at least one of the following outcome measures listed below was required to be obtainable: total efficiency, adverse reaction rate, recurrence rate, clinical efficacy rate, improvement rate, curing rate, IgE levels, Dermatology Quality Life Index (DLQI), quality of life score (QoL), a total improvement of clinical signs and symptoms, disease activity control, and Itch Severity Score (ISS); (v) SRs published in English or Chinese were included. SRs that were noninterventional SRs, comments, guidelines, editorials, incomplete articles, proceedings, and responses were excluded. Disagreements were settled through discussion and consensus.

2.3. Data Abstraction

One reviewer extracted descriptive data using a standardized form, which was then verified by a second reviewer. The following data were extracted: author, published country, age, study type, number of trials included, interventional methods, comparator, outcomes, methods for primary study quality assessment, etc. Based on what the review authors reported, data from these SRs were expressed as standardized mean differences, weighted mean differences, odds ratios (ORs), or relative risks (RRs). Whenever possible, the results of the meta-analysis are also presented with 95% confidence intervals (CIs).

2.4. Quality of Included SRs
2.4.1. Methodological Quality of Included SRs

Two reviewers independently assessed the methodological quality of these SRs using the Assessment of Multiple Systematic Reviews 2 (AMSTAR 2) tool. The AMSTAR has a strong face and content validity for measuring the methodological quality of SRs as a methodological quality evaluation tool [42, 43]. In 2017, the expert group of AMSTAR reported on the updating and adaptation of AMSTAR [44] to allow for a more detailed assessment of SRs that includes randomized or nonrandomized studies of healthcare interventions, or both. There are 16 items in the AMSTAR 2 tool. If the item explanation was satisfied, the judgment was “Y”; if the item was answered correctly with limited information, it was evaluated as “Partial Yes”; and if the item was not subject to relevant evaluation or was evaluated incorrectly, it was evaluated as “No”. The ratings of overall confidence on the AMSTAR2 scale ranged from high to critically low. The presence of more than one critical flaw, with or without noncritical flaws, indicated critically low; one critical flaw, with or without noncritical weaknesses, indicated low quality; more than one noncritical weakness indicated moderate quality; and no or one noncritical weakness indicated high quality.

2.4.2. Reporting Quality of Included SRs

Two reviewers independently evaluated the reporting quality of the included SRs using PRISMA 2020 checklists [45, 46]. The PRISMA 2020 checklist consists of 27 items and includes items deemed essential for the transparent reporting of an SR. It was evaluated as “Yes” if the item was answered as completely correct and well documented; it was evaluated as “Partial Yes” if the item was answered correctly with limited evidence; it was evaluated as “No” if the item was not subjected to relevant evaluation or was evaluated incorrectly.

2.4.3. Risk of Bias of Included SRs

Using the risk of bias in systematic reviews (ROBIS) tool, two reviewers independently assessed the risk of bias of the included SRs [47]. The ROBIS tool contains the following 2 phases with 4 domains: “study eligibility criteria,” “identification and selection of studies,” “data collection and study appraisal,” and “synthesis and findings.” Each domain contains signaling questions as well as a judgment of concerns about the domain’s risk of bias, and the results are regarded as “high risk,” “low risk,” or “unclear risk.”

2.4.4. Evidence Quality of Included SRs

The GRADE (Grading of Recommendations Assessment, Development, and Evaluation) was used to assess the evidence quality of the included SRs [48]. GRADE pro 3.2 software includes five downgrading factors (bias risk, inconsistency, indirectness, imprecision, and publication bias) and three upgrading factors (a large magnitude of the effect, the influence of all plausible residual confounding, and the dose-response gradient). In this study, only SRs included in RCTs are included, so only 5 downgrading factors need to be analyzed. Among them, inconsistency was mainly divided into no (I2 < 50%), serious (50% < I2 < 80%), and very serious (I2 > 80%). Imprecision determined whether the total sample size met the optimal information size, and publication bias was determined according to the funnel plot. Each outcome of the included studies was evaluated by two reviewers. Disagreements were settled through consensus or by consulting a third investigator.

All included SRs were evaluated by two independent reviewers. We used SPSS 23.0 statistical software (IBM Corp., Armonk, New York, USA) for consistency analysis, and the kappa consistency test was used to evaluate [49]. Inconsistent results were reevaluated by another reviewer.

3. Results

A total of 212 articles were detected in the initial search. 92 duplicate articles were excluded by using EndNote and manual searches. After screening the titles and abstracts, 93 articles were excluded. Then, full texts were screened according to the inclusion and exclusion criteria, and 4 articles were excluded. Ultimately, 23 SRs were included in the present study. Figure 1 presents the flow of studies through the selection process.

3.1. Basic Characteristics of the Included SRs

The 23 included SRs were published between 2009 and 2021. Among the 23 SRs, 5 English papers and 18 Chinese papers were included, and the number of RCTs examined in the SRs ranged from 5 to 16. All reviews were conducted in China, and a total of 11 outcome indicators were reported. In terms of intervention measures, among the 23 SRs, 10 SRs examined autohemotherapy at acupoints [2830, 33, 3539, 41], 6 SRs examined acupuncture [22, 24, 25, 27, 34, 40], 3 SRs examined catgut embedding at acupoints [26, 31, 32], 1 SR examined bloodletting therapy [20], 1 SR examined auriculotherapy [21], 1 SR examined cupping [23], and 1 SR examined acupoint stimulation [19]. For the assessment of methodological quality, 17 SRs used the Cochrane risk of bias tool [1923, 2529, 31, 33, 36, 37, 3941], 4 SRs used the Jadad scale [32, 34, 35, 38], and 2 SRs used the Jadad scale and Cochrane risk of bias tool [24, 30]. All SRs included a meta-analysis; 9 out of 23 SRs included subgroup analysis [19, 23, 24, 29, 30, 3841]; and 13 SRs included sensitivity analysis [20, 21, 2527, 2932, 3740]. The general characteristics of these SRs are summarized in Table 1.

3.2. Quality of Included SRs
3.2.1. Methodological Quality of Included SRs

Table 2 shows the rating overall confidence of the individual quality components based on the AMSTAR 2 tool. All SRs were of critically low quality. Items 2, 3, 7, 9, 10, and 16 were rated as particularly low quality. Among the 23 SRs, twenty (86.96%) SRs used the Patient Interventions Control Outcomes (PICO) description as an organizing framework for a study question and used a satisfactory technique for assessing the risk of bias (RoB). All review authors applied appropriate methods for statistical synthesis. However, only a small proportion (8.70%) of these SRs provided the protocol or registered information [22, 23]. No SR explained the selective inclusion of the study designs and assessed the potential impact of RoB in individual studies on the results of the meta-analysis. In addition, no SR provided a list of excluded studies and justified the exclusions. Only 2 (8.70%) SRs reported potential sources of conflict of interest [21, 23]. A summary of the AMSTAR 2 results is shown in supplementary file 2. Overall, the methodological quality of the included SRs was unsatisfactory. According to statistical analysis, the consistency coefficient kappa between the two reviewers was 0.908 (), which indicated that the evaluation of the two reviewers was independently balanced.

3.2.2. Reporting Quality of Included SRs

Table 3 shows the individual quality components of PRISMA. The consistency coefficient kappa between the two reviewers was 0.750 (). Among the 23 SRs, the title, eligibility criteria, study characteristics, and introduction were all well-reported (100%). For the introduction of these SRs, 22 (95.65%) SRs [1938, 40, 41] described the rationale for the review in the context of what is already known and provided precise and explicitly framing questions in objectives. For the methods of these SRs, the search strategy, data items, synthesis method, reporting bias assessment, and certainly assessment were reported inadequately. Regarding the search strategy, none of the SRs reported using filters or restrictions. Regarding the data items, more than half (60.87%) of the SRs [2023, 2932, 34, 3741] listed and defined all outcome indicators, but none of the SRs listed and defined all other variables. Over 56% of the SRs [2426, 29, 3135, 3739, 41]failed to describe the methods used to assess the risk of bias due to missing results in a synthesis, and over 90% of the SRs [19, 21, 2341] failed to describe methods for evaluating the quality of evidence for each outcome. Regarding the results of the SRs, the flow diagram and text did not clearly describe the process of report selection in 2 (8.70%) SRs [24, 32]. Only 1 (4.35%) SR [20] provided the results of individual studies. Similarly, only 2 (8.70%) SRs presented reporting biases [20, 32] and 2 (8.70%) SRs showed certainly evidence [20, 22]. No sensitivity analysis was presented. For the discussion of these SRs, nearly all (95.65%) SRs [1936, 3841] discussed the limitations of the evidence and the implications of the findings for practice, policy, and future research. However, fewer than 15% of them interpreted the results based on other evidence and discussed limitations in the study process. In addition, 3 (13.04%) SRs [20, 22, 23] provided registration information. Only 1 (4.35%) SR [22] provided access to the protocol or stated that there was no protocol and described the relevant content and modification. Over 90% of these SRs [19, 20, 22, 2441] failed to describe sources of funding and other support. The proportion of the individual components of PRISMA of included SRs is summarized in Figure 2. Overall, the reporting quality of the included SRs was low.

3.2.3. Risk of Bias of Included SRs

Figure 3 presents the results of the risk of bias of the included SRs by using the ROBIS tool, which contains 2 phases with 4 domains. Phase 1 was not performed in our study. Domain 1 assessed concerns regarding the specification of study eligibility criteria, and 16 of 23 (69.57%) SRs were rated as having a low risk of bias [1921, 2427, 29, 30, 32, 3641]. Seven of 23 (30.43%) SRs were rated as having a high risk of bias [22, 23, 28, 31, 3335]. Domain 2 assessed concerns regarding methods used to identify and select studies. All SRs (100%) were at high risk of bias. Domain 3 assessed concerns regarding methods used to collect data and appraise studies; 12 (52.17%) SRs had a low risk of bias [1923, 25, 26, 29, 31, 36, 38, 41], 5 (21.74%) SRs had a high risk of bias [24, 3235], and 6 (26.09%) SRs had an unclear risk of bias [27, 28, 30, 37, 39, 40]. Domain 4 assessed concerns regarding the synthesis and findings: 22 (95.65%) SRs had a high risk of bias [19, 20, 2241], and 1 (4.35%) SR had an unclear risk of bias [21]. The individual quality components of ROBIS of included SRs are summarized in Table 4. The final phase considered the overall risk of bias of these SRs to be high. The consistency coefficient kappa between the two reviewers was 0.827 ().

3.2.4. Evidence Quality of Included SRs

This study includes 23 SRs involving 55 outcomes. Table 5 shows the individual quality components of GRADE of included SRs. The consistency coefficient kappa between the two reviewers was 0.844 (). For all outcomes, there were 3 outcomes with moderate-quality evidence [34], which included the total efficiency rate using the symptom score reducing index (SSRI), recurrence rate, and curing rate. The results of three moderate-quality studies showed that the total effective rate and curing rate of the acupuncture group were higher than those of the western medicine group, and the recurrence rate was lower than that of the western medicine group. There were 6 outcomes with low-quality evidence and 46 outcomes with very low-quality evidence. Figure 4 presents the quality of the evidence for the included SRs. In addition, 14 outcomes had a large amount of heterogeneity (I2 > 50%), and 2 had a very large amount of heterogeneity (I2 > 80%). The elevated risk of bias, imprecision, and public bias were the main reasons for downgrading. Significant heterogeneity downgraded inconsistency, and imprecision was downgraded because the total sample size did not meet the optimal information size.

3.2.5. Evidence from Quantitative Research Syntheses

In the 23 SRs, there were 11 outcome indicators, including total efficiency, adverse reaction rate, recurrence rate, clinical efficacy rate, improvement rate, curing rate, IgE levels, DLQI, QoL, total improvement in clinical signs and symptoms, and disease activity control. The types of intervention are acupuncture and related therapies, including bloodletting, cupping, auriculotherapy, acupoint injection, autohemotherapy, and catgut embedding. In the 23 SRs, the efficacy of acupuncture therapy in CU treatment (acupuncture alone or acupuncture-based combination therapies) yielded superior results to the control treatments. Considering the wide range of acupuncture and related therapies, we conducted a categorical analysis. When acupuncture was compared with antihistamines, one SR found no significant difference in the total effective rate (OR = 2.19, 95%CI (0.79 to 6.07), , I2 = 47%) [40], and another SR reported the opposite conclusion (RR = 1.21, 95%CI (1.00 to 1.46), ) [19]. Additionally, the other two SRs found acupuncture to be more effective on global symptom improvement (RR = 1.37, 95%CI (1.11 to 1.70), , I2 = 23%) [22] and curing rate (RR = 2.14, 95%CI (1.64 to 2.79), , I2 = 32%) [24]. These 4 SRs also analyzed the efficacy between acupuncture with antihistamines and antihistamines, and the results showed that added acupuncture was more effective (RR = 1.77, 95%CI (1.41 to 2.22), , I2 = 0%; RR = 1.20, 95%CI (1.07 to 1.35), , I2 = 0%; OR = 6.59, 95%CI (2.69 to 16.16), , I2 = 0%; RR = 2.03, 95%CI (1.35 to 3.06), , I2 = 48%) [19, 22, 24, 40]. In addition, there was no statistical significance when cupping was compared with antihistamines (RR = 1.10, 95%CI (0.97 to 1.25), , I2 = 52%) [23]. However, the efficacy of cupping combined with antihistamines was better than that of antihistamines alone (RR = 1.18, 95%CI (1.01 to 1.39), , I2 = 67%) [23]. Autohemotherapy was the most frequently used therapy among the included SRs. Overall, 73.08% of these results were effective. Interestingly, when catgut embedding was compared with other therapies, the difference in efficacy was statistically significant (RR = 1.1, 95%CI (1.03 to 1.16), , I2 = 30%) [26], but when it was combined with antihistamines, the result was reversed (RR = 1.14, 95%CI (0.98 to 1.34, ) [26]. This result may be related to the low sample size and low quality of the reviews. In addition, we also examined the efficacy of bloodletting, auriculotherapy, and acupoint injection. More details are provided in Table 6.

4. Discussion

When there is a need to quickly gather evidence to inform new policies or procedures, existing research syntheses are available [5053]. SRs, which have the highest level of evidence, are increasingly used for evidence-based decision-making [53]. Currently, acupuncture in CU has been widely used in clinical practice in China [54]. A variety of SRs about CU interventions has been published with varying recommendations of treatment effectiveness. In this evidence-based review, we analyzed the evidence provided at the SR level of acupuncture and related therapies in CU and assessed the methodological and reporting quality of these SRs. The results showed that there were some deficiencies in the quality of methodology and reporting. The results of the AMSTAR 2 indicated that the authors should complete the protocol or register in the Cochrane Library or PROSPERO website in advance to reduce the risk of bias. Second, all SRs did not provide a list of excluded studies and justify the exclusions, which may lead to bias in the findings. In addition, the selection of study types for inclusion, the source of research funding for randomized trials in included SRs, and the potential impact of RoB in individual studies on the results of the meta-analysis were not well explained, which can directly reduce the rating overall confidence. Therefore, authors should adequately report these items in future studies. Notably, AMSTAR 2 was published in 2017. Considering that the majority of the included SRs in this study were published before 2017, the low quality of methodological evaluation may be related to a failure to meet the latest quality standards.

From the summary of PRISMA and ROBIS, some reporting shortcomings should be more noticeable in future research. Regarding the methods of the SRs, the authors should report the existence of the review protocol and full electronic search strategy for at least one major database, and they should describe the methods of analysis in detail. In addition, when reporting the results, the authors should assess ROB across studies and give results of additional analyses, if done (e.g., sensitivity or subgroup analyses, meta-regression). For the discussion of the SRs, it is suggested that the authors should assess the strength of evidence for each main outcome and consider their relevance to key groups. In addition, readers and decision-makers in health care can judge whether evaluation bias and conflicts of interest exist, and it is important for review authors to record the sources of funding reported in every study.

We summarized the current evidence of acupuncture and related therapies in CU for a wide range of outcome indicators. On the whole, compared with the control group, most of these results indicated that the experimental group was effective. According to the GRADE analysis, compared with the quality of different acupuncture treatments, the results of three moderate-quality outcomes showed that the total effective rate and curing rate of the acupuncture group were higher than those of the western medicine group [34], and the recurrence rate was lower than that of the western medicine group [34]. However, the evidence for other acupuncture treatments was low [24, 26, 27, 29, 31, 41] or very low-quality [1933, 3541], and no significant qualitative differences were found between the different treatment options. The factors that led to the (very) low-quality evidence may be as follows: the original studies were of poor quality, and all outcome measures were significantly at risk of bias due to the randomization, allocation concealment, and blinding of the included original studies. However, this has a certain relationship with the difficulty of acupuncture itself to strictly blind the patients. In addition, the research methods of these SRs were incomplete implementations. For 88% of the outcome indicators, the total sample size of the included SRs did not meet the optimal information size, and the confidence interval was wide, which seriously affected the imprecision of the outcome indicators. In addition, 82% of the outcome indicators have a large possibility of publication bias due to the small number of included primary studies and positive results.

The pathogenesis of CU is still unclear. At present, antihistamines are used to complete symptom control. However, many patients have an inadequate response to the majority of these drugs [14], which causes recurring attacks of CU. The above results can provide an evidence-based reference for acupuncture and related therapies in the treatment of CU. In clinical practice, when the patient is in an inadequate response to antihistamines, the doctors can choose to apply acupuncture and related therapies to treat CU based on comprehensive consideration of the patients’ symptoms and preferences. When the symptoms of urticaria become recurrent or more severe, the evidence support that acupuncture and related therapies combined with another active therapy can improve symptom and reduce the recurrence rate [1925, 27, 29, 34, 39, 40]. Additionally, the results suggest that we need to pay more attention to the quality of the primary study and strengthen training to meet relevant requirements on methodologies and reporting of the SRs and address these existing deficiencies of the SRs in future research.

Our study had some limitations. First, while all attempts were made to search and access all relevant literature, it is possible that some publications may have been missed in the search process due to language restrictions. Second, although uniform training was performed before our study, consistency analysis was performed after our study. The personal beliefs of authors can influence their judgment, and the study was also limited due to the subjectivity of quality evaluation.

5. Conclusions

Acupuncture and related therapies for the treatment of CU are supported by low-quality evidence-based medicine. However, considering the poor quality of these SRs, we suggest that studies with more rigorous designs, larger sample sizes, and higher methodological and reporting quality are necessary to provide stronger evidence.

Data Availability

All relevant data are within the paper and its supporting information files.

Conflicts of Interest

The authors declare that they have no conflicts of interest.

Authors’ Contributions

YZS, WC, and CXL contributed to the conception and design. ZHZ, SJC, and QY contributed to the acquisition of data. XJX, YH, LW, and LXZ contributed to the analysis and interpretation of data; YZS and WC drafted the manuscript; PSH, ZPG, and YL revised the manuscript. All authors read and approved the final manuscript. YZS, WC, and CXL contributed equally to the study.

Acknowledgments

This study was supported by the National Natural Science Foundation of China (No. 82105026), National Key Research and Development Program of the Ministry of Science and Technology of the People’s Republic of China (No. 2017YFC1703605), and China Postdoctoral Science Foundation (No. 2021M693787). The funders had no role in study design, data collection and analysis, decision to publish, or preparation of the manuscript.

Supplementary Materials

Supplemental 1: search strategy for PubMed. Supplemental 2: the summary of the AMSTAR 2 results.() (Supplementary Materials)