Abstract

Background. Acupuncture had long been a primary treatment in the healthcare system of China. In recent years, there were more and more network meta-analyses (NMAs) in the field of acupuncture and moxibustion, but the quality evaluation of NMAs was rare. Objectives. The goal of this study was to evaluate the methodological and reporting quality of NMAs and summarize the effects of different treatments of acupuncture and moxibustion. Methods. PubMed, Embase, the Cochrane Library, China National Knowledge Infrastructure Database (CNKI), WanFang Database (WF), Chinese Scientific Journal Database (VIP), and Chinese Biomedical Literature Database (CBM) were searched from inception to January 2020 without any language restriction. In addition, the unpublished studies and the references of initially included literature were also retrieved manually. We included all relevant NMAs treated with acupuncture and moxibustion; other therapies such as traditional Chinese medicine and Western medicine may also be included, but at least three types fall under the category of acupuncture in each NMA. Outcome indicators were not limited. We selected AMSTAR2 and PRISMA-NMA to evaluate the methodological and reporting quality of eligible studies, respectively. Results. In total, 29 NMAs were included finally, including 12 Chinese references and 17 English references. All eligible studies were published from May 2013 to August 2019. The number of interventions was between 4 and 22. The number of clinical trials included ranged from 10 to 121, with a total of 1098 clinical trials. The NMAs were involved in up to 23 diseases, knee osteoarthritis and primary dysmenorrhea covered with 3 NMAs separately, others focusing on chronic functional constipation, lumbar disc herniation, chronic fatigue syndrome, and the like. The Jadad scale and RoB scale were used as the bias risk assessment tools. Among them, 7 articles adopted the Jadad scale, 22 articles adopted the RoB scale (1 article adopted both the Jadad scale and RoB scale), and only 1 article did not mention the risk assessment tool. The AMSTAR2 methodological evaluation showed that the highest score was 13.5 points and the lowest was 4, with an average of 8.64 and a median of 9.5. According to the quality criteria, only one of them was in high quality, twenty-four were in medium quality, and four were in low quality. The PRISMA-NMA reporting quality evaluation showed that the highest score was 29 points and the lowest was 13.5, with an average of 23.62 and a median of 24.5; severe flaws also existed in some items, especially in “Structured summary,” “Protocol and registration,” “Search,” “Data collection process,” “Data items,” “Additional analyses,” “Risk of bias across studies,” and “Results of additional analyses.” Conclusion. The number of NMAs in the field of acupuncture and moxibustion was still in the initial stage. Overall, their methodology and reports were of moderate quality. However, severe flaws also existed in some items. Because the eligible NMAs were limited, the conclusion needed further research to confirm its authenticity and reliability.

1. Introduction

Acupuncture has a history of more than 4,000 years and is now commonly used in medical care in China. As a traditional oriental therapy, it has been widely used globally [1]. With the development of evidence-based acupuncture and moxibustion, acupuncture and moxibustion is more and more popular in clinics because of its simple treatment, quick effect, and nontoxic side effects. However, there are many kinds of acupuncture and moxibustion intervention methods, and the indications are similar. Traditional meta-analysis can only achieve pairwise direct comparison of intervention measures, but it cannot carry out indirect comparison of intervention measures without a direct comparative study, and let alone comparative analysis of various intervention measures; so, it is difficult to solve this problem.

Network meta-analyses (NMAs) are still named multiple-treatments meta-analysis (MTM) or mixed treatment comparison meta-analyses (MTC). Meta-analysis is an analytical method that evaluates the relative efficiency of treatments and synthesizes evidence using a randomized controlled trial network diagram. This method is based on the combination of traditional direct/head-to-head comparison and indirect comparison for meta-analysis, which can simultaneously compare the efficacy of three or more interventions. It was developed on the basis of classical meta-analysis, which resulted in a consistent and complete set of comparisons based on all available evidence from existing research studies [25]. The NMA provides evidence for clinical drug selection by quantifying different interventions to treat the same disease and ranking the benefits [6]. As an emerging evidence synthesis tool, NMAs are becoming more and more popular, which can make more decisions or choices than classic meta-analysis [710].

In recent years, NMA has made some progress in the field of acupuncture and moxibustion, and the number of publication of NMAs related to acupuncture and moxibustion is on the rise, but its quantity is still small, and its quality still lacks systematic evaluation. This study through retrieving NMAs of acupuncture and moxibustion published a comprehensive understanding of the present situation, and apply AMSTAR2 (a measurement tool to assess the methodological quality of systematic reviews) and PRISMA-NMA (PRISMA extension statement for reporting of systematic reviews incorporating network meta-analyses of health care interventions: checklist and explanations) to assess the methodological and reporting quality of the published NMAs in acupuncture and moxibustion field systematically, in order to offer reference to improve the quality of acupuncture and moxibustion in NMA.

2. Materials and Methods

2.1. Information Sources

PubMed, Embase, the Cochrane Library, China National Knowledge Infrastructure Database (CNKI), WanFang Database (WF), Chinese Scientific Journal Database (VIP), and Chinese Biomedical Literature Database (CBM) were searched from inception to January 2020 without any language restriction. In addition, the unpublished studies and the references of initially included literature were also retrieved manually. The comprehensive search strategy for PubMed is presented in Table 1. The retrieval of other electronic databases was similar to PubMed, which adopts the combination of subject words and keywords.

2.2. Eligible Criteria

We included all relevant NMAs treated with acupuncture and moxibustion; other therapies such as traditional Chinese medicine and Western medicine may also be included, but at least three types fall under the category of acupuncture in each NMA. Outcome indicators were not limited, while language limited in Chinese and English.

2.3. Exclusion Criteria

(1) Duplicate detection and republished literature. (2) Theoretical research. (3) Reviews, conference papers, and abstracts. (4) Incomplete data of the results. (5) Acupuncture interventions <3. (6) Non-Chinese and English literature.

2.4. Study Selection and Data Extraction

The search was conducted by NoteExpress 5.3.0 literature management software. NoteExpress 5.3.0 automatic duplicate check function was used and combined with manual duplicate check to eliminate the duplicate research. By reading the title and abstract, we excluded studies that obviously do not meet the inclusion criteria; downloaded and read the full text of the remaining studies to judge; and for research with incomplete data report, tried to contact the author and gain the complete data. The three evaluators (TY, XW, and JY) screened and extracted the literatures back to back independently according to the inclusion and exclusion criteria, and cross-checked the results. The included details were as follows: author information, year of publication, sample size, disease, type of study, diagnostic criteria, number of original study, description of interventions, number of interventions, comparators, outcome measures, and risk assessment tools for bias and adverse reactions. When there was any disagreement, it was resolved by the fourth researcher (JX).

2.5. Methodological and Reporting Quality Assessment Tools

Three independent researchers (TY, XW, and JY) evaluated the methodological and reporting quality back to back. The AMSTAR2 tool contained 16 aspects [11]. For each aspect, when the answer was “yes,” the score was 1, and when the answer was “no,” “cannot answer,” or “not applicable,” the score was 0. The total score of an NMA was calculated by counting the number of “yes” in 16 items on a scale of 0–16. A score of 12–16 was rated as “high quality,” a score of 7–11 was rated as “medium quality,” and a score of 0–6 was rated as “low quality.” The PRISMA-NMA contained 32 items [12]. For each item, a score of “1” means full compliance, “0.5” means partial compliance, and “0” means noncompliance [13, 14]. The total PRISMA-NMA score of an NMA was calculated by accumulating the scores of each item, with a range of 0–32. A score of 26–32 was rated as “high quality,” a score of 20–25.5 as rated as “medium quality,” and a score of 0–19.5 as rated as “low quality.” When there was any disagreement, it was resolved by the fourth researcher (JX).

2.6. Data Analysis

We analyzed the characteristics of included studies through descriptive statistical methods. All the data used were counted in the Excel 2007 spreadsheet. We described the dichotomous data in terms of number and percentage and the continuous variables in terms of median with interquartile range (IQR). And we calculated the number of papers per item, its percentage, and 95% confidence intervals. We summarized the scores according to the quality evaluation tool. AMSTAR2 and PRISMA-NMA scored 16 and 32, respectively. Finally, we calculated the total score through adding a list of each component.

3. Results

3.1. Search Results

242 related references through searching seven electronic databases and manual searches were retrieved. After reading the title and abstract, a total of 29 studies were included [1543]. The literature screening process is shown in Figure 1.

3.2. Study Characteristics

The characteristics of all included NMAs were documented. After screening, 29 NMAs finally met the inclusion criteria, including 12 Chinese references (including 1 master’s thesis) and 17 English references. All eligible studies were published from May 2013 to August 2019. The study contained 4–22 types of treatments and 10–121 RCTs for 1098 RCTs in total. The NMAs were involved in up to 23 diseases, knee osteoarthritis and primary dysmenorrhea covered with 3 NMAs separately, others focusing on chronic functional constipation, lumbar disc herniation, chronic fatigue syndrome, and the like. The Jadad scale and RoB scale were used as the bias risk assessment tools. Among them, 7 articles adopted the Jadad scale, 22 articles adopted the RoB scale (1 article adopted both the Jadad scale and RoB scale), and only 1 article did not mention the risk assessment tool. The characteristics of the eligible studies are presented in Table 2.

3.3. Methodological Quality Assessment

According to the AMSTAR2 checklist, the median score and IQR of eligible NMAs was 9.5 (6.5–10.75), and the details are presented in Tables 3 and 4. The item with the best degree of compliance was “comprehensive literature search” (100%) (Table 4). Longitudinal analysis, a good degree of compliance was with item 9 (93.1%), item 11 (79.31%), and items 1, 5, and 6 (75.86%). However, a poor degree of compliance was with item 2 (17.24%) and item 3 (6.9%), and the worst degree of compliance was with item 7 (0%) and item 10 (0%).The details are presented in Tables 3 and 4 and Figure 2.

3.4. Reporting Quality Assessment

According to the PRISMA-NMA checklist, the median score and IQR of eligible NMAs was 24.5 (20.5–26.5).The item with the best degree of compliance was item 1 “Title” (100%). Longitudinal analysis, a good degree of compliance was with item 4 (96.55%), item 12 (96.55%), item 25 (96.55%), item 18 (93.10%), item 26 (93.10%), item 3 (89.65%), item 19 (89.65%), and item 24 (86.21%). However, a poor degree of compliance was with item 23 (27.59%), item 2 (17.24%), item 5 (17.24%), item 10 (17.24%), and item 11 (10.34%), and the worst degree of compliance was with item S4 (3.45%). There are significant problems in the reporting methods and results sections, such as underreporting or selective reporting. Three studies [29, 39, 42] (10.34%) reported evaluating the risk of bias within individual studies in the methods section (item 12), but did not really evaluate it in the results section (item 19). Horizontal analysis, there were four NMAs (13.79%) that scored less than 20 points, with a minimum score of 13.5. The details are presented in Tables 5 and 6.

4. Discussion

4.1. Summary of Main Findings

The goal of this cross-sectional survey was to evaluate the methodological and reporting quality of NMAs and summarize the effects of different treatments of acupuncture and moxibustion (Table 6). A total of 29 acupuncture NMAs were included in this study. From the perspective of the number and publication time, the development of NMA in acupuncture and moxibustion was still in the initial stage, with a small number, but it had shown a trend of gradual growth. In terms of disease, more than half were chronic pain. In terms of intervention measures, more than half were acupuncture combined therapy. It was not difficult to find that NMAs of acupuncture and moxibustion was still limited to several diseases, and there were still large gaps in many aspects.

The methodological quality of NMAs was important, so we evaluated the methodological quality of NMAs in acupuncture and moxibustion according to AMSTAR2 tool. The results showed some methodological deficiencies, particularly with regard to item 2, item 3, item 7, and item 10. The highest NMA score for each item was 13.5, the lowest was 2.5, and the median and IQR was 9.5 (6.5–10.75), indicating average methodological quality.

The quality evaluation of the report showed that the quality of acupuncture NMAs was generally acceptable, indicating that the NMA researchers had a high level of evidence-based medical knowledge and scientific research literacy. Some items need to be improved, particularly with regard to the structured summary (item 2), protocol and registration (item 5), search (item 8), data collection process (item 10), data items (item 11), additional analyses (item 16), risk of bias across studies (Results section) (item 22), and results of additional analyses (item 23). From the perspective of a single NMA, the highest score was 29, the lowest was only 13.5, and the median and IQR was 9.5 (6.5–10.75), showing that the quality of reports included in the study was of average quality. PRISMA-NMA checklist was helpful to improve the reporting quality of acupuncture NMAs. Therefore, it was necessary to improve the comprehensiveness and standardization of the report.

4.2. Strengths and Limitations

First, this was the first study that evaluated the methodological and reporting quality of NMAs comprehensively, which complied with the methodological and reporting guidelines in the field of acupuncture and moxibustion. Even though there were two studies regarding the methodological or reporting quality of NMAs in TCM, one English article excluded acupuncture and moxibustion [44] and one Chinese article included acupuncture and moxibustion literature incompletely [45]. Second, compared with published quality studies of NMAs in acupuncture and moxibustion, this review implemented a more comprehensive and detailed literature retrieval strategy. In addition, the unpublished studies and the references initially were also retrieved manually. As a result, the results were more credible.

This research also had presented some limitations: first, there was no specific methodological quality assessment tool for NMAs, even though AMSTAR2 was generally used in the quality evaluation of systematic review and meta-analysis. Second, in this study, the quality of each NMA was quantified by the assignment method, and there were some controversies on whether the weight of each item was consistent. Third, even if a comprehensive literature search strategy was used, there was no guarantee that all relevant literatures were identified. Finally, since only Chinese and English studies were included, there may be a lack of data to influence the results.

5. Conclusion

The NMAs methodological and report quality related to acupuncture and moxibustion were general, and there was still room for improvement in some aspects. For example, the researchers should design the scheme in advance before carrying out the study, design and carry out the study strictly in accordance with PICOS, and present the network structure, so as to improve the prospective and reliability of the study. Considering the importance of PRISMA-NMA checklist to NMA, we advise that the researchers should strictly follow the PRISMA-NMA checklist when writing a NMA.

Abbreviations

AMSTAR2:A measurement tool to assess the methodological quality of systematic reviews
PRISMA-NMA:PRISMA extension statement for reporting of systematic reviews incorporating network meta-analysis of health care interventions
NMA:Network meta-analysis.

Data Availability

The data used to support the findings of this study are available from the corresponding author upon request.

Conflicts of Interest

The authors declare that they have no conflicts of interest.

Authors’ Contributions

JX obtained funds for this study. JX and TY made contributions to the conception and design of the review. FYJ revised the search strategy. TY, JY, and XW searched, extracted, and analyzed data for studies. YT, JY, XW, and JX conducted methodological and reporting quality evaluation. FYJ, HXZ, KL, and LLX assisted. Ting Yuan wrote the original draft. All authors reviewed the manuscript, agreed to all the contents, and agreed the submission.

Acknowledgments

This study was supported by Jiangxi University of Traditional Chinese Medicine 1050 youth talent project (5141900101), “One Thousand Talents Plan” for Introducing and Training High-level Innovative and Entrepreneurial Talents in Jiangxi Province—The First Batch of Training High-level Scientific and Technological Innovative Talents (Youth) Project (jxsq2019201104), and Natural Science Youth Foundation Key Projects of Jiangxi Province (20192ACB21007).