Evidence-Based Complementary and Alternative Medicine

Evidence-Based Complementary and Alternative Medicine / 2013 / Article

Review Article | Open Access

Volume 2013 |Article ID 708251 | https://doi.org/10.1155/2013/708251

Alex Moroz, Brian Freed, Laura Tiedemann, Heejung Bang, Melanie Howell, Jongbae J. Park, "Blinding Measured: A Systematic Review of Randomized Controlled Trials of Acupuncture", Evidence-Based Complementary and Alternative Medicine, vol. 2013, Article ID 708251, 12 pages, 2013. https://doi.org/10.1155/2013/708251

Blinding Measured: A Systematic Review of Randomized Controlled Trials of Acupuncture

Academic Editor: Gerhard Litscher
Received17 Oct 2012
Revised06 Jan 2013
Accepted06 Jan 2013
Published03 Mar 2013


Background. There is no agreement among researchers on viable controls for acupuncture treatment, and the assessment of the effectiveness of blinding and its interpretation is rare. Purpose. To systematically assess the effectiveness of blinding (EOB) in reported acupuncture trials; to explore results of RCTs using a quantitative measure of EOB. Data Sources. A systematic review of published sham RCTs that assessed blinding. Study Selection. Five hundred and ninety studies were reviewed, and 54 studies (4783 subjects) were included. Data Extraction. The number of patients who guessed their treatment identity was extracted from each study. Variables with possible influence on blinding were identified. Data Synthesis. The blinding index was calculated for each study. Based on blinding indexes, studies were congregated into one of the nine blinding scenarios. Individual study characteristics were explored for potential association with EOB. Limitations. There is a possibility of publication or reporting bias. Conclusions. The most common scenario was that the subjects believed they received verum acupuncture regardless of the actual treatment received, and overall the subject blinding in the acupuncture studies was satisfactory, with 61% of study participants maintaining ideal blinding. Objectively calculated blinding data may offer meaningful and systematic ways to further interpret the findings of RCTs.

1. Introduction

The presence of a viable control is important for any research study, allowing for valid comparison to the condition of interest. Randomized controlled trials (RCTs) are the primary way that many areas of research examine the comparative effectiveness of verum and control conditions. Patient blinding in these studies is essential for gathering reliable results that can be expanded upon. However, blinding success in nonpharmacologic interventions, such as acupuncture, can be difficult to assess, and results may not be applicable from one study to the next, due to differences in control methods and in how blinding success is determined, if it is considered at all. The CONSORT 2010 Statement on blinding suggests that it is important to include how blinding was attempted, but there is no mention in the statement of how to assess whether blinding was successful [1]. Without knowing if the specific nonpharmacologic blinding techniques used are valid overall, the information collected may or may not be reliable. We believe it is imperative that blinding success be assessed toward better understanding of the reliability of results.

With respect to acupuncture research specifically, developing a control procedure that is physiologically inert and indistinguishable from true treatment has proven to be a challenge due to the very nature of traditional acupuncture. It has resulted in a variety of control methods that up until now have not been compared in terms of blinding success [25]. The traditional techniques of acupuncture (penetrating needles at acupoints for different organ systems and deqi) are the verum conditions of most acupuncture studies. Control conditions include penetrating needles at “nonpoints” and “wrong points,” commercially-developed nonpenetrating devices, homemade nonpenetrating needles, and toothpicks or cocktail sticks [69]. “Nonpoints” are points not used for any purpose in traditional acupuncture. For the “wrong points,” sham acupuncture is done at points thought to affect a different body system than the one targeted by the verum condition. However, the “wrong point” method may cause a physiological effect similar to that of acupuncture and, therefore, may be more appropriately considered to be verum than sham acupuncture [10].

To date, there is no standard or universally accepted sham procedure for acupuncture research and no quantitative comparison of blinding between the above sham methods. This may contribute to why there is a discrepancy between the clinically recognized effectiveness of acupuncture and the relative lack of research supporting it [11]. Methodological progress for blinding characteristics, including the amount of disclosure to study participants, the variables to be collected, the analytic design, and the interpretation strategy, is needed in validation studies of sham control procedures [12].

The present meta-analysis systematically examines the status of blinding in sham acupuncture RCTs via a numerical measure of blinding index. Our primary aim is to empirically evaluate the validity (via effectiveness of blinding) of sham control techniques in order to quantitatively assess blinding across available studies. We hope to determine the reliability of the results of studies that used different sham techniques, so that we may learn which sham methods are most useful for future acupuncture research. We also believe that our systematic review is just the beginning of increasing the validity of the assessment of quantitatively assessed blinding practices in acupuncture research. Our methods can be further extended as a model to assess other nonpharmacologic treatments’ blinding techniques.

2. Methods

2.1. Data Sources and Searches

PubMed, Embase, and Web of Knowledge databases were searched for scientific articles using the keywords “acupuncture,” “sham acupuncture,” or “sham procedure.” A revised search was also performed with Ovid Medline using the keywords “acupuncture,” “sham acupuncture,” or “placebo acupuncture.” Eligible studies were those that were randomized controlled in humans and were published in English between 1985 and 2011. Our search was not limited by patient diagnosis or by the part of the body where acupuncture was administered.

2.2. Study Selection

A study was considered eligible for initial screening if the authors stated that they evaluated blinding. One of the authors (Moroz, Freed, or Tiedemann) determined if the study reported data on effectiveness of blinding (EOB) by specifically asking participants if they thought the treatment used verum (V, real needle used or needle penetrated the skin) or sham (S) needling techniques, with or without an option to say they did not know (DK). If the blinding evaluation data was not included or was unclear, the authors were contacted twice by e-mail and/or telephone and asked for additional blinding evaluation data or clarification. Studies whose authors did not respond or responded but no longer had access to the original data were excluded. Studies that used a credibility questionnaire, asking patients to choose whether or not they had received treatment based on the principles of Chinese medicine or another type of acupuncture, were excluded as well. In these studies, the patients’ ability to distinguish between verum and sham needling was not directly addressed and the questionnaire could be misinterpreted. If EOB was measured multiple times within the same study, the data collected at the end of the study or the data collected from the insertion (as opposed to sensation of deqi or needle withdrawal) arm of the study would be used. In one work that reported results of two separate studies using different acupuncture points, one on the patients’ back and the other on the upper extremity, the blinding index calculations were done independently [58].

2.3. Data Extraction

For the studies that were included in this meta-analysis, the number of patients who responded as V, S, or DK was extracted from each trial. Additionally, the following variables were extracted from each of the included studies (we hypothesized a priori that these may be associated with EOB): year of publication, subject only or staff and subject blinding, time of blinding assessment, assessment of deqi, type of sham control device used, use of penetrating or nonpenetrating sham control, patient diagnosis, and number of days without acupuncture experience prior to participation. Research staff members that were blinded were those involved in data analysis and interpretation, not those administering acupuncture.

2.4. Data Synthesis and Analysis

Statistical analyses were carried out using the blinding index (BI) in order to objectively assess EOB [63, 64]. Blinding index estimates the degree of potential unblinding beyond chance for each arm in a given study by counting the excessive numbers of correct guesses. Blinding index values are always between −1 and 1, where 1 corresponds to all correct guesses, whereas −1 corresponds to all incorrect or opposite guesses. If 50% of patient responses are correct and 50% are incorrect, then ; this is indicative of random guessing and thus is an ideal blinding scenario. Another plausible scenario indicating effective blinding is that patients tend to believe they received active treatment regardless of actual treatment received, which may reflect patients’ wish to receive active intervention. In this case, blinding index will have a positive value in the active treatment arm and a negative value in the sham treatment arm, where this scenario is denoted later as unblinded/opposite.

Verum and sham acupuncture groups were each assigned a separate blinding index value. Based on the calculated blinding index value combinations for the two treatment arms, nine possible blinding scenarios were proposed (Table 1). For classification purposes, we decided to consider unblinded; random guesses; opposite guesses [13, 64, 65]. (Remark: this cutoff value was based on authors’ consensus and used as a general tool for classification and explanation; it should not be interpreted as an absolute indication of blinding effectiveness.)

Experimental arm (verum)Control arm (sham)Possible blinding and clinical effectiveness interpretationsTrials number (%)

Random guessRandom guessIdeal, possibly most ideal from the scientific or statistical perspective8 (15)
Random guessOpposite guessRare2 (4)
Random guessUnblindedPossibly little treatment effect and completely no effect in control arm3 (6)
UnblindedUnblindedPossibly problematic. Treatment effect in experimental arm and no treatment effect in control arm (e.g., patients tend to know what to expect)4 (7)
UnblindedOpposite guessIdeal (e.g., patients tend to have wishful thinking, strong placebo effect, and any treatment administered is perceived as real treatment)25 (46)
UnblindedRandom guessPossibly problematic. Treatment effect in experimental arm and no treatment effect in control arm (e.g., patients do not know what to expect in the absence of treatment)12 (22)
Opposite guessOpposite guessRare0 (0)
Opposite guessRandom guessRare0 (0)
Opposite guessUnblindedNo treatment effect at all; patients may have low expectations0 (0)

Individual variables hypothesized to potentially impact EOB were compared by their average VBI and SBI scores, weighted by sample size. The weighted averages were used to determine the overall blinding index scenario for each variable. Blinding scenarios were also compared to the overall outcome of each study. Finally, we looked for patterns of possible association of study design factors with EOB based on blinding index values and scenarios. The factors included were based on data extraction criteria, sample size, timing of blinding assessment, blinded parties, sensation assessed, subject’s status, subject’s experience, and sham device used.

3. Results

3.1. Data Search

Using our search inclusion criteria, 590 peer reviewed journal articles were found, with 186 of these reporting blinding data in RCTs. 133 studies were excluded from the review, most often due to a lack of patient guess of treatment allocation. One article reported two distinct studies that were included separately [58]. Fifty-four studies were included in our final analysis, with a total of 4783 patients (Figure 1).

3.2. Blinding Index Calculations

The blinding index values (point and interval estimates) computed from all 54 studies can be found in Figure 2. The average weighted blinding index values for the entire review were 0.34 for verum and −0.20 for sham groups, respectively. Overall, a correct guess is quite common in the verum arm, and opposite guess is not uncommon in the sham arm.

After grouping studies into the nine possible blinding scenarios based on the blinding index (Table 1), 33 out of 54 (61%) of the studies might be adequately blinded. Of these, 70% (23/33) had a positive outcome reported overall; similarly, 62% (13/21) with less ideal blinding had an overall positive outcome (Table 3). Unblinded/opposite for V, S is most common, with 46% of the studies belonging to this scenario, followed by unblinded/random with 22%.

3.3. Design Characteristics and Effectiveness of Blinding

The variables hypothesized to affect blinding were compared by their average VBI and SBI values, and blinding scenarios in Table 2.

Study characteristicsNumber of studiesBI ( )BI ( )BI
( )
Blinding scenario

Sample size
Blinding assessed
Blinded parties
 Subjects + research staff320.26−0.170.09Unblinded/random
Assessed for sensation of
Subject’s status
Subject’s acupuncture experience
Sham device used
 Custom 140.48−0.170.31Unblinded/random
 Penetrating 120.16−0.080.08random/random
 Toothpick or cocktail stick60.55−0.330.22Unblinded/opposite

 *Raw data available in Table 4.

Study VBIVBI 95% CISBISBI 95% CIScenario Study outcome

Assefi et al. 2005 [14]75−0.17−0.49 to to 0.49 Random/unblindedNegative
Kaptchuk et al. 2008 [15]1480.12−0.07 to 0.31−0.52−0.69 to −0.35Random/oppositePositive
Kennedy et al. 2008 [9]450.18−0.13 to 0.49−0.38−0.70 to −0.05Random/oppositePositive
Barlas et al. 2006 [6]48−0.09NA−0.06NARandom/randomPositive
Deng et al. 2008 [16]530.08−0.30 to 0.460.19−0.18 to 0.55Random/randomNegative
Enblom et al. 2008 [7]800.20.00 to 0.400.1−0.13 to 0.33Random/randomPositive
Endres et al. 2007 [17]403−0.16−0.35 to 0.03−0.04−0.24 to 0.16Random/randomPositive
Haake et al. 2007 [8]692−0.01−0.09 to to 0.19Random/randomPositive
Harris et al. 2009 [18]200.2−0.26 to 0.66−0.2−0.57 to 0.17Random/randomPositive
Shin et al. 2010 [19]42−0.05−0.35 to 0.180.05−0.23 to 0.33Random/randomNegative
Zaslawski et al. 1997 [20]640.12−0.37 to 0.130.03−0.28 to 0.21Random/randomPositive
Deng et al. 2007 [21]670−0.29 to 0.290.460.17 to 0.75Random/unblindedNegative
Jubb et al. 2008 [22]510.18−0.14 to 0.510.30.03 to 0.56Random/unblindedPositive
Alecrim-Andrade et al. 2006 [23]240.210.11 to 0.33−0.29−0.37 to −0.20Unblinded/oppositeNegative
Berman et al. 2004 [24]2830.730.63 to 0.82−0.48−0.61 to −0.35Unblinded/oppositePositive
Brinkhaus et al. 2006 [25]2050.440.30 to 0.57−0.24−0.42 to −0.05Unblinded/oppositePositive
Enblom et al. 2011 [26]1900.72 0.60 to 0.83−0.6 −0.74 to −0.46Unblinded/oppositePositive
Fink et al. 2001 [27]641NA−0.75−0.98 to −0.52Unblinded/oppositePositive
Goddard et al. 2005 [28]490.580.26 to 0.91−0.76−1.01 to −0.50Unblinded/oppositePositive
Goldman et al. 2008 [29]1180.420.24 to 0.60 −0.62−0.77 to −0.47Unblinded/oppositeNegative
Itoh et al. 2006 [30]190.480.14 to 0.81−0.32−0.70 to 0.07Unblinded/oppositePositive
Itoh et al. 2008 [31]240.560.01 to 1.10−0.5−1.10 to 0.10Unblinded/oppositePositive
Lee et al. 2008 [32]890.910.79 to 1.03−0.68−0.90 to −0.47Unblinded/oppositePositive
Lee et al. 2011 [33]350.23−0.06 to 0.51−0.56−0.80 to −0.31Unblinded/oppositePositive
Park et al. 2002 [34]580.380.20 to 0.56−0.31−0.48 to −0.14Unblinded/oppositePositive
Park et al. 2005 [35]940.470.32 to 0.61−0.31−0.48 to −0.13Unblinded/oppositeNegative
Shen et al. 2009 [36]120.630.24 to 1.01−0.5−0.99 to −0.01Unblinded/oppositePositive
Sherman et al. 2002 [37]520.650.34 to 0.96−0.52−0.80 to −0.24Unblinded/oppositePositive
Smith et al. 2007 [38]271NA−1NAUnblinded/oppositePositive
So et al. 2009 [39]3700.520.43 to 0.61−0.36−0.46 to −0.25Unblinded/oppositeNegative
Streitberger and Kleinhenz 1998 [40]600.80.65 to 0.95−0.57−0.77 to −0.36Unblinded/oppositePositive
Streitberger et al. 2004 [41]2120.220.06 to 0.37−0.25−0.38 to −0.11Unblinded/oppositeNegative
Tong et al. 2010 [42]631NA−0.81−1.06 to −0.56Unblinded/oppositePositive
Tough et al. 2009 [43]370.570.30 to 0.75−0.67−0.93 to −0.40Unblinded/oppositePositive
Venzke et al. 2010 [44]510.480.15 to 0.81−0.42−0.78 to −0.05Unblinded/oppositeNegative
White et al. 1996 [45]91NA−0.6−1.30 to 0.10Unblinded/oppositeNegative
White et al. 2000 [46]440.610.35 to 0.87−0.26−0.56 to 0.04Unblinded/oppositeNegative
White et al. 2007 [47]370.770.55 to 0.98−0.29−0.61 to 0.03Unblinded/oppositePositive
Alecrim-Andrade et al. 2008 [48]360.260.14 to 0.39−0.12−0.23 to −0.00Unblinded/randomNegative
Harris et al. 2005 [49]760.25−0.09 to 0.59−0.13−0.50 to 0.23Unblinded/randomNegative
Lao et al. 1999 [50]390.470.17 to 0.780.05−0.21 to 0.31Unblinded/randomPositive
Linde et al. 2005 [51]2010.50.38 to 0.63−0.06−0.26 to 0.15Unblinded/randomNegative
Nabeta and Kawakita 2002 [52]340.410.01 to 0.81−0.18−0.61 to 0.26Unblinded/randomPositive
Shen and Goddard 2007 [53]150.780.37 to 1.190−0.80 to 0.80Unblinded/randomPositive
Streitberger et al. 2008 [54]200.550.29 to 0.81−0.150.46 to 0.16Unblinded/randomPositive
Takakura and Yajima 2007 [55]600.60.40 to 0.800.17−0.08 to 0.42Unblinded/randomPositive
Takakura and Yajima 2008 [56]1140.370.20 to 0.54−0.12−0.30 to 0.06Unblinded/randomPositive
Tan et al. 2009 [57]200.250.04 to 0.460.1−0.12 to 0.32Unblinded/randomPositive
Tsukayama et al. 2006 BL23 [58]200.4−0.00 to 0.800−0.44 to 0.44Unblinded/randomNegative
Wasan et al. 2010 [59]400.80.61 to 0.990−0.31 to 0.31Unblinded/randomNegative
Chae et al. 2011 [60]280.570.14 to 1.000.710.35 to 1.08Unblinded/unblindedPositive
Kreiner et al. 2010 [61]320.340.11 to 0.570.440.22 to 0.66Unblinded/unblindedPositive
Tsukayama et al. 2006 L14 [58]201NA0.3−0.12 to 0.72Unblinded/unblindedNegative
Wayne et al. 2008 [62]140.22−0.19 to 0.630.4−0.30 to 1.10Unblinded/unblindedPositive

StudyTotal Party blindedTime of blinding assessment (days from beginning of study)Deqi assessment (yES/nO)Sham devicePenetration of shamStudy participant diagnosisDays since last acupuncture experience

Shen and Goddard 2007 [53]15Staff + subject0 NOCNPChronic TM painUnknown
Itoh et al. 2006 [30]19Staff + subject21 NOCNPChronic LBPUnknown
Nabeta and Kawakita
2002 [52]
34Subject28 NOCNPChronic neck and shoulder painUnknown
Tough et al. 2009 [43]37Subject14 NOCNPWhiplashUnknown
Fink et al. 2001 [27]64Staff + subject14 NOCNPTension headacheUnknown
Shen et al. 2009 [36]12Staff + subject0YESCNPChronic myofacial pain of jaw muscles>365
Kreiner et al. 2010 [61]32Staff + subject0 NOCNPNormal>365
Lao et al. 1999 [50]39Staff + subject0.2YESCNPAfter oral surgery Pain>365
Goddard et al. 2005 [28]49Subject0YESCNPNormal>365
Jubb et al. 2008 [22]51Subject63YESCNPKnee OA>365
Itoh et al. 2008 [31]24Staff + subject35 NOCNPKnee OA180
Deng et al. 2008 [16]53Subject30 NOCNPAfter thoracotomy Pain42
Takakura and Yajima
2007 [55]
60Staff + subject0 NOCNPNormal0
Takakura and Yajima
2008 [56]
114Staff + subject0YESCNPNormal0
Smith et al. 2007 [38]27Staff + subject28 NOPNPTMJUnknown
Tan et al. 2009 [57]20Subject0 NOPNPNormal>365
Kennedy et al. 2008 [9]45Subject42 NOPNPAcute LBP>365
Park et al. 2002 [34]58Staff + subject84 NOPNPAcute stroke>365
Park et al. 2005 [35]94Staff + subject14 NOPNPStroke>365
Enblom et al. 2011 [26]190Staff + subject35 NOPNPRadiation-induced nausea365
Tsukayama et al. 2006
BL23 [58]
20Staff + subject0 NOPNPNormal0
Tsukayama et al. 2006
L14 [58]
20Staff + subject0 NOPNPNormal0
Chae et al. 2011 [60]28Subject0 NOPNPNormal0
Shin et al. 2010 [19]42Staff + subject21YESPENPDry eyeUnknown
Tong et al. 2010 [42]63Subject15YESPENPDPN~50%
Zaslawski et al. 1997 [20]64Subject28YESPENPStress~50%
Harris et al. 2005 [49]76Staff + subject21 NOPENPFibromyaligia>365
Haake et al. 2007 [8]692Staff + subject168 NOPENPChronic LBP>365
Linde et al. 2005 [51]201Staff + subject168YESPENPMigraine HA365
Brinkhaus et al. 2006 [25]205Subject56YESPENPChronic LBP365
Endres et al. 2007 [17]403Staff + subject180YESPENPTension headache365
Alecrim-Andrade et al. 2006 [23]24Staff + subject84YESPENPMigraine HA84
Alecrim-Andrade et al. 2008 [48]36Subject84YESPENPMigraine HA84
Lee et al. 2011 [33]35Staff + subject70 NOPENPChronic pelvic pain42
Lee et al. 2008 [32]89Subject70 NOPENPChronic prostatitis42
White et al. 2007 [47]37Subject0YESSNPChronic Pain/NormalUnknown
So et al. 2009 [39]370Staff + subject0 NOSNPInfertilityUnknown
Wayne et al. 2008 [62]14Staff + subject28 NOSNPEndometriosis>365
Streitberger et al. 2008 [54]20Subject0 NOSNPNormal>365
Wasan et al. 2010 [59] 40Subject0YESSNPChronic LBP>365
Barlas et al. 2006 [6]48Staff + subject0YESSNPNormal>365
Streitberger and Kleinhenz 1998 [40]60Subject0NOSNPNormal>365
Enblom et al. 2008 [7]80Subject0NOSNPNormal>365
Kaptchuk et al. 2008 [15]148Staff + subject42NOSNPIBS>365
Goldman et al. 2008 [29]118Staff + subject28YESSNPArm pain365
Venzke et al. 2010 [44]51Subject168YESSNPHot flashes180
Streitberger et al. 2004 [41]212Staff + subject1YESSNPpostoperative nausea180
Deng et al. 2007 [21]67Subject42YESSNPHot flashes42
White et al. 1996 [45]9Staff + subject42NOTNPTension headache>365
Harris et al. 2009 [18]20Staff + subject0NOTNPFibromyalgia>365
White et al. 2000 [46]44Staff + subject7YESTNPTension headache>365
Sherman et al. 2002 [37]52Subject0NOTNPChronic LBP>365
Assefi et al. 2005 [14]75Staff + subject84NOTNP, PFibromyalgia>365
Berman et al. 2004 [24]283Subject182YESTNP, PKnee OA>365

Sham devices: C: custom sham, P: park sham, PEN: penetrating sham, S: streitberger sham, and T: toothpick.

Of the 54 studies, 22 studies used a commercially developed sham control device, 14 studies used a custom-made sham control device, 12 used penetrating sham control, and 6 used a toothpick or cocktail stick. According to their averaged blinding index scenarios, all of the sham control devices with the exception of custom devices seemed to be effective in blinding the subjects, with the penetrating sham controls providing relatively more effective blinding.

In looking at the penetrating versus nonpenetrating dichotomy in a more direct way, the nonpenetrating group was unblinded/opposite, the mixed penetrating/nonpenetrating group was unblinded/opposite, and the penetrating group was random/random. All of these scenarios indicated effective blinding.

Studies with a greater number of subjects had a greater unblinding in the verum group. Measurements assessed later tended to have more ideal effectiveness of blinding than measurements assessed immediately. Interestingly, there was a higher tendency in both arms, when only the subjects—not staff—were blinded, to believe they received the verum treatment. The deqi group had slightly better EOB than the insertion/puncture group, and the symptomatic group had more ideal EOB than the healthy group.

Twenty-four studies used acupuncture-naïve subjects, and 19 used subjects with prior acupuncture experience. Eleven of the studies were excluded from this section of the review due to unknown prior experience or mixed experience of the subjects. According to their averaged blinding index scenarios, both groups were unblinded/random.

4. Discussion

This systematic review of 54 randomized controlled acupuncture studies showed that overall 61% of the studies (as a conservative estimate) meeting our inclusion/exclusion criteria were effectively blinded. The most common scenario encountered was unblinded in the acupuncture group and opposite guess in the sham acupuncture group, which could be indeed interpreted as “well-blinded.” In this scenario there may be a psychological phenomenon of “wishful thinking.” A majority of people, in both the verum and sham groups, guessed that they received real acupuncture. Thus, guesses are inflated towards real acupuncture in both study arms. It is also possible that once a needle is administered, a subject believes it is real acupuncture, or subjects may not know what to expect, creating a similar trend, or there is a strong placebo effect.

A similar pattern emerges when looking at the V and S groups individually; the average VBI was “unblinded” and the average SBI was “opposite.” Is it possible for one not to know when a needle is penetrating his or her skin? Perhaps the answer is “no,” given that unblinded V may mean that subjects know when their skin is being penetrated by a needle and thus increases the chance a subject chooses the V acupuncture group over the S acupuncture group upon questioning. Is it possible for one to know when a needle is not penetrating their skin? The answer seems to be “no” again; opposite guesses in S may indicate that subjects are not able to tell if they are not being penetrated by a needle, and thus are truly guessing.

Most sham control devices with the exception of custom devices were effective in blinding the subjects (Table 2). Since there was a great diversity within the custom sham group a more in depth case by case analysis of each device could be performed, but at this time there does not seem to be compelling evidence supporting the use of custom sham devices. Even though commercial sham devices and toothpick/cocktail stick devices appear to provide effective blinding, the penetrating sham controls provided relatively more effective blinding.

By comparing study blinding and study outcomes, the majority of studies reported positive outcomes, regardless of the degree of guess correctness. This leads us to believe that there is no obvious association between EOB and reported study outcomes. The current literature provides conflicting evidence so the direction of bias may be specific to context or treatment [69, 1449, 5162] or random.

Exploration of individual variables and their possible effect on EOB indicated that some design characteristics such as larger sample size, symptomatic subjects, and later assessment were associated with more effective blinding, and these may be encouraged to be further evaluated and considered in designing future acupuncture trials.

4.1. Recommendations for Future Acupuncture Research

The effect size was not a part of the extracted information from the reviewed studies. This poses an interesting idea for future investigation. Additional future research should include a sufficiently powered, prospective randomized trial comparing the EOB of different methods and sham devices by direct comparison, as well as investigating the influence of practitioner behavior, the patient’s expectations and beliefs, and the location of treatment points on blinding effectiveness. A good treatment should have a greater treatment effect than placebo effect. It is important that we collect more data in this field, especially qualitative data (e.g., reasons of guessing in a particular way). It is possible that the reasons for correct guesses in individual trials may be more revealing than our numbers. Individual trialists should be willing to share their experiences with others, as individual trialists and patients must have greater insight and more stories to tell in specific conditions within the trial than meta-analysis and readers. This is particularly important, because any analysis of available numeric data on blinding is destined to be prone to some biases.

4.2. Systematic Review Limitations

There are several potential weaknesses of this review that we recognize. Language bias may be a possibility given that we included only English language publications while acupuncture is a popular treatment modality in Asia and Europe. There is also a possibility of publication bias, which is an inherent problem in virtually all systematic reviews or meta-analyses. Moreover, it is possible that some investigators collected blinding data, saw the data, and decided not to report it in the paper, particularly when blinding was shown to be unsatisfactory. Also, for classification and decision making purposes, we used categorization/dichotomization (e.g., numerical BI cutoff of 0.2) and summary statistics (e.g., average BI). Alternative ways of analyzing the data could yield different results. Finally, in this systematic review, we used the conventional terms commonly used in the literature and equated “unblinded” with “correctly guessed,” whereas there could be other reasons for correct guesses. With its limitations, to our knowledge, this is the first systematic review based on empirical blinding data.

In conclusion, based on the status of blinding, the most common scenario encountered was a more correct guess in the real acupuncture group and an opposite guess in the sham acupuncture group, and the overall subject blinding of the evaluated acupuncture studies was satisfactory. In addition, quantitatively calculated blinding data, ideally together with more qualitative data for individual trials, may offer meaningful means to further interpret the findings of RCTs and improve the practice in the direction of a higher validity.


J. J. Park developed the Park Sham Device via AcuPrime in UK and was supported by the National Institute of Dental and Craniofacial Research of the National Institute of Health under award number K12DE022793. The content is solely the responsibility of the authors and does not necessarily represent the official views of the National Institute of Health.


The authors acknowledge the assistance of Hannah McFadden during the preparation of the paper.


  1. Yeung and C. Albert, “New guidelines for trial reporting—CONSORT, 2010,” British Medical Journal, vol. 340, Article ID c332, 2010. View at: Google Scholar
  2. A. J. Vickers, “Placebo controls in randomized trials of acupuncture,” Evaluation & the Health Professions, vol. 25, pp. 421–435, 2002. View at: Google Scholar
  3. C. Vincent, “Credibility assessment in trials of acupuncture,” Complementary Medicine Research, vol. 4, pp. 8–11, 1990. View at: Google Scholar
  4. R. Wood and G. Lewith, “The credibility of placebo controls in acupuncture studies,” Complementary Therapies in Medicine, vol. 6, no. 2, pp. 79–82, 1998. View at: Publisher Site | Google Scholar
  5. I. Boutron, D. Moher, D. G. Altman, K. F. Schulz, and P. Ravaud, “Extending the CONSORT statement to randomized trials of nonpharmacologic treatment: explanation and elaboration,” Annals of Internal Medicine, vol. 148, no. 4, pp. 295–309, 2008. View at: Google Scholar
  6. P. Barlas, S. L. H. Ting, L. S. Chesterton, P. W. Jones, and J. Sim, “Effects of intensity of electroacupuncture upon experimental pain in healthy human volunteers: a randomized, double-blind, placebo-controlled study,” Pain, vol. 122, no. 1-2, pp. 81–89, 2006. View at: Publisher Site | Google Scholar
  7. A. Enblom, M. Hammar, G. Steineck, and S. Börjeson, “Can individuals identify if needling was performed with an acupuncture needle or a non-penetrating sham needle?” Complementary Therapies in Medicine, vol. 16, no. 5, pp. 288–294, 2008. View at: Publisher Site | Google Scholar
  8. M. Haake, H. H. Müller, C. Schade-Brittinger et al., “German Acupuncture Trials (GERAC) for chronic low back pain: randomized, multicenter, blinded, parallel-group trial with 3 groups,” Archives of Internal Medicine, vol. 167, no. 17, pp. 1892–1898, 2007. View at: Publisher Site | Google Scholar
  9. S. Kennedy, G. D. Baxter, D. P. Kerr, I. Bradbury, J. Park, and S. M. McDonough, “Acupuncture for acute non-specific low back pain: a pilot randomised non-penetrating sham controlled trial,” Complementary Therapies in Medicine, vol. 16, no. 3, pp. 139–146, 2008. View at: Publisher Site | Google Scholar
  10. T. Lundeberg, I. Lund, J. Näslund, and M. Thomas, “The Emperor's sham—wrong assumption that sham needling is sham,” Acupuncture in Medicine, vol. 26, no. 4, pp. 239–242, 2008. View at: Publisher Site | Google Scholar
  11. A. Moroz, “Issues in acupuncture research: the failure of quantitative methodologies and the possibilities for viable, alternative solutions,” American Journal of Acupuncture, vol. 27, no. 1-2, pp. 95–103, 1999. View at: Google Scholar
  12. J. Park and H. Bang, “Methodological advances needed in analysis and interpretation of sham acupuncture validation studies,” Acupuncture in Medicine, vol. 29, no. 3, pp. 168–169, 2011. View at: Google Scholar
  13. J. Park, H. Bang, and I. Cañette, “Blinding in clinical trials, time to do it better,” Complementary Therapies in Medicine, vol. 16, no. 3, pp. 121–123, 2008. View at: Publisher Site | Google Scholar
  14. N. P. Assefi, K. J. Sherman, C. Jacobsen, J. Goldberg, W. R. Smith, and D. Buchwald, “A randomized clinical trial of acupuncture compared with sham acupuncture in fibromyalgia,” Annals of Internal Medicine, vol. 143, no. 1, pp. 10–19, 2005. View at: Google Scholar
  15. T. J. Kaptchuk, J. M. Kelley, L. A. Conboy et al., “Components of placebo effect: randomised controlled trial in patients with irritable bowel syndrome,” British Medical Journal, vol. 336, no. 7651, pp. 999–1003, 2008. View at: Publisher Site | Google Scholar
  16. G. Deng, V. Rusch, A. Vickers et al., “Randomized controlled trial of a special acupuncture technique for pain after thoracotomy,” Journal of Thoracic and Cardiovascular Surgery, vol. 136, no. 6, pp. 1464–1469, 2008. View at: Publisher Site | Google Scholar
  17. H. G. Endres, G. Böwing, H. C. Diener et al., “Acupuncture for tension-type headache: a multicentre, sham-controlled, patient-and observer-blinded, randomised trial,” Journal of Headache and Pain, vol. 8, no. 5, pp. 306–314, 2007. View at: Publisher Site | Google Scholar
  18. R. E. Harris, J. K. Zubieta, D. J. Scott, V. Napadow, R. H. Gracely, and D. J. Clauw, “Traditional Chinese acupuncture and placebo (sham) acupuncture are differentiated by their effects on μ-opioid receptors (MORs),” NeuroImage, vol. 47, no. 3, pp. 1077–1085, 2009. View at: Publisher Site | Google Scholar
  19. M. S. Shin, J. I. Kim, M. S. Lee et al., “Acupuncture for treating dry eye: a randomized placebo-controlled trial,” Acta Ophthalmologica, vol. 88, no. 8, pp. e328–e333, 2010. View at: Publisher Site | Google Scholar
  20. C. Zaslawski, C. Rogers, M. Garvey, D. Ryan, C. X. Yang, and S. P. Zhang, “Strategies to maintain the credibility of sham acupuncture used as a control treatment in clinical trials,” Journal of Alternative and Complementary Medicine, vol. 3, no. 3, pp. 257–266, 1997. View at: Google Scholar
  21. G. Deng, A. Vickers, S. Yeung, and B. Cassileth, “Randomized, controlled trial of acupuncture for the treatment of hot flashes in breast cancer patients,” Journal of Clinical Oncology, vol. 25, no. 35, pp. 5584–5590, 2007. View at: Publisher Site | Google Scholar
  22. R. W. Jubb, E. S. Tukmachi, P. W. Jones, E. Dempsey, L. Waterhouse, and S. Brailsford, “A blinded randomised trial of acupuncture (manual and electroacupuncture) compared with a non-penetrating sham for the symptoms of osteoarthritis of the knee,” Acupuncture in Medicine, vol. 26, no. 2, pp. 69–78, 2008. View at: Google Scholar
  23. J. Alecrim-Andrade, J. A. Maciel, X. C. Cladellas, H. R. Correa-Filho, and H. C. Machado, “Acupuncture in migraine prophylaxis: a randomized sham-controlled trial,” Cephalalgia, vol. 26, no. 5, pp. 520–529, 2006. View at: Publisher Site | Google Scholar
  24. B. M. Berman, L. Lao, P. Langenberg, W. L. Lee, A. M. K. Gilpin, and M. C. Hochberg, “Effectiveness of acupuncture as adjunctive therapy in osteoarthritis of the knee. A randomized, controlled trial,” Annals of Internal Medicine, vol. 141, no. 12, pp. 901–910, 2004. View at: Google Scholar
  25. B. Brinkhaus, C. M. Witt, S. Jena et al., “Acupuncture in patients with chronic low back pain: a randomized controlled trial,” Archives of Internal Medicine, vol. 166, no. 4, pp. 450–457, 2006. View at: Publisher Site | Google Scholar
  26. A. Enblom, A. Johnsson, M. Hammar, G. Steineck, and S. Borjeson, “The nonpenetrating telescopic sham needle may blind patients with diferrent characteristics and experiences when treated by several therapists,” Evidence-Based Complementary and Alternative Medicine, vol. 2011, Article ID 185034, pp. 1–12, 2011. View at: Publisher Site | Google Scholar
  27. M. Fink, C. Gutenbrunner, J. Rollnik, and M. Karst, “Credibility of a newly designed placebo needle for clinical trials in acupuncture research,” Forschende Komplementarmedizin und Klassische Naturheilkunde, vol. 8, no. 6, pp. 368–372, 2001. View at: Publisher Site | Google Scholar
  28. G. Goddard, Y. Shen, B. Steele, and N. Springer, “A controlled trial of placebo versus real acupuncture,” Journal of Pain, vol. 6, no. 4, pp. 237–242, 2005. View at: Publisher Site | Google Scholar
  29. R. H. Goldman, W. B. Stason, S. K. Park et al., “Acupuncture for treatment of persistent arm pain due to repetitive use: a randomized controlled clinical trial,” Clinical Journal of Pain, vol. 24, no. 3, pp. 211–218, 2008. View at: Publisher Site | Google Scholar
  30. K. Itoh, Y. Katsumi, S. Hirota, and H. Kitakoji, “Effects of trigger point acupuncture on chronic low back pain in elderly patients—a sham-controlled randomised trial,” Acupuncture in Medicine, vol. 24, no. 1, pp. 5–12, 2006. View at: Google Scholar
  31. K. Itoh, S. Hirota, Y. Katsumi, H. Ochi, and H. Kitakoji, “Trigger point acupuncture for treatment of knee osteoarthritis—a preliminary RCT for a pragmatic trial,” Acupuncture in Medicine, vol. 26, no. 1, pp. 17–26, 2008. View at: Google Scholar
  32. S. W. H. Lee, M. L. Liong, K. H. Yuen et al., “Acupuncture versus sham acupuncture for chronic prostatitis/chronic pelvic pain,” American Journal of Medicine, vol. 121, no. 1, pp. 79.e1–79.e7, 2008. View at: Publisher Site | Google Scholar
  33. S. W. H. Lee, M. L. Liong, K. H. Yuen, W. S. Leong, N. K. Khan, and J. N. Krieger, “Validation of a sham acupuncture procedure in a randomised, controlled clinical trial of chronic pelvic pain treatment,” Acupuncture in Medicine, vol. 29, no. 1, pp. 40–46, 2011. View at: Publisher Site | Google Scholar
  34. J. Park, A. White, C. Stevinson, E. Ernst, and M. James, “Validating a new non-penetrating sham acupuncture device: two randomised controlled trials,” Acupuncture in Medicine, vol. 20, no. 4, pp. 168–174, 2002. View at: Google Scholar
  35. J. Park, A. R. White, M. A. James et al., “Acupuncture for subacute stroke rehabilitation: a sham-controlled, subject- and assessor-blind, randomized trial,” Archives of Internal Medicine, vol. 165, no. 17, pp. 2026–2031, 2005. View at: Publisher Site | Google Scholar
  36. Y. F. Shen, J. Younger, G. Goddard, and S. Mackey, “Randomized clinical trial of acupuncture for myofascial pain of the jaw muscles,” Journal of Orofacial Pain, vol. 23, no. 4, pp. 353–359, 2009. View at: Google Scholar
  37. K. J. Sherman, C. J. Hogeboom, D. C. Cherkin, and R. A. Deyo, “Description and validation of a noninvasive placebo acupuncture procedure,” Journal of Alternative and Complementary Medicine, vol. 8, no. 1, pp. 11–19, 2002. View at: Google Scholar
  38. P. Smith, D. Mosscrop, S. Davies, P. Sloan, and Z. Al-Ani, “The efficacy of acupuncture in the treatment of temporomandibular joint myofascial pain: a randomised controlled trial,” Journal of Dentistry, vol. 35, no. 3, pp. 259–267, 2007. View at: Publisher Site | Google Scholar
  39. E. So, E. Ng, Y. Wong, W. Yeung, and P. Ho, “A randomized double blind comparison of real and placebo acupuncture in IVF treatment,” Human Reproduction, vol. 24, pp. 341–348, 2009. View at: Google Scholar
  40. K. Streitberger and J. Kleinhenz, “Introducing a placebo needle into acupuncture research,” The Lancet, vol. 352, no. 9125, pp. 364–365, 1998. View at: Publisher Site | Google Scholar
  41. K. Streitberger, M. Diefenbacher, A. Bauer et al., “Acupuncture compared to placebo-acupuncture for postoperative nausea and vomiting prophylaxis: a randomised placebo-controlled patient and observer blind trial,” Anaesthesia, vol. 59, no. 2, pp. 142–149, 2004. View at: Publisher Site | Google Scholar
  42. Y. Tong, H. Guo, and B. Han, “Fifteen-day acupuncture treatment relieves diabetic peripheral neuropathy,” Journal of Acupuncture and Meridian Studies, vol. 3, no. 2, pp. 95–103, 2010. View at: Publisher Site | Google Scholar
  43. E. A. Tough, A. R. White, S. H. Richards, B. Lord, and J. L. Campbell, “Developing and validating a sham acupuncture needle,” Acupuncture in Medicine, vol. 27, no. 3, pp. 118–122, 2009. View at: Publisher Site | Google Scholar
  44. L. Venzke, J. F. Calvert, and B. Gilbertson, “A randomized trial of acupuncture for vasomotor symptoms in post-menopausal women,” Complementary Therapies in Medicine, vol. 18, no. 2, pp. 59–66, 2010. View at: Publisher Site | Google Scholar
  45. A. R. White, C. Eddleston, R. Hardie, K. L. Resch, and E. Ernst, “A pilot study of acupuncture for tension headache, using a novel placebo,” Acupuncture in Medicine, vol. 14, no. 1, pp. 11–15, 1996. View at: Google Scholar
  46. A. R. White, K. L. Resch, J. C. K. Chan et al., “Acupuncture for episodic tension-type headache: a multicentre randomized controlled trial,” Cephalalgia, vol. 20, no. 7, pp. 632–637, 2000. View at: Publisher Site | Google Scholar
  47. P. White, G. Lewith, and P. Prescott, “Should we recruit patients or healthy volunteers for acupuncture studies of chronic pain?” Clinical Journal of Pain, vol. 23, no. 8, pp. 714–719, 2007. View at: Publisher Site | Google Scholar
  48. J. Alecrim-Andrade, J. A. Maciel-Júnior, X. Carnè, G. M. Severino Vasconcelos, and H. R. Correa-Filho, “Acupuncture in migraine prevention: a randomized sham controlled study with 6-months posttreatment follow-up,” Clinical Journal of Pain, vol. 24, no. 2, pp. 98–105, 2008. View at: Publisher Site | Google Scholar
  49. R. E. Harris, X. Tian, D. A. Williams et al., “Treatment of fibromyalgia with formula acupuncture: investigation of needle placement, needle stimulation, and treatment frequency,” Journal of Alternative and Complementary Medicine, vol. 11, no. 4, pp. 663–671, 2005. View at: Publisher Site | Google Scholar
  50. L. Lao, S. Bergman, G. R. Hamilton, P. Langenberg, and B. Berman, “Evaluation of acupuncture for pain control after oral surgery: a placebo-controlled trial,” Archives of Otolaryngology, vol. 125, no. 5, pp. 567–572, 1999. View at: Google Scholar
  51. K. Linde, A. Streng, S. Jürgens et al., “Acupuncture for patients with migraine: a randomized controlled trial,” Journal of the American Medical Association, vol. 293, no. 17, pp. 2118–2125, 2005. View at: Publisher Site | Google Scholar
  52. T. Nabeta and K. Kawakita, “Relief of chronic neck and shoulder pain by manual acupuncture to tender points—a sham-controlled randomized trial,” Complementary Therapies in Medicine, vol. 10, no. 4, pp. 217–222, 2002. View at: Publisher Site | Google Scholar
  53. Y. F. Shen and G. Goddard, “The short-term effects of acupuncture on myofascial pain patients after clenching,” Pain Practice, vol. 7, no. 3, pp. 256–264, 2007. View at: Publisher Site | Google Scholar
  54. K. Streitberger, J. Steppan, C. Maier, H. Hill, J. Backs, and K. Plaschke, “Effects of verum acupuncture compared to placebo acupuncture on quantitative EEG and heart rate variability in healthy volunteers,” Journal of Alternative and Complementary Medicine, vol. 14, no. 5, pp. 505–513, 2008. View at: Publisher Site | Google Scholar
  55. N. Takakura and H. Yajima, “A double-blind placebo needle for acupuncture research,” BMC Complementary and Alternative Medicine, vol. 7, article 31, 2007. View at: Publisher Site | Google Scholar
  56. N. Takakura and H. Yajima, “A placebo acupuncture needle with potential for double blinding—a validation study,” Acupuncture in Medicine, vol. 26, no. 4, pp. 224–230, 2008. View at: Publisher Site | Google Scholar
  57. C. W. Tan, L. Christie, V. St-Georges, and N. Telford, “Discrimination of real and sham acupuncture needles using the park sham device: a Preliminary Study,” Archives of Physical Medicine and Rehabilitation, vol. 90, no. 12, pp. 2141–2145, 2009. View at: Publisher Site | Google Scholar
  58. H. Tsukayama, H. Yamashita, T. Kimura, and K. Otsuki, “Factors that influence the applicability of sham needle in acupuncture trials: two randomized, single-blind, crossover trials with acupuncture—experienced subjects,” Clinical Journal of Pain, vol. 22, no. 4, pp. 346–349, 2006. View at: Publisher Site | Google Scholar
  59. A. D. Wasan, J. Kong, L. D. Pham, T. J. Kaptchuk, R. Edwards, and R. L. Gollub, “The impact of placebo, psychopathology, and expectations on the response to acupuncture needling in patients with chronic low back pain,” Journal of Pain, vol. 11, no. 6, pp. 555–563, 2010. View at: Publisher Site | Google Scholar
  60. Y. Chae, S. I. Um, S. H. Yi et al., “Comparison of biomechanical properties between acupuncture and non-penetrating sham needle,” Complementary Therapies in Medicine, vol. 19, no. 1, pp. S8–S12, 2011. View at: Publisher Site | Google Scholar
  61. M. Kreiner, A. Zaffaroni, R. Alvarez, and G. Clark, “Validation of a simplified sham acupuncture technique for its use in clinical research: a randomised, single blind, crossover study,” Acupuncture in Medicine, vol. 28, no. 1, pp. 33–36, 2010. View at: Publisher Site | Google Scholar
  62. P. M. Wayne, C. E. Kerr, R. N. Schnyer et al., “Japanese-style acupuncture for endometriosis-related pelvic pain in adolescents and young women: results of a randomized sham-controlled trial,” Journal of Pediatric and Adolescent Gynecology, vol. 21, no. 5, pp. 247–257, 2008. View at: Publisher Site | Google Scholar
  63. H. Bang, L. Ni, and C. E. Davis, “Assessment of blinding in clinical trials,” Controlled Clinical Trials, vol. 25, no. 2, pp. 143–156, 2004. View at: Publisher Site | Google Scholar
  64. H. Bang, S. P. Flaherty, J. Kolahi, and J. Park, “Blinding assessment in clinical trials: a review of statistical methods and a proposal of blinding assessment protocol,” Clinical Research and Regulatory Affairs, vol. 27, no. 2, pp. 42–51, 2010. View at: Publisher Site | Google Scholar
  65. J. Kolahi, H. Bang, and J. Park, “Towards a proposal for assessment of blinding success in clinical trials: up-to-date review,” Community Dentistry and Oral Epidemiology, vol. 37, no. 6, pp. 477–484, 2009. View at: Publisher Site | Google Scholar

Copyright © 2013 Alex Moroz et al. This is an open access article distributed under the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.

Related articles

No related content is available yet for this article.
 PDF Download Citation Citation
 Download other formatsMore
 Order printed copiesOrder

Related articles

No related content is available yet for this article.

Article of the Year Award: Outstanding research contributions of 2021, as selected by our Chief Editors. Read the winning articles.