Research Article | Open Access
Masako Nishiwaki, Miho Takayama, Hiroyoshi Yajima, Morihiro Nasu, Jian Kong, Nobuari Takakura, "The Japanese Version of the Massachusetts General Hospital Acupuncture Sensation Scale: A Validation Study", Evidence-Based Complementary and Alternative Medicine, vol. 2017, Article ID 7093967, 7 pages, 2017. https://doi.org/10.1155/2017/7093967
The Japanese Version of the Massachusetts General Hospital Acupuncture Sensation Scale: A Validation Study
Acupuncture sensations are considered essential in producing the treatment effect of acupuncture. The Massachusetts General Hospital Acupuncture Sensation Scale (MASS) is a frequently used scale in acupuncture research to measure acupuncture sensations. We translated the MASS into Japanese (Japanese MASS) based on Beaton’s guidelines. 30 acupuncturists evaluated the relevancy and meaning of the 12 descriptors included in the Japanese MASS. The content validity ratios for 10 of the 12 descriptors were 0.33 or greater. 42 healthy subjects then evaluated acupuncture sensations evoked by manual acupuncture at LI4 using the Japanese MASS. Cronbach’s alpha was 0.86. The correlation coefficient of total MASS scores and total Short Form McGill Pain Questionnaire scores and MASS indices and sensory visual analogue scores were 0.78 and 0.80, respectively. Factor analysis loaded the 12 descriptors onto two meaningful factors. This study demonstrated that the Japanese MASS has good reliability, content validity, criterion-related validity, and construct validity. Therefore, the Japanese MASS is a valid and reliable instrument for use with Japanese populations.
Acupuncture sensations, or de qi, felt by patients are sensations elicited with needling and are traditionally considered to play an important role in acupuncture treatment [1–3]. Characteristics of de qi elicited with needling depend on where and how a needle is applied and on the patient’s state of health . The Standards for Reporting Interventions in Clinical Trials of Acupuncture recommend that manuscripts related to clinical acupuncture trials report de qi, demonstrating its importance in acupuncture research .
Studies suggest that there are close associations between acupuncture sensations and acupoints [6–8], anatomical structures (e.g., the tissues and the nerves stimulated with needle insertion) [9, 10], depth of needle insertion , and methods of acupuncture [6, 12–14]. Physiological responses with de qi, including blood flow [15, 16] and brain activity (observed using electroencephalography  or functional magnetic resonance imaging [18–20]), have also been reported. Studies investigating physiological or psychological responses induced with acupuncture sensations, as well as those focused on the characteristics of acupuncture sensations per se using globally common questionnaires, are indispensable in evaluating the relationship between acupuncture sensations and the effects of acupuncture.
The Acupuncture Sensation Scale (ASS) , the Massachusetts General Hospital Acupuncture Sensation Scale (MASS) , and the Southampton Needle Sensation Questionnaire (SNSQ)  were developed to qualify and quantify acupuncture sensations and investigate their relationship to the effect of acupuncture. The development of the ASS was based on McGill’s Pain Questionnaire (MPQ) , the MASS on traditional Chinese medicine descriptions of de qi sensation [2, 6, 22], and the SNSQ on interviews with patients . They have been validated for use in clinical acupuncture trials and physiological studies.
In order to use these acupuncture sensation scales to investigate patients’ acupuncture sensations in languages other than English, the scales must be translated into the respective language. The ASS, MASS, and SNSQ have been translated into Korean , Chinese , and German , respectively. Some of these translations have been used in acupuncture studies evaluating de qi [27, 28].
However, only the ASS has been translated into Japanese, with the descriptors adopted from the Japanese version of the MPQ . Although the MPQ in Japanese was validated, the Japanese version of the ASS has not yet been validated. Therefore, researchers whose native language is Japanese are not able to conduct acupuncture trials with Japanese patients using a validated Japanese acupuncture sensation scale. As the MASS is the most well-known and frequently used scale, we decided to develop a validated MASS in Japanese rather than validating the ASS in Japanese.
In this study we aimed to develop a Japanese language version of the MASS, the English version of which is the most used sensation scale in acupuncture studies [2, 3, 18]. We translated the original MASS into Japanese (Japanese MASS) based on Beaton’s guidelines and tested both its reliability and its validity.
This cross-cultural adaptation of the MASS from English to Japanese comprised translation (Beaton’s stages 1–4) and validation and reliability tests (Beaton’s stage 5) . The study was approved by the Ethics Committee of Tokyo Ariake University of Medical and Health Sciences (TAU). The study objectives and protocol were fully explained to each participant using a written form, and all participants provided written consent.
2.1. Translation and Production of the Prefinal Version of the Japanese MASS
The original English version of the MASS was translated into Japanese according to a five-step process for cross-cultural adaptation, as described by Beaton et al. .
In the first stage, two acupuncture researchers whose native language was Japanese made forward translations from English to Japanese. In the second stage, the two independent translations made by the acupuncture researchers were synthesised into a Japanese version (synthesised version), based on a discussion between the two translators. In the third stage, two native-born Americans whose native language was English produced two English backtranslations of the synthesised version. One backtranslator was an acupuncturist. The other was not an acupuncturist and had no medical/acupuncture background and no prior knowledge of or information about the MASS. In the fourth stage, an expert committee comprised of three licensed acupuncturists (all of whom held doctoral degrees in medicine) from the TAU Department of Acupuncture and Moxibustion, along with a Boston-based Japanese acupuncturist who had obtained her acupuncture license in Massachusetts, reviewed the original MASS, the forward translations, the synthesised version, and the two backtranslations. They discussed the material for semantic, idiomatic, experimental, and conceptual equivalences between the original MASS and the synthesised version to develop a prefinal version of the Japanese MASS.
2.2. Content Validity
In the fifth stage, we recruited 30 licensed acupuncturists (19 males, 11 females) to evaluate the content validity of the Japanese MASS, based on Lawshe’s approach . The mean age (± standard deviation [SD]) of these acupuncturists was years and the mean experience in acupuncture treatment was years. We asked the acupuncturists to evaluate the necessity for each descriptor (heaviness, dull pain, tingling, warmth, sharp pain, numbness, throbbing, deep pressure, aching, soreness, cold, and fullness/distension) as an acupuncture sensation on a five-point scale: “agree,” “somewhat agree,” “neutral,” “somewhat disagree,” or “disagree.”
2.3. Internal Consistency, Criterion-Related Validity, and Construct Validity
We recruited 42 healthy subjects (20 males, 22 females, aged years) who had normal Japanese language ability and no nervous system disorders. Half of these 42 subjects had previous acupuncture experience and knowledge about acupuncture (including acupuncture sensations or de qi), whereas the others had no experience of acupuncture and no knowledge about acupuncture. The study was conducted in a laboratory at TAU, Tokyo, Japan. The acupuncture room was kept at 24–26°C. Acupuncture was administered by five acupuncturists with years of acupuncture experience.
Subjects lay on their back on a bed with their arms resting along their sides. An acupuncturist disinfected the subject’s skin with ethanol, administered the needle using the tapping-in method to penetrate the skin, and then applied the rotation technique at LI4 (large-intestine meridian) [6, 7, 12, 14, 16, 21, 24, 25, 27, 29] on the right or left hand. After the needle was inserted 10 mm perpendicularly to the skin’s surface, the acupuncturist rotated the needle handle by 180 degrees clockwise and anticlockwise at 1 Hz for 2 minutes  and then removed it. The needles were stainless steel (0.18 mm diameter, 40 mm length; Seirin, Shizuoka, Japan).
To exclude bias from acupuncture naïve or nonnaïve subjects, we randomly assigned acupuncture-experienced and non-acupuncture-experienced subjects to right LI4 or left LI4 groups using a web-generated table of random numbers (http://randomization.com).
Immediately after the removal of the needle, subjects completed three questionnaires on their acupuncture sensations during the needle rotation: (1) the sensory visual analogue scale (S-VAS), on which subjects rated the intensity of sensations from 0 (no sensation) to 100 (most severe sensation, unbearable); (2) the Japanese MASS (prefinal version), on which subjects rated the magnitude of each acupuncture sensation corresponding to the 12 descriptors on a numerical rating scale (NRS) from 0 (none) to 10 (strongest imaginable); if the subjects felt other sensations besides the 12 descriptors, they described the additional sensation and rated it on the NRS; (3) the Japanese version of the Short Form MPQ (SF-MPQ), which consists of the same sensory and affective descriptors as the original English version of the SF-MPQ. As the reliability of the Japanese SF-MPQ has been demonstrated, it may be used to evaluate criterion-related validity to show the concurrent validity of the Japanese MASS.
2.4. Statistical Analysis
To evaluate the need for each descriptor as an acupuncture sensation and the content validity of the Japanese MASS, we calculated the content validity ratio (CVR) from the scores of all 30 acupuncturists using the formula used by Lawshe . A 5% significance level represents a CVR of 0.33 when there are 30 acupuncturists .
To estimate the reliability of the Japanese MASS, we calculated the Cronbach’s alpha coefficient using the scores obtained from all subjects to assess internal consistency. To evaluate the criterion-related validity to show concurrent validity, we calculated Pearson’s correlation coefficients between the total MASS scores and S-VAS scores, total SF-MPQ scores, and the sensory and affective dimension scores in SF-MPQ, respectively. We also calculated Pearson’s correlation coefficients between the MASS indices and the same scores. The MASS index is an exponential smoothing average, which is considered a more reliable method to measure the magnitude of overall acupuncture sensations than total MASS scores. We calculated the MASS indices using scores for the 12 descriptors .
Based on the original MASS study , we conducted an exploratory factor analysis for the Japanese MASS to demonstrate construct validity. Factor extraction was performed using a principal component method, in which the number of factors was determined by scree plot criteria with eigenvalues. We conducted a varimax rotation involving Kaiser normalisation below 25 iterations for convergence.
Statistical analyses were performed with SPSS Version 24 (SPSS Japan Inc., IBM Company, Tokyo, Japan).
3.1. Translation into Japanese
Figure 1 shows the Japanese MASS. In the forward translation, “soreness” in the English MASS was translated as “feeling of languid” in Japanese. However, “feeling of languid” was backtranslated as “sluggish” in English. Therefore, the expert committee changed “feeling of languid” to “muscular pain” in Japanese, which was backtranslated as “soreness” in English. There were no revisions required for other descriptors during the translation process.
3.2. Content Validity
Figure 2 shows the CVRs calculated from the data reported by the 30 acupuncturists for each descriptor in the Japanese MASS. Overall, the CVRs for the 12 descriptors ranged from 0.07 for “fullness/distension” to 1.00 for “heaviness.” We found significant validity for the 10 CVRs, with the exception of “fullness/distension” and “cold.” In particular, the descriptors “heaviness,” “tingling,” “dull pain,” and “warmth” which showed high validity.
3.3. S-VAS Scores, Total MASS Score, and MASS Index
The mean ± SD intensity of acupuncture sensations marked on the S-VAS by the 42 subjects was .
For the 42 subjects, the mean ± SD intensity for the total MASS score was . The mean ± SD of MASS indices calculated from the scores for the 12 descriptors was . The largest mean sensation intensity among the 12 descriptors was for “heaviness” and the second largest was for “tingling.” The smallest mean sensation intensity was for “cold” and the second smallest was for “sharp pain” (Figure 3).
In terms of the frequency of occurrence, “tingling” was felt by 70% of subjects, showing the highest occurrence of the 12 descriptors. Less than 5% of subjects felt “cold.” The frequencies and intensities of the sensations for the 12 descriptors were positively correlated; that is, the more frequent the occurrence, the higher the intensity (Figure 3). Subjects reported few sensations other than the 12 descriptors. The mean ± SD occurrence of sensations in the descriptors reported by a single subject was .
There were no significant differences between males and females, or between right and left LI4 for the S-VAS scores, total MASS scores, and MASS indices.
3.4. Internal Consistency
Overall, Cronbach’s alpha coefficient for the 12 descriptors was 0.860. The largest value for 11 descriptors was 0.865 with the exclusion of “cold,” the second largest was 0.856 with the exclusion of “soreness,” and the smallest was 0.835 with the exclusion of “warmth.” The corrected item-total correlation for “cold” was 0.154, while the others ranged from 0.427 to 0.734.
3.5. Criterion-Related Validity
Total MASS scores and MASS indices correlated significantly with total SF-MPQ scores, S-VAS scores, and SF-MPQ sensory dimension scores. The Pearson’s correlation coefficient between total MASS scores and SF-MPQ affective dimension scores was larger than that between MASS indices and affective dimension scores (Table 1).
3.6. Construct Validity (Factor Analysis)
The factor analysis using a scree plot criterion loaded the 12 descriptors onto two factors. The cumulative contribution ratio was 0.561. The Kaiser-Meyer-Olkin statistic, which indicates sampling adequacy, was 0.796, and Bartlett’s test of sphericity was (). Factors 1 and 2 were classified as “varying pain sensation” and “pressure sensation depth under the skin” (Table 2).
|Note: factor loadings over 0.45 are exclusively represented.|
The original English MASS was translated into Japanese to develop a reliable and valid Japanese MASS for use in evaluating acupuncture sensations, or de qi, in acupuncture trials with Japanese populations. The results obtained from our reliability and validation tests showed that the Japanese MASS is a reliable and valid questionnaire to evaluate acupuncture sensations associated with needle insertion or manipulation in a Japanese sample. Our study yielded the first reliable and valid Japanese version of the MASS.
4.1. Translation into Japanese
In the Oxford English Dictionary, “soreness” is defined as “a raw or painful place on the body.” Translating “soreness” into “feeling of languid” in Japanese might have been biased due to the translator’s previous experience with acupuncture. However, this biased translation was highlighted in the backtranslation and amended by the expert committee. There was no discrepancy between the two backtranslators in the backtranslation process, even though one backtranslator had a medical/acupuncture background, which is not consistent with Beaton’s guidelines. This suggests that the backtranslation process avoided information bias and elicited no unexpected meanings for the descriptors. Given the amendment of the initial forward translation of “soreness” in the third and fourth stages of our cross-cultural adaptation process, we believe that the meanings of the descriptors in the Japanese MASS are equivalent to those in the English MASS.
4.2. Content Validity
A relatively large proportion of acupuncturists in our study disagreed with “cold” being an acupuncture sensation, which might be because patients and subjects rarely reported a “cold” sensation during needling. In the original and Chinese MASS study, “cold” was included because coldness and warmth are two of the earliest sensations to be symptom-specific based on the vacuity-repletion theory  in ancient literature [2, 22], which is core concept in acupuncture treatment. Therefore, although the frequency and intensity of “cold” sensations felt by subjects during needling were distinctively smaller when compared with the other descriptors, as well as in previous studies [2, 18, 25], we included “cold” to be consistent with original version of the MASS.
The CVR for “fullness/distension” was lower than that for “cold.” However, we found the average intensity and frequency of “fullness/distension” were higher than those of “cold”; therefore, we kept the descriptor in the MASS.
4.3. Internal Consistency
The overall Cronbach’s alpha coefficient for the descriptors (0.860) was between 0.70 and 0.95, indicating that the Japanese MASS had good internal consistency . This was further supported by the fact that there were almost no changes in the Cronbach’s alpha coefficients when individual descriptors were deleted in turn. The alpha coefficient exceeded 0.860 only when “cold” was deleted, and the extremely low corrected item-total correlation for “cold” indicated that “cold” was a heterogeneous descriptor. However, we left “cold” in place to compare with the other versions of the MASS.
4.4. Criterion-Related Validity
The criterion-related validity of the Japanese MASS was demonstrated by strong correlations between total MASS scores and total SF-MPQ scores and between total MASS scores and the SF-MPQ sensory dimension scores. These strong correlations were expected, as four descriptors classified as sensory from the SF-MPQ are included in the MASS, and the descriptors in the ASS are closely related to those in the MPQ . Acupuncture sensations involve a pain component and it could be said that the MASS is another form of a questionnaire that explores various aspects of pain sensation, especially deep, dull pain.
The SF-MPQ affective dimension scores were moderately correlated with the total Japanese MASS scores, whereas no correlation was found with the Chinese MASS . Approximately 70% of Chinese acupuncturists reported that their patients liked de qi. In contrast, most Japanese patients seem to not like the acupuncture sensations, similar to many people in the US . Japanese people may recognise de qi as an uncomfortable sensation and may also perceive it negatively.
The stronger correlation between the S-VAS scores and the MASS indices than between the S-VAS scores and total MASS scores indicates that the Japanese MASS indices expressed the total amount of acupuncture sensations well. Therefore, the Japanese MASS works as well as the original MASS in studying acupuncture sensations.
4.5. Construct Validity (Factor Analysis)
Although the 12 descriptors were classified into five factors in the Chinese MASS validation study, they were classified into two meaningful factors in the Japanese MASS, which corresponded to the two factors in the original MASS . This indicates that the Japanese MASS retained the same factor structure as the original MASS.
The Japanese MASS presented in this study will allow Japanese acupuncture researchers to investigate acupuncture sensations in clinical studies with Japanese populations. In order to understand acupuncture sensations in terms of clinical efficacy from an ethnical perspective, further exploration is necessary to compare the de qi profiles in the Japanese MASS with those obtained by the original MASS and the Chinese MASS.
Our study demonstrated that the Japanese MASS has good reliability, content validity, criterion-related validity, and construct validity. Therefore, the Japanese MASS is a valid and reliable instrument for use with Japanese populations.
Conflicts of Interest
The authors declare that they have no conflicts of interest.
The authors express their appreciation to all participants of this study, especially the translators. Jian Kong is supported by R01AT006364, R01 AT008563, R21AT008707, R61AT009310, and P01 AT006663 from NIH/NCCIH. They also thank Joel Park and Courtney Lang at Massachusetts General Hospital for their help in polishing English.
- H. MacPherson and A. Asghar, “Acupuncture needle sensations associated with De Qi: a classification based on experts' ratings,” Journal of Alternative and Complementary Medicine, vol. 12, no. 7, pp. 633–637, 2006.
- R. B. Spaeth, S. Camhi, J. A. Hashmi et al., “A longitudinal study of the reliability of acupuncture deqi sensations in knee osteoarthritis,” Evidence-Based Complementary and Alternative Medicine, vol. 2013, Article ID 204259, 12 pages, 2013.
- G.-X. Shi, Q.-Q. Li, C.-Z. Liu et al., “Effect of acupuncture on Deqi traits and pain intensity in primary dysmenorrhea: analysis of data from a larger randomized controlled trial,” BMC Complementary and Alternative Medicine, vol. 14, article 69, 2014.
- S. K. Lu, B. H. Hu, and Z. F. Zhang, Eds., A Hundred Kinds of Needling Methods, China Medical Science Press, Beijing, China, 1988 (Chinese).
- H. MacPherson, D. G. Altman, R. Hammerschlag et al., “Revised STandards for Reporting Interventions in Clinical Trials of Acupuncture (STRICTA): extending the CONSORT statement,” Journal of Evidence-Based Medicine, vol. 3, no. 3, pp. 140–155, 2010.
- J. Kong, D. T. Fufa, A. J. Gerber et al., “Psychophysical outcomes from a randomized pilot study of manual, electro, and sham acupuncture treatment on experimentally induced thermal pain,” The Journal of Pain, vol. 6, no. 1, pp. 55–64, 2005.
- K. K. S. Hui, E. E. Nixon, M. G. Vangel et al., “Characterization of the ‘deqi’ response in acupuncture,” BMC Complementary and Alternative Medicine, vol. 7, article 33, 2007.
- K. Zhou, J. Fang, X. Wang et al., “Characterization of De Qi with electroacupuncture at acupoints with different properties,” The Journal of Alternative and Complementary Medicine, vol. 17, no. 11, pp. 1007–1013, 2011.
- Y. Guo and J. Q. Fang, Eds., Experimental Acup-Moxibustion Science, China Medical Science Press, Beijing, China, 3rd edition, 2012 (Chinese).
- K. M. Wang, S. M. Yao, Y. L. Xian, and Z. L. Hou, “A study on the receptive field of acupoints and the relationship between characteristics of needling sensation and groups of afferent fibres,” Scientia Sinica B, vol. 28, no. 9, pp. 963–971, 1985.
- J. J. Park, M. Akazawa, J. Ahn et al., “Acupuncture sensation during ultrasound guided acupuncture needling,” Acupuncture in Medicine, vol. 29, no. 4, pp. 257–265, 2011.
- A. Y. Leung, J. Park, G. Schulteis, J.-R. Duann, and T. Yaksh, “The electrophysiology of De Qi sensations,” The Journal of Alternative and Complementary Medicine, vol. 12, no. 8, pp. 743–750, 2006.
- J. Lee, V. Napadow, J. Kim et al., “Phantom acupuncture: dissociating somatosensory and cognitive/affective components of acupuncture stimulation with a novel form of placebo acupuncture,” PLoS ONE, vol. 9, no. 8, Article ID e104582, 2014.
- N. Salih, P. I. Bäumler, M. Simang, and D. Irnich, “Deqi sensations without cutaneous sensory input: results of an RCT,” BMC Complementary and Alternative Medicine, vol. 10, article 81, 2010.
- M. Sandberg, T. Lundeberg, L.-G. Lindberg, and B. Gerdle, “Effects of acupuncture on skin and muscle blood flow in healthy subjects,” European Journal of Applied Physiology, vol. 90, no. 1-2, pp. 114–119, 2003.
- T.-C. Kuo, C.-W. Lin, and F.-M. Ho, “The soreness and numbness effect of acupuncture on skin blood flow,” The American Journal of Chinese Medicine, vol. 32, no. 1, pp. 117–129, 2004.
- S. Sakai, E. Hori, K. Umeno, N. Kitabayashi, T. Ono, and H. Nishijo, “Specific acupuncture sensation correlates with EEGs and autonomic changes in human subjects,” Autonomic Neuroscience: Basic and Clinical, vol. 133, no. 2, pp. 158–169, 2007.
- V. Napadow, R. P. Dhond, J. Kim et al., “Brain encoding of acupuncture sensation&coupling on-line rating with fMRI,” NeuroImage, vol. 47, no. 3, pp. 1055–1065, 2009.
- K. K. S. Hui, J. Liu, O. Marina et al., “The integrated response of the human cerebro-cerebellar and limbic systems to acupuncture stimulation at ST 36 as evidenced by fMRI,” NeuroImage, vol. 27, no. 3, pp. 479–496, 2005.
- X. Wang, S.-T. Chan, J. Fang et al., “Neural encoding of acupuncture needling sensations: evidence from a fMRI study,” Evidence-Based Complementary and Alternative Medicine, vol. 2013, Article ID 483105, 15 pages, 2013.
- C. A. Vincent, P. H. Richardson, J. J. Black, and C. E. Pither, “The significance of needle placement site in acupuncture,” Journal of Psychosomatic Research, vol. 33, no. 4, pp. 489–496, 1989.
- J. Kong, R. Gollub, T. Huang et al., “Acupuncture De Qi, from qualitative history to quantitative measurement,” The Journal of Alternative and Complementary Medicine, vol. 13, no. 10, pp. 1059–1070, 2007.
- P. White, F. Bishop, H. Hardy et al., “Southampton needle sensation questionnaire: development and validation of a measure to gauge acupuncture needle sensation,” The Journal of Alternative and Complementary Medicine, vol. 14, no. 4, pp. 373–379, 2008.
- H. Park, J. Park, H. Lee, and H. Lee, “Does Deqi (needle sensation) exist?” American Journal of Chinese Medicine, vol. 30, no. 1, pp. 45–50, 2002.
- D. T. W. Yu, A. Y. M. Jones, and M. Y. C. Pang, “Development and validation of the Chinese version of the Massachusetts General Hospital Acupuncture Sensation Scale: an exploratory and methodological study,” Acupuncture in Medicine, vol. 30, no. 3, pp. 214–221, 2012.
- D. Pach, C. Hohmann, R. Lüdtke, F. Zimmermann-Viehoff, C. M. Witt, and C. Thiele, “German translation of the southampton needle sensation questionnaire: use in an experimental acupuncture study,” Forschende Komplementarmedizin, vol. 18, no. 6, pp. 321–326, 2011.
- J. Park, H. Park, H. Lee, S. Lim, K. Ahn, and H. Lee, “Deqi sensation between the acupuncture-experienced and the Naïve: a Korean study II,” The American Journal of Chinese Medicine, vol. 33, no. 2, pp. 329–337, 2005.
- D. T. W. Yu and A. Y. M. Jones, “Physiological changes associated with de qi during electroacupuncture to LI4 and LI11: a randomised, placebo-controlled trial,” Acupuncture in Medicine, vol. 31, no. 2, pp. 143–150, 2013.
- H. Tsukayama, H. Yamashita, T. Kimura, and K. Otsuki, “Factors that influence the applicability of sham needle in acupuncture trials: two randomized, single-blind, crossover trials with acupuncture-experienced subjects,” The Clinical Journal of Pain, vol. 22, no. 4, pp. 346–349, 2006.
- D. E. Beaton, C. Bombardier, F. Guillemin, and M. B. Ferraz, “Guidelines for the process of cross-cultural adaptation of self-report measures,” Spine, vol. 25, no. 24, pp. 3186–3191, 2000.
- C. H. Lawshe, “A quantitative approach to content validity,” Personnel Psychology, vol. 28, no. 4, pp. 563–575, 1975.
- H. Ishida, “Shinge-hen,” in The Modern Japanese Translation of Huangdi Neijing the Suwen, H. Ishida, Ed., vol. 2, chapter 54, pp. 270–273, Toyo Gakujutsu Shuppansha, Ichikawa, Japan, 1992 (Japanese).
- M. Tavakol and R. Dennick, “Making sense of Cronbach's alpha,” International Journal of Medical Education, vol. 2, pp. 53–55, 2011.
- K. K. K. S. Hui, T. N. Sporko, M. G. Vangel, M. Li, J. Fang, and L. Lao, “Perception of Deqi by Chinese and American acupuncturists: a pilot survey,” Chinese Medicine, vol. 6, article no. 2, 2011.
Copyright © 2017 Masako Nishiwaki et al. This is an open access article distributed under the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.