Development and Preliminary Validation of a Stroke Physical Activity Questionnaire

Phusuttatam, Thunyakamon; Saengsuwan, Jittima; Kittipanya-ngam, Pajeemas

doi:https://doi.org/10.1155/2019/6764834

Stroke Research and Treatment

On this page

Abstract Introduction Methods Results Discussion Conclusion Data Availability Disclosure Conflicts of Interest Acknowledgments Supplementary Materials References Copyright Related Articles

Research Article | Open Access

Volume 2019 | Article ID 6764834 | https://doi.org/10.1155/2019/6764834

Development and Preliminary Validation of a Stroke Physical Activity Questionnaire

Thunyakamon Phusuttatam,¹Jittima Saengsuwan,^1,2and Pajeemas Kittipanya-ngam¹

Academic Editor: Augusto Fusco

Received21 Apr 2019

Revised12 Jun 2019

Accepted15 Jun 2019

Published25 Jul 2019

Abstract

Objective. The aim of the current study was to develop and then to determine preliminary validity of a physical activity questionnaire specifically designed for ambulatory, chronic stroke patients. Methods. This cross-sectional study recruited 92 chronic stroke patients. In Phase I, the SPAQ was developed and its content validity index (CVI) determined. In Phase II, we examined (1) the concurrent validity of SPAQ vs. the International Physical Activity Questionnaire-Short Form (IPAQ-SF); (2) the convergent validity of SPAQ vs. the Functional Ambulation Category (FAC), vs. the six-minute walk test (6-MWT), vs. the timed up and go test (TUGT), vs. the Motricity Index, vs. the National Institutes of Health Stroke Scale (NIHSS), and vs. the Modified Rankin Scale (MRS); and (3) the discriminant validity of the SPAQ vs. the Montreal Cognitive Assessment (MoCA). The concurrent validity and convergent and divergent validity were analysed using Spearman’s rank correlation coefficient. The agreement between the SPAQ and IPAQ-SF was assessed using the Kappa statistic. Results. The SPAQ has 12 items in 3 main components which covers low (7 items), moderate (3 items), and vigorous (2 items) physical activity. The SPAQ had a CVI of 0.93. Spearman’s correlation coefficient () for SPAQ vs. IPAQ-SF was 0.53 (p < 0.001). The SPAQ yielded substantial agreement with the IPAQ-SF (Kappa = 0.65). For convergent validity, the SPAQ had a moderate correlation with the 6-MWT, MI, NIHSS, FAC, TUGT, and MRS (p < 0.05). For discriminant validity, the SPAQ was not correlated with the MoCA ( = 0.061, p = 0.68). Conclusion. The SPAQ can be used to practically assess physical activity in chronic stroke patients, and it has acceptable psychometric properties which are comparable to other standard physical activity questionnaires. Future work should determine the validity of the SPAQ using an objective device such as an accelerometer.

1. Introduction

Stroke is a leading cause of death and disability worldwide and continues to have a high incidence in low- to middle-income countries [1]. There is ample evidence that stroke patients have low levels of physical activity and are more inactive than their healthy peers [2–4]. The inactivity of stroke patients may be the result of factors related to stroke (i.e., fatigue, lack of cardiovascular fitness, or coordination problems) together with personal factors (i.e., motivation or being elderly) [5–7].

One aim of stroke rehabilitation is to enable the stroke survivor to develop and maintain an active lifestyle that meets the recommended physical activity and exercise guidelines, so as to improve physical function and prevent stroke recurrence and other cardiac events [8]. Physical activity in patients after stroke has been shown to have a positive effect on poststroke depression [9, 10], social participation [11], health related quality of life [12], and cognitive function [13, 14]. Physical activity may prevent recurrent stroke and decrease poststroke mortality by modifying stroke risk factors (e.g., by decreasing blood pressure, decreasing the total cholesterol/HDL ratio, decreasing BMI, and improving aerobic capacity [15]). Physical inactivity is a stroke risk factor [16].

Since the questionnaire is a practical and relatively inexpensive way of assessing physical activity, one might expect that several questionnaires would have been developed to assess physical activity among stroke patients. In fact, few questionnaires have been developed that specifically assess physical activity in this population. The questionnaires currently in use among stroke patients include the Physical Activity Scale for the Elderly (PASE) [17], the Physical Activity Scale for Individuals with Physical Disabilities (PASIPD) [18], the Human Activity Profile (HAP) developed for people with pulmonary problems, and the International Physical Activity Questionnaire Short Form (IPAQ-SF) for the general population [19].

The aim of the current study was to develop and then to determine preliminary validity of a physical activity questionnaire specifically designed for ambulatory, chronic stroke patients with no communication problems

2. Methods

This was a cross-sectional study conducted at the Rehabilitation Unit, Srinagarind Hospital, Thailand, between December 2017 and October 2018. The study was reviewed and approved by the Khon Kaen University Ethics Committee for Human Research (Ref. HE 601395). All participants signed informed consent prior to participation.

3. Subjects

The eligible population comprised chronic stroke patients who had had a haemorrhagic or ischaemic stroke 3 or more months previously. Patient inclusion criteria were (a) age ≥ 18 years, (b) ability to walk at least 10 m, and (c) ability to communicate. Exclusion criteria were (a) severe perceptual or communication problems and/or (b) severe concurrent cardiac or pulmonary disease.

Based on an in-depth interview, 15 stroke patients were recruited for development of the preliminary version of SPAQ1 (i.e., the Pre-SPAQ1). Thirty patients were then recruited to pilot the questionnaire so as to determine any deficits and to improve understanding (Pre-SPAQ2). The Pre-SPAQ2 was then evaluated for content validity by 5 experts and the final SPAQ was developed thereafter. Finally, 47 patients were recruited to determine the validity of the SPAQ. The sample size for the validation study was determined assuming a correlation coefficient of 0.4 between the SPAQ and the International Physical Activity Questionnaire Short Form (IPAQ-SF). This assumption was based upon the correlation coefficient between the IPAQ-SF and the Global Physical Activity Questionnaires (GPAQ) of 0.45-0.65 [20]. We hypothesised that in stroke patients the correlation might be lower than the general population. Alpha was set at 0.05, and β at 0.2. The sample size formula was sample size = 0.5ln[(1+r)(1-r)] [21], which yielded a total sample size of 47 patients.

4. Study Protocol

The study was divided into two main phases: Phase I was development of the stroke physical activity questionnaire (SPAQ), and Phase II determination of the validity of the SPAQ.

Phase 1 (development) comprised 4 steps.

Step 1. Defining the concept of physical activity after stroke based on five aspects (viz. daily activities, household activities, outdoor activities, exercise, and hobbies).

Step 2. Conducting in-depth interviews with 15 post-stroke patients according to the five preceding concepts. The preliminary version of SPAQ (Pre-SPAQ1) was then developed based on the in-depth interview.

Step 3. Improving the Pre-SPAQ1 with respect to the preliminary version of the SPAQ2 (Pre-SPAQ2) by using the Pre-SPAQ1 to interview 30 patients. The data were then analysed with respect to the level and frequency of each activity.

Step 4. Rating the Pre-SPAQ2 for content validity by five experts. Expertise was accepted if the general rehabilitation doctor, cardiac rehabilitation doctor, neurologist, physical therapist, or home care nurses had at least 5 years’ experience caring for stroke patients. The final stroke physical activity questionnaire (SPAQ) was then developed with input from experts.

Phase 2 (Validation) determined concurrent validity of the SPAQ with the International Physical Activity Questionnaire-Short Form (IPAQ-SF). The convergent and discriminant validity were determined using data from the SPAQ as correlated with (a) the six-minute walk test (6-MWT), (b) the timed up and go test (TUG), (c) the Motricity Index, (d) the Functional Ambulation Category (FAC), (e) the National Institute of Health Stroke Scale (NIHSS), (f) the Modified Rankin Scale (MRS), and (g) the Montreal Cognitive Assessment (MoCA).

5. Statistical Analysis

SPSS version 23.0 was used for statistical analysis (IBM SPSS Statistics for Windows, Version 23.0. Armonk, NY: IBM Corp.). Normality was tested using the Kolmogorov-Smirnov test. Data that deviate from a normal distribution are reported as median and interquartile range (IQR). The association of reported participation in different activities with gait aids was analysed using the chi-squared test. Fisher’s exact test was used when the number in each cell of the table was less than 5. The Mann-Whitney U test was used to compare average time in each activity in patients who were able to independently walk or used a gait aid. Spearman’s rank correlation coefficient () was used to study the relationship between time reported at moderate and high intensity and measured variables such as NIHSS or Motricity Index: between 0.1 and 0.3 indicated a weak relationship; 0.3 to 0.5 a moderate relationship; between 0.5 and 1 a strong relationship; and 1 a perfect relationship [22].

Agreement between the SPAQ and the IPAQ-SF was determined using the Kappa statistic. We classified patients into 2 groups according to whether or not they met (or did not meet) the Center of Disease Control and Prevention (CDC) guidelines for individuals to have moderate to vigorous physical activity at least 150 min/week [23]. We determined the amount of exercise based on the duration of moderate and vigorous physical activity indicated in the questionnaire. A Kappa between 0 and 0.20 indicated slight agreement; 0.21 to 0.40 fair agreement; 0.41 to 0.60 moderate agreement; 0.61 to 0.80 substantial agreement; and 0.81 to 1.00 almost perfect agreement [24].

6. Results

6.1. Phase 1: Development of the Questionnaire

The preliminary in-depth interview of 15 poststroke patients comprised 21 questions. The SPAQ (Pre-SPAQ1) covered daily living (3 questions), household activities (5 questions), outdoor activities (4 questions), exercise (6 questions), and hobbies (3 questions).

The Pre-SPAQ1 was then piloted in 30 poststroke patients to assess the comprehensibility of the questionnaires. Some of the questions were not always completed by several of the patients, so we either adjusted or reduced some questions. Finally, there were 12 questions (preliminary version of SPAQ (Pre-SPAQ2)) and we arranged the questions based on intensity of exercise as light, moderate, and vigorous (i.e. 7, 3, and 2 questions, respectively).

The Pre-SPAQ2 was then assessed for content validity by five experts. The mean content validity index was 0.93 (range 0.8 to 1.0). We then made additional adjustments to the questionnaire based on the expert comments to improve the comprehensibility of the questions.

6.2. Phase 2: Validation of the Stroke Physical Activity Questionnaire

In Phase 2, we had 47 poststroke patients (age 57.5 ± 10.1 years; BMI 24.7 ± 4.3 kg/m²; mean ± standard deviation). Most were men (72.3%). The median poststroke time was 16.5 months. Thirty-nine patients (83 %) had had an ischaemic stroke and 30 (63.8 %) experienced left hemiparesis. Most patients used gait aids (55.3 %) and 42.6 % had not exercised prior to the stroke (Table 1).

According to the questionnaire results, the respective median time for light, moderate, and vigorous physical activity was 590, 0, and 0 min/week in patients with gait aids and 895, 70, and 0 min/week in patients who walked independently. Patients who walked independently were more likely to participate in light housework, light gardening, grocery shopping, and moderate intensity exercise. Patients with gait aids spent significantly less time in light housework, gardening work, grocery shopping, and moderate intensity exercise compared to those who walked independently. There was only a small proportion of patients who participated in high intensity exercise; thus there was no significant difference between patients who walked independently and those who used gait aids. Time in physical activity was 18.7 % of awake time in patients who walked independently and 13.8 % in patients with gait aids. The time without activity was approximately 12.7 and 13.5 h/day of awake time, and inactive time accounted for 81.3 % and 86.2 % of awake time in patients who walked independently and in patients who used gait aids, respectively (Tables 2 and 3).

7. Validity

7.1. Concurrent Validity and Agreement of SPAQ vs. IPAQ-SF

Concurrent validity tests whether the new questionnaire correlated well with previously established measurements. Table 4 shows moderate correlation ( = 0.53, p < 0.001) between time spent in moderate and vigorous physical activity (MVPA) as obtained from the SPAQ and International Physical Activity Questionnaire-Short Form (IPAQ-SF). The SPAQ identified nine patients who met the physical activity recommendation per the CDC whereas the IPAQ-SF identified 13 patients who met the recommendations. The kappa value was 0.65 (p < 0.0001) (Table 5).

7.2. Convergent Validity

Convergent validity tests whether theoretical constructs are indeed related to the questionnaire endpoint. Table 6 shows the correlation between the SPAQ and different measures of motor strength (Motricity Index), ambulatory level (FAC), balance (TUG), aerobic fitness (6-MWT), stroke severity level (NIHSS), and disability level (MRS). The SPAQ yielded a moderate correlation with the 6-MWT ( = 0.45, p = 0.001), Motricity Index ( = 0.43, p = 0.002), and the FAC (0.37, p = 0.01). The SPAQ was inversely correlated with the NIHSS ( = -0.48, p = 0.001), MRS ( = -0.38, p = 0.008), and TUG ( = -0.36, p = 0.013).

8. Discriminant Validity

Convergent validity tests whether constructs that are theoretically expected not to be related to the endpoint of the questionnaire are, in fact, not related. Table 6 (lower panel) shows the correlation between the SPAQ and the MoCA. No significant correlation was found between SPAQ and MoCA.

9. Discussion

Assessment of physical activity is of great importance when assessing patient status, particularly as a guide to interventions and follow-up after rehabilitation. The aim of the current study was to develop and then to determine preliminary validity of a physical activity questionnaire specifically designed for ambulatory, chronic stroke patients with no communication problems. We developed a questionnaire after conducting in-depth interviews and running a pilot. The result was a 12-question SPAQ tool covering the intensity, duration, and frequency of physical activity. The data obtained from the SPAQ revealed the time spent at each intensity of physical activity per week. Since patients typically spend more time doing low-intensity physical activities, there are more questions related to low-intensity physical activity than moderate- and high-intensity activities.

According to the questionnaire, we found that most patients engaged in low-level physical activity. Only one-fifth (21.2 %) of the patients met the WHO global recommendations for physical activity [23]. Patients reported a median of 5 min of moderate physical activity per week. Inactive time accounted for 81.3 % to 86.2 % of total awake time with an average of 12.7 to 13.5 hours of inactive awake time. Our findings are consistent with previous reports of stroke patients using an accelerometer where inactivity accounted for 81.6 to 84.1 % of awake time [25], and the average inactive awake time was 13 hours per week [12, 26].

Concurrent validity indicated a strong relationship ( = 0.53) between the SPAQ and IPAQ-SF, as found between the IPAQ-SF vs. GPAQ among normal subjects (r = 0.45 to 0.65) [20]. Such a correlation coefficient is considered acceptable, considering the physical activity questionnaire should have a correlation greater than 0.50 when determining the relationship with other standard questionnaires [27]. We also found substantial agreement between the SPAQ vs. IPAQ-SF (kappa = 0.65) in terms of recommended physical activity per the WHO global recommendations (150 min/week of moderate to vigorous exercise). As for convergent validity, we found that SPAQ had a moderate correlation with proposed components (viz. FAC, Motricity Index, MRS, TUG-T, 6MWT, and NIHSS).

The correlation of SPAQ vs. 6-MWT in this study (= 0.45, p = 0.001, 47 chronic stroke patients) was comparable to that of PASE vs. 6-MWT ( = 0.47, p < 0.001, 49 chronic stroke patients) [28] and higher than PASIPD vs. 6-MWT (r = 0.31, p = 0.057, 40 chronic stroke patients) [12]. Based on the recommended acceptable correlation coefficient of more than 0.30 for physical activity questionnaires and physical functioning and health variables [27], the SPAQ demonstrated an acceptable correlation in every domain.

We used the MoCA to study the discriminant validity, which aimed to examine whether measurements that are not correlated by theory were also not correlated in the study. The MoCA—the cognitive level—is theoretically not related to physical activity; thus, as expected, there was no correlation between SPAQ and MoCA in our study.

Our study had several limitations. Although the questionnaires and fitness levels were considered adequate for the assessment of concurrent validity, standard measures (e.g. accelerometers) should be used to confirm the concurrent validity, and a reliability study should be done. The questionnaire was developed and validated for only ambulatory stroke patients with no communication problems, so further validation of the questionnaire will be required if it is to be used among patients with communication problems.

10. Conclusion

The SPAQ can be used to practically assess physical activity among chronic stroke patients and it has comparable psychometric properties to other standard physical activity questionnaires. Future work should determine the validity of the SPAQ using an objective device such as accelerometer.

Data Availability

The data used to support the findings of this study are included within the article.

Disclosure

Parts of this work were previously presented as an abstract at the CaREhab Conference in Singapore, February 2019.

Conflicts of Interest

The authors declare that they have no conflicts of interest.

Acknowledgments

The authors thank Mr. Bryan Roderick Hamman for assistance with the English-language version of the manuscript under the aegis of the Khon Kaen University Publication Clinic. This study received grant support from the Faculty of Medicine, Khon Kaen University, Thailand (Grant number IN61208).

Supplementary Materials

Supplementary material 1: Stroke Physical Activity Questionnaire Thai version. Supplementary material 2: Stroke Physical Activity Questionnaire English version. (Supplementary Materials)

References

V. L. Feigin, G. A. Roth, M. Naghavi, P. Parmar, R. Krishnamurthi, S. Chugh et al., “Global burden of stroke and risk factors in 188 countries, during 1990-2013: a systematic analysis for the global burden of disease study 2013,” Lancet Neurol, vol. 15, no. 9, pp. 913–924, 2016.
View at: Publisher Site | Google Scholar
C. English, G. N. Healy, A. Coates, L. K. Lewis, T. Olds, and J. Bernhardt, “Sitting time and physical activity after stroke: physical ability is only part of the story,” Topics in Stroke Rehabilitation, vol. 23, no. 1, pp. 36–42, 2015.
View at: Publisher Site | Google Scholar
L. A. Hale, J. Pal, and I. Becker, “Measuring free-living physical activity in adults with and without neurologic dysfunction with a triaxial accelerometer,” Archives of Physical Medicine and Rehabilitation, vol. 89, no. 9, pp. 1765–1771, 2008.
View at: Publisher Site | Google Scholar
K. R. Zalewski and L. Dvorak, “Barriers to physical activity between adults with stroke and their care partners,” Topics in Stroke Rehabilitation, vol. 18, no. 1, pp. 666–675, 2011.
View at: Publisher Site | Google Scholar
S. Thilarajah, B. F. Mentiplay, K. J. Bower et al., “Factors associated with post-stroke physical activity: a systematic review and meta-analysis,” Archives of Physical Medicine and Rehabilitation, vol. 99, no. 9, pp. 1876–1889, 2018.
View at: Publisher Site | Google Scholar
C. U. Persson, P. Hansson, G. Lappas, and A. Danielsson, “Physical activity levels and their associations with postural control in the first year after stroke,” Physical Therapy in Sport, vol. 96, no. 9, pp. 1389–1396, 2016.
View at: Publisher Site | Google Scholar
A. Danielsson, C. Meirelles, C. Willen, and K. S. Sunnerhagen, “Physical activity in community-dwelling stroke survivors and a healthy population is not explained by motor function only,” PM&R, vol. 6, no. 2, pp. 139–145, 2014.
View at: Publisher Site | Google Scholar
S. A. Billinger, R. Arena, J. Bernhardt et al., “Physical activity and exercise recommendations for stroke survivors: a statement for healthcare professionals from the American Heart Association/American Stroke Association,” Stroke, vol. 45, no. 8, pp. 2532–2553, 2014.
View at: Publisher Site | Google Scholar
J. J. Eng and B. Reime, “Exercise for depressive symptoms in stroke patients: a systematic review and meta-analysis,” Clinical Rehabilitation, vol. 28, no. 8, pp. 731–739, 2014.
View at: Publisher Site | Google Scholar
S. E. Aaron, C. M. Gregory, and A. N. Simpson, “Lower odds of poststroke symptoms of depression when physical activity guidelines met: national health and nutrition examination survey 2011–2012,” Journal of Physical Activity & Health, vol. 13, no. 8, pp. 903–909, 2016.
View at: Publisher Site | Google Scholar
A. O. Obembe and J. J. Eng, “Rehabilitation interventions for improving social participation after stroke: a systematic review and meta-analysis,” Neurorehabilitation and Neural Repair, vol. 30, no. 4, pp. 384–392, 2015.
View at: Publisher Site | Google Scholar
D. Rand, J. J. Eng, P. Tang, C. Hung, and J. Jeng, “Daily physical activity and its contribution to the health-related quality of life of ambulatory individuals with chronic stroke,” Health and Quality of Life Outcomes, vol. 8, no. 1, p. 80, 2010.
View at: Publisher Site | Google Scholar
L. E. Oberlin, A. M. Waiwood, T. B. Cumming, A. L. Marsland, J. Bernhardt, and K. I. Erickson, “Effects of physical activity on poststroke cognitive function a meta-analysis of randomized controlled trials,” Stroke, vol. 48, no. 11, pp. 3093–3100, 2017.
View at: Publisher Site | Google Scholar
A. Wong, A. Y. Lau, E. Lo et al., “Relations between recent past leisure activities with risks of dementia and cognitive functions after stroke,” PLoS ONE, vol. 11, no. 7, Article ID e0159952, 2016.
View at: Publisher Site | Google Scholar
P. L. Prior, V. Hachinski, R. Chan et al., “Comprehensive cardiac rehabilitation for secondary prevention after transient ischemic attack or mild stroke: psychological profile and outcomes,” Journal of Cardiopulmonary Rehabilitation and Prevention, vol. 37, no. 6, pp. 428–436, 2017.
View at: Publisher Site | Google Scholar
M. J. O'Donnell, S. L. Chin, S. Rangarajan et al., “Global and regional effects of potentially modifiable risk factors associated with acute stroke in 32 countries (INTERSTROKE): a case-control study,” The Lancet, vol. 388, no. 10046, pp. 761–775, 2016.
View at: Publisher Site | Google Scholar
R. A. Washburn, W. Zhu, E. McAuley, M. Frogley, and S. F. Figoni, “The physical activity scale for individuals with physical disabilities: development and evaluation,” Archives of Physical Medicine and Rehabilitation, vol. 83, no. 2, pp. 193–200, 2002.
View at: Publisher Site | Google Scholar
R. J. van den Berg-Emons, A. A. L'Ortye, L. M. Buffart et al., “Validation of the physical activity scale for individuals with physical disabilities,” Archives of Physical Medicine and Rehabilitation, vol. 92, no. 6, pp. 923–928, 2011.
View at: Publisher Site | Google Scholar
C. L. Craig, A. L. Marshall, M. Sjoestroem, A. E. Bauman, M. L. Booth, B. E. Ainsworth et al., “International physical activity questionnaire: 12-country reliability and validity,” Medicine & Science in Sports & Exercise, vol. 35, pp. 1396–1396, 2003.
View at: Google Scholar
F. C. Bull, T. S. Maslin, and T. Armstrong, “Global Physical Activity Questionnaire (GPAQ): nine country reliability and validity study,” Journal of Physical Activity & Health, vol. 6, no. 6, pp. 790–804, 2009.
View at: Publisher Site | Google Scholar
S. Hulley, S. Cummings, W. Browner, D. Grady, and T. Newman, Designing Clinical Research, Lippincott Williams & Wilkins, Philadelphia, USA, 1st edition, 2013.
G. Corder and D. Foreman, Nonparametric Statistics Form Non-Statisticians: A Step-By-Step Approach, John Wiley and Sons, Inc, New Jersey, NJ, USA, 1st edition, 2009.
World Health Organization, “Global recommendations on physical activity for health,” https://www.who.int/dietphysicalactivity/global-PA-recs-2010.pdf, 2010.
View at: Google Scholar
J. R. Landis and G. G. Koch, “The measurement of observer agreement for categorical data,” Biometrics, vol. 33, no. 1, pp. 159–174, 1977.
View at: Publisher Site | Google Scholar
S. Mudge, P. A. Barber, and N. S. Stott, “Circuit-based rehabilitation improves gait endurance but not usual walking activity in chronic stroke: a randomized controlled trial,” Archives of Physical Medicine and Rehabilitation, vol. 90, no. 12, pp. 1989–1996, 2009.
View at: Publisher Site | Google Scholar
D. Rand, J. J. Eng, P.-F. Tang, J.-S. Jeng, and C. Hung, “How active are people with stroke? Use of accelerometers to assess physical activity,” Stroke, vol. 40, no. 1, pp. 163–168, 2009.
View at: Publisher Site | Google Scholar
L. Forsén, N. W. Loland, A. Vuillemin et al., “Self-administered physical activity questionnaires for the elderly: a systematic review of measurement properties,” Sports Medicine, vol. 40, no. 7, pp. 601–623, 2010.
View at: Publisher Site | Google Scholar
M. Lindahl, L. Hansen, A. Pedersen, T. Truelsen, and G. Boysen, “Self-reported physical activity after ischemic stroke correlates with physical capacity,” Advances in Physiotherapy, vol. 10, no. 4, pp. 188–194, 2009.
View at: Publisher Site | Google Scholar

Copyright

Copyright © 2019 Thunyakamon Phusuttatam et al. This is an open access article distributed under the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.

PDF Download Citation

Download other formats

Order printed copies

Views

2551

Downloads

1441

Citations