Research Article | Open Access
Reliability of a Simple Physical Therapist Screening Tool to Assess Errors during Resistance Exercises for Musculoskeletal Pain
The main objective was to investigate the intra- and intertester reliability of a simple screening tool assessing errors in exercise execution by visual observation. 38 participants with no previous resistance exercise experience practiced for two weeks four typical upper limb exercises using elastic tubing. At 2-week follow-up, the participants were invited for a test-retest assessment on errors in technical execution. The assessment was based on ordinal deviation of joint position from neutral of the shoulder, elbow, and wrist in a single plane by visual observation. Moderate intratester reliability weighted kappa (w) score ranging from 0.50 (0.21–0.71) to 0.57 (0.24–0.82) for observer 1 and a fair to moderate intratester reliability w score ranging from 0.27 (0.09–0.43) to 0.52 (0.15–0.86) for observer 2 across the four exercises was observed. For intertester reliability moderate to substantial mean w scores were found between the two observers, slightly improving from round one to round two ranging from 0.40 (0.20–0.59) to 0.68 (0.45–0.91) in round one to 0.52 (0.20–0.80) to 0.69 (0.39–0.86) in round two. The exercise error assessment demonstrated fair to substantial intratester and intertester reliability, which is congruent with previously published studies. Hence the simplicity of defining a neutral joint position for each of the involved joints in the exercise and categorizing the deviation in “some deviation” and “substantial deviation” to either side in a single plane is a viable and inexpensive solution when assessing for errors during exercise.
With an estimated cost between 0.5% and 2% of the Gross National Product  musculoskeletal disorders comprise one of the most common and costly public health problems in Europe today . In particular, the occurrence of neck and shoulder pain has been progressively increasing through the past 10–20 years  and is now only surpassed by low back pain, which still is the most common reported multifactorial musculoskeletal disorder [4, 5]. In occupations characterized by repetitive movement tasks and sustained low force muscular contractions, such as computer and laboratory work, neck and shoulder pain is of substantial concern [6, 7] as pain not only decreases wellbeing at work [1, 8–10] but could also affect productivity and accuracy, something of utmost importance in the work of laboratory technicians , while potentially leading to an increased sense of stress . Indeed, the socioeconomic consequences of musculoskeletal pain and discomfort are considerable.
Possible intervention strategies for reducing the adverse effects of monotonous repetitive work include strengthening exercises of the painful muscles and a substantial number of studies within the past 10 years provide ample evidence for the effectiveness and the clinical relevance of physical activity in the form of resistance training modalities to manage musculoskeletal discomfort and pain [1, 13–16]. In accordance with this, our lab has previously shown strong effects on neck and shoulder pain with resistance exercise using elastic tubing, dumbbells and kettlebells [15, 17–25] among office workers and lab technicians with substantial and clinically relevant pain reduction. However, one possible concern with integrated resistance exercise as an intervention strategy in the working environment is the potential harmful effects of incorrectly executed exercises. Although, to own knowledge, no cause and effect studies provide evidence that wrongly executed strengthening exercises musculoskeletal problems it may be speculated that improper exercise form potentially increases the risk of sprains, strains, tendonitis, bursitis, or impingement of joints and ligaments as well as muscle contusions and general overuse injuries [26, 27]. For instance, the impingement of the subacromial bursa lying between the coracoacromial ligament and the supraspinatus muscle  is a relatively common problem in rehabilitation of athletes and it may occur with overuse and/or lack of scapulae-humeral rhythm during shoulder abduction movements as seen in swimmers [29–32].
To minimize mistakes potentially leading to overuse injuries, physical therapists and physical trainers may benefit from a consensus about the correct technical execution of the exercise by utilizing an assessment tool. Additionally, no standardized assessment tools exist at present, whereas the main objective of this study was to develop and investigate the intra- and intertester reliability of a simplistic assessment tool to assess errors in exercise execution to help clinicians evaluate technical errors in exercise execution. Four commonly used elastic tubing exercises for upper limb musculoskeletal pain were included.
2.1. Study Design
We recruited 38 participants (laboratory technicians and office workers) from a pool of approximately 200 people at a large pharmaceutical company in Copenhagen, Denmark, in the fall of 2012. Inclusion criteria were (1) a history of neck or shoulder pain, (2) female aged 18–67 years, and (3) no prior experience exercising with elastic tubing. Exclusion criteria were (1) resting blood pressure higher than 160/100, (2) pregnancy, and (3) life-threatening disease or other adverse health conditions and contraindications towards resistance exercise. The participants were recruited based on their answers to a recruitment-screening questionnaire sent out by email. Participants meeting the inclusion criteria were allocated to either a personal + video instruction group or a video-based instruction group as part of a randomized controlled trial that will be published in a separate article. Following two weeks of practicing four different elastic tubing exercises targeting the shoulder, arm, and hand musculature, all participants were invited to participate in intra- and intertester reliability examination of the assessment tool for errors made during exercise execution.
2.2. Ethical Approval and Trial Registration
All participants () were informed about the main objective and content of the project and gave written informed consent to participate in the study, which conformed to the Declaration of Helsinki. The study was approved by the Local Ethical Committee (H-3-2010-062). Table 1 shows baseline demographics.
2.3. Exercise Error Assessment
Together with two physical therapists we developed a simple standardized operating procedure for assessing the number of errors in four common shoulder, arm, and hand exercises using elastic tubing. The four exercises were (1) bilateral raise, (2) unilateral external shoulder rotation, (3) unilateral wrist extension, and (4) bilateral scapular retraction (Figures 1(a)–1(d)). Each exercise was described by joint (wrist, elbow, and shoulder) and ordinal deviation from the neutral position in a single plane, by visual observation. For each joint the examiners had to evaluate by how much the position of the joint deviated from neutral, as well as to what side from neutral, during exercise execution each joint deviation was chosen based on best practice and instructional experience with these exercises. The possible deviations were denoted as “no deviation,” “some deviation,” or “substantial deviation.” The assessment score “no deviation” was given the value “0,” the score “some deviation” was given the value either “+1” or “−1” depending on which direction the deviation had and “substantial deviation” was given the value “+2” or “−2” again depending on the direction of the deviation equaling five different possible scores (−2, −1, 0, 1, and 2) with “0” being neutral or the defined ideal for each of the exercise specific subdomains. Figure 2 shows an example of the elbow position in the shoulder external rotation exercise for the right side. Table 2 lists the subdomains for each exercise. Figures 1(a)–1(d) show the four exercises with the ideal defined technique (no deviation) in the start and end position and video instructional material can be seen online here: http://www.jobogkrop.dk/Ondt-i-muskler-og-led/Ondt-i-nakke-skulder-og-arm/Elastikoevelser-for-nakke-skulder-og-arm.
|(a) Exercise: Bilateral raise|
|(b) Exercise: Bilateral shoulder retraction|
|(c) Exercise: Unilateral shoulder external rotation|
|(d) Exercise: Unilateral Wrist Extension|
For the reliability assessment each participant was invited in for assessment on two separate occasions with at least one day in between, by two trained physical therapists. They were asked to perform 2 × 10 repetitions of each exercise in a slow and controlled manner taking approximately 1-2 sec. for the concentric portion of the lift and 1-2 sec for the eccentric portion. For the unilateral exercises (unilateral shoulder external rotation and unilateral wrist extension) the participant used the dominant arm. One set of each exercise was demonstrated facing the examiners and one set was demonstrated in a side-view profile. The two examiners conducting the assessment were positioned in the room in such a way that they could not see what the other examiner was noting. The examiners were instructed to not talk about the exercise execution during or after the assessment. Furthermore, the examiners were instructed to not provide any feedback to the participant on the execution of each exercise. Finally a standard operating procedure was followed to make sure each assessor evaluated the appropriate technical aspects of each exercise.
Approximately 200 people received information email about the study. 49 people agreed to answer a baseline-screening questionnaire and 38 were invited to participate in the study. One person was excluded due to lack of answering the screening questionnaire and four people did not show up for the second assessment by the examiners due to sickness unrelated to the study. Thus, 37 people completed the first assessment and 33 people completed the second assessment.
Dropouts from the two-week familiarization training were invited to participate in the test-retest assessment to avoid selection bias. Intra- and intertester reliabilities were determined by weighted Kappa (w) analysis of the SAS statistical software (SAS institute, Cary, NC, version 9.2). Landis and Koch have previously defined w > 0.80 as almost perfect, 0.60 ≤ w < 0.80 as substantial, 0.40 ≤ w < 0.60 as moderate, 0.21 ≤ w < 0.40 as fair, and w < 0.20 as slight agreement . Further, we calculated an intraclass correlation coefficient (ICC) between the two examiners and two rounds from the mean error assessment scores of each exercise.
In general, we found a moderate intratester reliability mean w score ranging from 0.50 (0.21–0.71) to 0.57 (0.24–0.82) for observer 1 and a fair to moderate intratester reliability mean w score ranging from 0.27 (0.09–0.43) to 0.52 (0.15–0.86) for observer 2 across the four exercises. Similarly, a moderate to substantial intertester reliability mean w was found between the two observers, slightly improving from round one to round two ranging from 0.40 (0.20–0.59) to 0.68 (0.45–0.91) in round one to 0.52 (0.20–0.80) to 0.69 (0.39–0.86) in round two. Table 2 summarizes the intra- and intertester reliability w scores in each of the observed subdomains of the four different exercises (Figures 1(a)–1(d)) and Table 3 summarizes ICC for the four exercises.
This study shows fair to substantial intra- and intertester reliability of a very simple design assessment protocol of errors performed during commonly used elastic tubing exercises for musculoskeletal pain of the neck/shoulder, arm, and hand. The results show that physical therapists and physical trainers, with little practice, are able to spot errors in trainee exercise execution in a reliable way once a consensus about correct technical execution has been formed. Our results compliment previous findings in intertester reliability of movement assessments with similar results. For instance, the Melbourne Assessment of Unilateral Upper Limb Function for children with neurological impairments was found to have moderate to high interrater reliability by visual observation  and similarly; the Movement Assessment Battery for Chinese preschool children (Movement ABC) has also shown good intertester reliability . Movement assessments targeted at the healthy adult population, like the Functional Movement Screen (FMS), have shown equally good intratester and intertester reliability [32, 36, 37] indicating that assessing movement by visual observation, between testers as well for the same tester, is a usable tool when physical trainers, physical therapists, and movement coaches correct technical exercise execution.
Overall our study shows a moderate reliability of the assessment tool. Noteworthy is the increase in intertester reliability from round 1 to round 2 in three of the four exercises indicating that examiners increase their accuracy in error spotting, which can be construed as a basic visual discrimination task. That kind of perceptual learning has been shown to improve with practice and can be viewed as local (in a retinotopic sense), as well as specific to the orientation of the visual target [38, 39], arguably the case in our study.
The one exercise not showing an improvement in intertester reliability between rounds is the bilateral scapulae retraction exercise. Investigating the subdomains indicates that the wrist and elbow position assessment decreases between the two examiners, that is, intertester reliability, from round 1 to round 2. It could be speculated that, because the majority of movement happens around the shoulder joint in the Bilateral raise, examiners might unintentionally pay more attention to that, because of the dynamic movement happening in that joint (shoulder) compared to the smaller joints (wrist and elbow), which primarily hold a static position throughout the movement. Directing attention towards the major moving part of the body could be an indication of momentary attentional drift of the examiner , but it still remains unclear why there is a drop in w intertester reliability score in this particular exercise from round 1 to round 2. The intratester reliability of the two examiners in this study was fair to moderate with observer two showing poorer reproducibility of the assessment, especially when assessing unilateral shoulder external rotation. The lack of consistency between rounds for examiner 2 is difficult, if not impossible, to explain but again may be related to an attentional drift of the mind resulting in momentary inattentiveness.
Strengths of the present study include the number of participants being assessed and the simple assessment design. A further strength is that we assessed both intra- and intertester reliabilities, which gives information about reproducibility over time as well as between different assessors. Limitations to the present study include the lack of objective assessment measures, for example, joint angle kinematics and the constrained exercises demonstration of each participant limited to 2 sets of 10 repetitions. Had the exercise performance of each participant been recorded and the examiners allowed to see the video multiple times as well as having the option of slowing and freeze framing the video sequences the reliability scores might have been higher. Furthermore, having video footage of the examiners performing the assessments would have allowed our lab to analyze behaviour and state of attention. However, in most settings where time and equipment are limited such options are not viable, and simple screening tools are needed. Finally, it could be argued that the study is limited by only testing the reliability between two assessors. Theoretically, these reviewers could have been of above average visual assessment ability, which could provide a skewed result of inter- and intrarater reliability.
In conclusion the exercise error assessment demonstrated fair to substantial intratester and intertester reliability, which is congruent with previously published studies on movement assessment reliability, hence the simplicity of defining a neutral joint position for each of the involved joints in the exercise and categorizing the deviation in “some deviation” and “substantial deviation” to either side in a single plane is a viable solution when assessing simple exercises for errors during execution.
Conflict of Interests
The authors declare that there is no conflict of interests regarding the publication of this paper.
The authors thank physical therapists Teis Finsen and Tine Thomsen for valuable practical help during the project.
- S. Kilbom, T. Armstrong, P. Buckle et al., “Musculoskeletal disorders: work-related risk factors and prevention,” International Journal of Occupational and Environmental Health, vol. 2, no. 3, pp. 239–246, 1996.
- L. Punnett and D. H. Wegman, “Work-related musculoskeletal disorders: the epidemiologic evidence and the debate,” Journal of Electromyography and Kinesiology, vol. 14, no. 1, pp. 13–23, 2004.
- P. Hakala, A. Rimpelä, J. J. Salminen, S. M. Virtanen, and M. Rimpelä, “Back, neck, and shoulder pain in Finnish adolescents: national cross sectional surveys,” British Medical Journal, vol. 325, no. 7367, pp. 743–745, 2002.
- C. Ihlebaek, S. Brage, and H. R. Eriksen, “Health complaints and sickness absence in Norway, 1996–2003,” Occupational Medicine, vol. 57, no. 1, pp. 43–49, 2007.
- R. Ferrari and A. S. Russell, “Neck pain,” Best Practice and Research Clinical Rheumatology, vol. 17, no. 1, pp. 57–70, 2003.
- P. Côté, J. D. Cassidy, L. J. Carroll, and V. Kristman, “The annual incidence and course of neck pain in the general population: a population-based cohort study,” Pain, vol. 112, no. 3, pp. 267–273, 2004.
- P. Côté, G. van der Velde, J. D. Cassidy et al., “The burden and determinants of neck pain in workers. Results of the bone and joint decade 2000–2010 task force on neck pain and its associated disorders,” Journal of Manipulative and Physiological Therapeutics, vol. 32, no. 2, pp. S70–S86, 2009.
- O. Gureje, M. von Korff, G. E. Simon, and R. Gater, “Persistent pain and well-being: a World Health Organization study in primary care,” Journal of the American Medical Association, vol. 280, no. 2, pp. 147–151, 1998.
- A. Stewart and J. E. Ware, Measuring Functioning and Well-Being, Duke University Press, 1992, http://www.rand.org/pubs/commercial_books/CB361.html#src=mobile.
- A. F. Mannion, A. Elfering, R. Staerkle et al., “Outcome assessment in low back pain: how low can you go?” European Spine Journal, vol. 14, no. 10, pp. 1014–1026, 2005.
- M. Plebani and P. Carraro, “Mistakes in a stat laboratory: types and frequency,” Clinical Chemistry, vol. 43, no. 8, pp. 1348–1351, 1997.
- J. G. Arena, R. A. Sherman, G. M. Bruno, and J. D. Smith, “The relationship between situational stress and phantom limb pain: cross-lagged correlational data from six month pain logs,” Journal of Psychosomatic Research, vol. 34, no. 1, pp. 71–77, 1990.
- M. Hagberg, K. Harms-Ringdahl, R. Nisell, and E. Wigaeus Hjelm, “Rehabilitation of neck-shoulder pain in women industrial workers: a randomized trial comparing isometric shoulder endurance training with isometric shoulder strength training,” Archives of Physical Medicine and Rehabilitation, vol. 81, no. 8, pp. 1051–1058, 2000.
- M. K. Zebis, L. L. Andersen, M. T. Pedersen et al., “Implementation of neck/shoulder exercises for pain relief among industrial workers: a randomized controlled trial,” BMC Musculoskeletal Disorders, vol. 12, article 205, 2011.
- L. L. Andersen, C. A. Saervoll, O. S. Mortensen, O. M. Poulsen, H. Hannerz, and M. K. Zebis, “Effectiveness of small daily amounts of progressive resistance training for frequent neck/shoulder pain: randomised controlled trial,” Pain, vol. 152, no. 2, pp. 440–446, 2011.
- M. Viljanen, A. Malmivaara, J. Uitti, M. Rinne, P. Palmroos, and P. Laippala, “Effectiveness of dynamic muscle training, relaxation training, or ordinary activity for chronic neck pain: randomised controlled trial,” British Medical Journal, vol. 327, no. 7413, pp. 475–477, 2003.
- K. Jay, M. Schraefel, C. H. Andersen et al., “Effect of brief daily resistance training on rapid force development in painful neck and shoulder muscles: randomized controlled trial,” Clinical Physiology and Functional Imaging, vol. 33, no. 5, pp. 386–392, 2013.
- K. Jay, M. D. Jakobsen, E. Sundstrup et al., “Effects of kettlebell training on postural coordination and jump performance: a randomized controlled trial,” Journal of Strength and Conditioning Research, vol. 27, no. 5, pp. 1202–1209, 2013.
- K. Jay, D. Frisch, K. Hansen et al., “Kettlebell training for musculoskeletal and cardiovascular health: a randomized controlled trial,” Scandinavian Journal of Work, Environment and Health, vol. 37, no. 3, pp. 196–203, 2011.
- L. L. Andersen, M. K. Zebis, M. T. Pedersen et al., “Protocol for work place adjusted intelligent physical exercise reducing musculoskeletal pain in Shoulder and neck (VIMS): a cluster randomized controlled trial,” BMC Musculoskeletal Disorders, vol. 11, article 173, 2010.
- L. L. Andersen, K. Hansen, O. S. Mortensen, and M. K. Zebis, “Prevalence and anatomical location of muscle tenderness in adults with nonspecific neck/shoulder pain,” BMC Musculoskeletal Disorders, vol. 12, article 169, 2011.
- C. H. Andersen, L. L. Andersen, O. S. Mortensen, M. K. Zebis, and G. Sjøgaard, “Protocol for Shoulder function training reducing musculoskeletal pain in shoulder and neck: a randomized controlled trial,” BMC Musculoskeletal Disorders, vol. 12, article 14, 2011.
- L. L. Andersen, M. B. Jørgensen, A. K. Blangsted, M. T. Pedersen, E. A. Hansen, and G. Sjøgaard, “A randomized controlled intervention trial to relieve and prevent neck/shoulder pain,” Medicine and Science in Sports and Exercise, vol. 40, no. 6, pp. 983–990, 2008.
- L. L. Andersen, J. L. Andersen, C. Suetta, M. Kjaer, K. Søgaard, and G. Sjøgaard, “Effect of contrasting physical exercise interventions on rapid force capacity of chronically painful muscles,” Journal of Applied Physiology, vol. 107, no. 5, pp. 1413–1419, 2009.
- L. L. Andersen, M. Kjaer, K. Søgaard, L. Hansen, A. I. Kryger, and G. Sjøgaard, “Effect of two contrasting types of physical exercise on chronic neck muscle pain,” Arthritis Care and Research, vol. 59, no. 1, pp. 84–91, 2008.
- J. J. Wilson and T. M. Best, “Common overuse tendon problems: a review and recommendations for treatment,” The American Family Physician, vol. 72, no. 5, pp. 811–818, 2005.
- L. Almekinders, “Tendinitis and other chronic tendinopathies,” Journal of the American Academy of Orthopaedic Surgeons, vol. 6, no. 3, pp. 157–164, 1998.
- J. Salata M, J. Nho S, J. Chahal et al., “Arthroscopic anatomy of the subdeltoid space,” Orthopedic Reviews, vol. 5, no. 3, article e25, 2013.
- R. Paine and M. L. Voight, “The role of the scapula,” International Journal of Sports Physical Therapy, vol. 8, no. 5, pp. 617–629, 2013.
- D. King, “student writing contest winner: glenohumeral joint impingement in swimmers,” Journal of Athletic Training, vol. 30, no. 4, pp. 333–337, 1995.
- S. Rupp, K. Berninger, and T. Hopf, “Shoulder problems in high level swimmers: impingement, anterior instability, muscular imbalance?” International Journal of Sports Medicine, vol. 16, no. 8, pp. 557–562, 1995.
- R. Shultz, S. C. Anderson, G. O. Matheson, B. Marcello, and T. Besier, “Test-retest and interrater reliability of the functional movement screen,” Journal of Athletic Training, vol. 48, no. 3, pp. 331–336, 2013.
- J. R. Landis and G. G. Koch, “The measurement of observer agreement for categorical data,” Biometrics, vol. 33, no. 1, pp. 159–174, 1977.
- M. Randall, J. B. Carlin, P. Chondros, and D. Reddihough, “Reliability of the melbourne assessment of unilateral upper limb function,” Developmental Medicine and Child Neurology, vol. 43, no. 11, pp. 761–767, 2001.
- S. M. K. Chow and S. E. Henderson, “Interrater and test-retest reliability of the Movement Assessment Battery for Chinese preschool children,” The American Journal of Occupational Therapy, vol. 57, no. 5, pp. 574–577, 2003.
- C. A. Smith, N. J. Chimera, N. J. Wright, and M. Warren, “Interrater and intrarater reliability of the functional movement screen,” Journal of Strength and Conditioning Research, vol. 27, no. 4, pp. 982–987, 2013.
- J. E. Elias, “The inter-rater reliability of the functional movement screen within an athletic population using untrained raters,” Journal of Strength and Conditioning Research, 2013.
- A. Karni and D. Sagi, “Where practice makes perfect in texture discrimination: evidence for primary visual cortex plasticity,” Proceedings of the National Academy of Sciences of the United States of America, vol. 88, no. 11, pp. 4966–4970, 1991.
- G. Wallis and E. T. Rolls, “. Invariant face and object recognition in the visual system,” Progress in Neurobiology, vol. 51, no. 2, pp. 167–194, 1997.
- C. Wardak, S. Denève, and S. Ben Hamed, “Focused visual attention distorts distance perception away from the attentional locus,” Neuropsychologia, vol. 49, no. 3, pp. 535–545, 2011.
Copyright © 2014 Kenneth Jay et al. This is an open access article distributed under the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.