Reliability Exercise for the Polymyalgia Rheumatica Classification Criteria Study: The Oranjewoud Ultrasound Substudy
Objective. A study supported by the EULAR and the ACR being conducted to establish classification criteria for polymyalgia rheumatica (PMR) will include ultrasound examination of the shoulders and hips. Ultrasound (US) depicts glenohumeral joint effusion, biceps tenosynovitis, subdeltoid bursitis, hip joint synovitis, and trochanteric bursitis in PMR. These findings may aid in distinguishing PMR from other diseases. The purpose of this study was to assess standards and US interreader agreement of participants in the PMR classification criteria study. Methods. Sixteen physicians in four groups examined shoulders and hips of 4 patients and 4 healthy adults with ultrasound. Overall agreement and interobserver agreement were calculated. Results. The overall agreement (OA) between groups was 87%. The OA for healthy shoulders was 88.8%, for healthy hips 100%, for shoulders with pathology 85.2%, and 74.3% for hips with pathology, respectively. Conclusion. There was a high degree of agreement found for the examination of healthy shoulders and pathologic hips. Agreement was moderate for pathologic shoulders and perfect for healthy hips. US of shoulder and hips performed by different examiners is a reliable and feasible tool for assessment of PMR related disease pathology and can be incorporated into a classification criteria study.
Polymyalgia rheumatica (PMR) is the most common inflammatory rheumatic disease in the elderly. There is considerable uncertainty related to diagnosis and outcomes in patients presenting with the polymyalgic syndrome. Therefore the European League against Rheumatism (EULAR) and the American College of Rheumatology (ACR) are supporting a study that is being conducted to establish classification criteria for PMR . Ultrasound (US) of shoulders and hips was selected among several other candidate criteria that had been recruited by a Delphi survey from a work group of 27 international physicians with interest in PMR (rheumatologists, nonrheumatologists, statisticians, and methodologists) who met to pursue a consensus-based process for the development of classification criteria in PMR in Cambridge, UK, in 2005. Participants agreed that currently, there was no role for routine use of magnetic resonance imaging or positron emission tomography in the evaluation of suspected PMR. However musculoskeletal US was thought to have utility as a diagnostic criterion for PMR due to widespread availability, feasibility, and results of preliminary studies in this condition [2–7].
US depicts characteristic pathologic findings of shoulder joints and the hip region that may aid in distinguishing PMR from other diseases that may mimic it. Typical findings on US include glenohumoral joint synovitis, subdeltoid bursitis, and biceps tendon tenosynovitis of the shoulders [2–6]. These findings occur in most PMR patients, but they are often mild in contrast to elderly onset rheumatoid arthritis (EORA). In the hips, US often reveals hip joint synovitis and trochanteric bursitis in patients with PMR [2, 5, 7].
Before considering US as a potential tool for the classification, there was agreement regarding the need to standardise the examination and assess the inter-observer agreement for distinguishing lesions typical of PMR from other lesions like extensive effusion or bursitis, rotator cuff tears, rotator cuff calcifications, and osteoarthritis.
To pursue this aim, members of centres participating in the PMR classification criteria study met to standardise the US examination in PMR patients for each centre and work towards achieving a high degree of inter-reader agreement in the US examination for the PMR classification criteria study. The study included both, patients with PMR and other related diseases with shoulder pain such as rheumatoid arthritis (RA). While we present a validation study, our study does not address the question of defining US characteristics of PMR.
2.1. Study Protocol
Fifteen rheumatologists and one radiologist of varying US experience from the participating centres (listed in the Acknowledgement) met in Oranjewoud (The Netherlands) to evaluate the possibility of including US in the upcoming PMR classification criteria study. The meeting took place just following the 12th EULAR basic ultrasound course and before the annual EULAR congress in June 2006. The 16 physicians were assigned to 4 groups. Each group consisted of four assessors: one supervisor, one performing the US scans, one measuring distances between structures, and one documenting the results. The ultrasound examiners were—with the exception of the supervisor—alumni of the basic EULAR sonography course. None were career specialists in the field of musculoskeletal ultrasonography. Prior to the patient examination, each supervisor gave a review of the US examination including specifics of the standard scans to be performed for operator training. Standard definitions and pathologic findings were demonstrated. The group as a whole decided if they regarded the US findings as normal or abnormal. To ensure standardised documentation, each participant was given a report sheet that listed possible pathologic findings recorded as “yes” or “no,” indicating the presence or absence of each particular finding. All US assessments were performed independently by each group without contact with other groups and without knowledge of the patient's disease or joint status. Each group was given a maximum of 15 minutes (shoulder joint) and 10 minutes (hip joint), respectively, for US examination per joint region, rotating on a preset plan from one US station to the next. Each healthy control and each patient was assigned to every US station in rotation.
The focus of the US examinations was the shoulder and hip joints. Two joints (one shoulder and one hip) were examined in 4 healthy controls and in 4 patients with RA or PMR. Four rounds (normal shoulder, normal hip, pathologic shoulder, and pathologic hip) for 4 groups were necessary to ensure that an examination was performed by each group, control/patient and joint region. Patients and healthy individuals were recruited from the Medisch Centrum Leeuwarden by GAWB. All patients gave their consent to participation in the study. Subject characteristics were as in Tables 1 and 2.
Examinations were performed with several US machines which are in use in practice and simulate the conditions of the PMR classification study. A first phase showed a good reliability between different US machines . Stations 1 and 2 were equipped with a Mylab 70 (Esaote, Genoa, Italy). Shoulder and hip joints were examined with a linear array probe (LA 18–6 MHz), respectively. Station 3 was equipped with a Logic E (General Electrics, Milwaukee, USA), and Station 4 with a Voluson I (General Electrics, Milwaukee, USA). Shoulder and hip joints were examined with a linear array probe (12L-RS, 13–5 MHz), respectively. Two equipment specialists each from Esaote and General Electrics were present to help in case of problems with regard to machine adjustments during the examinations.
Scanner settings were uniform for all measurements: frequency setting, B-mode gain, and 100%; one focus point position in the region of measurement. An introduction to the US device was given to the observers prior to US examinations.
Standard scans according to the EULAR guidelines for musculoskeletal US were applied . For the shoulder joint, examination of the biceps tendon, rotator cuff (subscapularis, supraspinatus, and infraspinatus tendons), glenohumeral joint, acromioclavicular joint, humeral bone surface subacromial-subdeltoid bursa, and axillary artery was required. Effusion, synovitis, tenosynovitis, bone erosions, osteophytes, and bursitidies were evaluated. Evaluation of the hip joints included assessment for effusion, synovitis, osteoarthritis, and trochanteric bursitis.
Synovitis, effusion, tenosynovitis, and erosions were defined according to the OMERACT definitions for musculoskeletal ultrasound .
2.3. Statistical Analysis
Overall agreement and inter-observer agreement values were calculated. All analyses were calculated with Statistical Product and Service Solutions (SPSS) 15.0 (Chicago, Ill, USA).
The overall agreement between the 4 groups of sonographers with regard to all normal and pathologic findings for shoulders and hips was 87%, reflecting substantial agreement.
For the healthy controls, overall agreement for the shoulder was 88.8% (substantial agreement). For the hip joint, overall agreement was 100% (perfect agreement). For the patient group, overall agreement for the shoulder joint was 85.2% (moderate agreement). For the pathologic hip joint, overall agreement was 74.3% (substantial agreement).
The overall agreements with regard to the different pathologies for the shoulder and hip joints are displayed in Tables 3 and 4. Figures 1(a) and 1(b) show typical joint pathologies in PMR patients, for example, subdeltoid bursitis and hip joint synovitis.
US plays an important role in the detection of many inflammatory processes. It is important to rigorously evaluate the utility of US in PMR since there are no recognised laboratory or imaging diagnostic tests for the condition. Recent studies have shown the importance of US in depicting characteristic pathologies that aid in distinguishing PMR from other mimicking diseases. The most frequent US soft tissue alterations in patients with PMR have been described for the shoulders (subdeltoid bursitis, tenosynovitis of the biceps tendon, and glenohumeral synovitis) [2–6] and for the hips (synovitis and trochanteric bursitis) [2, 5, 7].
However, to date US has not yet been included in any diagnostic or classification criteria for PMR. Therefore, a study group has met to evaluate the possibility of including US in the upcoming PMR classification criteria study. US examinations were performed according to the EULAR criteria . Our standardized protocol and study results demonstrated that US examinations for both, shoulder and hip joints can be a useful tool for assessment of pathology in a future study of PMR; since it can be performed by competent examiners across centres with reliable result.
Some former studies have investigated the prevalence for the detection of inflammatory changes of the shoulder [2–6] as well as the hip [2, 5, 7] joints in PMR by US. Cantini et al. performed a case control study of shoulder US in PMR patients including 57 consecutive patients with PMR and 114 controls with bilateral shoulder pain and stiffness due to RA, psoriatic arthritis, spondyloarthritis, osteoarthritis, fibromyalgia, or connective tissue disease. Twenty-four PMR patients were also examined with MRI. Bursitis was detected in all patients, glenohumeral synovitis in 88%, and biceps tendon tenosynovitis in 88% . Other authors have reported that inflammatory changes are discrete in PMR but much more severe in EORA . Frediani et al.  performed a case control study that describes the US findings in 50 consecutive patients with PMR, spondyloarthritis, and RA, respectively. They detected biceps tenosynovitis in 44% of spondyloarthritis patients and in 38% of patients with RA, glenohumoral joint synovitis in 16% of patients with spondyloarthritis, and in 54% of patients with RA, and subdeltoid bursitis in 34% of patients with spondyloarthritis and in 44% of patients with RA.
Since a high prevalence of synovitis is seen in shoulder and hip joints of PMR patients, the current study focused on whether overall agreement could be achieved between readers for typical joint pathologies seen in PMR patients (including subdeltoid bursitis, biceps tenosynovitis, and glenohumoral synovitis for the shoulder joint, resp.). We found substantial overall agreement for detecting these pathologies in normal as well as pathologic shoulder and hip joint pathologies. These findings underline the fact that US is not as observer dependent as formerly thought . Furthermore, our results demonstrated the feasibility of performing a multicentre study using US for evaluation of patients with shoulder and hip lesions typical of PMR.
US is an important imaging tool for the visualisation of inflammation in shoulders and hips joints of PMR. In this exercise substantial agreement was found for the examination of healthy shoulders and pathologic hips. Agreement was moderate for pathologic shoulders and perfect for healthy hips. The results of this study confirm that US can be evaluated as a diagnostic technique in studies of PMR including the PMR classification criteria study.
The Polymyalgia Rheumatica EULAR/ACR Classification Criteria Group would like to acknowledge Esaote, Biomedica SA, Genoa, Italy, and General Electrics, Fairfield, Connecticut, USA for providing ultrasound equipments and technical support for this project. Funding for the organization and facilities were provided by a Grant from the American College of Rheumatology to Doctors Matteson and Dasgupta. Other participants in this exercise included: Fabrizio Cantini, Italy; Annamaria Lagnocco, Italy; Pierre Duhaut, France; Pierluigui Macchioni, Italy; Maria Antonietta D‘Agostino, Italy; Massimiliano Parodi, Itlay; Andy Abril, USA; Hans Weiss, Austria; Richard Wakefield, UK; Michael Schirmer, Austria; Edith Wipfler, Austria; Christropher Schaufelberger, Peter Balint, Hungary; Marina Backhaus, Germany; All authors contributed equally in this paper.
B. Dasgupta, C. Salvarani, M. Schirmer et al., “Developing classification criteria for polymyalgia rheumatica: comparison of views from an expert panel and wider survey,” The Journal of Rheumatology, vol. 35, no. 2, pp. 270–277, 2008.View at: Google Scholar
J. M. Koski, “Ultrasonographic evidence of synovitis in axial joints in patients with polymyalgia rheumatica,” British Journal of Rheumatology, vol. 31, no. 3, pp. 201–203, 1992.View at: Google Scholar
U. Lange, M. Piegsa, J. Teichmann, and G. Neeck, “Ultrasonography of the glenohumeral joints—a helpful instrument in differentiation in elderly onset rheumatoid arthritis and polymyalgia rheumatica,” Rheumatology International, vol. 19, no. 5, pp. 185–189, 2000.View at: Publisher Site | Google Scholar
F. Cantini, C. Salvarani, I. Olivieri et al., “Shoulder ultrasonography in the diagnosis of polymyalgia rheumatica: a case-control study,” The Journal of Rheumatology, vol. 28, no. 5, pp. 1049–1055, 2001.View at: Google Scholar
B. Frediani, P. Falsetti, L. Storri et al., “Evidence for synovitis in active polymyalgia rheumatica: sonographic study in a large series of patients,” The Journal of Rheumatology, vol. 29, no. 1, pp. 123–130, 2002.View at: Google Scholar
G. Coari, F. Paoletti, and A. Iagnocco, “Shoulder involvement in rheumatic diseases. Sonographic findings,” The Journal of Rheumatology, vol. 26, no. 3, pp. 668–673, 1999.View at: Google Scholar
F. Cantini, L. Niccoli, C. Nannini et al., “Inflammatory changes of hip synovial structures in polymyalgia rheumatica,” Clinical and Experimental Rheumatology, vol. 23, no. 4, pp. 462–468, 2005.View at: Google Scholar
M. A. D'Agostino, R. Wakefield, M. Backhaus et al., “Combined evaluation of influence of sonographer and machine type on the reliability of power Doppler ultrasonography for detecting, scoring and scanning synovitis in rheumatoid arthritis patients. Results of an intermachine reliability exericise,” Annals of the Rheumatic Diseases, vol. 67, supplement 2, p. 421, 2008.View at: Google Scholar
R. J. Wakefield, M.-A. D'Agostino, A. M. Iagnocco et al., “The OMERACT Ultrasound Group: status of current activities and research directions,” The Journal of Rheumatology, vol. 34, no. 4, pp. 848–851, 2007.View at: Google Scholar
A. K. Scheel, W. A. Schmidt, K.-G. A. Hermann et al., “Interobserver reliability of rheumatologists performing musculoskeletal ultrasonography: results from a EULAR “Train the trainers” course,” Annals of the Rheumatic Diseases, vol. 64, no. 7, pp. 1043–1049, 2005.View at: Publisher Site | Google Scholar