Table of Contents Author Guidelines Submit a Manuscript
International Journal of Rheumatology
Volume 2015 (2015), Article ID 834070, 4 pages
Research Article

Improving the Measurement of Disease Activity for Patients with Rheumatoid Arthritis: Validation of an Electronic Version of the Routine Assessment of Patient Index Data 3

1Dartmouth-Hitchcock Medical Center, Lebanon, NH, USA
2Dartmouth-Hitchcock Medical Center, Geisel School of Medicine at Dartmouth, Hanover, NH, USA
3Dartmouth-Hitchcock Medical Center, The Dartmouth Institute for Health Policy & Clinical Practice, Geisel School of Medicine, Hanover, NH, USA

Received 10 September 2015; Accepted 21 October 2015

Academic Editor: Malcolm Smith

Copyright © 2015 Ruthie M. Chua et al. This is an open access article distributed under the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.


Introduction. An electronic Routine Assessment of Patient Index Data 3 (RAPID 3) was incorporated into our electronic health records (EHRs) which did not replicate the visual presentation of the paper version. This study validated the electronic RAPID 3 compared to the paper version. Methods. Rheumatoid arthritis (RA) patients () completed both the electronic RAPID 3 online in the week prior to and a paper version on the day of their clinic visit. Results. Paired t-test showed no significant difference ( value = 0.46) between versions. Conclusion. The electronic version of RAPID 3 is valid and can be easily integrated in care of RA patients.

1. Background

Patient-reported outcome measures (PROMs) are common tools used to inform about and improve patient care. They provide valuable longitudinal information about a patient’s experience with his or her health and treatment. Because the patient is the only source of information, it is not subjected to interobserver variability and may be more sensitive for monitoring disease activity over time. PROMs also engage patients to become more active participants in monitoring their health [13].

As electronic health records (EHRs) proliferate, there is increased interest in electronic versions of PROMs that can be easily deployed through the web or touch screen tablets and seamlessly integrated into the patient chart. Automated PROMs increase compliance by decreasing administrative burden and facilitate efficient collection and interpretation of data. However, many of the more commonly used tools that have been validated as paper tools have not been tested in their electronic versions to see if they produce different results. Consequently, the migration of PROMs from the paper to the electronic version requires evidence of measure equivalence especially if there is modification of the visual presentation such as splitting a questionnaire into multiple screens or using scrollbars [4, 5].

The routine assessment of patient index (RAPID 3) is a validated PROM for rheumatoid arthritis (RA) that evaluates a patient’s physical function, pain, and global health estimate. Previous studies that compared RAPID 3 to Clinical Disease Activity Index (CDAI) and Disease Activity 28 (DAS 28) showed significant correlation in disease activity severity. RAPID 3 is scored 0–30 with scores >12 indicating high disease activity, 6.1–12 moderate disease activity, 3.1–6 low disease activity, and ≤3 disease remission [68].

Dartmouth-Hitchcock Medical Center is currently using Epic version 2014 (Epic Systems Corporation, Verona, WI) as its EHR platform. An electronic version of the RAPID 3 was developed that could be integrated into routine patient care and research. Due to technical constraints when the questions were incorporated into the EHR, there were several significant formatting modifications which changed the layout of the questionnaire. The aim of this study was to validate the performance characteristics of the EHR version of RAPID 3 compared to the paper version in adult RA patients at an academic medical center.

2. Method

Our study was granted exemption by Committee for the Protection of Human Subjects of Dartmouth College. Between March and June 2014, we prospectively identified patients ≥ 18 years old with seropositive and seronegative rheumatoid arthritis who were presenting to the rheumatology clinic at Dartmouth-Hitchcock Medical Center. We verified the diagnosis via chart review and included only those who are active users of the patient portal of the EHRs and had at least 2 visits related to rheumatoid arthritis. The patient portal allows patients web-based access to their medical records, provides a confidential channel for physicians and patients to communicate, and allows providers to send online questionnaires for patients to complete prior to their appointment. Electronic versions of the RAPID 3 were sent to patients via the patient portal one week prior to their clinic appointment. They were alerted by an email reminder and a telephone call that a questionnaire was available to them. They completed the questionnaire online anytime in the week prior to the appointment. On the day of the clinic visit, we identified patients who had completed the electronic version and asked them to complete the paper version of the RAPID 3 in the reception area. We included the first fifty sequential patients to complete both the paper and electronic versions in the study (Figure 1). We excluded patients who did not complete the questionnaire for either the paper or electronic version. The EHR automatically calculated and interpreted the disease activity score for the electronic RAPID 3. Results of the paper form were manually calculated. A paired -test was used to compare samples, with values of ≤0.05 considered significant.

Figure 1: Flowchart of patient recruitment.

3. Results

Of the fifty patients included in the study, 66% were female. The average age was 58.8 years old (range 30–81 years) and average time lapse between answering the electronic and paper versions was 4.2 days (range 0–7 days). A paired -test did not show statistically significant difference between the mean total RAPID 3 of the paper and electronic versions (). This was also true for each component score for RAPID 3. The means for the total RAPID 3 for both paper and EHR versions were 9.57 (±6.45) and 9.75 (±6.46), respectively, indicating moderate disease activity (Table 1).

Table 1: Comparison of paper and electronic health record versions of RAPID 3.

4. Discussion

We found no significant difference in responses between the electronic and paper versions of the RAPID 3 questionnaire. The data entered by patients in electronic RAPID 3 is automatically stored, calculated, and interpreted in the EHR, thus eliminating the need for manual calculation, manual entry, and storage of paper scores. Additionally, patients may answer questions at home, and this can save time in a busy clinic and improve patient flow. Prior electronic scores can be easily reviewed and this facilitates comparison of disease activity over time. The EHR has the ability to prompt patients when questionnaires are incomplete and improved compliance has been previously demonstrated with electronic versions [9, 10]. It is interesting to note that the mean RAPID 3 showed moderate disease activity. However, some of the patients may have chronic pain and functional impairment which could drive their RAPID 3 score. Hence, the RAPID 3 is more useful for monitoring disease activity over time.

Our study has several limitations. While a randomized crossover study design for measuring equivalence was considered, due to the large catchment area and long travel distances for appointments, we felt that this was not feasible in our setting. In a randomized crossover study, participants are randomly assigned to either the paper or electronic version of the questionnaire. After a predetermined period of time, they are assigned to the other version and the results are then compared [4]. Although the sample size was relatively small, the study was adequately powered to show a difference in a paired -test. There was no qualitative component to the study to help understand patients’ personal experiences in terms of acceptance, convenience, and preference when answering the questionnaires. A review of our panel shows that only 52.5% of our 1130 RA patients have an active EHR patient portal. Therefore, our study population may not be representative of all RA patients in the practice as the sample only included participants active on the patient portal and, therefore, represent a computer-literate subpopulation of patients. This may introduce selection bias, as we do not know how patients who use the portal differ from other RA patients or why the 15 patients did not complete the electronic version. There was a mean of 4.2 days in between the electronic and paper versions. We were unable to complete a sensitivity analysis to see if longer times between completing the electronic and paper versions affected the scores.

Most rheumatologists still rely on nonquantitative assessments of RA activity despite studies showing that quantitative tools are associated with tighter control and thus better outcome [1113]. In our academic center, documentation is mostly narrative, and if quantitative tools are used, they are not standardized in formatting and not easily searchable. By validating the electronic version, practitioners now have a convenient tool to monitor disease activity and ultimately improve quality of care. The electronic version of the RAPID 3 continues to be utilized in our clinic and we have expanded its use to include a tablet which can be used at point of care for those patients without EHR portal access. The information gathered from electronic RAPID 3 can be retrieved easily for individual patient care, clinical research, quality improvement, and innovations in care delivery.

5. Conclusion

Scores on an electronic version of RAPID 3 are not statistically significant from the validated paper version and can streamline patient reported data collection to improve the quality of patient care in a rheumatology clinic.

Conflict of Interests

The authors declare that there is no conflict of interests regarding the publication of this paper.


  1. L. E. Hart, P. Tugwell, W. W. Buchanan, G. R. Norman, E. M. Grace, and D. Southwell, “Grading of tenderness as a source of interrater error in the Ritchie articular index,” Journal of Rheumatology, vol. 12, no. 4, pp. 716–717, 1985. View at Google Scholar · View at Scopus
  2. I. Castrejón and T. Pincus, “Assessing remission in rheumatoid arthritis on the basis of patient reported outcomes—advantages of using RAPID3/MDHAQ in routine care,” Bulletin of the Hospital for Joint Diseases, vol. 72, no. 2, pp. 136–141, 2014. View at Google Scholar
  3. K. L. Haywood, “Patient-reported outcome I: measuring what matters in musculoskeletal care,” Musculoskeletal Care, vol. 4, no. 4, pp. 187–203, 2006. View at Publisher · View at Google Scholar · View at Scopus
  4. S. J. Coons, C. J. Gwaltney, R. D. Hays et al., “Recommendations on evidence needed to support measurement equivalence between electronic and paper-based patient-reported outcome (PRO) measures: ISPOR ePRO Good Research Practices Task Force report,” Value in Health, vol. 12, no. 4, pp. 419–429, 2009. View at Publisher · View at Google Scholar · View at Scopus
  5. C. J. Gwaltney, A. L. Shields, and S. Shiffman, “Equivalence of electronic and paper-and-pencil administration of patient-reported outcome measures: a meta-analytic review,” Value in Health, vol. 11, no. 2, pp. 322–333, 2008. View at Publisher · View at Google Scholar · View at Scopus
  6. T. Pincus, C. J. Swearingen, M. Bergman, and Y. Yazici, “RAPID3 (Routine Assessment of Patient Index Data 3), a rheumatoid arthritis index without formal joint counts for routine care: proposed severity categories compared to disease activity score and clinical disease activity index categories,” Journal of Rheumatology, vol. 35, no. 11, pp. 2136–2147, 2008. View at Publisher · View at Google Scholar · View at Scopus
  7. T. Pincus, M. J. Bergman, Y. Yazici, P. Hines, K. Raghupathi, and R. Maclean, “An index of only patient-reported outcome measures, routine assessment of patient index data 3 (RAPID3), in two abatacept clinical trials: similar results to disease activity score (DAS28) and other RAPID indices that include physician-reported measures,” Rheumatology, vol. 47, no. 3, pp. 345–349, 2008. View at Publisher · View at Google Scholar · View at Scopus
  8. T. Pincus, V. Furer, E. Keystone, Y. Yazici, M. J. Bergman, and K. Luijtens, “RAPID3 (Routine Assessment of Patient Index Data 3) severity categories and response criteria: similar results to DAS28 (Disease Activity Score) and CDAI (Clinical Disease Activity Index) in the RAPID 1 (Rheumatoid Arthritis Prevention of Structural Damage) clinical trial of certolizumab pegol,” Arthritis Care and Research, vol. 63, no. 8, pp. 1142–1149, 2011. View at Publisher · View at Google Scholar · View at Scopus
  9. F. Salaffi, S. Gasparini, and W. Grassi, “The use of computer touch-screen technology for the collection of patient-reported outcome data in rheumatoid arthritis: comparison with standardized paper questionnaires,” Clinical and Experimental Rheumatology, vol. 27, no. 3, pp. 459–468, 2009. View at Google Scholar · View at Scopus
  10. A. S. Wilson, G. D. Kitas, D. M. Carruthers et al., “Computerized information-gathering in specialist rheumatology clinics: an initial evaluation of an electronic version of the Short Form 36,” Rheumatology, vol. 41, no. 3, pp. 268–273, 2002. View at Publisher · View at Google Scholar · View at Scopus
  11. Y. P. M. Goekoop-Ruiterman, J. K. de Vries-Bouwstra, C. F. Allaart et al., “Clinical and radiographic outcomes of four different treatment strategies in patients with early rheumatoid arthritis (the BeSt study): a randomized, controlled trial,” Arthritis and Rheumatism, vol. 58, no. 2, supplement, pp. S126–S135, 2008. View at Publisher · View at Google Scholar · View at Scopus
  12. C. Grigor, H. Capell, A. Stirling et al., “Effect of a treatment strategy of tight control for rheumatoid arthritis (the TICORA study): a single-blind randomised controlled trial,” The Lancet, vol. 364, no. 9430, pp. 263–269, 2004. View at Publisher · View at Google Scholar · View at Scopus
  13. T. Pincus and O. G. Segurado, “Most visits of most patients with rheumatoid arthritis to most rheumatologists do not include a formal quantitative joint count,” Annals of the Rheumatic Diseases, vol. 65, no. 6, pp. 820–822, 2006. View at Publisher · View at Google Scholar · View at Scopus