Abstract
An evaluation scale is indispensable for the promotion of continuing, effective postkidney transplantation self-management behaviors. We aimed to develop and validate a new self-management scale for kidney transplant recipients to improve their long-term outcomes and prevent the recurrence of CKD complications. Two hundred and thirty-nine Japanese patients who had undergone kidney transplantation were recruited from three hospitals. The scale’s validity and reliability were evaluated using exploratory factor analysis and intraclass correlation coefficients. Scale items were selected by considering the findings of a clinical importance survey of 17 experts. Four subscales were identified by the factor analysis: “Self-monitoring” (six items), “Self-care behavior in daily living” (seven items), “Early detecting and coping with abnormalities after kidney transplantation” (four items), and “Stress management” (three items). Cronbach’s alpha coefficients for the subscales ranged from 0.61 to 0.87. A further four items with high clinical importance were added to the scale. The final scale comprised 24 items within four subscales and four single items. The intraclass correlation ranged from 0.88 to 0.95 in each subscale and the weighted kappa coefficient for the single items ranged from 0.33 to 0.72. The validity and reliability of our new self-management scale were confirmed by a patient-based field survey.
1. Introduction
The incidence of acute rejection after kidney transplantation has decreased due to dramatic progress in immunosuppressive drug therapy. However, despite recent improvements in short-term survival rates, long-term survival remains unchanged [1] and improving this remains a challenge for the future. Kidney transplantation outcomes relate not only to immunosuppressive factors such as rejection but also to nonimmunosuppressive factors such as nonadherence and post-transplantation chronic kidney disease (CKD) [2]. Required recipient self-management measures include taking immunosuppressive drugs, monitoring for adverse events, preventing infections, self-monitoring, physical activity, managing nutrition, and regular consultation with specialists [3–5]. These multiple self-management requirements are considered valuable for maintaining good health and minimizing the recurrence of CKD complications [6, 7].
However, our previous study [8] showed that the degree of adherence to self-management regimens is negatively correlated with time since transplantation, and we must therefore continuously promote effective self-management, taking into account each patient’s circumstances. An evaluation scale is indispensable both to increase patients’ motivation to self-manage and to provide them with accurate feedback on their efforts.
There is an existing evaluation scale that has been confirmed valid and reliable for kidney transplant recipients [9]. The scale has three dimensions: problem-solving, patient-provider relationship, and self-care behaviors. On this scale, the self-care behavior dimension contains items such as taking medication, measuring the amount of urine, and palpating the transplant site but does not include CKD management. To improve long-term outcomes, minimizing the recurrence of CKD complications is vitally important. It is therefore necessary to develop valid instruments to evaluate self-management behaviors that include CKD management, especially self-monitoring of vital signs, physical activity [10], and nutritional management [11]. Furthermore, the new instrument must not only cover the recipient’s therapeutic behaviors, but must also be suitable for use in clinical settings.
We believe that the development of such an instrument could be useful for identifying individual problems as well as for evaluating the effectiveness of educational interventions for self-management. Hence, the aim of this study was to develop and validate a new self-management scale for kidney transplant recipients to improve their long-term outcomes and prevent the recurrence of CKD complications.
2. Subjects and Methods
2.1. Study Design
To develop an instrument to quantify self-management in kidney transplant recipients, we first devised the questionnaire items, then tested the questionnaire, and finally surveyed the items’ clinical importance. Ethical approval of this study was granted by the ethics committees of The University of Tokyo, Toranomon Hospital, Kitasato University and St. Marianna University School of Medicine.
2.2. Questionnaire Item Generation
Questionnaire item generation was conducted based on a review of the literature [3, 9–15] and 14 pamphlets for kidney transplant recipients provided by different institutions. Forty-eight candidate items were generated as an initial draft. The content validity of these 48 items was evaluated by a panel of 12 experts (five transplant surgeons, one kidney physician, four nurses, and two transplant coordinators) interviewed with open-ended questions asking whether the wording was adequate and the number of items either excessive or deficient. Items that were thought unnecessary were deleted, to produce a list of 45. To determine their validity, five kidney transplant recipients evaluated the appropriateness of each item using a four-point Likert-type categorical scale (1: Not applied, 2: Barely applied, 3: Fairly applied, 4: Strongly applied). From these evaluations, a preliminary questionnaire of 45 items was compiled.
2.3. Questionnaire Testing
2.3.1. Participants
Kidney transplant recipients visiting outpatient clinics who met our inclusion criteria were consecutively enrolled in the survey. Eligible subjects were 20 years of age or older, had functioning grafts, could read and understand Japanese, and were judged by a physician to be mentally and physically capable of completing the questionnaire. The survey was conducted from August to November 2010 at Toranomon Hospital, Kitasato University Hospital and St. Marianna University Hospital. These institutions are located in Tokyo and Kanagawa Prefecture, Japan.
2.3.2. Data Collection and Procedure
A cross-sectional, anonymous, self-administered questionnaire survey was conducted. To determine the test-retest reliability of our scale, the questionnaire was reissued two weeks after completion of the first survey. The questionnaire, cover letter, and retest consent form were distributed to the patients at outpatient visits. The cover letter explained that the survey was confidential and anonymous. Consent to participate in the survey was indicated by completion and return of the questionnaire. When the questionnaire was returned with a signed retest consent form, we considered the patient to have consented to participate in the retest. A questionnaire was then mailed to those patients along with a stamped return envelope within two weeks of receiving consent. One reminder was given to patients who did not return the questionnaire within four weeks.
The questionnaire elicited information on the following:(1)self-management of kidney transplant, assessed using the 45 items described previously;(2)demographic factors (age, sex, education, employment status, marital status, and living situation);(3)clinical factors (time since transplantation, donor source, dialysis experience, dialysis period, cause of end-stage kidney disease, complications, serum creatinine level, and estimated glomerular filtration rate (eGFR)).
2.3.3. Statistical Analysis
We conducted an item reduction procedure to improve the scale’s psychometric properties and usability in clinical settings. First, items indicating a ceiling or floor effect (more than 80% of responses were 1 or 4 on the four-point Likert-type scale) were eliminated to enhance sensitivity. Next, principal factor analyses with varimax rotation were performed to conceptualize the preliminary list as a few independent factors. Data for patients who had no missing data for any of the 45 self-management items were included in the factor analyses. Items contributing to more than one factor or items with factor loading of less than 0.4 were eliminated from the scale. The tentative multi-item factor structure that best fitted a compromise of these criteria across all assessments was then tested by means of a multitrait analysis program [16]. This technique is based on item-scale correlations and examines convergent and discriminatory validity. The scaling success rate in these analyses was defined as the proportion of items that correlated more highly or significantly more highly with their own scale than with all other scales. To determine internal consistency, Cronbach’s alpha coefficients were calculated [16]. Test-retest reliability was determined by calculating intraclass correlation coefficients with a 2-week administration interval. Weighted kappa coefficients were calculated to measure the test-retest reliability of the items added because of their clinical importance.
The statistical software program SPSS 14.0 for Windows (IBM, NY, USA) was used for all analyses.
2.4. Clinical Importance Survey
Separate from the survey, to assess their clinical importance, the 45 self-management items were assessed by 17 experts (five transplant surgeons, two transplant recipient coordinators, nine nurses, and one pharmacist). The items were scored on a six-point Likert-type categorical scale (1: most important, 2: important, 3: slightly important, 4: secondary, 5: not important, and 6: incomprehensible). The items for which more than 80% of the experts rated their importance as “1: most important” or “2: important” were included in the final scale, even if they had been previously eliminated in the item reduction procedure.
3. Results
3.1. Survey Participants
Two hundred and thirty-nine of 360 eligible patients (response rate 66.4%) participated in the questionnaire survey. Of the 239 initial responses, 235 patients agreed to take the retest and 218 questionnaires were returned.
The respondents’ characteristics are summarized in Table 1. Their mean age was 50.0 years (standard deviation (SD) = 11.7), most were men (62.3%) and the mean time since transplantation was 98.6 months (SD = 91.7). The organ source was a living donor in 75.3% of cases. The mean serum creatinine level was 1.5 mg/dL (SD = 0.7) and the mean eGFR was 42.5 mL/min/1.73 m2 (SD = 17.1). Most of the respondents (84.1%) had stage 3T CKD.
3.2. Ceiling Effect/Floor Effect
There were seven items for which more than 80% of the respondents selected “4: Strongly applied.” These items were removed from the scale. There were no items for which more than 80% of the respondents selected “1: Not applied.”
3.3. Tests of Internal Structure between Items
Following the principal exploratory factor analyses, we eliminated 18 items that had factor loadings lower than 0.4 for any factor or that were highly related to multiple factors. We adopted 20 items and repeated the analyses. As a result, we confirmed 20 items for four factors (Table 2). These factors were named “self-monitoring,” “self-care behavior in daily living,” “early detecting and coping with abnormalities post-kidney transplantation,” and “stress management.”
A summary of the multitrait scaling, internal consistency, and reliability analysis results is shown in Table 3. The convergent correlation coefficient reached 0.4 for all items except for one in the “stress management” subscale. The item/scale discriminatory validity was also satisfactory. Only in the “early detecting and coping with abnormalities post-kidney transplantation” subscale did certain items fail to correlate more highly or significantly more highly with their hypothesized scale than with competing scales (100% success rate).
3.4. Clinical Importance Survey
Table 4 shows a list of the items that were evaluated as clinically important. Thirteen items were rated “1: most important” or “2: important” by more than 80% of the experts in the clinical importance survey. Among these items, nine had already been included, but four had been excluded by ceiling effect removal (“I take the dosage of immunosuppressive drugs as prescribed”) or by the principal factor analysis “I take immunosuppressive drugs at the prescribed time,” “I check the remaining quantity of the immunosuppressive drugs,” and “I take fluids to prevent dehydration”. We subsequently included these four items in our final self-management scale because of their clinical importance.
3.5. Reliability
Cronbach’s alpha coefficient for each factor ranged from 0.61 to 0.87 (Table 3). The intraclass correlation coefficients for each factor in the test-retest examination ranged from 0.86 to 0.95 (Table 5). The weighted kappa coefficients for the test-retest reliability of the four items included following the clinical importance survey ranged from 0.33 to 0.72.
4. Discussion
This study developed a scale to evaluate post-kidney transplantation self-management. The scale comprises 24 items, 20 of which are divided into four subscales obtained by factor analysis and four of which are single items with high clinical importance. The number and validity of the items in the scale were based on the literature review, consultation with kidney transplantation experts, evaluation by kidney transplant recipients and a clinical importance survey (Figure 1). The scale includes items concerning self-management of post-transplantation CKD and other clinically important items. Having a smaller rather than larger number of items is recognized as an advantage, because less time is needed to administer the scale to patients.
An important item in this scale is “I cut down on sodium intake.” In post-transplantation CKD patients, hypertension is a common problem and an independent risk factor for both allograft failure and mortality [17]. Low sodium intake in combination with antihypertensive treatment appears to efficiently control blood pressure in recipients with hypertension [11]. The Asian population, including the Japanese, and the populations of Canada, Colombia, Hungary, Bassiano (Italy), Poland, and Portugal as reported in INTERSALT [18], are noted for consuming excessive sodium. In addition, almost all CKD patients have sodium-sensitive hypertension [19]. Therefore, cutting down on sodium consumption is one of the most important self-management behaviors following kidney transplantation and is meaningful worldwide. Another distinctive item is “I perform gargling and hand washing.” In Japan, water gargling is generally accepted and strongly recommended as a cost-effective and preventive modality against upper respiratory tract infection [20]. For these reasons, daily gargling is recommended for transplant recipients.
The explanatory factor analyses identified a four-factor structure. This structure is easily interpreted and was therefore considered valid. Multitrait scaling analyses showed that most item-scale correlation coefficients met the standards of convergent and discriminatory validity.
The four items added by expert opinion based on their clinical importance were considered indispensable because they correspond with life changes after transplantation, preventing rejection and maintaining kidney blood flow. Because nonadherence of transplant recipients to prescribed medical regimens has been identified as a major cause of allograft failure [21], three items (accurate dose, accurate intake timing, and self-management of immunosuppressive drugs) were added. Also, because adherence to fluid intake is statically a significant predictor of better graft function in kidney transplant recipients from longitudinal research [22], another item (fluid intake to prevent dehydration) was added. Because we could not confirm the factor validity of the four items included based on their clinical importance, we treated them as single items instead of summing them as subscale scores.
The internal consistencies of the subscales, as assessed by Cronbach’s alpha coefficients, were well above the preferred level of 0.70, except for the “stress management” subscale. This factor comprised only three items. The intraclass correlation coefficients following the test-retest examination were also well above the preferred level of 0.7. Moreover, it was judged that the four single items had satisfactory test-retest reliability. The weighted kappa coefficients showed high levels of absolute and substantial agreement except for one item, “I take immunosuppressive drugs as prescribed,” which was slightly low (weighted kappa = 0.33). This is likely to be a result of the high agreement low kappa paradox, which arises when some scores are underrepresented within the sample so, despite high levels of absolute agreement, the weighted kappa coefficient is low [23]. Actually, the responses to this item were strongly skewed and had a confirmed ceiling effect.
Overall, we consider the reliability of our scale to have been confirmed. We believe that these 24 items enable accurate evaluation of the effectiveness of educational interventions for self-management. The kidney transplant recipient participants in this study were at no specific stage post-transplantation, because our previous study [8] showed that the degree of adherence to self-management regimens is negatively correlated with time after transplantation. Although one factor, “Self-monitoring,” is more meaningful for assessing interventions within one-year post-transplantation, some factors, such as “Self-care behavior in daily living,” “Early detecting and coping with abnormalities postkidney transplantation,” and “Stress management,” are more useful for assessing interventions in recipients who have already returned to their daily activities. Therefore, with the combination of these factors, this scale has the flexibility to be applied to recipients at various stages after transplantation. The scale can also be utilized by healthcare providers for the early identification of recipients with poor self-management, enabling immediate interventions to improve their scores. Recipients can use the scale not only to help them remember their self-management behaviors at consultations with specialists, but also to take action for themselves.
This study has several limitations. First, the participants enrolled in the survey were outpatients who were prescribed immunosuppressive drugs for the maximum period (2 months) possible at our research institutions. These patients were therefore likely to be a compliant and well-controlled group, whereas in practice physicians may need to attend to less compliant patients. Second, the patients were recruited from only three facilities in a single country. This sample is not sufficiently representative to allow generalization of our findings to other populations. Furthermore, because we used self-administered questionnaires, it is possible that the patients overestimated their self-care behaviors. Third, we did not confirm criterion-related validity, and, though we examined the test-retest reliability, the scale may be weak at detecting minor change. Fourth, the study did not include any patient-healthcare staff relationship measure. For example, the doctor-patient relationship may affect self-management behaviors. In future studies, we must investigate this issue. Finally, though the scale was found to be satisfactory for evaluating self-management behaviors in the study participants, it may reflect characteristics of the Japanese population. To determine the generalizability of the results, further studies must be conducted in other countries.
In the future, to obtain more a generalizable scale, this instrument must be evaluated by recipients followed up by a greater number of hospitals, including noncompliant patients who do not have regular follow-up visits. In this way, the scale will be validated for the evaluation of self-management behaviors in a wide spectrum of recipients. It will also be necessary to explore factors related to self-management and specific clinical markers of graft survival, and to develop a comprehensive self-management intervention program for each factor. Because the items under “Stress management” were not evaluated by a mental health specialist, we plan to involve a psychologist or clinical psychotherapist in our team for further studies. Despite these limitations, we believe that our scale can be used to evaluate the effectiveness of self-management interventions for kidney transplant recipients.
This study developed a new self-management scale for kidney transplant recipients. Its validity and reliability were confirmed through a patient-based field survey. In the future, we will use this scale to evaluate the effectiveness of our educational interventions in clinical settings. The scale has the potential to be valuable in assessing interventions that may prevent the recurrence of postkidney transplantation CKD and avoiding exacerbation of CKD complications, leading to improvement of long-term graft survival.
Acknowledgments
The authors would like to thank Dr. Nakamura, Dr. Marui, Dr. Sasaki, Dr. Ishii, Dr. Takeuchi, RN Munemura, RTC Noguchi, RTC Nakanome, and RTC Ikeda for their excellent advice. The kidney transplant recipients who participated in this study are also warmly thanked.