Development and Validation of the Tibetan Primary Care Assessment Tool

Wang, Wenhua; Shi, Leiyu; Yin, Aitian; Lai, Youwen; Maitland, Elizabeth; Nicholas, Stephen

doi:https://doi.org/10.1155/2014/308739

BioMed Research International

On this page

Abstract Introduction Materials and Methods Results Discussion Conclusion Abbreviations Ethical Approval Acknowledgments References Copyright Related Articles

Research Article | Open Access

Volume 2014 | Article ID 308739 | https://doi.org/10.1155/2014/308739

Development and Validation of the Tibetan Primary Care Assessment Tool

Wenhua Wang,¹Leiyu Shi,²Aitian Yin,¹Youwen Lai,³Elizabeth Maitland,⁴and Stephen Nicholas^5,6,7

Academic Editor: Beatriz Gonzalez López-Valcárcel

Received06 Feb 2014

Accepted08 May 2014

Published21 May 2014

Abstract

Objective. To develop a primary care assessment tool in Tibetan area and assess the primary care quality among different healthcare settings. Methods. Primary care assessment tool-Tibetan version (PCAT-T) was developed to measure seven primary care domains. Data from a cross-sectional survey of 1386 patients was used to conduct validity and reliability analysis of PCAT-T. Analysis of variance was used to conduct comparison of primary care quality among different healthcare settings. Results. A 28-item PCAT-T was constructed which included seven multi-item scales and two single-item scales. All of multi-item scales achieved good internal consistency and item-total correlations. Scaling assumptions tests were well satisfied. The full range of possible scores was observed for all scales, except first contact and continuity. Compared with prefecture hospital (77.42) and county hospital (82.01), township health center achieved highest primary care quality total score (86.64). Conclusions. PCAT-T is a valid and reliable tool to measure patients' experience of primary care in the Tibet Autonomous Region. Township health center has the best primary care performance compared with other healthcare settings, and township health center should play a key role in providing primary care in Tibet.

1. Introduction

Mounting evidence has demonstrated that primary care may contribute to better health outcomes and lower health care costs [1–9]. The US Institute of Medicine defined primary care as “the provision of integrated, accessible health care services by clinicians who are accountable for addressing a large majority of personal health care needs, developing a sustained partnership with patients, and practicing in the context of family and community” [10, 11]. In order to assess and monitor primary care performance, there is a need to develop valid and reliable instruments. Highlighting the fallibility of relying on unidimensional proxies for primary care, Safran et al. showed the value of measuring primary care in multidimensional terms that parallel its formal definition [12]. Under this framework, several multiscale instruments were developed, including the primary care assessment tool (PCAT) [12, 13]. PCAT was developed by Johns Hopkins Primary Care Policy Center and included four unique domains (first contact, longitudinality, comprehensiveness, and coordination) as well as three derivative attributes (family centeredness, community orientation, and cultural competence) [13]. Its original English version has been translated into Spanish, Portuguese, Chinese (PCAT-C), and Korean versions, and testing has confirmed that all of these versions have good validity and reliability in terms of congruence between the theoretically derived measures and the empiric results in terms of the underlying structure of the principal primary care domains [14–18].

During the past four years of national health reform, the Chinese central government and the Tibet Autonomous Region (TAR) government have made significant investments in the Tibetan health system, especially in the primary care system. In the context of rapidly changing health care investments, there is a particular need to measure primary care performance in the TAR. PCAT-C provides a starting point for assessing primary care in the TAR. The original English PCAT version, with more than 90 items, was found unsuitable for China’s health system, resulting in a PCAT-C modified to fit the local Chinese context [17]. Unfortunately, we cannot use PCAT-C directly in TAR. First, most residents cannot speak Chinese. Second, there are substantial differences in Chinese versus Tibetan health care providers, especially with respect to geographic accessibility. TAR is located in southwestern China, with an average elevation of 4900 meters and an area about one-eighth of China’s area. The geography of this large and inaccessible province contrasts with its population of roughly three million, which, although overwhelmingly Tibetan, also comprises Monpas, Qiang, and Lhobas indigenous populations, as well as Han and Hui people. Therefore, a PCAT-Tibetan (PCAT-T) is required.

TAR has seven prefectures, 74 counties, 691 towns, and 5254 villages. In each prefecture, there are two types of prefecture level hospitals: prefecture people’s hospitals, which provide western medicine service, and prefecture Tibetan medicine hospitals, which provide traditional Tibetan medicine services. In this paper, we do not differentiate between the two types of prefecture hospitals. Every county has one county hospital, every town has one township health center, and every administrative village has one village clinic. Except for village clinics, all of these organizations are main primary care providers. The low medical capacity of village clinics is reflected in their doctors’ rudimentary three months’ training in a prefecture health school, a fixed monthly salary of only 600 RMB, and the lack of adequate village medical center sites, with many village doctors practicing from their own houses. Seeking jobs outside the health system, many village physicians practiced medicine part-time. For these reasons, we excluded village clinics from the study.

While the primary care system in TAR shares similarities with inland China, there is the need both to translate the PCAT-C into Tibetan and to assess differences between the TAR and inland Chinese health care systems. In addition to setting out the modified PCAT-T version of the PCAT-C instrument, our study conducts a psychometric evaluation of the PCAT-T version, based on a sample of 1440 interviews across 11 health care sites, ranging from prefecture level hospitals to township health centers. The paper then compares primary care achievement across these different health care settings, encompassing sites in two of the seven TAR prefectures.

2. Materials and Methods

2.1. Measurement

The adaption of PCAT-C for the TAR was carried out in the following steps. First, the PCAT-C was translated into Tibetan by two health staff, who were fluent in both Tibetan and Chinese. Second, seven health experts reviewed the translated PCAT-T version. The expert panel included two health service researchers, who are familiar with Tibet’s primary care system, one county health bureau director, one prefecture health bureau director, and three doctors from local TAR hospitals. Experts rated the necessity of each item using a five-point scale (1 = “irrelevant” to 5 = “100% relevant”). They were also asked to add items they believed were necessary based on the Tibetan local environment. Consensus was reached regarding whether to alter, remove, or add an item. Third, pilot testing was conducted through qualitative interviews with 20 random patients for further modification. Finally, through in-depth discussions with two health services researchers, the final PCAT-T was constructed.

The expert review scored highly (more than 4 in a scale of 1 “not relevant” to 5 “highly relevant” to Tibet) the 36-item PCAT-C as a measure of patients’ experience of primary health care quality covering the six domains of first contact, continuity, coordination, comprehensiveness, family-centeredness, and community orientation. Since the PCAT-C items were deemed appropriate for the Tibetan context, all the PCAT-C items were retained in the PCAT-T version.

The PCAT-C version reduced both the items and the principal primary care domains compared with the original English version (see Appendix 1 of Yang et al. [17]). The expert panel recommended one item from the English version to be reinstated in the PCAT-T version to measure “cultural competence” in the patients’ experience of primary care quality. Culturally competent care refers to care that honors and respects the beliefs, interpersonal styles, attitudes, and behaviors of people receiving health care. The item to measure cultural competence sought information on whether a patient would recommend their primary care provider to a friend or relative. In common with the original English PCAT version, but not the PCAT-C, this item in the PCAT-T reflects the diverse cultures and languages in TAR.

Further, the expert review recommended a number of additional questions, including a measure of the regular source of care. In the original English version, there are three questions to measure regular source of care and two questions in PCAT-C. Based on the results of group discussion, the expert panel took the view that two or three questions were difficult for local patients to understand and so recommended just one question to measure regular source of care. Four additional questions were used to measure patients’ satisfaction of their regular source of care. A four-point Likert-type scale was applied to measure certainty as to whether a service was received, ranging from “1” (“definitely not”) to “4” (“definitely”). A neutral response of “not sure/don’t remember” was provided for the lack of knowledge about a characteristic.

2.2. Data Collection

A stratified, purposive sampling approach was used to select study sites. Socioeconomic and geographic factors were employed to ensure our selection of sampling sites represented Tibetan health agencies at each level. Given the socioeconomic level and geographic location of TAR’s seven prefectures, two prefectures were sampled: Shigatse, located in western Tibet with 18 low socioeconomic level counties, and Linzhi in eastern Tibet with seven high socioeconomic level counties.

In Shigatse prefecture, we selected two prefecture level hospitals; at the county level, the Jiangzi and Lazi hospitals; and two health centers per sampling county at the township level. In Linzhi prefecture, we selected two prefecture level hospitals; the Gongbujiangda county hospital; and two township health centers in Gongbujiangda county. The sample sizes were comparable to three key studies that showed 300 interviews were required at each sample site for comparison analysis [19–21]. Considering some collected questionnaires may contain missing data, 10 additional questionnaires were conducted at each township health centre, 20 additional questionnaires at each county hospital, and 30 additional questionnaires at each prefecture level hospital. Overall, four prefecture level hospitals (720), three county hospitals (360 interviews), and six township health centers (360 interviews) yielded 1440 interviews.

The data were collected between September and October 2013 by trained interviewers from the local health bureau through face-to-face interviews with patients 18 years old or older, who had completed their visits to township health centers or hospital outpatients. Patients were given small gifts (worth 10 RMB) of appreciation upon completion of the interview. While 1440 questionnaires were administered, 54 questionnaires were deleted due to missing data, leaving 1386 completed questionnaires.

2.3. Analysis

For consistency with methods used in PCAT studies in other countries, we assigned a median value of 2.5 to “not sure/don’t remember” answers, to be consistent with the methods used in PCAT studies in other countries [17]. We imputed missing values using multiple regression based on the assumption that missing values are random. We converted Likert scales to scores ranging from 25 to 100 by dividing the Likert scale by 4 and multiplying by 100. Then, we conducted the validation of PCAT-T in the following steps. First, we used factor analysis (principal component analysis and varimax rotation) to measure construct validity. To attain the best fitting structure and the correct number of factors, the following criteria were used: eigenvalues > 1.0 and factor loadings > 0.35. Before conducting the factor analysis of the PCAT-T, the Kaiser-Meyer-Olkin and Bartlett’s test was calculated to evaluate whether the sample was large enough to perform a satisfactory factor analysis.

Second, the data’s internal consistency reliability was assessed by Cronbach alphas and item-total correlation. For a scale to be considered sufficiently reliable, an alpha value of 0.7 is recommended. All the retained items should exceed the minimum acceptable item-total correlation of 0.30.

Next, item-convergent validity and item-discriminant validity were tested by item-scale correlations and scaling success rate. The range of item-scale correlations was used to test equal item-scale correlation, while intraclass correlation was used to measure equal item variance. Then, descriptive statistics were performed for the revised primary care scales, including mean, standard deviation, median, and interscale correlation. Finally, primary care achievement of different health care settings was compared using analysis of variance [12, 18, 22].

3. Results

As reported above, a total of 1386 completed interviews were used. Table 1 shows patients’ sociodemographic (age, gender, education, occupation, income, and marital status), health care utilization (PCP visits and inpatient experience), and self-assessed health details. Among 1386 participants, 46.4% were male; the average age was 41.72 years old; 76.9% were married; about one-third received junior high school education or above; and 80.4% were employed. For nearly half the sample, the annual household income was above 30000 RMB. One-fifth of the sample received inpatient care during the past year, and more than 60% reported good, self-rated health. TAR’s recent census does not include information on the sociodemographics of Tibetan population over 18 years old. However, the average family income in the census and our sample (RMB 30000) and the gender breakdown (51% versus 46% male) were roughly comparable. Our sample had a higher education level than the Tibetan population because we sampled only patients over 18 years old (who had the opportunity to complete a junior high school education).

3.1. Construct Validity

The calculated Kaiser-Meyer-Olkin and Bartlett’s test statistic was 0.923 with a value < 0.001, indicating that the sample was large enough to perform a satisfactory factor analysis. All 37 items were included in the principal component analysis and nine components were derived, based on the criteria that the eigenvalues were larger than 1.0. Nine items from the original scale were eliminated, because their secondary loadings were > 0.35: two items from first contact-accessibility, three items from original continuity, one item from original comprehensiveness, two items from original coordination, and one item from family centeredness. Factor loadings of all the retained items ranged from 0.40 to 0.81, above the standard of 0.35.

Finally, a 28-item instrument was constructed which included seven multi-item scales and two single-item scales. The seven multi-item scales were first contact and continuity (6 items); comprehensiveness (medical care) (4 items); comprehensiveness (social care) (3 items); first contact (access) (2 items); coordination (2 items); family centeredness (5 items); and community orientation (4 items). Two single-item scales were same doctor and stableness (see Table 2).

3.2. Internal Consistency

The overall Cronbach alpha coefficient of PCAT-T was 0.92. Cronbach alpha coefficient results were above 0.7 for all multi-item scales, except first contact (access) scale (0.63). As presented in Table 2, the corrected item-total correlations ranged from 0.42 to 0.74, far above the standard of 0.30.

3.3. Scaling Assumption Testing

All item-scale correlations exceeded 0.6, with the majority above 0.7, except two items in the first contact and continuity scale (0.61, 0.67) and one item in the family centeredness scale (0.68). All items had higher correlation with their own scale than with other scales, achieving 100% scaling success. As shown in Table 3, all scales demonstrated a relatively narrow range of item-scale correlations (from 0.01 for “first contact (access)” to 0.15 for “comprehensiveness (medical care)”).

3.4. Descriptive Features of the PCAT-T

We identified seven multi-item scales and two single-item scales, which explained 60.7% of the common variance in the responses to 28 of the original 37 items in the PCAT-T. The alpha coefficient of each scale substantially exceeded its correlation with all other primary care scales. Table 4 presents estimates of central tendency and dispersion of score distribution for seven multi-item scales and two single-item scales. The full range of possible scores was observed for all scales, except first contact and continuity.

3.5. Primary Care Quality in Various Settings

Table 5 shows that township health centers achieved the highest total primary care quality score (86.64), followed by district hospitals (82.01), while prefecture hospitals achieved the lowest scores (77.42). For each scale, township health centers also achieved the highest scores, with the exception of the same doctor and stableness scales.

4. Discussion

The PCAT-T is not a simple translation of the PCAT-C into Tibetan. The expert review identified key modifications to the PCAT-C version to reflect the Tibetan context. A standard psychometric evaluation method was then used to evaluate the PCAT-T version. Overall, the PCAT-T achieved good validity and reliability. The final PCAT-T consisted of seven multi-item scales and two single-item scales. Although the final PCAT-T scales were not completely consistent with the PCAT theoretical domains, the final nine scales covered seven domains suggested by PCAT. Three scales (first contact, continuity, and coordination) in PCAT were split into five scales (first contact and continuity, first contact (access), coordination, the same doctor, and stableness) in the PCAT-T and one scale (comprehensiveness) in PCAT was represented by two scales (comprehensiveness (medical care) and comprehensiveness (social care)) in the PCAT-T. Family centeredness and cultural competency in PCAT were integrated into family centeredness in the PCAT-T. There was no difference in the community orientation scale between PACT-C and PACT-T. All seven multi-item scales achieved relatively good internal consistency. Therefore, PCAT-T is a valid and reliable tool to measure patients’ experience of primary care in the TAR.

In the TAR, township health centers, county hospitals, and prefecture hospitals are the main primary care providers. From the PCAT-T results among different healthcare settings, we found that township health centers achieved the highest score, especially on the scale of first contact (access), which means patients can receive health care without waiting for a long time (less than thirty minutes) and can receive the needed service more easily in township health centers. As residents in the TAR are distributed across a dispersed area, geographic accessibility of health care is a significant problem. To address this problem, the TAR has invested in capacity building for township health centers to provide better primary care, because residents in a town are concentrated and the health staff in township health centers are more familiar with the covered residents. For township health centers, the national government funds staff salaries, infrastructure, and equipment cost, supporting, on average, 4.6 health staff and providing 7-day 24-hour health services. Our results provide evidence that TAR’s investment in township health centers achieved a better outcome than county hospitals (with their larger geographical cover) and prefecture-level hospitals (with the most difficult geographic accessibility and less familiar doctor-patient relationships).

However, township health centers received the lowest score on the scale of the same doctor. To promote capacity building, TAR’s regional health policy allocated new medical graduates to township health centers, but these doctors quickly transferred to upper level health facilities as opportunities arose. This led to township health centers’ low score on the scale of same doctor.

Our study has several limitations. First, patient-reported measurement is subject to recall bias. Some aspects of technical quality cannot be assessed from patients’ perceptions, because of their limited clinical knowledge. Despite these issues, patient reporting is widely accepted as a method of measuring aspects of care important to patients [23]. Second, we assumed that each respondent had experiences of visiting other health care providers in addition to their regular source of care, because this was a precondition to measure achievement of coordination scale. Based on our study experience, most respondents had such experience. Even if respondents did not have such experience, we argue they could make a judgment about items in the coordination scale on the basis of their past experiences in their regular source of care. Third, we only used one question (what is the place you usually go to when getting sick or seeking an advice for health?), instead of the original three questions to measure patients’ regular source of care. Given the challenges stemming from geographic accessibility, patients generally have no choice but to attend the site that is the closest in distance to their place of residence. Fourth, three scales (first contact (access), the same doctor, and stableness) have lower correlations with other scales. Most of these correlation coefficients are below 0.2 and the alpha coefficient of first contact (access) scale is below 0.7, which suggest further improvement of PCAT-T, especially in the scale of first contact (access).

5. Conclusion

PCAT-T is a valid and reliable tool to measure patients’ experience of primary care in the Tibet Autonomous Region. Township health center has the best primary care performance compared with other healthcare settings, and township health center should play a key role in providing primary care in Tibet. Using the PCAT-T, future work should be conducted to analyse two key aspects of TAR health care reform and performance. The first is to explore characteristics from the provider level and organization level that lead to different primary care performance. The other is to examine the extent to which the principal scales of primary care can be linked to health outcomes.

Abbreviations

PCAT:	Primary care assessment tool
PCAT-C:	Primary care assessment tool-Chinese version
PCAT-T:	Primary care assessment tool-Tibetan version.

Ethical Approval

This study received ethics approval from each of the participating health facilities. The authors also received an official letter from regional health bureau to collect patient data from the health facilities.

Conflict of Interests

The authors declare that there is no conflict of interests regarding the publication of this paper.

Authors’ Contribution

Wenhua Wang contributed to conception, design, acquisition of data, analysis and interpretation of data, drafting of the paper, revising of the paper, and final approval. Leiyu Shi contributed to conception, design, drafting of paper, revising of the paper, and the final approval. Aitian Yin contributed to conception, design, revising of the paper, and the final approval. Youwen Lai contributed to conception, design, acquisition of data, revising of the paper, and final approval. Elizabeth Maitland contributed to conception, design, revising of the paper, and final approval. Stephen Nicholas contributed to conception, design, revising of the paper, and final approval.

Acknowledgments

The authors thank Associate Professor Liu Xiaoyun at the China Center for Health Development Studies in Peking University for his contribution to initial planning of this study and expert feedback on their adaptation and validation work. They are grateful to and appreciative of Professor Wang Jian of Center for Health Management and Policy at Shandong University for his helpful comments during their paper preparation period.

References

L. Shi, “The relationship between primary care and life chances,” Journal of Health Care for the Poor and Underserved, vol. 3, no. 2, pp. 321–335, 1992.
View at: Google Scholar
B. Starfield, “Primary care: is it essential?” The Lancet, vol. 344, no. 8930, pp. 1129–1133, 1994.
View at: Google Scholar
L. Shi, “Primary care, specialty care, and life chances,” International Journal of Health Services, vol. 24, no. 3, pp. 431–458, 1994.
View at: Google Scholar
P. Franks and K. Fiscella, “Primary care physicians and specialists as personal physicians: health care expenditures and mortality experience,” The Journal of Family Practice, vol. 47, no. 2, pp. 105–109, 1998.
View at: Google Scholar
F. L. Farmer, C. S. Stokes, R. H. Fiser, and D. P. Papini, “Poverty, primary care and age-specific mortality,” The Journal of Rural Health, vol. 7, no. 2, pp. 153–169, 1991.
View at: Google Scholar
L. Shi, B. Starfield, B. Kennedy, and I. Kawachi, “Income inequality, primary care, and health indicators,” The Journal of Family Practice, vol. 48, no. 4, pp. 275–284, 1999.
View at: Google Scholar
L. Shi and B. Starfield, “Primary care, income inequality, and self-rated health in the United States: a mixed-level analysis,” International Journal of Health Services, vol. 30, no. 3, pp. 541–555, 2000.
View at: Google Scholar
B. Starfield, L. Shi, and J. Macinko, “Contribution of primary care to health systems and health,” Milbank Quarterly, vol. 83, no. 3, pp. 457–502, 2005.
View at: Publisher Site | Google Scholar
A. B. Bindman, K. Grumback, D. Osmond, K. Vranizan, and A. L. Stewart, “Primary care and receipt of preventive services,” Journal of General Internal Medicine, vol. 11, no. 5, pp. 269–276, 1996.
View at: Publisher Site | Google Scholar
Institute of Medicine, Defining Primary Care: An Interim Report, National Academy Press, Washington, DC, USA, 1994.
J. S. Donaldson, K. D. Yordy, and K. N. Lohr, Primary Care: Americas Health in a New Era, National Academy Press, Washington, DC, USA, 1996.
D. G. Safran, M. Kosinski, A. R. Tarlov et al., “The primary care assessment survey: test of data quality and measurement performance,” Medical Care, vol. 36, no. 5, pp. 728–739, 1998.
View at: Google Scholar
L. Shi, B. Starfield, and J. Xu, “Validating the adult primary care assessment tool,” The Journal of Family Practice, vol. 50, no. 2, pp. 161–175, 2001.
View at: Google Scholar
P. Tomas, B. Marcela, and M. Gladys, “Patients' perspective in Chilean primary care: a questionnaire validation study,” International Journal for Quality in Health Care, vol. 21, no. 1, pp. 51–57, 2009.
View at: Publisher Site | Google Scholar
M. I. Pasarin, S. Berra, L. Rajmil, M. Solans, C. Borrell, and B. Starfield, “An instrument to evaluate primary health care from the population perspective,” Atencion Primaria, vol. 39, no. 8, pp. 395–401, 2007.
View at: Publisher Site | Google Scholar
E. Harzheim, B. Starfield, L. Rajmil, C. Álvarez-Dardet, and A. T. Stein, “Internal consistency and reliability of Primary Care Assessment Tool (PCATool-Brasil) for child health services,” Cadernos de Saúde Pública, vol. 22, no. 8, pp. 1649–1659, 2006.
View at: Publisher Site | Google Scholar
H. Yang, L. Shi, L. A. Lebrun, X. Zhou, J. Liu, and H. Wang, “Development of the Chinese primary care assessment tool: data quality and measurement properties,” International Journal of Quality in Health Care, vol. 25, no. 1, pp. 92–105, 2013.
View at: Publisher Site | Google Scholar
J. H. Lee, Y.-J. Choi, N. J. Sung et al., “Development of the Korean primary care assessment tool—measuring user experience: tests of data quality and measurement performance,” International Journal for Quality in Health Care, vol. 21, no. 2, pp. 103–111, 2009.
View at: Publisher Site | Google Scholar
L. Shi, B. Starfield, J. Xu, R. Politzer, and J. Regan, “Primary care quality: community health center and health maintenance organization,” Southern Medical Journal, vol. 96, no. 8, pp. 787–795, 2003.
View at: Publisher Site | Google Scholar
S. Y. Wong, K. Kung, S. M. Griffiths et al., “Comparison of primary care experiences among adults in general outpatient clinics and private general practice clinics in Hong Kong,” BMC Public Health, vol. 10, article 397, 2010.
View at: Publisher Site | Google Scholar
H. X. Harry, S. W. Wang, C. S. Martin et al., “Patients' experiences in different models of community health centers in southern China,” Annals of Family Medicine, vol. 11, no. 6, pp. 517–526, 2013.
View at: Publisher Site | Google Scholar
R. Likert, “A technique for the measurement of attitudes,” Archives of Psychology, vol. 140, article 1, 1932.
View at: Google Scholar
R. Martin, “Measuring and improving patient experience in primary care,” Primary Health Care Research & Development, vol. 13, no. 2, pp. 103–105, 2002.
View at: Publisher Site | Google Scholar

Copyright

Copyright © 2014 Wenhua Wang et al. This is an open access article distributed under the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.

PDF Download Citation

Download other formats

Order printed copies

Views

2102

Downloads

1125

Citations