Abstract
Introduction. Healthcare reforms launched in the early 2000s in Québec, Canada, involved the implementation of new forms of primary healthcare (PHC) organizations: Family Medicine Groups (FMGs) and Network Clinics (NCs). The objective of this paper is to assess how the organizational changes associated with these reforms have impact on patients’ experience of care, use of services, and unmet needs. Methods. We conducted population and organization surveys in 2005 and 2010 in two regions of the province of Québec. The design was a before-and-after natural experiment. Changes over time between new models and other practices were assessed using difference-in-differences statistical procedures. Results. Accessibility decreased between 2003 and 2010, but less so in the treatment than in the comparison group. Continuity of care generally improved, but the increase was less for patients in the treatment group. Responsiveness also increased during the period and more so in the treatment group. There was no other significant difference between the two groups. Conclusion. PHC reform in Québec has brought about major organizational changes that have translated into slight improvements in accessibility of care and responsiveness. However, the reform does not seem to have had an impact on continuity, comprehensiveness, perceived care outcomes, use of services, and unmet needs.
1. Introduction
Most industrialized countries are facing efficiency problems in their healthcare systems associated with increased longevity and levels of morbidity among their populations. Issues with healthcare systems raise concerns since some countries, such as Canada and the United States, rank lower than other countries on most performance indicators, particularly those related to patients’ experience of care, even though they devote comparable or even higher levels of financial resources [1].
To rectify the situation, different measures have been proposed such as implementing practices that are based on Patient-Centered Medical Home and Chronic Care Model concepts [2, 3]. These proposals recognize the widely accepted notion that primary care is the cornerstone in higher-performing healthcare systems [4, 5]. Search for solutions culminated in 2007 with the Institute for Health Improvement launching the Triple Aim Initiative [6]. The initiative promotes and supports projects aimed at achieving better population health and patients’ experience of care and enhancing healthcare costs containment. The Canadian Foundation for Healthcare Improvement recently adopted the initiative, and some provinces, including Québec, are currently participating in a Triple Aim Collaborative [7].
Several components of the Triple Aim Initiative are found in the reforms initiated in Québec in the early 2000s. The aims of the reforms were to improve accessibility and continuity of primary healthcare and patients’ experience of care while reducing unnecessary use of services, particularly among specialists services. The reforms also aimed to strengthen coordination of care in order to enable patients to navigate more easily the complexity of the healthcare system. Two new types of primary healthcare (PHC) organizations were created: Family Medicine Groups (FMGs) and Network Clinics (NCs) [8, 9]. FMGs incorporate elements of the Chronic Care and the Medical Home Models [3, 10, 11]. A typical FMG consists of 6 to 10 physicians working with nurses and providing services for 8,000 to 15,000 registered patients. FMGs, contracting with the Provincial Ministry of Health and Social Services, agree to increase services provision (e.g., extended opening hours and 24/7 phone access) in exchange for complementary public funding to support computerization and provide the means to hire additional staff, such as nurses. At onset of the reform, the target was to establish 300 FMGs in the province of Québec; as of 2010, there were 217 accredited FMGs. An FMG can comprise more than one clinical setting since it is based on physician participation and engagement, regardless of practice setting. Hence, an FMG can encompass more than one clinic with participating doctors. A complementary organizational model of PHC is the Network Clinic (NC), which was mainly implemented in Montréal under the initiative of the Regional Health Agency. Those clinics more specifically aim to foster accessibility through walk-in visits and provide access to diagnostic support, such as X-rays and lab tests, and to specialists. The distinction between FMG and NC is often blurred, as many clinics have both gained status and became FMG-NC. These clinics thus benefit from two sources of funding, provincial and regional.
In 2004, a local coordination structure was also created in Québec: 95 Health and Social Services Centres were entrusted by the Ministry of Health and Social Services to create local service networks [12]. The centres serve geographically defined populations. They were created by law, merging, on a geographical basis, long-term care hospitals, Local Community Services Centres, and, in most cases, acute care hospitals. Local Community Services Centres are public organizations created in the early 1970s under the governance of provincial authorities. They provide health and social services and physicians are generally paid on salary. Health and Social Services Centres were also mandated to increase collaboration among local actors and consequently to support the development of new emerging forms of local PHC organizations.
Structural changes and implementation processes associated with healthcare reforms have been well documented [13–16]. Likewise, outcomes associated with the introduction of innovative practices or new organizational models have been widely assessed [13, 14, 17–19]. However, the impact of PHC reforms on patients’ experience of care and use of services at the population level has received much less attention [5, 18–21]. The Québec reform provided us with an opportunity to address that question. In close collaboration with the Regional Health and Social Services Agencies of Montréal and Montérégie, we conducted two studies in 2005 and 2010 to assess the evolution of PHC organizations and their impacts on patients’ experience of care, unmet needs, and use of services in both regions [22, 23].
In a previous article, we discussed effects of the reforms on PHC organizations’ changes [16]. Specifically, we found that significant organizational changes had resulted from the reform, mainly due to implementation of new types of PHC organizations (FMGs, NCs, and FMG-NCs). We also found that the main mediating factor accounting for these changes was the mimetic influence of exemplary PHC organizations that acted as promoters for implementation of the new models supported by provincial and regional policies.
The question raised in the conclusion of that article was whether these organizational changes had translated into improvement of patients’ experience of care, unmet needs, and use of services. This is the issue we address in the current paper. More specifically, this paper aims to compare FMGs, NCs, and FMG-NCs implemented since 2003 with the other PHC organizations, in regard to the evolution of patients’ experience of care, unmet needs, and use of services between 2003 and 2010.
2. Methods
2.1. Design
The study design corresponded to a before-and-after natural experiment in which FMGs, NCs, and FMG-NCs constituted the treatment group, and the other clinics formed the comparison group (Figure 1). The intervention was the reforms initiated in 2003 as described in the Introduction. The study consisted of population and organization surveys carried out in 2005 and 2010 in the two most populous regions of the province of Québec: Montréal and Montérégie [23]. In both years, surveys were conducted on two independent population samples. They included 9,206 adults aged 18 or more in 2005 with a response rate of 64% and 9,180 adults in 2010 with a response rate of 56%. Concurrent surveys of all PHC organizations were also carried out; they included 659 organizations in 2005 with a response rate of 71% and 606 organizations in 2010 with a response rate of 62%. These surveys were preceded by telephone calls to the receptionists of all PHC practices to gather basic information such as type of practice (FMG, NC, and FMG-NC), number of physicians, presence of a nurse, and offering walk-in services.
The population samples were stratified but not proportionally with approximately 400 respondents in each of the 23 territories. Data were weighted to account for unequal selection probabilities resulting from this type of sampling. Poststratification weighting was applied for age and sex, based on 2006 and 2010 estimated Canadian census data. The population-based questionnaire focused on respondents’ attachment to a PHC organization, healthcare and preventive services received, and experience of care in the two years preceding the surveys as well as unmet needs in the previous six months. The questionnaire drew mainly from two validated instruments, the Primary Care Assessment Survey and the Primary Care Assessment Tool, to which we added questions when a topic had not been addressed [24–26]. In addition to experience of care, the population questionnaire inquired about individual characteristics such as sex, age, education level, economic status, perceived health, and presence of morbidities [23].
The two organizational surveys included all PHC organizations in the two regions (659 in 2005 and 606 in 2010). In every organization, a key informant, usually the physician responsible for professional and administrative matters, completed the questionnaire. The organization questionnaire explored elements of vision, resources, structure, and practices [16]. Population and organization surveys were linked through respondents’ identification of their regular source of primary care in the two years preceding the survey (2003–2005 and 2008–2010 corresponding to the T0 and T1 in Figure 1). Failing to identify a usual source of care, the respondent identified the PHC organization most frequently attended in the past two years. This organization was then regarded as the respondent’s usual source of care.
2.2. Organizational Variables
PHC organizations were classified into two groups: a treatment group that included FMGs, NCs, and FMG-NCs; a comparison group that included Local Community Services Centres (LCSC) and Family Medicine Units (FMU) that were not FMG or NC, and all other clinics. Family Medicine Units are academic training units that are likely to reflect family medicine current practice and thus espouse hosting PHC organizations’ dominant philosophy of care. In 2010, among the different PHC organization models in Montréal and Montérégie, group and solo practices were dominant, as they represented 73% of all organizations and were the usual source of care for 50% of all service users. They are privately run by physicians, who are paid on a fee-for-service basis. Costs are shared but not income. For the purposes of this study, FMGs implemented in Local Community Services Centres and Family Medicine Units were included in the treatment group. In the 2010 survey, there were only 11 LCSCs in the treatment group and 38 in the comparison group. The number of FMU remains marginal in the study as 9 were in the treatment group and only 3 in the comparison group.
Among the numerous indicators derived from responses to the organizational questionnaire, we retained the following (Appendix A):(i)number of physicians;(ii)presence of nurses;(iii)number of information technologies used in the practice;(iv)blood taking services available in the building;(v)radiology services available in the building;(vi)collaboration with other PHC clinics;(vii)collaboration with hospitals;(viii)predominant type of visits in the practice (walk-in, by-appointment, and mixed);(ix)scope of diagnostic, therapeutic, and preventive services offered.
At the time of the 2005 survey, 50 FMGs had already been implemented. Therefore, the baseline data were adjusted for those FMGs to take into consideration time elapsed between accreditation and the survey. We simply applied a regression analysis using pace of change of FMGs accredited after 2005 to those accredited before [16].
2.3. Experience of Care and Utilization-Related Variables
Patients’ experience of care, unmet needs, and use of services were reported by population survey respondents. Experience of care variables were accessibility, continuity, comprehensiveness, responsiveness, and perceived care outcomes [27]. Operationalization details for these variables are presented in Appendix B.
We selected 28 indicators of experience of care and grouped them under five dimensions: accessibility (6 items), continuity (5 items), comprehensiveness (5 items), responsiveness (7 items), and perceived care outcomes (5 items). We carried out factor analyses within each of the five dimensions and calculated Cronbach’s alpha on 2010 population data. Cronbach’s alpha was over .60 for four of the five dimensions: continuity (.61), comprehensiveness (.79), responsiveness (.63), and care outcomes (.82). Cronbach’s alpha was low for accessibility (.30), presumably reflecting the great heterogeneity of the indicators and the formative nature of this index [28]. In fact, the accessibility items are not highly correlated, which characterizes composite indices rather than reflective scales [29]. Experience of care variables were expressed as scores on a 10-point scale.
Aside from experience of care, which referred to the two years preceding the surveys, the population questionnaire contained information on use of emergency rooms, hospital admissions, visits to usual source of care, having a family doctor, reporting unmet needs, preventive care received, and sociodemographic characteristics of respondents [23]. Presence of unmet needs in the six months preceding the survey as well as utilization variables was dichotomous variables.
2.4. Data Analysis
To test for differences between 2003 and 2010 in the treatment and comparison groups, we applied the difference-in-differences technique and matched individuals with the propensity score method to ensure better comparability of the two groups [30–34]. This method is particularly well suited to compare change in outcomes over time between individuals exposed to an intervention (the treatment group) and individuals that are not (comparison group) [32]. Individuals were matched on the basis of propensity scores that estimate the probability that an individual with given characteristics is attached to a PHC organization of the treatment group [35]. The propensity score acts as a balancing score that renders the distribution of observed baseline covariates similar between the treatment group and the comparison group [36]. Analyses were carried out using STATA (version 13). Groups were compared after matching by propensity scores with -tests. The results showed no significant values, indicating that covariates were successfully balanced.
3. Results
3.1. Differences between Treatment and Comparison Groups on Selected Organizational Characteristics
As shown in Table 1, the treatment and comparison groups differ considerably with respect to the organizational characteristics presented, particularly in 2010.
Treatment group’s PHC practices are larger and include virtually no solo practices. They are more likely to have nurses and use information technologies and to have blood taking and radiology services available in the building. They have also developed greater collaboration with other PHC practices and hospitals and offer a wider scope of diagnostic, therapeutic, and preventive procedures. Finally, a higher percentage of practices in the treatment group provide a mix of visits (walk-in and by-appointment) at the expense of other types of visits. This percentage increased between 2005 and 2010 from 27.3% to 47.1% whereas the percentage of those providing predominantly walk-in visits decreased from 24.0% to 9.9%. All other indicators followed the same trend, increasing between 2005 and 2010 in the treatment group but remaining more stable or even decreasing in the comparison group. For example, collaboration with other PHC practices decreased with 15.8% in the comparison group while it increased with 19.0% in the treatment group. We observe the same pattern for collaboration with hospitals, although the decrease was less in the comparison group (−7.1%).
3.2. Differences between Treatment and Comparison Groups in the Evolution of Patients’ Experience of Care, Unmet Needs, and Use of Services
Table 2 shows the evolution of patients’ experience of care, unmet needs, and use of services between 2003–2005 and 2008–2010, as well as differences in evolution between the two groups.
The most surprising results concern accessibility which declined between 2003–2005 and 2008–2010 in both groups. However, it declined to a lesser degree for individuals attached to practices of the treatment group than those of the comparison group. This result, in favor of the treatment group, is shown with the positive and significant DD (+0.18; ) and corresponds to a 2.5% change. The percentage of change was calculated by dividing the DD value by the mean baseline scores of both groups. Indicators that contributed the most to this change were the possibility of seeing another doctor when the regular doctor was not available and adequate opening hours of the clinic (data not presented).
A positive DD value is also noted for responsiveness that, unlike accessibility, increased during the same period in the two groups but at a higher pace in the treatment group (+0.09; ), corresponding to a 1.0% change. Indicators that contributed the most to this change are shorter waiting time between arrival at the clinic and seeing the doctor and courtesy of reception (data not presented).
Continuity generally improved between 2003–2005 and 2008–2010. At the two periods, the comparison group performed better, as shown by the negative and significant differences. The differences widened over the years but not enough to reach the level of statistical significance for the DD value (−0.16; ). Indicators that contributed the most to the differences at the two periods are “how long the patient goes to his usual source of primary care” and “whether he always sees the same doctor at this place” (data not presented).
Perceived care outcomes scores slightly improved over time and more so in the comparison group. Although the difference between the two groups was significant in 2008–2010 (), the DD value failed to reach statistical significance (−0.10; ).
The score for comprehensiveness was higher in the comparison than in the treatment group at the two time periods but especially in 2008–2010 (). Still, the change over time was not sufficient to yield a significant DD value. For all other indicators of experience of care, unmet needs, and use of services, DD values were not statistically significant.
4. Discussion
Implementation of FMGs in the two regions has brought about important organizational changes, mainly in terms of resources and structures of PHC organizations [16]. However, the impact of these organizational changes on patients’ experience of care, unmet needs, and use of services has been so far limited. The major impact has been on accessibility. Introduction of the new PHC models has potentially slowed deterioration in accessibility. A second positive effect is on responsiveness. A main contributing factor to increased responsiveness was decreased waiting time between arrival of a patient in a doctor’s office and the face-to-face encounter with the doctor. In that sense, this indicator of responsiveness partly mirrors accessibility.
The positive effect of FMGs and NCs on continuity has been less important than that of other PHC organizations. This result raises concerns because FMGs have been entrusted with improving continuity of care. Again, we should recall that main contributing factors to the relative decrease of continuity in the treatment group were not being able to always see the same doctor at the regular source of care and length of time a patient has been attached to the clinic. We further explored the lower increase in continuity in the treatment group by carrying out separate DD analyses for PHC organizations accredited between 2003 and 2006 and those accredited between 2007 and 2010. The DD values were very close in both analyses for all indicators except continuity which showed a negative and significant DD value for clinics of the treatment group accredited between 2007 and 2010 (data not presented). This difference could be explained partly by the fact that NCs which focus on accessibility more than continuity were mostly implemented during that period. An additional factor is that incentive-linked enrolment of patients presenting specific condition was implemented for all PHC organizations in 2003, thus lessening the advantageous position of FMGs and NCs [37].
Our findings for continuity contradict those reported by Tourigny et al. in their study of 1275 patients followed in five FMGs implemented in two regions of Québec [38]. That study found an increase in continuity but a decrease in accessibility after 18 months among the patients enrolled in those FMGs. Our study is not comparable to the Tourigny et al. study in many regards, namely, number of FMGs studied, inclusion of NCs as new forms of PHC organizations, and regions covered by the study [23, 38]. Our population sample size and the large number of PHC organizations surveyed allow us to generalize with a fair degree of confidence, at least for the two regions under study.
Like most studies that have examined continuity of care, our measures of continuity focus on face-to-face contacts with the regular doctor and not on group continuity (seeing different doctors or other professionals in the same clinic). This is presumably the reason why studies report solo practices ranking the highest in terms of patient experience of care, with the exception of accessibility [39]. This limitation in the measure of continuity has been raised by authors and new measures have been proposed to correct it [40, 41]. Finally, our results did not show that FMGs and NCs have had an impact on use of services or unmet needs.
Our results concur with results reported by other studies on the partial failure of FMGs and NCs in Québec to attain expected results. In a study involving the entire population of Quebec, Dunkley-Hickine found that a higher degree of physician participation in FMGs in a geographical area was not associated with improved accessibility for the population living in this area [42]. In a cohort study based on large administrative data banks and involving 79 FMGs implemented between 2002 and 2004, Strumpf observed no improvement in accessibility but a slight reduction in use of primary care and specialist services [43]. Using the same data bank, Héroux et al. followed a cohort of 122,722 patients enrolled in 79 FMGs compared to 675,102 who were not [44]. They found a small reduction in number of emergency visits for those in FMGs but, as in our study, no change in hospitalizations.
We must recognize that the PHC reform in Quebec has added resources: financial, technical, and human. However, the impact on the population has been limited. This contrasts, for example, with Ontario where the introduction of major financial incentives for physicians to adopt other modes of payment and particularly per capita and blended payment schemes has led to major changes in medical practice [9, 21, 45–47].
A survey conducted by the Commonwealth Fund in 2013 compared different countries and three Canadian provinces on several indicators of experience of care [48]. The report shows that experience of care reported by Quebec respondents, particularly regarding accessibility, was generally less favorable in Quebec than in Ontario, Alberta, and the rest of Canada. Since most of these indicators are related to PHC services, they reflect a relatively poor performance, given the major financial efforts devoted to reforms in this sector.
A recent report of the Auditor General of Quebec to the National Assembly drew a very critical picture of the situation around the implementation of FMGs and NCs [49]. It concluded that FMGs and NCs failed to fully attain the objectives set by the Ministry. It attributes this failure mainly to the absence of clear rules, guidelines, and incentives and to the lack of control of the Ministry and Regional Agencies over the implementation. This has left FMGs and NCs free to develop on their own without having to be fully accountable to the Ministry or the Regional Agencies.
A recent report of the C. D. Howe Institute by Forget has come to similar conclusions [50]. It underlines several factors that have slowed implementation of FMG policies, notably inadequate development of teamwork, and low registration rate of patients.
Overall, the results reported in this paper indicate that reforms in PHC organizations initiated in 2003 have brought about some expected benefits but have yet to be completed. The reported impact of the new types of PHC organizations on accessibility and responsiveness partly fulfills the objectives set at onset of the reforms. Other results raise concerns, notably regarding continuity, use of services, and unmet needs.
5. Limitations
Our study has limitations. A population survey lends itself to possible recall bias by respondents reporting on their experience of care. If present, the bias should be equally distributed among respondents. Recall bias is more likely to occur when referring to single events taking place at a given point in time than with an overall experience extending over a certain period of time.
We did not have information on nonrespondents to the population surveys, which could also introduce a bias. However, the bias was minimized by applying a weighting procedure that adjusts samples’ data to the 2005 and 2010 populations’ data. In addition, our analyses control for individual characteristics of respondents.
Measures of experience of care are expressed from the patient’s points of view and perceptions. In that sense, these measures are subjective but still the most appropriate to describe experience of care. Our scales are largely derived from two validated instruments, PCAT and PCAS [24–26]. We constructed new scales of experience of care that were more adapted to our context. Following the classical theory of measure, we carried out factor analyses and calculated Cronbach’s alpha coefficients. The coefficient values were generally close to the commonly accepted level (.70) except for accessibility which was low (.30) [51]. We have kept accessibility in our analyses, considering it is a formative composite index in which the items composing the index are not necessarily correlated nor substitutable with each other, as is the case in reflective scales [29].
The imputation method we applied to nonrespondents of the PHC organization survey might have introduced a bias since it is based on respondent data. As pointed out by Haziza, the nonresponse bias may be greater with nonresponse than with imputation, particularly when basic information is available, as was the case here from calls to receptionists prior to the organization survey [52]. We tested the magnitude of a possible bias by carrying out statistical analyses with respondent organizations only. The results obtained were similar to those with grouped respondents and nonrespondents [16].
6. Conclusion
In spite of important resources devoted to PHC reforms initiated in Quebec in the early 2000s, results so far have been limited in terms of experience of care improvement, reduction of utilization of services, and unmet needs. This is the conclusion we can draw when we compare the evolution of PHC practices that have become FMGs, NCs, or FMG-NCs to that of PHC practices that have not acquired such status. This is not to say that the healthcare system has not improved as a whole. For example, we observed that continuity has increased at the system level but more so in practices that were not FMG, NC, or FMG-NC. Accessibility has decreased but much less than it would have without FMGs, NCs, and FMG-NCs. These examples highlight the fact that it is hazardous and perhaps unfair to attribute specific effects of a social policy to a single intervention, be it success or failure. Social systems are complex and their components are highly interactive. The effect of a change may induce other, often unexpected, changes. Finally, implementing policy changes takes time [53]. This is the conclusion of a recent article that assessed one of the earliest and largest medical home pilots implemented in the United States [54]. How long it takes to realistically and fairly assess sustainable benefits brought about by primary care reforms such as the ones initiated in Québec in the early 2000s remains difficult to ascertain.
Appendices
A. Organizational Variables
See Table 3.
B. Experience of Care Indices
See Table 4.
Conflict of Interests
The authors declare that they have no competing interests.
Acknowledgments
The authors acknowledge the contribution of the following collaborators and researchers associated with the project: D. Roy, J. L. Denis, P. Lamarche, M. D. Beaulieu, J. Haggerty, D. Roberge, J. Côté, M. Hamel, F. Goulet, M. Drouin, J. Rodrigue, L. Côté, and O. Lemoine. The study benefited from the financial contributions of the Canadian Institutes of Health Research (no. PHE 101905), the Fonds de Recherche du Québec-Santé (FRQS) (no. 22032), the Agence de la Santé et des Services Sociaux de Montréal, and the Agence de la Santé et des Services Sociaux de la Montérégie. Isabelle Rioux provided editorial assistance and Sylvie Gauthier revised the text.