Table of Contents
International Journal of Family Medicine
Volume 2016, Article ID 4929432, 9 pages
Research Article

General Practitioners’ Participation in a Large, Multicountry Combined General Practitioner-Patient Survey: Recruitment Procedures and Participation Rate

1Netherlands Institute for Health Services Research (NIVEL), P.O. Box 1568, 3500 BN Utrecht, Netherlands
2Department of Sociology and Department of Human Geography, Utrecht University, Heidelberglaan 2, 3584 CS Utrecht, Netherlands
3Hochschule Fulda University of Applied Sciences, Leipziger Straße 123, 36037 Fulda, Germany

Received 3 December 2015; Accepted 14 February 2016

Academic Editor: Hakan Yaman

Copyright © 2016 Peter P. Groenewegen et al. This is an open access article distributed under the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.


Background. The participation of general practitioners (GPs) is essential in research on the performance of primary care. This paper describes the implementation of a large, multicountry study in primary care that combines a survey among GPs and a linked survey among patients that visited their practice (the QUALICOPC study). The aim is to describe the recruitment procedure and explore differences between countries in the participation rate of the GPs. Methods. Descriptive analyses were used to document recruitment procedures and to assess hypotheses potentially explaining variation in participation rates between countries. Results. The survey was implemented in 31 European countries. GPs were mainly selected through random sampling. The actual implementation of the study differed between countries. The median participation rate was 30%. Both material (such as the payment system of GPs in a country) and immaterial influences (such as estimated survey pressure) are related to differences between countries. Conclusion. This study shows that the participation of GPs may indeed be influenced by the context of the country. The implementation of complex data collection is difficult to realize in a completely uniform way. Procedures have to be tuned to the context of the country.

1. Background

The participation of general practitioners (GPs) is essential in research on the performance of primary care. Rates of participation are not always satisfactory. This paper describes and analyses the rate of participation of GPs in a large multicountry study, the QUALICOPC (Quality and Costs of Primary Care) study.

The study consists of linked surveys among GPs and their patients, with the patient survey implemented through field workers that visited the practices, in 31 European countries between 2011 and 2013. Recruitment of GPs to participate was challenging and rates of participation varied per country and were often relatively low. A brief summary of the study design of the QUALICOPC study is provided in Box 1. This paper describes the recruitment procedures and the rates of participation of GPs, providing the background information to publications based on the QUALICOPC study and to future users of the data or researchers who want to organize a comparable survey. We also analyse explanations for the varying participation rates to contribute to our knowledge on the implementation of complex international field studies.

Box 1: Study design of the QUALICOPC study.

As in any study of nonresponse, willingness to participate will vary between individuals. However, in multicountry studies there might also be systematic variation related to influences that are the same for all GPs in a country. This variation might be related to the way the study is implemented, but also to position of GPs. Consequently, response may be selective within each country, but also between countries.

Studies that review survey participation rates of GPs usually review single survey, single country studies. These studies may be relevant for comparison, but only to a limited extent. Nevertheless, they may be helpful in developing ideas about between country variation. Recent review studies of health care worker response rates [1] or GP response rates [2] show that monetary and nonmonetary incentives increase response, as do prior contact by telephone and email and reminders. A systematic review [1] also reports on differences between countries, with European countries showing higher response rates compared to the US, Canada, and Australia/New Zealand. However, they focus on explaining differences by characteristics of the surveys, leaving country context out.

Our research questions are as follows:(1) How was recruitment of GPs in the QUALICOPC study organized and what is the resulting participation rate per country?(2) How does the willingness of GPs to participate in this combined GP and patient survey vary between countries and how can this variation be explained?We hypothesize that differences in participation rate between countries are related to material and immaterial influences [6] and that these may be dependent on the implementation of the study and on country context.

1.1. Material Influences

(H1)Participation in a study, such as QUALICOPC, takes time and effort that could also be used in other activities. Hence, there are opportunity costs that may vary systematically between countries. If the opportunity costs for time spent in participating are higher, we expect less GPs to do it. The implication is that in countries where GPs earn more they might be less willing to participate (country context).(H2)In countries where GPs are paid fee for service, the opportunity costs of participation are more salient and they might be more used to receive material incentives. We expect lower participation among GPs in these countries and more participation in countries where GPs are salaried (country context).(H3)In countries where GPs were given a material incentive for participation in the study, we expect a higher willingness to participate (study implementation).

1.2. Immaterial Influences

Immaterial influences could work via several mechanisms: survey fatigue [7], prosocial (or altruistic) attitudes [8], and contribution to a collective good [9].(H4)The number of requests to participate in surveys might differ between countries and lead to “survey fatigue.” Willingness to participate will be lower in countries with a large survey load, assuming that GPs have a bigger chance to have been invited to participate in surveys in the past (country context).(H5)By participating in our study GPs do others (the researchers or the country coordinators that have approached them) a favour. GPs may be more inclined to do so if they have prosocial attitudes. Countries differ in the prevalence of these attitudes. Hence, if GPs live in a country with in general more prosocial attitudes, we expect them to be more willing to participate (country context).(H6)Participation in our study contributes to a collective good, by providing information about the position of GPs that may be used by, for example, professional organizations to plea for better conditions for primary care. We hypothesize that GPs feel more induced to contribute to the collective good when their numbers are smaller, when their structural position is weaker (country context), and when an appeal has been done by their professional organization to participate (study implementation).

2. Methods

2.1. Recruitment Procedures and Participation Rate

The intended recruitment procedures have been described elsewhere [4]. We briefly summarize them here. In Section 3 we will describe the actual implementation.

Data were to be collected in 31 European countries and three non-European countries. This paper is limited to a description of the European countries. Descriptions of the recruitment in Australia and Canada are described elsewhere [10, 11]. The target response was 220 GPs per country, except for the four smallest countries where the target was 75. In each country we aimed to draw a nationally representative sample of GPs with one GP per practice. In countries with no national register, multistage sampling was to be used, for example, by combining registers from different regions. Furthermore, in large countries with regional differences in health care system, several representative regions would be selected and subsequently GPs within these regions. The information on the actual implementation of the study was provided by the national coordinators. Based on their information on the numbers of GPs invited and the actual number in the database we calculated the participation rate.

Ethical approval for the QUALICOPC study was acquired in accordance with the legal requirements in each country.

2.2. Data to Assess the Hypotheses

To assess the hypotheses, data were derived from several sources:(i)GP income: estimates from the PHAMEU database [3, 12].(ii) Material incentive: see Table 1.(iii) GP payment system: fee-for-service, mixed systems of capitation and fee-for-service, or salary, PHAMEU database [3].(iv) Survey pressure: indicated by the number of abstracts reporting on GP surveys at WONCA Europe conferences from 2008 to 2012, grouped in 5 categories (additional file 1, in Supplementary Material available online at Abstracts have a lower threshold than published research papers. As the conferences are always in a different place in Europe, the bias towards Northwest Europe is less strong.(v) Prosocial attitudes: percentage of the general population saying that most people can be trusted. Most recent data are available per country (oldest 1990–94, 2 countries, most recent 2010–2014, 10 countries), 9 countries missing [13].(vi) Number of GPs per country: calculated from GPs per 100.000 population [3] and country population [14].(vii) Structural position of GPs: overall strength of the structure of primary care [3, 15].(viii) Appeal by professional organizations: letter of recommendation used in recruitment (see Table 1).Apart from the variables in the hypotheses we have included two confounders: total population size and gross domestic product (as deviation from the EU28) [14].

Table 1: Recruitment procedures.
2.3. Analyses

Due to the small number of countries we have analysed the data pragmatically in order to assess the hypotheses. We have grouped the countries according to participation rate into three groups: participation rates low (≤20%), medium (21–50%), and high (>50%). As a sensitivity test we have also used alternative cut-off points (≤30%, 31–60%, and >60%). For each of the groups we have computed the average or percentage of the explanatory variables from the hypotheses and the two confounders.

3. Results

3.1. Implementation of the Study and Participation Rates

The aim was to include at least all EU member states. However, for different reasons the study could not be implemented in France.

We have categorized the actual sampling procedure. In four small countries the (almost) entire population of GPs was invited to participate (A in Table 1). Random sampling was used in the majority of countries, either from a national sample (17 countries, B) or from preselected regions (4 countries, C). In four countries random sampling was complemented with opportunity sampling (D) and in two countries opportunity sampling [16] was used (E). In most countries selected GPs were contacted through a mix of letters, email, and telephone contact. One or more reminders were sent to the selected GPs. The implementation of the patient survey was in most cases done through field workers who visited the practice to invite patients to fill in the patient questionnaire (with the exception of Denmark and England and a part of Norway and Sweden). Approximately half of the countries used a material incentive, but in widely different form and size. Just under half of the countries used a support letter from a professional organization. The median duration of the data collection, including the patient survey, was half a year.

Table 2 gives the estimated participation rates. These vary from less than 10% in Austria, Belgium, Germany, Ireland, and Sweden to over 70% in Cyprus, Greece, Iceland, Malta, and Spain.

Table 2: Participation rates of GPs in a combined GP and patient survey in 35 countries.

For most countries we could check the representativeness of the participating GPs by comparing them with regard to age and gender to national statistics (additional file 2) [17]. Average age of the participating GPs differed from the national average by five years or more in Hungary, Italy, and Spain. The percentage of female GPs in our study differs by ten percentage points or more in Greece, FYR Macedonia, the Netherlands, Slovenia, and Spain.

3.2. Assessing the Hypotheses

Table 3 contains the information to evaluate the hypotheses.

Table 3: Assessment of the hypotheses—average (and range) or percentage of the independent variables by three categories of participation rates.

Average GP income is highest in the low response group. There is no trend over all three groups. This partly confirms hypothesis 1.

A material incentive was more often provided in the low response group and less often in the medium and high response groups, opposite to the relationship of hypothesis 2.

In the higher response groups the percentage of countries with predominantly salaried GPs is higher. This confirms our hypothesis that opportunity costs are related to participation. However, there is no trend for the percentage of countries with predominant pure fee-for-service or mixed payment systems. Therefore the hypothesis is partly confirmed.

Survey load is lowest in the high response group, but there is no trend over all three groups, which partly confirms hypothesis 4.

The percentage of people that say most people can be trusted is lower in higher response groups. Hypothesis 5 was therefore refuted.

For hypothesis 6, we used three indicators. In the low response group the average number of GPs per country is higher, but there is no trend over all groups, which partly confirms the hypothesis. GPs in countries with a weaker primary care system seem to be more inclined to participate than those in countries with stronger primary care. In contrast with the hypothesis, the share of countries using a support letter is lowest in the high response group and highest in the low response group. In summary, hypothesis 6 is partly confirmed.

As far as the two confounders are concerned, we see a trend towards smaller countries and countries with lower GDP per capita in the higher response groups.

4. Discussion

The actual implementation of the QUALICOPC study differs somewhat from the intended approach. The country coordinators received uniform instructions but also had to take into account the feasibility of the suggested procedures in the context of their own country. Moreover, the financial resources for the implementation of the study in each country were modest. Consequently, the country coordinators had to use their creativity to come as close as possible to the suggested procedures within the restrictions of time, money, and national circumstances. In some countries there has been a stepwise deviation from the original instructions when it turned out that these were not resulting in the expected participation.

A large number of European GPs have participated in the study with a multiple of patients. In half of the countries a participation rate among GPs of over 30% was realized. Against the background of decreasing participation rates of physicians in surveys and large differences between countries [1], this is satisfactory. Random sampling of GPs was realized in two-thirds of countries. Only in France we did not succeed in implementing the study. We have, after several attempts, not been able to locate a team that was willing and able to take up the national coordination for data collection in that country, within the requested time and financial frame.

Differences in procedures and in participation rates between countries might have led to unknown levels of bias. We have attempted to estimate bias by comparing the samples of GPs to national data on age and gender of the population of GPs. The participating GPs are by and large representative on age and gender for the population of GPs in their country. Nevertheless, the low response and the efforts to reach the target suggest self-selection, probably in the direction of GPs who are interested in research and developing their profession.

An important reason for low response was probably that participation involved allowing field workers to come to the practice to ask patients to fill in a questionnaire. GPs may have many reasons not to want this [18]. They may see it as a disruption of practice routines, expect question from their patients, and so forth. Moreover, the combination with a patient survey changes the context of the survey from individual GPs to the practice. Filling in a questionnaire by a GP is an individual decision but becomes a practice decision when patients are also involved and field workers visit the practice. Primary care practices are organizations (although often small) and, in general, organizations behave differently from individuals. They tend to follow procedures to decide about requests, with as a result a longer procedure and lower chance of spontaneous participation compared to individuals [1922]. GP participation is also related to the role of other staff that have to pass a request or a questionnaire on to the GP [10, 23]. In sum, limitations for the QUALICOPC study are the differences in implementation between countries and low participation rate of GPs.

The QUALICOPC study includes enough countries to analyse differences in participation rates between countries. It was hypothesized that material and immaterial influences would affect the decision to participate. The participation rate was indeed related to material factors, but not always in the hypothesized direction. The response rate to the QUALICOPC study was low in countries with higher average income of GPs and in countries where a material incentive was more often used. The latter might be a case of reverse causation in that coordinators might have proposed material incentives because of their estimation that participation rates are low in their country. Moreover, in some countries it might have become common practice to give a material incentive for research participation, perhaps as a reaction to decreasing participation rates. However, we have not found information in the literature on such practices. Participation was high in countries with predominantly salaried GPs.

The response rate was also found to be related to immaterial factors. In the high response group survey load was lower. Prosocial attitudes were related to response rates but in the opposite direction of the hypothesis. We only had information for a selection of countries and prosocial attitudes were measured in the population and not amongst GPs. The propensity to participate as a contribution to the collective good was analysed through the number of GPs, the strength of primary care, and the support of a professional organization. For support letters this hypothesis was refuted and for the two other operationalizations it was partly confirmed. The collective good mechanism is usually connected to much smaller numbers than we find with the number of GPs in a country. However, the professions seem to be able to produce collective goods in the situation of much larger numbers of coproducers [24, 25]. As concerns the role of strong primary care, it could also be reasoned that in countries with stronger primary care systems GPs are more aware of their leading position internationally and want to confirm that by participating in international research. However, this alternative line of reasoning was not confirmed by our analysis.

A limitation of the test of our hypotheses is that some country coordinators have introduced material and immaterial incentives in the course of recruitment in response to disappointing participation. Consequently, the effects of incentives might be underestimated.

An alternative grouping of the countries according to participation rates, as a sensitivity analysis, largely gave the same results, but for hypothesis 2 there was a less clear gradient with the percentage with a material incentive nearly the same in the low and middle response categories; also for hypothesis 3 there was a less clear gradient with the lowest percentage salaried GPs in the middle response category.

Although we were able to include a large number of countries in this study, from a purely statistical point of view it still is a limitation that the number of countries is too small to do multivariable analysis. As the independent variables are correlated, we are not able to determine which is the most important. The response pattern could also be influenced by other unmeasured variables or confounders. We have looked at two confounders and they differ between the response groups. However, the number of countries is too small to assess the hypothesized effects while taking confounders into account. Participation was higher in smaller and less wealthy countries. However, we prefer the interpretation in terms of our hypotheses over an interpretation in terms of variables that have no specified mechanism.

For the design of future international studies on general practice with comparable size and complexity, we think the following issues are important. The first one is slightly trivial, but important: a comparable study in the future should be able to allocate more money for the implementation of the study. Secondly, the role of national coordinators is of prime importance, because the knowledge of the national situation is indispensable. Thirdly, piloting of the fieldwork procedures should be done in all countries involved. Finally, it might be considered to subcontract the actual fieldwork in a country to a specialized organization, under the direct supervision of the national coordinator; this could improve the uniformity of the fieldwork, although subcontracting is known to bring its own problems often. Our analysis confirmed a lack of strong evidence about the role of material and immaterial incentives in increasing the willingness of GPs to participates. The current systematic reviews on these subjects do not adequately take the country context into account. However, we hope that future reviews of participation rates and response rates will specifically analyse country or health care system differences.

5. Conclusion

This paper adds to the literature on survey participation by GPs by bringing in the international comparative perspective. Existing reviews do not address the country context. This study shows that country context has an influence on participation rates. The implication from the QUALICOPC study is that a uniform approach of data collection across countries is almost impossible to achieve. Procedures have to be tuned to the context of the country and available financial resources.

Conflict of Interests

No competing interests exist.

Authors’ Contribution

Peter P. Groenewegen designed the analysis and drafted the paper; Stefan Greß designed the analysis and reviewed the drafts of the paper; Willemijn Schäfer collected the data, assisted in the analysis, and reviewed the drafts of the paper.


The authors thank their partners in the QUALICOPC project for their role throughout the study and their coordination of the data collection: W. Boerma (the Netherlands, NIVEL, Coordinator); J. De Maeseneer, E. De Ryck, L. Hanssens, A. Van Pottelberge, and S. Willems (Belgium); S. Greß and S. Heinemann (Germany); G. Capitani, S. De Rosis, A. M. Murante, S. Nuti, C. Seghieri, and M. Vainieri (Italy); D. Rotar Pavlič and I. Švab (Slovenia); and M. Van den Berg, T. Van Loenen, and D. Kringos (the Netherlands). The authors would like to thank the coordinators of the data collection in each country: K. Hoffmann and M. Maier (Austria); P. Salchev (Bulgaria); G. Samoutis (Cyprus); B. Seifert and N. Šrámková (Czech Republic); J. Reinholdt Jensen and P. Vedsted (Denmark); M. Lember and K. Põlluste (Estonia); E. Kosunen (Finland); K. Stavric (FYR Macedonia); C. Lionis (Greece) and I. Rurik (Hungary); J. Heimisdóttir and O. Thorgeirsson (Iceland); C. Collins (Ireland); G. Ticmane (Latvia); S. Macinskas (Lithuania), M. Aubart, J. Leners, and R. Stein (Luxembourg); G. Bezzina and P. Sciortino (Malta); T. Bjerve Eide and H. Melbye (Norway); M. Oleszczyk and A. Windak (Poland); L. Pisco (Portugal) and D. Farcasanu (Romania); T. Dedeu (Spain); E. Jurgova (Slovakia); C. Björkelund and T. Faresjö (Sweden); T. Bisschoff and N. Senn (Switzerland); M. Akman (Turkey); and C. Sirdifield and N. Siriwardena (UK). Finally, the authors would like to thank all persons who have supported the national coordinators in the data collection in their countries. This paper is based on the QUALICOPC (Quality and Costs of Primary Care in Europe) project, cofunded by the European Commission under the Seventh Framework Programme (FP7/2007–2013) under Grant Agreement 242141.


  1. J. V. Cook, H. O. Dickinson, and M. P. Eccles, “Response rates in postal surveys of healthcare professionals between 1996 and 2005: an observational study,” BMC Health Services Research, vol. 9, article 160, 2009. View at Publisher · View at Google Scholar · View at Scopus
  2. S. W. Pit, T. Vo, and S. Pyakurel, “The effectiveness of recruitment strategies on general practitioner's survey response rates—a systematic review,” BMC Medical Research Methodology, vol. 14, article 76, 2014. View at Publisher · View at Google Scholar · View at Scopus
  3. “PHAMEU database,” 2014,
  4. W. L. A. Schäfer, W. G. W. Boerma, D. S. Kringos et al., “QUALICOPC, a multi-country study evaluating quality, costs and equity in primary care,” BMC Family Practice, vol. 12, article 115, 2011. View at Publisher · View at Google Scholar · View at Scopus
  5. W. L. A. Schäfer, W. G. W. Boerma, D. S. Kringos et al., “Measures of quality, costs and equity in primary health care instruments developed to analyse and compare primary care in 35 countries,” Quality in Primary Care, vol. 21, no. 2, pp. 67–79, 2013. View at Google Scholar · View at Scopus
  6. S. R. Porter, “Raising response rates: what works? Multiple surveys of students and survey fatigue,” New Directions for Institutional Research, vol. 121, pp. 15–21, 2004. View at Google Scholar
  7. S. R. Porter, M. E. Whitcomb, and W. H. Weitzer, “Multiple surveys of students and survey fatigue,” New Directions for Institutional Research, vol. 2004, no. 121, pp. 63–73, 2004. View at Publisher · View at Google Scholar
  8. B. S. Frey and S. Meier, “Pro-social behavior in a natural setting,” Journal of Economic Behavior and Organization, vol. 54, no. 1, pp. 65–88, 2004. View at Publisher · View at Google Scholar · View at Scopus
  9. M. Olson, The Logic of Collective Action, Harvard University Press, Cambridge, Mass, USA, 1971.
  10. A. Parkinson, L. Jorm, K. A. Douglas et al., “Recruiting general practitioners for surveys: reflections on the difficulties and some lessons learned,” Australian Journal of Primary Health, vol. 21, no. 2, pp. 254–258, 2015. View at Publisher · View at Google Scholar
  11. S. T. Wong, L. W. Chau, W. Hogg et al., “An international cross-sectional survey on the Quality and Costs of Primary Care (QUALICO-PC): recruitment and data collection of places delivering primary care across Canada,” BMC Family Practice, vol. 16, article 20, 2015. View at Publisher · View at Google Scholar
  12. D. S. Kringos, W. G. Boerma, Y. Bourgueil et al., “The European primary care monitor: structure, process and outcome indicators,” BMC Family Practice, vol. 11, article 81, 2010. View at Publisher · View at Google Scholar · View at Scopus
  13. World Values Survey, August 2014,
  14. Eurostat, August 2014,
  15. D. Kringos, W. Boerma, Y. Bourgueil et al., “The strength of primary care in Europe: an international comparative study,” British Journal of General Practice, vol. 63, no. 616, pp. e742–e750, 2013. View at Publisher · View at Google Scholar · View at Scopus
  16. A. Brady, “Opportunity sampling,” in The Sage Dictionary of Social Research Methods, V. Jupp, Ed., Sage, Thousand Oaks, Calif, USA, 2006. View at Publisher · View at Google Scholar
  17. S. Barclay, C. Todd, I. Finlay, G. Grande, and P. Wyatt, “Not another questionnaire! Maximizing the response rate, predicting non-response and assessing non-response bias in postal questionnaire studies of GPs,” Family Practice, vol. 19, no. 1, pp. 105–111, 2002. View at Publisher · View at Google Scholar · View at Scopus
  18. E. Hummers-Pradier, C. Scheidt-Nave, H. Martin, S. Heinemann, M. M. Kochen, and W. Himmel, “Simply no time? Barriers to GPs' participation in primary health care research,” Family Practice, vol. 25, no. 2, pp. 105–112, 2008. View at Publisher · View at Google Scholar · View at Scopus
  19. A. L. Stinchecombe, The Logic of Social Research, University of Chicago Press, Chicago, Ill, USA, 2005.
  20. G. Snijkers, G. Haraldsen, J. Jones, and D. K. Willimack, Designing and Conducting Business Surveys, John Wiley & Sons, Hoboken, NJ, USA, 2013. View at Publisher · View at Google Scholar
  21. Y. Baruch and B. C. Holtom, “Survey response rate levels and trends in organizational research,” Human Relations, vol. 61, no. 8, pp. 1139–1160, 2008. View at Publisher · View at Google Scholar · View at Scopus
  22. E. F. Lewis, M. Hardy, and B. Snaith, “Estimating the effect of nonresponse bias in a survey of hospital organizations,” Evaluation & the Health Professions, vol. 36, no. 3, pp. 330–351, 2013. View at Publisher · View at Google Scholar · View at Scopus
  23. T. S. Flanigan, E. McFarlane, and S. Cook, “Conducting survey research among physicians and other medical professionals: a review of current literature,” in Proceedings of the American Statistical Association Survey Research Methods Section (AAPOR '08), pp. 4136–4147, New Orleans, La, USA, May 2008.
  24. J. G. Lulofs, “Professies en de markt voor vertrouwensgoederen,” in Modellen in de Sociologie, S. Lindenberg and F. N. Stokman, Eds., Van loghum Slaterus, Deventer, Netherlands, 1983. View at Google Scholar
  25. E. A. Krause, Death of the Guilds: Professions, States and the Advance of Capitalism, 1930 to the Present, Yale University Press, New Haven, Conn, USA, 1986.