Quality Indicators of Pharmaceutical Care for Integrative Healthcare: A Scoping Review of Indicators Developed Using the Delphi Technique

Table 1

Summary of the studies included in this scoping review (n = 31).

No.	Author(s)	Objectives of the study	Participants	Data collection	Main results
Pharmaceutical services relevant to medications
1	Fernandes et al. [26]	To develop performance indicators to improve clinical pharmacy practice and patient care	The Delphi rounds were completed by 26 pharmacists: experience of 6–10 (n = 2), 11–15 (n = 5), and greater than 15 years (n = 19)	A working group of frontline clinical and hospital pharmacists from all over Canada systematically developed a comprehensive list of potential performance indicators. Three authors conducted a comprehensive literature review to create an inventory of candidate performance indicators. The crude performance indicators were rated against a list of 11 ideal attribute criteria. Three authors extracted 8 thematic critical activity areas that the final list should contain. A modified Delphi technique of three rounds with an in-person meeting was followed to develop the final list.	The final list contained 8 performance indicators grouped into 6 categories: discharge medication reconciliation, admission medication reconciliation and best possible medication history, interprofessional patient care rounds, pharmaceutical care, bundle of critical activity areas, and patient education/discharge counselling.
2	Shawahna [23]	Development of a core set of key performance indicators to measure impact of pharmacists in caring for patients	Pharmacists (n = 25), nurses (n = 4), physicians (n = 4), and doctorates (n = 7)	A formal consensus technique using the Delphi technique (literature search, interviews with 14 pharmacists, neurologists, nurses, and patients, and a three-round Delphi technique among 40 panelists).	The final core list contained 8 key performance indicators in the following thematic areas: pharmaceutical care (n = 3), patient education/counselling/reconciliation (n = 2), medication reconciliation and best possible medication history (n = 1), interprofessional patient care (n = 1), and competence and performance efficiency/patient satisfaction (n = 1).
3	Krzyżaniak et al. [33]	To identify and develop a core list of essential roles and activities that could serve as performance indicators of pharmacy services in Poland	Pharmacist/director of pharmacy (n = 9), pharmacists in academia (n = 2), neonatologist/doctor (n = 1), and nurse/midwife (n = 1)	A literature review was conducted. Healthcare providers were consulted for suitability of the potential indicators identified. A modified Delphi technique of two consecutive online rounds was conducted.	The final core list contained 23 performance indicators for quality pharmaceutical care grouped into structure (n = 9), process (n = 9), and outcome (n = 5) indicators.
4	Cillis et al. [34]	Developing and validating a benchmarking tool to measure clinical pharmacy activities	17 pharmacists who provided services in geriatrics, surgery, intensive care unit, cystic fibrosis, nonpatient-centered activities, anticoagulation, antibiotic therapy management group, and nutrition	A narrative literature review was conducted by the authors to identify and collect potential performance indicators. Two focus groups were held to refine the list of collected performance indicators. A three-round Delphi technique was followed to achieve consensus on a final core list.	The final core list contained 10 quality indicators grouped into 6 areas: medication reconciliation at admission, patient monitoring, information provided to the healthcare team, patient education, discharge and transfer medication counselling, and adverse drug reaction monitoring.
5	Ng and Harrison [35]	Identification of a list of key performance indicators that could be measured to demonstrate contributions of pharmacists in patient care	Respondents (n = 44) were chief medical officers (n = 12), director of nursing (n = 5), chief pharmacists (n = 15), quality and risk managers (n = 8), and senior management team members (n = 4).	Potential items were collected from the literature and presented to the panelists. The panelists rated the items in a Delphi technique.	Performance indicators were ranked by scores of relevance and measurability. Indicators included: chart review, medication reconciliation, prescribing errors, clinical pharmacy interventions, medication card provision, correct pediatric medication orders, adjustment or review of toxic or subtherapeutic doses, patient reviews, provision of written information to patients, and patient counselling.
6	Lima et al. [36]	Development of a set of key performance indicators for services related to medication management	The working group consisted of university professors and researchers in clinical pharmacy (n = 2), doctoral students (n = 2), and clinical pharmacists (n = 4). The questionnaire was administered on pharmacists (n = 82).	Iterative rounds were conducted to identify potential performance indicators. The indicators identified were rated by experts for 7 attributes using a Likert-scale of 5 points in 2 iterative Delphi technique rounds. An online questionnaire was administered on 82 pharmacists.	The final core list contained 6 performance indicators grouped in the following categories: pharmaceutical consultation, interventions accepted by the prescriber, therapy problems solved, assessment of patient clinical status, satisfaction of the patient, and quality of life of the patient.
7	De Bie et al. [37]	To develop a system of quality indicators for pharmaceutical care	The first Delphi round was completed by 16 panelists and the second round was completed by 151 pharmacists.	A thorough literature review was conducted to compose an initial list of indicators. A two-round Delphi technique was followed to develop and validate the final list of indicators. Indicators in the final list were used to collect data from 30 pharmacies.	The final list consisted of 42 quality indicators grouped into 6 categories: patient counselling, clinical risk management, compounding, dispensing of medication, monitoring of medication use, and quality management.
8	Grey et al. [38]	To seek confirmation of stakeholders and rank in order of importance a list of characteristics of good pharmaceutical care	The first round was completed by 23 participants who were dispensing general practitioners or practice managers (n = 3), community pharmacists (n = 8), community pharmacy dispensing assistants (n = 2), community pharmacy board members (n = 1), large chain community pharmacy executives (n = 2), and laypersons (n = 7).	A postal questionnaire was sent to community pharmacists and dispensing doctors to identify characteristics of good pharmaceutical care. In-depth case studies of community pharmacists (n = 3) and dispensing doctors (n = 4) were conducted. A two-round Delphi technique was then followed to confirm and rank in order of importance a list of characteristics of good pharmaceutical care.	The final list contained 23 characteristics of good pharmaceutical care grouped into 4 categories: patient safety dispensing, patient–provider interaction, workplace culture, and public health.
9	Clay et al. [39]	Development of a checklist of pharmacist interventions while providing patient care services	The final list received input from more than 200 stakeholders over a period of 4 years.	A list of items was collected through expert group meetings, literature review, and refinement of the items through iterative rounds including face-to-face meetings, conference calls, and receiving public comments.	The final list contained 9 critical components: replicability, patient population, patient and other data sources, environment, delivery, frequency and duration, pharmacist role and responsibility, attribution, and unique attributes.
10	Richardson [57]	To develop indicators for referral to an outpatient service providing CAM modalities by considering the research evidence for the effectiveness of these modalities	General practitioners (n = 71) were surveyed and healthcare professional panelists (n = 27) took part in the modified Delphi technique.	General practitioners were surveyed for their opinions with regard to referring patients to outpatient services providing CAM modalities. A modified Delphi technique was used to develop indicators for referral to an outpatient service providing CAM modalities.	The panelists agreed on developing indicators for referral to services providing CAM modalities like acupuncture, homeopathy, and osteopathy in conditions like allergic conditions (rheumatoid arthritis, osteoarthritis, asthma, chronic obstructive airways disease, and rhinitis), back pain, neurologic conditions, palliative care, irritable bowel syndrome and reflux oesophagitis, eczema, emotional disorders, eye & mouth disorders, prolapse/endometriosis/menstrual problems, headaches, stress/fatigue, insomnia, hypertension, skeletal problems, strokes, tinnitus, viral conditions, and common childhood disorders.
11	Mackinnon and Hepler [40] (review other studies)	Developing a list of clinical indicators of preventable drug-related morbidity in older adults	The panelists were physicians (n = 6) and a clinical pharmacist (n = 1).	The literature was reviewed to identify scenarios that represented potential outcomes and patterns of care that were thought to be possible preventable drug-related morbidity situations in older adults. A modified Delphi technique was followed among the panelists to develop the final list of clinical indicators of preventable drug-related morbidity in older adults.	The panelists agreed on 52 scenarios representing possible preventable drug-related morbidity situations in older adults.
12	Pyne et al. [44]	To develop a valid and usable list of quality indicators to detect and treat depression in patients	The panelists were physicians (n = 6), psychiatrists (n = 4), and a clinical pharmacist (n = 1).	The literature was reviewed to collect potential quality indicators for detection of depression in patients. A modified Delphi technique was followed to develop the final list of quality indicators in detecting and treating depression in patients.	The final list contained 59 quality indicators grouped into 6 categories: general indicators for depression treatment in patients, bereavement, substance abuse, viral infections, cognitive impairment, and mental health drug interactions.
13	Morris and Cantrill [41]	To assess if a series of preventable drug-related morbidity indicators used in the United States were applicable to the United Kingdom after transferring from the United States to the United Kingdom healthcare facilities	A panel of 16 members: general practitioners (n = 6) and primary care pharmacists (n = 10)	Preventable drug-related morbidity indicators were taken from previous studies and presented to the panelists for consensus.	The final list contained 19 indicators of possible preventable drug-related morbidity situations in older adults in the United Kingdom healthcare settings.
14	Morris et al. [42]	Description of the process of developing and validating a series of indicators that could be used to prevent drug-related morbidity	General practitioners (n = 6) and pharmacists (n = 10)	Indicators selected were validated in a preliminary step. A two-round Delphi technique was followed among the panelists to develop the final list.	The final list contained 29 indicators. Of those, 19 were originally developed in the United States practice and 10 were generated by the panelists for the United Kingdom practice.
15	Robertson and MacKinnon [43]	Development of clinical indicators of preventable drug-related morbidity in older adults	Two separate specialists panels: geriatricians (n = 6) and clinical pharmacologists (n = 6); general practitioners (n = 12) participated in the focus group.	The Delphi technique was followed in 2 separate specialist panels to develop and achieve consensus on the clinical indicators. General practitioners participated in the focus group to assess the applicability of the indicators in Canada practice.	The final list contained 52 clinical indicators of preventable drug-related morbidity in older adults that can be applied to Canada practice.
16	Currie et al. [45]	Development of guidelines to document elements needed to record care provided by pharmacists to allow assessment of quality of care	Pharmacists (n = 9) and experts (n = 8)	The literature was reviewed and an initial list was compiled. A group of pharmacists validated the list. A three-round Delphi technique followed by group meetings was conducted among the panelists to achieve consensus on the final list.	The final list contained elements of documentation as a tool to evaluate documentations (n = 14). This list might serve as a tool to assess the quality of care provided and documented by pharmacists.
17	Malone et al. [46]	Development of a list of clinically important drug-drug interactions that could be encountered and detected by pharmacist through a computerized pharmacy system.	The panelists were physicians (n = 2), clinical pharmacists (n = 2), and one expert in drug-drug interactions (n = 1).	The literature was reviewed.	The final list consisted of 56 drug-drug interactions. Consensus was achieved to consider 25 drug-drug interactions as clinically important.
18	Puumalainen et al.	Development of a validated and easy to use patient counselling quality assurance tool for pharmacists	Two separate panels: practicing pharmacists (n = 10) and academic and professional experts (n = 10)	The panelists developed indicators for the tool. The Delphi technique was followed among the panelists to develop the final tool.	The final tool contained 16 indicators grouped into 3 quality groups relevant to patient (n = 4), process (n = 6), and learning and innovations (n = 6).
19	Byrne et al. [58]	Developing core competencies in natural health products that future pharmacists should possess	The panelists (n = 17) were pharmacy educators, academic administrators, and representatives from Canadian pharmacy organizations. All panelists had interest in natural health products.	A list of potential competencies was compiled from previous qualitative and survey studies. A four-round Delphi technique was followed among the panelists to develop and achieve consensus on the final list.	The final list contained competencies grouped into 3 areas: knowledge of natural products when providing pharmaceutical care, access to and critical appraisal of information sources, and provision of appropriate patient education on effects, adverse reactions, and interactions of natural health products.
20	Bowie et al. [47]	Development and prioritization of a list of safety-critical issues to be addressed in the first period of general practice training	General practitioner educators (n = 127) and specialty trainees (n = 9).	Items and themes were generated and refined using a mixed method which included iterations in small group meetings, a modified Delphi technique, and interviews.	The final list contained 47 safety-critical issues organized under 14 themes: prescribing safely (n = 6), dealing with medical emergency (n = 4), specific clinical management (n = 1), dealing effectively with results of investigation requests (n = 2), patient referrals (n = 4), effective & safe communication (n = 3), consulting safely (n = 3), ensuring confidentiality (n = 2), awareness of the implications of poor record keeping (n = 5), raising awareness of personal responsibility (n = 4), dealing with child protection issues (n = 3), enhancing personal safety (n = 3), emphasizing the importance of the learning environment (n = 4), and safe use of practice computerized systems (n = 3).
21	Fernandez-Llamazares et al. [48]	Designing and achieving consensus on a pediatric pharmaceutical care model	A panel of experts (n = 50) from 20 different hospitals	Items were developed using an iterative process. A two-round Delphi technique was followed among the panelists to achieve consensus.	The final model contained 39 items grouped used in basic validation (n = 17), intermediate level (n = 13), and advanced level (n = 9).
22	Floor-Schreudering et al. [49]	Development of drug-drug interaction management guidelines to support healthcare professionals in clinical practice	A panel (n = 23) was voted in the Delphi rounds. The panelists included pharmacists, physicians, educators, and clinical pharmacologists.	The panelists expressed their views and opinions on a list of potential items relevant to management of drug-drug interactions.	The final list contained 15 elements in a standardized report which included quality of evidence for harm, level of evidence, pharmacological plausibility, seriousness, incidence of outcomes, clinical impact on the population, susceptibility factors, clinical impact on the patient, strength of recommendations, what to manage, when to start management, how to monitor, when to stop management, a set of communication tools, and a brief summary.
23	Tonna et al. [50]	Development of guidelines to facilitate service redesign around pharmacist prescribing	A panel (n = 35) included directors of pharmacy (n = 4), directors of hospital services (n = 3), chairmen of area drug and therapeutics committee (n = 4), nonpharmacist authors of local nonmedical prescribing policy (n = 5), pharmacist authors of local nonmedical prescribing policy (n = 10), and pharmacist prescribers (n = 15).	Statements were presented to the panelists in the two-round Delphi technique.	The final list contained 27 statements which were related to two domains: service development and pharmacist prescribing role development. Service development included succession planning (n = 8), multiprofessional working (n = 6), quality evaluation (n = 2), practice development (n = 2), and outcome measures (n = 1). Pharmacist prescribing role development included education (n = 7) and future orientation of service (n = 1).
24	Aljamal et al. [51]	Development and examination of appropriateness of indicators of medication reconciliation	A panel (n = 65) contained hospital pharmacists with pharmacy degree only (n = 4), postgraduate diploma (n = 34), master’s degree (n = 23), and other degrees (n = 4).	An initial list of indicators was presented to the panelists. Consensus was achieved in a two-round Delphi technique.	The final list contained 41 indicators grouped into collecting (n = 16), checking (n = 12), communicating (n = 7), and entire process (n = 6).
25	Satibi et al. [52]	Development of performance indicators to measure quality of pharmacy services	The panel (n = 15) included pharmacist practitioners at primary health centers (n = 12), representative of the regency health office (n = 3), and chairperson of the province health service quality agency (n = 1).	The literature was reviewed and an initial list was compiled. A group of pharmacists validated the list. A three-round Delphi technique followed by group meetings was conducted among the panelists to achieve consensus on the final list.	The final list contained 26 indicators of drug management, 19 indicators of clinical pharmacy services, and 2 indicators of overall pharmacy performance.
26	Rocha et al. [53]	Development and validation of a tool to support pharmaceutical counselling of patients with regard to medications	The panel (n = 29) included pharmacists with basic pharmacy degree (n = 2), specialization course (n = 13), and master’s or doctoral degree (n = 14).	Iterations, repeated meetings, and Delphi technique rounds were used to develop and validate the tool that can be used to support pharmaceutical counselling of patients with regard to medications.	The final tools contained 3 components: suggestions for questions, dispensing process reasoning, and suggestions for counselling.
Pharmaceutical services relevant to CAM
1	Im et al. [6]	Development of an evaluative scale to measure the effects of horticultural therapy in practical settings	Horticultural therapists (n = 779) answered open-end questionnaire. In-depth interviews were conducted with horticultural therapists (n = 20). Panelists (n = 24) participated in the Delphi technique.	Items collected from the interviews and the literature were presented to the panelists in the Delphi technique.	The final list of effects of horticultural therapy was categorized into 4 aspects: physical (n = 27 items), cognitive (n = 25 items), psychoemotional (n = 24 items), and social (n = 22).
2	van Overveld et al. [54]	Development of multidisciplinary quality indicators for measurement of quality of integrated oncological care	Two separate panels: medical specialists (n = 18) and allied health practitioners (n = 11)	Items collected from the interviews and the literature were presented to the panelists in the Delphi technique.	The final list contained structure, process, and outcome indicators. The list of medical specialists contained 5 outcome and 13 process indicators. The list of the allied health professionals contained 3 structure, 19 process, and 5 outcome indicators.
3	Shawahna et al. [55]	Development of a list of using harms and benefits of using fenugreek for breastfeeding women that need to be discussed during clinical consultations	Two separate panels of healthcare providers (n = 56) and breastfeeding women (n = 65)	Potential items were collected from the literature and interviews and presented to the panelists. The panelists rated the items in a Delphi technique.	The final list contained 34 items grouped into harms (n = 21) and benefits (n = 13).
4	Shawahna and Al-Atrash [56]	Development of a list of knowledge items that healthcare providers and CAM practitioners need to know on the benefits of exercise as a CAM modality in cancer	The panel (n = 65) included healthcare providers and CAM practitioners.	Items collected from the interviews and the literature were presented to the panelists in two-round Delphi technique.	The final list contained 45 items grouped into 6 categories: general items (n = 9), effects on the immune system (n = 16), anticancer treatment (n = 12), metastasis (n = 3), tumor metabolism (n = 3), and release of myokines (n = 2).
5	Guangyi et al. [59]	Development of a list of traditional Chinese medicine symptoms and signs for screening chronic low back pain	Panelists (n = 13) were experts in orthopedics, massage, and acupuncture.	Items collected from the interviews and the literature were presented to the panelists in the Delphi technique.	The final list contained 35 diagnostic characteristics grouped into pain characteristics (n = 8), associated factors (n = 11), and physical and tongue diagnostic expressions (n = 16).

CAM: complementary and alternative medicine.