Abstract
Objective. To compare the Business process management and the analytic hierarchy process as the tools of process performance assessment. Instruments and Methods. Case study of the attention process of rheumatology patients. Business process management and analytic hierarchy process were applied to assess the redesign of the attention process. The two methods were compared. The data were obtained through personal observations, an interview with a Colombian health insurer’s senior executive, and retrospective documentary analysis. Results. Both methods assessed the process redesign as an improvement. While Business process management made a qualitative evaluation, the analytic hierarchy process allowed for a quantitative approach. Conclusions. Business process management is helpful in process performance assessment because it offers a conceptual foundation. Analytic hierarchy process is a complement which makes the intuitions based on business process management rigorous.
1. Introduction
A business process (henceforth, a process) is a group of activities that create value for the client. Beginning with Taylor’s pioneering work [1], a process design is known to be a source of competitive advantage [2] and dynamic capabilities [3, 4]. The assessment of a process performance is a complex issue. Performance management is a fundamental part of business process management (BPM). However, the research on process performance measurement is inadequate [5]. The literature does not present effective tools to measure the nonfinancial aspects of the process performance, and, as a result, the performance measurement systems are not focused upon business processes [6].
Yen [7] makes a nonsystematic review of the process evaluation methods. The classic one is the BPM which suggests to analyze a process architecture, cycle time, cycle cost, and process participants [8]. BPM is good when the process assessment is limited to time and cost criteria. But this method has no metrics to evaluate a process architecture and participants and does not allow to integrate an opinion of the process client into the decision. In multicriteria situations the literature suggests using a balanced scorecard. This tool has two drawbacks. First, it lacks the method to integrate the four perspectives in one easy to interpret measure. Second, it needs a previous evaluation of the client value created as a result of the process redesign. Such evaluation may be expensive and time-consuming. The strength of the BPM-balanced scorecard approach is its good theoretical foundation in terms of the variables intended to measure the process performance. The weakness is the lack of operationalization of qualitative variables like process architecture.
An alternative to the BPM and scorecard approach is the analytic hierarchy process (AHP), which, according to Yen, can improve the process performance evaluation. AHP is used for a range of decision problems [9]. Sarkis reports a successful use of AHP in an environmental assessment of business processes [10]; Sharma and Bhagwat used AHP and balanced scorecard in performance evaluation of a supply chain in a range of firms [11]. According to this literature, the strong side of AHP is its quantitative nature and rigorous treatment of preferences. The weakness of AHP is its agnosticism regarding the selection of variables the researcher feeds into the model.
So we formulate this research question: which method, the classical BPM or the AHP, is better for process performance evaluation?
To answer this question, we use the results of a natural experiment conducted by a major Colombian health insurer, who redesigned his process of attention of rheumatologic patients. Rheumatologic diseases constitute a problem of individual health and an economic problem. It is imperative to improve the health status of the population and to control the costs of care. In Colombian health system the insurance companies face this task. These companies sell the policies to the persons. Every citizen has the obligation to be affiliated with some health insurance company. The insurer negotiates the medical services with inpatient and outpatient medical providers and offers their services to the affiliates. The insurers are balancing between two rival goals. Their long-term success is to keep its affiliates healthy, to practice prevention, and to authorize medical services without restriction. However, their short-term success consists in restricting the use of medical services. If the insurer does not resolve these two tasks, he risks to increase the burden of disease and to lose the participation in the market.
In order to respond to the aforementioned dilemma, the insurer redesigned the care process of rheumatologic patients. The ex post version of the process allowed the insurer to improve a range of economic indicators. However, when the ex post process was analyzed from a broader perspective that included hospitalizations, doubts arose about the effectiveness of the redesign because the hospitalization rate increased by 49,8%. With an adequate management of the patients, the hospitalizations should have decreased. The second research was formulated: which version of the process, ex ante or ex post, is better for the insurer?
This paper makes two contributions. First, we show a way to improve the care of patients with rheumatoid arthritis. Second, we make a comparative analysis of BPM and AHP as process performance evaluation tools.
2. Method
We used an exploratory case study method [13] to compare BPM and AHP as process evaluation tools. To analyze the data and to make comparative analysis, we used quantitative and qualitative techniques.
The data were obtained from the insurer in an aggregated anonymized presentation, so there was no need to request informed consent from the patients. The observations of 2011 correspond to a care process before the redesign, and the observations of 2012 to 2015, to a redesigned process.
The research project on the forms of contracting in health supply chain was approved by the research and ethics committee of the institution of the authors and by the insurer.
We began describing the attention process ex ante and ex post. We applied the BPM method to evaluate the change in process performance. We identified the decision criteria for process assessment: process architecture, time, cost, and responsibilities of the participants in the process before and after the redesign. We managed to express the cost, time, and participants in a quantitative way. We could not find a metric to measure the structure of the process. Nevertheless, we managed to make an overall evaluation of the change in process performance.
Then we applied the AHP method to evaluate the change in process performance. We used the same variables of time, cost, and participants and, instead of the process architecture, included the outpatient and inpatient consultation rate. We conducted an interview with a senior executive of the insurer and asked him to express his preferences as to the decision criteria: time, cost, participants, inpatient, and outpatient consultation rate. A matrix was created according to Saaty's fundamental scale, which used nine levels of preference for each pair of criteria (Table 1) [12].
We used the data from the matrix to calculate a geometric mean as a proxy to eigenvector. The eigenvector is the representation of the preferences as to the above-mentioned criteria [14]. Then we calculated the eigenvalue, λmax of the matrix, in order to use it for calculating the consistency index (CI) and the consistency ratio (CR). CI is formulated by Saaty [12] as
It is a product of simulations with different sizes () of random matrices. CI adopts the value of zero when the preferences are totally consistent. CI is used to calculate CR.Mean random CI is taken from Saaty [12].
As the best value of CI is zero, the same happens with CR. When the preferences are totally consistent, CR adopts the value of zero; when they are inconsistent, CR adopts the value of one. Thus, the interpretation of CI and CR is similar, but CR is more intuitive because it is restricted to zero and one.
We used a mathematics website http://comnuan.com/cmnn01002/ to calculate eigenvalues.
Once the preferences toward the decision criteria were analyzed, we turned to study how comfortable the insurer felt with the two versions of the care process. We asked the insurer’s officer to state his attitude to both editions of the care process, bearing in mind only the cost as a criterion. The above-mentioned fundamental scale matrix was used to gather the data. Then we asked the officer to do the same, bearing in mind the criteria of time, participants, inpatient, and outpatient consultation rate. The five matrices, one for each criterion, resulted. We calculated eigenvectors, max, CI, and CR for both versions of the care process.
Once we understood the officer’s preferences toward the decision criteria and toward both versions of the process in the light of five criteria, we turned to the global evaluation of process performance.
We assembled a new matrix which combined the mathematical expressions of the preferences as to the criteria (Pc) with the preferences toward both versions of the care process (Pp). Then we multiplied Pp × Pc and summed the rows. The sum of the rows is the global weight of the preferences of the insurer as to both versions of the care process.
Finally, we made a comparative analysis of two methods.
3. Results
The ex ante version of the process was beginning when an affiliate felt discomfort and was seeking medical help. Usually, the affiliate requested an appointment with a general practitioner of an outpatient clinic, owned by the insurer, in which the person was georeferenced. The general practitioner determined whether the case merited consultation with a rheumatologist and, if positive, referred the patient to a hospital specialist. While the outpatient clinic is owned by the insurer, the hospital is an independent provider.
When the patient arrived at the rheumatologist consulting room in a hospital, the rheumatologist would make the diagnosis, formulate the prescription, and explain the treatment plan for the patient. The treatment started with the disease-modifying antirheumatic drugs that can be managed at home. When these medicines ceased to work, the rheumatologist switched to biotechnological medicines which required some special conditions and were applied in outpatient clinics inside the network of the insurer. Once the treatment was started, the rheumatologist continued to quote the patient to periodic checks. The checks are used to monitor health status of the patient and to renew the medical formula. Since rheumatoid arthritis has no definitive cure, appointments with the rheumatologist became cyclical and permanent.
After the consultation with the general practitioner of the outpatient clinic, the patient left the network controlled by the insurer and entered a hospital, where the insurer could not influence the care model. The lack of control manifested itself in several ways. The insurer had to tolerate that patients with related diseases, such as osteoarthrosis, were often referred to rheumatologist instead of physiatrist and orthopedist. The incorrect referrals occurred due to mistakes of general practitioners. But rheumatologists, far from streamlining the situation, accepted to attend the patients with osteoarthrosis. While the recommended frequency of visits is one in three months, rheumatologists quoted patients with a higher frequency. This behavior of the rheumatologists contributed to saturating the rheumatology service and pushed the waiting time to 60 days. The delay was generating dissatisfaction among the insurer’s patients. In addition, a rheumatologist appointment costs 25% more than one with a physiatrist or orthopedist. Another manifestation of lack of control over the model was the separate medical records which existed for the same patient in the outpatient clinic and the hospital.
Based on the analysis of the process, the insurer redesigned it. The beginning of the process did not change. The differences begin when the general practitioner decides to refer the patient to a rheumatologist. Before the redesign, the key role in the process was performed by the hospitals, where rheumatologists worked. This was questionable because the rheumatologic patients are hospitalized only in exceptional cases. 18 part-time rheumatologists participated in the process, which made it difficult for the insurer to implement any proprietary model of care. After the redesign, the patient, instead of a hospital, is referred to a special outpatient rheumatologic provider (In Spanish: Punto de desinstitucionalización, PDI) created by the insurer, where three full-time rheumatologists serve exclusively the patients of the insurer with the clinical diagnosis of autoimmune disease. The PDI is not owned by the insurer but is in a strategic alliance with him, so the patient remains in the insurer's network. Thanks to this, the possibility for an osteoarthrosis patient to be attended by a rheumatologist was eliminated. The PDI specialist, when acknowledging the mistake of the general practitioner, refers the patient to the physiatrist. After the redesign, PDI specialists schedule follow-up appointments within the three-month recommended interval. To monitor the compliance, the insurer introduced a concentration indicator that shows the average times a patient is attended during three months. To further clear the rheumatology service, the insurer introduced the role of an experienced physician with some training in the rheumatology. He replaces the rheumatologist when the only thing the patient needs is to extend the prescription of the same medicine. As a result, the appointment waiting time was reduced from 60 to 30 days.
The insurer got access to the full medical records of its patients because the PDI uses the same medical records as the outpatient clinics.
The BPM analysis showed that the only BPM variable which clearly improved was the waiting time.
Apparently, the process architecture also improved, but we could not be sure due to a qualitative nature of the variable. The variable of participants has improved because the reduced number of the specialists allowed for better coordination, but we could not express the improvement numerically. The cost (unitary cost of a visit to a rheumatologist) showed a slight increase. Intuitively, the redesigned process performed better, but there was no solid ground to state this (Table 2).
We began the AHP assessing the weights of decision criteria. Based on the preferences reported in Table 1, we calculated the geometric mean as a proxy to the eigenvector and normalized it. The normalized values represent the weight every criterion has in the total decision. The most important criterion for the insurer was the cost, followed by the rate of inpatient and outpatient consultation, the number of rheumatologists, and waiting time for appointments (Table 3). The model did not allow us to introduce the patients’ opinion directly. But the interests of patients were taken into considerations by the insurer when he formulated his preferences.
We calculated the eigenvalue, CI, and CR. CI and CR are close to zero; this indicates a good consistence of preferences.
Then we studied the preferences of the insurer toward both versions of the care process for every one of the five criteria. Five individual matrices resulted (Table 4). In all five matrices the insurer preferred the redesigned version of the process. In all the cases λmax = 2; CI = CR = 0.
Finally, the eigenvectors calculated for both versions of the care process (Table 4) were combined with the eigenvector of preferences as to the four criteria (Table 3). The weight of every version of the care process in the light of every criterion was multiplied by the weight of every criterion. The rows were summed up to produce the global priority for both versions of the care process. Table 5 shows that the redesigned version is preferred to the first version of the process.
Both methods (BPM and AHP) delivered the same result: the redesign has improved the process for the insurer. While the BPM method was largely intuitive, the AHP appeared like a rigorous one.
4. Discussion
The ex post version of the care process offers a better overall performance than the ex ante version. This confirms the prediction of the transaction costs theory: vertical integration and the hybrid governance mode (strategic alliance) can reduce the transaction costs present in the market, when the object of the transaction is complex and the participants have incentive to behave opportunistically. This result is interesting for the Colombian health system that restricts the vertical integration for fear of reducing the autonomy of the patients when they make a choice of a service provider.
BPM and AHP delivered similar results in our case. That was not unexpected. In the Colombian health system, the insurer is focused on economic results, while the health improvement of the population is a secondary concern. This may explain why AHP, which has a broader perspective, produced the same result as BPM, narrowly focused on cost minimization.
Nevertheless, the two methods are quite different. BPM is conceptually strong because it represents a conceptual model of a generic business process from the process owner’s perspective. AHP has nothing similar. The decision maker must build a conceptual model of the process before using AHP. In our case, we used the BPM variables to build the AHP model.
But this conceptual advantage of BPM is offset by the qualitative treatment of BPM variables. It allows for a quantitative evaluation of cycle cost and time. But BPM offers no mechanism to bring all the metrics to a common denominator. A decision maker, who uses BPM, is unable to feed into the model her feelings about the weights the BPM variables have in the overall decision. AHP is strong where BPM is weak. AHP allowed us to allocate weights to decision criteria and, later, to both versions of the care process.
No method takes into consideration the patients’ concerns. BPM is focused on process efficiency and does not allow for clients’ opinion. AHP is more flexible and, when the decision maker formulates her preferences, can take the patient into consideration, but not explicitly. The inclusion of balanced scorecard, reported by Sharma and Bhagwat [11] can improve the situation because this tool has a client perspective. This balanced scorecard capacity is valuable for health care where technological and administrative interventions affect patients and should not be evaluated solely from a cost perspective.
The BPM, focused on efficiency, may be complemented with balanced scorecard, which allows for the clients’ opinion. This combination will not change the qualitative nature of BPM. But the combination of BPM with AHP may constitute an interesting tool for process performance evaluation.
This work has a limitation. The insurer detected an increase in the rate of inpatient consultations in 2011–2015. However, it is not known whether this increase is due to the change in the care process or a higher disease burden among new affiliates who joined the insurer after 2011. A future study should isolate that possible bias.
Conflicts of Interest
The authors declare that there are no conflicts of interest regarding the publication of this paper.