Strategic Improvement for Quality and Satisfaction of Hospital Information Systems

Kuo, Kuang-Ming; Liu, Chung-Feng; Talley, Paul C.; Pan, Su-Ya

doi:https://doi.org/10.1155/2018/3689618

Journal of Healthcare Engineering

On this page

Abstract Introduction Materials and Methods Results Discussion Conclusions Abbreviations Data Availability Conflicts of Interest Acknowledgments References Copyright Related Articles

Research Article | Open Access

Volume 2018 | Article ID 3689618 | https://doi.org/10.1155/2018/3689618

Strategic Improvement for Quality and Satisfaction of Hospital Information Systems

Kuang-Ming Kuo,¹Chung-Feng Liu,²Paul C. Talley,³and Su-Ya Pan⁴

Academic Editor: Ioannis G. Tollis

Received14 May 2018

Accepted09 Aug 2018

Published12 Sept 2018

Abstract

The purpose of our study aimed to identify attributes capable of improving physicians’ satisfaction levels with the use of a hospital information system (HIS). A model inclusive of system quality, information quality, and service quality related to an HIS is used to form antecedents of user satisfaction. Survey methodology was used to collect an attributive set representing the system quality, information quality, and service quality made available from 150 physicians at a large health-care system in southern Taiwan. Responses were segmented into low and high satisfaction and analyzed with partial least squares and importance-performance analysis. The results reveal that system quality, information quality, and service quality may be used to significantly predict physicians’ satisfaction. Two system quality attributes (reliability and response time) were identified as the highest priorities for intervention by low- and high-satisfaction users. Low-satisfaction users further expect improvement of the HIS service quality to take place. The subject health-care system should produce coping interventions for those high priorities to enhance the satisfaction of physicians.

1. Introduction

With the rising cost pressure and the increasing demand for improved health-care quality, the larger number of health-care facilities has forced the introduction of health information technologies (HIT) to resolve myriad problems that have arisen. By leveraging HIT with an integration of hospital functions and processes, hospital information systems (HIS) are frequently adopted among most health-care facilities to meet administrative requisites. A hospital’s goal of adopting HIS is expected to potentially fulfill the strategic goal of improving overall patient-care quality and reducing costs [1, 2]. Furthermore, HIS usually requires heavy capital investments with respect to both staff and information systems processing [1]. However, literature [3] from the information systems (IS) field has clearly pointed out that IS cannot improve organizational performance unless it is fully utilized. The question invariably is how to go about increasing HIS use which is an important issue for health-care facilities choosing HIS. The adoptive users of HIS, who are best seen as an integral part of an HIS [1], therefore play a pivotal role on whether or not an HIS can ever achieve the expected goals of improving overall patient-care quality and reducing costs.

Prior evidence has found that user satisfaction has a direct association with IS usage intentions [4]. Earlier studies [5] also found that the users’ level of satisfaction was positively related with their intention to continue the use of an IS. It is therefore entirely reasonable to assume that if users are more satisfied with HIS, the greater the likelihood there will be for them to integrate use of HIS for future patient care. Hence, it is strategically crucial for administrative managers to comprehend and to identify the primary HIS attributes perceived by users as becoming important, and to scrutinize how users perceive the given performance of those attributes once adoption is implemented.

The purpose of our study is therefore to determine the most important attributes of HIS related to perceived physicians’ satisfaction since they are the primary users of HIS, and they are heavily reliant on HIS to affect efficient patient diagnosis and meaningful treatment. By identifying the priorities of improvements for those given attributes, hospital administrators can plan and then take appropriate steps for intervention to occur that will improve HIS in order to fulfill physicians’ clinical requirements for providing better health-care quality.

1.1. Hospital Information Systems

According to the literature [1], a hospital information system (HIS) is defined as “the socio-technical subsystem of a hospital, which comprises all information processing as well as the associated human or technical actors in their respective processing roles” (p.30). HIS can also be defined as an integrated information system that supports the various information requirements of clinical services and hospital management [6]. Therefore, HIS ranges from a simple system, such as transaction processing systems, all the way up to complicated systems, such as clinical decision support systems [7]. More specifically, HIS may include various and diverse types of health-care IS [6, 7], such as clinical information systems, radiology information systems, laboratory information systems, pharmacy information systems, clinical decision support systems, nursing documentation systems, computerized physician order entry, patient-centered information systems, or administrative information systems, etc. No matter what type of HIS is in use, the primary aim of HIS is to contribute to better quality and efficient patient care [1]. In the beginning, HIS was aimed at supporting the regular fiscal operations and administrative aspects of a given hospital [2]. Nowadays, health-care facilities leverage their HIS mainly to focus on providing better possibilities for patient care and its management to take place, in addition to optimizing operational standards [2]. And, HIS has become an indispensable part of the diagnostic and treatment processes of health-care professionals [7].

Among those different types of HIS, computerized physician order entry (CPOE), just one type of HIS primarily used by physicians, is gaining popularity among health-care facilities. Via CPOE, physicians can acquire clinical decision support such as drug-drug interaction and drug-allergy checks during the relative diagnosis and treatment of patients [7]. Without these decision-support capabilities, patient safety can be compromised if physicians misunderstand or neglect such important sources of patient information. The importance of HIS for improving health-care quality is thus most evident. Therefore, the extent to which HIS fulfills its role and supports health-care services cannot be overemphasized.

Many studies have been conducted regarding HIS, and these studies have lain special focus on how to encourage HIS adoption [8]. The findings surely have advanced our knowledge toward what factors motivate health-care professionals for adopting an HIS. However, if the use of HIS is mandatory, the method of how to increase their usage is an additionally important issue that should be focused upon. Furthermore, since there are many types of HIS, with each possessing differing characteristics and users, it may be better to focus on specific HIS and possible users to better understand those determinants that can improve HIS use by its end users.

1.2. Information Systems Success Model

To assess the success of an IS, DeLone and McLean [9] introduced a taxonomy comprising six measures of IS success by reviewing 180 related studies. They proposed an IS success classification framework inclusive of system quality, information quality, use, user satisfaction, individual impact, and organizational impact. These dimensions are interrelated both causally and temporally. As shown in Figure 1, system quality and information quality together impact both use and user satisfaction which, in their turns, are precursors of individual impact. Finally, individual impact affects organizational impact. DeLone and McLean, however, do not empirically assess their model but suggest further refinement and corroboration of the model they proposed [9]. Numerous studies have extensively tested this model [10–15]. According to the previous literature, DeLone and McLean [16] included service quality as a new dimension to update their originally proposed model for better measuring IS success, further consolidating all the “impact” measures into one construct labeled as “net benefits” (Figure 1).

Prior literature has adopted the information systems success model (ISSM) for assessing the success of an IS including those used in the health-care field [11–13, 17]. The subjects of these studies included health-care professionals such as nursing staff or a mix of physicians, nurses, and other health-care professionals. To our knowledge, studies that mainly focused on the exclusive perceptions of physicians are scarce. These studies generally support the ISSM [9, 16]. However, in a meta-analysis of ISSM [14], the relationship between service quality and user satisfaction was not supported. One of the potential reasons that Petter and McLean [14] proposed for this is the possible consideration of population choice. It is therefore necessary to focus on a specific group of HIS users in order to gain a deeper knowledge of those associations observable among the constructs proposed by ISSM.

1.3. Importance-Performance Analysis

Stemming from the marketing discipline, the importance-performance analysis (IPA) technique has often been utilized for strategy formulation, leading to service improvement in different fields [12, 18, 19]. The IPA analyzes quality attributes on the basis of performance and importance dimensions. These two dimensions are then integrated into a matrix with four quadrants: (1) “Concentrate here,” (2) “Keep up the good work,” (3) “Low priority,” and (4) “Possible overkill.” Respondents are asked to indicate the relative attribute’s importance and performance merits. And, the mean values of importance and performance ratings are often used to divide the quadrants. The ratings of importance and performance are then plotted on the importance-performance grid (Figure 2). For example, if customers feel an attribute is important but its performance is low, then this attribute falls on the “I. Concentrate here” quadrant. Organizations would thus be best served to pay immediate attention to improve this desired attribute.

The original IPA is easy to implement for discovering strategies, leading to improved organizational services quality. The IPA, however, may suffer from the problem of ceiling effects which may inflate the importance ratings of most attributes [18] since the respondents have to rate importance and performance simultaneously [19]. In order to resolve this issue, some researchers adopted statistical methodologies such as correlation analysis [20], linear regression [21], and partial least squares [22] to derive relative importance.

In the health-care field, Cohen et al. [12] used IPA to assess important HIS attributes from a nursing perspective. They adopted partial least squares to derive the most important HIS attributes as high priorities for purposes of intervention. The findings of Cohen et al. [12] have absolutely added to the knowledge of the applicability of IPA, especially in health-care context. We, however, consider that if the perspectives of physicians, another key HIS user group, responsible for the diagnosis and treatment of patients should also be investigated, a more holistic view of the applicability of IPA in health-care context may then be acquired. Furthermore, healthcare facilities can take suitable interventions to improve HIS, thus achieving better patient care.

1.4. Conceptual Background

The ISSM, proposed by DeLone and McLean [9, 16], provides a useful framework for any understanding of the influence IS attributes have on user satisfaction. ISSM articulates that system quality, information quality, and service quality are important dimensions for users to evaluate the success of IS [9, 16]. The ISSM has been applied to various fields for IS evaluation, and it has been found applicable for health-care professionals [11–13, 17], including physicians [11].

To determine the most important attributes of HIS related to a physician’s satisfaction, we proposed a research framework based on the ISSM. As shown in Figure 3, the framework demonstrates that system quality of, information quality of, and service quality of an HIS are all expected to influence physicians’ satisfaction related to HIS. And, system quality, information quality, and service quality are composed of their respective attributes.

1.4.1. User Satisfaction

User satisfaction, referring to the affective response or attitude of physician users toward HIS, is considered an imperative indication of IS success [9, 16]. Lower levels of satisfaction, implying HIS may not meet the demands of physician users, can result in low or nonuse of HIS by physician users. In a health-care context, such inefficiencies in HIS usage may disturb the delivery of care and increase the possibility of new errors that may certainly jeopardize the quality and safety of patient health care [23]. On the other hand, an HIS that physician users are satisfied with can improve decision-making quality, performance, productivity, and effectiveness of patient-care tasks as the evidence illustrated [13, 17, 24].

According to the IS success model [9, 16] and other evidence [13, 17, 25], HIS attributes including system quality, information quality, and service quality may prove to be important antecedents of user satisfaction.

1.4.2. System Quality

System quality measures desired technical characteristics such as the reliability, response time, and functionality of an HIS [9, 16]. These attributes have been confirmed to be important predictors for health-care professionals to use HIT applications [13, 17, 26, 27]. An unreliable HIS may lead to several unintended consequences such as additional workload and negative emotional states for the health-care professionals involved [28, 29].

1.4.3. Information Quality

Information quality relates to the characteristics of the information output derived, such as sufficient detail, easy-to-read perception, and the completeness offered by an HIS [9, 16]. Previous studies have proved that health-care professionals can improve their ability to make better diagnoses, treatment plans, and patient-care provisions by acquiring patient information with the abovementioned attributes [13, 17]. By perceiving these tangible benefits, they can also have a better view of HIT [13]. Based on this finding, information quality should be seen as an important determinant of physician users’ satisfaction on HIS.

1.4.4. Service Quality

Service quality concentrates on the level of support, such as availability, responsiveness, and training opportunities, of physician users by the IS department [9, 16]. The marketing literature [30] has found strong empirical notice that improved service quality can lead to consumers’ behavioral intention. In the same vein, treating physician users as internal customers and providing them with appropriate support and training should attract them to use an HIS even when and if they encounter perceived problems. Previous evidence also demonstrates the association between service quality and HIT implementation and adoption [31].

Evidence [14] found that ISSM has been less validated as a whole in the form of a single study. Furthermore, the use of an HIS in the subject hospitals is mandatory; our study, therefore, does not incorporate the use construct but only constructs that are inclusive of system quality, information quality, service quality, and satisfaction as found in the proposed model. According to a meta-analysis of ISSM [14], system quality, information quality, and service all have a significant relationship with user satisfaction. The strength of these relationships is ranked as follows: system quality, information quality, and service quality. Our study adapted the operational definitions of previous ISSM [9, 16] and altered them for use in the HIS context as provided. The constructs used in our study and in their HIS-specific definition are demonstrated in Table 1.

2. Materials and Methods

2.1. Measures

Based on Churchill’s [32] suggestion for developing questionnaires, we derived survey items from prior validated surveys to establish an initial pool of survey items for each of the given constructs. An expert panel, including two experienced attending physicians employed by the subject health-care system and one health-care information management scholar, assessed the face and content validity of those proposed items. Suggestions offered by the panel were used to modify some ambiguous words in order to minimize plausible confusion during the survey’s subsequent administration.

The questionnaire utilized in our study consisted of two parts. The first part elicits physicians’ demographic information and the second section is designed to ascertain physicians’ perceptions regarding the four constructs to be explored (i.e., system quality, information quality, service quality, and satisfaction). Our investigated constructs were assessed by adopting existing instruments [10–12, 15, 33], and a seven-point Likert scale was used to measure the items (e.g., one for “strongly disagree,” four for “neither agree nor disagree,” and seven for “strongly agree”). Regarding the detailed sources of survey items, system quality was measured using three items based on Balaban et al. [10] and Xu et al. [33]. Information quality was measured using three items adapted from Bossen et al. [11] and Xu et al. [33]. Service quality was measured using four items in accordance with Balaban et al. [10] and Wang [15]. Satisfaction was measured using five items taken from Cohen et al. [12].

Following the suggestion of Straub [34], we conducted a pilot test to establish the scales via a sampling of 10 attending and resident physicians employed at the subject health-care system. Slight modifications of wordings were made to the given items, and Table 2 lists the revised final scale (Table 2) justified for further validation.

2.2. Sampling

A cross-sectional survey was conducted to obtain data from the physicians of a large health-care system located in southern Taiwan. The subject health-care system comprises three different scales of hospitals, which have nearly 1,600 beds and which employ about 190 full-time physicians. The study chose the subject health-care system for two primary reasons: (1) The subject hospitals provide nearly all necessary/comprehensive medical services to their respective communities, which attracts an average of 7,000 outpatients each day, and (2) the subject health-care system has adopted the same HIS, with slightly different functionalities according to the requirements of respective hospital, since 2004. Since patients may need to be transferred among these three hospitals, an effective HIS is critical for exchanging patient information among physicians. With a growing demand from both administration and health-care perspectives, the functionalities of a current HIS may not conform to the demands of physicians, who are the primary users.

Ethical approval from the institutional review board of the respective large Taiwanese hospital was acquired prior to the commencement of the study (IRB#: EMRP-105-128). Prior to the distribution of the questionnaires, the researchers successfully contacted the relevant medical service departments to ensure their collaboration. We then designated a coordinator for those units of the medical service departments which were involved and who were both voluntarily and willing to assist with the dispatching and collection of the questionnaires per se. Altogether, 180 questionnaires were dispatched to those units. Physicians were recruited by our coordinator to anonymously and voluntarily participate in our survey. In total, 150 useful responses were returned, resulting in a response rate of 78.95% (150/180), for the purpose of later analysis.

2.3. Statistical Analysis

In addition to descriptive statistics, several quantitative procedures were involved for importance-performance assessment in our study. First, exploratory factor analysis with oblique rotation was employed to prove construct validity before conducting IPA as suggested by literature [18]. Next, partial least squares (PLS) was used to test the measurement model and structural model. The measurement model articulates the associations between the latent variables and the measured variables, whereas the structural model articulates the associations between the exogenous and endogenous latent variables [35]. Importance scores and performance scores, of both construct level and item level, were then derived from PLS results, respectively [22, 36].

In order to obtain different perspectives from a traditional utilization of IPA, a median of factor means based on original 7-point scales of satisfaction (4.2) was used to split the data into high-satisfaction and low-satisfaction groups. An independent sample t-test was conducted to compare satisfaction mean scores between low- and high-satisfaction groups of respondents, and a significant difference was noted between low satisfaction (M = 2.98, SD = 0.85) and high satisfaction (M = 4.99, SD = 0.69); t (130.74) = −15.65, . IPA was then employed to compare the perceptions of HIS attributes by physician users between high-satisfaction and low-satisfaction groups. The above statistical analysis were conducted by using R software [37] with the matrixpls package [38].

3. Results

3.1. Descriptive Statistics

Of the 150 useable responses, most respondents were male (79.33%), close to the national average gender ration of physicians [39]. Nearly 82% of the respondents were aged between 30 and 49 years . All respondents have at least a college or university education, respective Taiwanese regulations respective to becoming physicians. General surgery accounts for the primary group of respondents (38.46%) and 68% of respondents are attending physicians. And, 78% of the respondents have more than 4 years of working experience in this particular health-care system. All physicians are mandated to use HIS when caring for their patient workload, showing that the respondents are suitable for the purposes of our study. Furthermore, high-satisfaction users seem to be senior (M: 2.27 vs. 2.13), and have more work experience (M: 3.69 vs. 3.64) and HIS usage frequencies (M: 3.57 vs. 3.52) than low-satisfaction users. Detailed demographic information of the respondents is shown in Table 3.

3.2. Exploratory Factor Analysis

Table 4 shows the reliability and validity results of the constructs used in this study. The Cronbach’s α values range from 0.83 to 0.95, indicating sufficient reliability [40]. The Kaiser-Meyer-Olkin (KMO) measure verifies the sampling adequacy with KMO = 0.94 [40]. Bartlett’s test of sphericity, χ² (105) = 2120.27, , demonstrating correlations of items, are sufficient for principal components analysis [40]. With oblique rotation, four factors with eigenvalues of at least one were extracted. Convergent validity can be confirmed if the items load significantly on their respective factors, while discriminant validity can be verified if each of the items load higher on its posited factors than on other factors [40]. Table 4 demonstrates that all items have loadings of >0.45 on their posited factors and load higher on the posited factors than on others. Reliability and construct validity are thus determined to be adequate in our study.

3.3. Partial Least Squares Modeling Results

In accordance with the suggested procedures for conducting PLS [35], we adopted a two-stage process, measurement model, and structural model, for assessing the PLS model. Furthermore, based on suggestions made in prior IPA literature [12, 36], HIS attribute and satisfaction scores were first rescaled from 0 to 100 by using the equation proposed by Anderson and Fornell [41] before conducting PLS analysis. Furthermore, in order to compare the perceptions of differing satisfaction level, the following data were analyzed with full, high-satisfaction, and low-satisfaction datasets, respectively.

3.3.1. Measurement Model

Regarding the evaluation of measurement model, reliability, convergent validity, and discriminant validity are often used to assess the measurement model of PLS per the suggested procedures by PLS guiding literature [35, 42]. We used composite reliability (CR) to evaluate the reliability [35, 42]. As shown in Table 5, the CR of four constructs in our study was larger than the suggested criterion of 0.7 [35]. As for convergent validity, the average variance extracted (AVE) of the constructs in our proposed model was larger than the threshold of 0.5 [42]. Furthermore, the matrix of inter-construct correlations (Table 6) demonstrated that the square root of AVE for each construct was higher than the association of the specific construct with any other constructs in our proposed model, thus revealing sufficient discriminant validity [42]. From the results, our study showed sufficient reliability and validity for the constructs to be investigated.

3.3.2. Structural Model

We evaluated the results of the structural model by assessing the size and significance of path coefficients (β), coefficient of determination (R²), the strength of each predictor variable in explaining endogenous variables (f²), predictive relevance (Q²), and the relative impact of the predictive relevance (q²) as per the suggestions of PLS literature [35]. First of all, we adopted a bootstrapping procedure to assess the structural model for testing the significance of each path coefficient. Figure 4 shows the results of structural model with path coefficient and probability values ( values). For the full-sample group, system quality predicted physicians’ satisfaction on HIS significantly and positively (β = 0.37, ), with a 95% bias-corrected and accelerated bootstrap confidence interval (BCa CI) of [0.11, 0.51]. Furthermore, both information quality (β = 0.26, , 95%) with a BCa CI of [0.08, 0.40] and service quality (β = 0.32, ) with a 95% BCa CI of [0.17, 0.66] also predicted satisfaction significantly and in an expected direction. In addition to the link between information quality and satisfaction in the low-satisfaction group and the link between service quality and satisfaction in the high-satisfaction group, all other paths between system quality→satisfaction, information quality→satisfaction, and service quality→satisfaction reveal significant and positive relationships.

We then assessed effect sizes including R², f², Q², and q². For the coefficient of determination (R²), the model explains about 74% of the determined variance in the physician users’ satisfaction regarding HIS usage. Furthermore, system quality, information quality, and service quality have f² effect sizes of 0.21, 0.12, and 0.14 for explaining satisfaction, respectively. These figures represent medium-large, small-medium, and small-medium effect sizes for system quality, information quality, and service quality, respectively [43]. As for predictive relevance, the Q² value of satisfaction (=0.53) is higher than zero, revealing that our model possesses sufficient predictive relevance for satisfaction construct [35]. The respective q² effect sizes for system quality, information quality, and service quality are 0.04, 0.10, and 0.08, indicating small to medium relative predictive relevance [43]. Table 7 depicts the summary of path coefficients and 95% bias-corrected and accelerated bootstrap confidence intervals for three groups of data.

3.4. Performance Indices and Prioritization of Improvement Areas

Following the suggestions of IPA literature [12, 18], PLS path coefficients were used as construct-level importance scores, whereas item-level importance scores were derived by multiplying each HIS attribute’s outer weights with relevant PLS path coefficients. Regarding construct level, performance scores were estimated as a weighted average of the attributes items, rescaled from the original 7-point scale into 0-to-100 point [41], whereas the item-level performance scores were simply calculated as the mean of rescaled scores of HIS attributes [36]. Crosshairs, using the mean values of importance and performance scores, were calculated to separate the latent variables and HIS attributes into four quadrants, namely, “I.Concentrate here,” “II. Keep up the good work,” “III. Low priority,” and “IV. Possible overkill.”

3.4.1. Construct-Level IPA Results

By first looking at construct-level IPA results, Table 8 clearly demonstrates that the performance of system quality and service quality for all three groups is below the mean performance index. Such results indicate that the system quality and service quality of an HIS have evident improvement potential for the subject health-care system. On the other hand, the information quality of an HIS is higher than the mean performance index for all three groups of data. Data from Table 8 were then transferred into the IPA grid representation. Figures 5(a)–5(c) depict the results of the IPA grids for the full-sample, low-satisfaction, and high-satisfaction data groups, respectively.

(a)

(b)

(c)

As shown in Figures 5(a) and 5(b), the full-sample and low-satisfaction groups of data have the same pattern of IPA grids. System quality and service quality were identified in the “Concentrate here” quadrant, whereas information quality was identified in the “Possible overkill” quadrant. Furthermore, Figures 5(b) and 5(c) show that low- and high-satisfaction group of respondents only had similar perceptions toward system quality, which was identified in the “Concentrate here” quadrant. Information quality was identified in the “Possible overkill” and “Keep up the good work” quadrants by low-satisfaction and high-satisfaction groups, respectively. Finally, service quality was identified in the “Concentrate here” and “Low priority” quadrants by low- and high-satisfaction groups, respectively. In order to identify those areas for interventions that are capable of improving overall system quality, information quality, and service quality of an HIS, the next step is to interpret the impact of all attributes for these three factors.

3.4.2. Attribute-Level IPA Results

Figures 6(a)–6(c) show the IPA grids of system quality, information quality, and service quality of an HIS from an attribute level for full-sample, low-satisfaction, and high-satisfaction groups of data, respectively. For the full-sample group, two system quality attributes, namely, SQ1 and SQ2, fall into the “Concentrate here” quadrant and one system quality attribute, namely, SQ3, falls into the “Keep up the good work” quadrant. The performance of SQ1 and SQ2 should therefore be improved, given their importance to engendering or maintaining user satisfaction. Furthermore, all three attributes of information quality fall into the “Possible overkill” quadrant, while all four attributes of service quality fall into the “Low priority” quadrant. It may indicate that the health-care system should consider reducing the organizational resources devoted to HIS information quality and service quality since physicians regard these two qualities as low priority or “Possible overkill” ones for improving HIS satisfaction. However, information quality and service quality are still considered to be critical determinants for HIS satisfaction, which was confirmed by means of the PLS analysis.

(a)

(b)

(c)

For low-satisfaction data, all three system quality attributes fall into the “Concentrate here” quadrant, while all three information quality attributes fall into the “Possible overkill” quadrant as with the full sample. Regarding service quality, two attributes, namely, SEQ3 and SEQ4, fall into the “Low priority” quadrant, while SEQ1 and SEQ2 fall into the “Concentrate here” and “Possible overkill” quadrant, respectively.

As for high-satisfaction data, two system quality attributes, namely, SQ1 and SQ2, were identified in the “Concentrate here” quadrant, while SQ3 was identified in the “Keep up the good work” quadrant. Regarding service quality, attributes including SEQ1, SEQ2, and SEQ3 fall into the “Low priority” quadrant, while SEQ4 falls into the “Possible overkill” quadrant. Item-level importance and performance scores are shown in Table 9.

3.4.3. Comparing Construct-Level and Item-Level IPA Results

Compared with the results of the construct-level IPA, one interesting point may be noted. The identified quadrant of services quality factor (“Concentrate here”) in Figure 5(a) is not the same as that of the identified quadrant, namely (“Low priority”), of service quality for the full sample in Figure 6(a). If we look further at the IPA grids of Figure 6(b) for the low-satisfaction group, we can find that one attribute of service quality, namely, SEQ1, did fall into the “Concentrate here” quadrant, while two other attributes of service quality (i.e., SEQ3 and SEQ4) fall into “Low priority” quadrant and one attribute, namely, SEQ2, falls into the “Possible overkill” quadrant. SEQ1 is clearly an important priority for intervention to enhance physician-user satisfaction according to such findings. It may imply that physicians concern whether services provided by the IT department are sufficient; however, physicians can figure out how to intuitively use common functions of their own HIS by themselves.

Another interesting point should be noted regarding the differing perspectives on information quality by low-satisfaction and high-satisfaction groups. For low-satisfaction users, information quality was identified in the “Possible overkill” quadrant, while information quality was identified in the “Keep up the good work” quadrant by high-satisfaction users. Based on the demographic information, high-satisfaction users tend to have higher experience levels.

4. Discussion

Based on the results, system quality, information quality, and service quality significantly predict the satisfaction level of users of an HIS. The importance of and performance of HIS attributes can be derived from those results.

The construct-level IPA results shown above suggest that special attention should be devoted to system quality factors, regardless of differing satisfaction status toward the HIS by physicians. Respective physicians usually need to make critical and near-immediate decisions about patient treatment plans, so an unreliable or unresponsive HIS is certainly unacceptable [11]. Furthermore, integration of various functionalities such as diagnosing, medication, and lab-test ordering is also essential for increasing system quality, leading to physician satisfaction. Therefore, without regular, positive experiences with an HIS and its operational reliability, response time, and sufficient functionalities, the availability of an HIS will still be incapable of satisfying physician users.

As for information quality, the health-care system is clearly performing well on this factor despite the considerations of low- and high-satisfaction users having differing attachments to such levels of importance. A plausible reason for such an incongruence may be that high-satisfaction users have more work and HIS usage experience than low-satisfaction users since they have greater relative understanding of how to access required information from an HIS. Since the adoption of an HIS in early 2004, the general health-care system has undergone numerous developments and modifications of HIS implementation. New information requirements of physicians have been proposed and implemented in HIS gradually, albeit junior physicians may not well understand all of those functionalities which provide differing patient information as required. They may instead consider those functionalities to be unnecessary and even that the health-care system places too many resources toward improvement of HIS information quality.

Furthermore, service quality is another important factor that cannot be ignored for low-satisfaction users. Despite high-satisfaction users regarding service quality factors to be of low importance, they still consider the health-care system to perform unsatisfactorily within this aspect. The health-care system should not overlook the implication of this finding. Low-satisfaction users tend to be less experienced, according to the demographic information, with HIS than high-satisfaction users; therefore, they may require more assistance with HIS usage. This concern may explain why service quality factor was identified in the “Concentrate here” quadrant by low-satisfaction physicians.

Based on item-level IPA results, the “Concentrate here” quadrant captured two system quality attributes, namely, SQ1 and SQ2, for both low- and high-satisfaction HIS users. These findings suggest that the subject health-care system should direct its special attention on the reliability (SQ1) and response time (SQ2) aspects of an HIS. Besides, SQ3 was identified in the “Concentrate here” and “Keep up the good work” quadrant by low-satisfaction and high-satisfaction physicians, respectively. A possible reason for such a differing perspective may be due to their differing HIS usage experiences, as mentioned in the previous section. In addition to having more experiences with HIS, high-satisfaction physicians may involve with the continuous development of HIS functions. They are therefore more knowledgeable about the available functions provided by HIS than low-satisfaction physicians.

The same rationale can be used to explain the differing perspectives of the identified quadrant of both information quality and service quality attributes by low- and high-satisfaction physicians. Regarding information quality attributes, IQ1, IQ2, and IQ3 fall into the “Possible overkill” quadrant for low-satisfaction physicians, while IQ1, IQ2, and IQ3 fall into the “Keep up the good work” quadrant for high-satisfaction physicians. With continued HIS development, increasing and differing angles of patient information can be used by physicians. Low-satisfaction physicians however may be confused by these differing patient-information retrieval approaches, and thus regard these attributes of information quality to be overdeveloped.

As for service quality attributes, low- and high-satisfaction physicians in general consider the factor to be of less importance when compared with system quality and information quality since physicians are professionals who may exhibit considerable competence in adapting to new technologies [44]. Such a competence in using an HIS can also explain why SEQ4 was identified in the “Possible overkill” quadrant by high-satisfaction physicians. SEQ1 (“insufficient support from IT department”) was however identified in the “Concentrate here” quadrant by low-satisfaction physicians. This finding may indicate less-experienced physicians still wanting to focus on their patient-caring jobs and not wanting to be interrupted by any HIS problems during their immediate care of patients.

5. Conclusions

Due to the rapid progress of information and communication technologies, developing and adopting an HIS has been one of the critical means of improving health-care quality for many health-care facilities. There are numerous studies focusing on the factors influencing physicians’ use of HIS from a varietal viewpoint of theories or models; however, exploring physicians’ behavior from the perspective of the differences between importance and performance is rarely discussed. Our study conducted an empirical investigation among physicians utilizing the IPA method in order to provide a successful attempt of explanation relevant to attributes.

Using IPA, our study has compared the importance and performance of factors contributing to HIS satisfaction, as perceived by low- and high-satisfaction users. The IPA grids have demonstrated that system quality and service quality fell into the “Concentrate here” quadrant and information quality into the “Possible overkill” quadrant from the perspective of high-satisfaction HIS users. As for low-satisfaction HIS users, system quality was found in the “Concentrate here” quadrant, service quality in the “Low priority” quadrant, and information quality in the “Keep up the good work” quadrant. Improving system quality of respective HIS is considered to be the most important integer toward evincing physician users’ satisfaction.

Via the use of IPA to explore the differences between the low- and high-satisfaction groups of HIS, physician users could add significant knowledge to further research studies in the area of IS success theory. Analyzing perceptions of quality in terms of differing segments can help hospitals to formulate developmental strategies of HIS to fulfill the demands of each specific segment. Based on these given findings, several implications can be derived. First, the health-care system may consider how to equip physicians with sufficient knowledge regarding the functions and information-retrieval approaches provided by their governing HIS. This implication is especially important for junior physicians; qualitative training sessions are thus a possible and a viable solution toward resolving this issue. Second, the IT department should provide sufficient support for physicians, especially for junior physicians interfacing the HIS, since patient care is the primary duty of physicians. If physicians can focus on patient care without other nonproductive interruptions, such as encountering HIS-usage problems, health-care quality can be substantially improved.

Some limitations should be noted in our study. First, our results are based on an HIS as implemented in a single health-care system in southern Taiwan; as a result, the generalizability may be somewhat limited. Furthermore, this study was conducted in a cross-sectional setting and might be unable to capture the varying perspectives of physicians over a period of time. Further efforts are thus necessary for the proposed model to be further implemented if longitudinal studies are to be conducted in the future.

Abbreviations

AVE:	Average variance extraction
BCa:	Bias-corrected and accelerated bootstrap
CI:	Confidence interval
CPOE:	Computerized physician order entry
CR:	Composite reliability
f2:	Strength of each predictor variable in explaining endogenous variables
HIS:	Hospital information systems
IPA:	Importance-performance analysis
IQ:	Information quality
ISSM:	Information system success model
KMO:	Kaiser-Meyer-Olkin
M:	Mean
:	probability values
PLS:	Partial least squares
Q²:	Predictive relevance
q²:	Relative impact of predictive relevance
R²:	Coefficient of determination
SAT:	Satisfaction
SD:	Standard deviation
SEQ:	Service quality
SQ:	System quality
t:	t statistics
β:	Path coefficients.

Data Availability

The data used to support the findings of this study are available from the corresponding author upon request.

Conflicts of Interest

No potential conflicts of interest were reported by the authors.

Acknowledgments

We would like to thank the E-Da Hospital of Taiwan for funding this research (Grant no. ISU-105-IUC-11).

References

R. Haux, A. Winter, E. Ammenwerth, and B. Brigl, Strategic Information Management in Hospitals: An Introduction to Hospital Information Systems, Springer Science & Business Media, New York, NY, USA, 2004.
P. L. Reichertz, “Hospital information systems–past, present, future,” International Journal of Medical Informatics, vol. 75, no. 3-4, pp. 282–299, 2006.
View at: Publisher Site | Google Scholar
F. D. Davis, R. P. Bagozzi, and P. R. Warshaw, “User acceptance of computer technology: a comparison of two theoretical models,” Management Science, vol. 35, no. 8, pp. 982–1003, 1989.
View at: Publisher Site | Google Scholar
S. Petter, W. DeLone, and E. McLean, “Measuring information systems success: models, dimensions, measures, and interrelationships,” European Journal of Information Systems, vol. 17, no. 3, pp. 236–263, 2008.
View at: Publisher Site | Google Scholar
A. Bhattacherjee, “Understanding information systems continuance: an expectation-confirmation model,” MIS Quarterly, vol. 25, no. 3, pp. 351–370, 2001.
View at: Publisher Site | Google Scholar
C. Liang, D. Gu, F. Tao, H. K. Jain, Y. Zhao, and B. Ding, “Influence of mechanism of patient-accessible hospital information system implementation on doctor–patient relationships: a service fairness perspective,” Information and Management, vol. 54, no. 1, pp. 57–72, 2017.
View at: Publisher Site | Google Scholar
M. M. Yusof, A. Papazafeiropoulou, R. J. Paul, and L. K. Stergioulas, “Investigating evaluation frameworks for health information systems,” International Journal of Medical Informatics, vol. 77, no. 6, pp. 377–385, 2008.
View at: Publisher Site | Google Scholar
P. W. Handayani, A. N. Hidayanto, A. A. Pinem, I. C. Hapsari, P. I. Sandhyaduhita, and I. Budi, “Acceptance model of a hospital information system,” International Journal of Medical Informatics, vol. 99, pp. 11–28, 2017.
View at: Publisher Site | Google Scholar
W. H. DeLone and E. R. McLean, “Information systems success: the quest for the dependent variable,” Information Systems Research, vol. 3, no. 1, pp. 60–95, 1992.
View at: Publisher Site | Google Scholar
I. Balaban, E. Mu, and B. Divjak, “Development of an electronic portfolio system success model: an information systems approach,” Computers and Education, vol. 60, no. 1, pp. 396–411, 2013.
View at: Publisher Site | Google Scholar
C. Bossen, L. G. Jensen, and F. W. Udsen, “Evaluation of a comprehensive ehr based on the Delone and Mclean model for is success: approach, results, and success factors,” International Journal of Medical Informatics, vol. 82, no. 10, pp. 940–953, 2013.
View at: Publisher Site | Google Scholar
J. F. Cohen, E. Coleman, and M. J. Kangethe, “An importance-performance analysis of hospital information system attributes: a nurses’ perspective,” International Journal of Medical Informatics, vol. 86, pp. 82–90, 2016.
View at: Publisher Site | Google Scholar
D. Garcia-Smith and J. A. Effken, “Development and initial evaluation of the clinical information systems success model (CISSM),” International Journal of Medical Informatics, vol. 82, no. 6, pp. 539–552, 2013.
View at: Publisher Site | Google Scholar
S. Petter and E. R. McLean, “A meta-analytic assessment of the Delone and Mclean is success model: an examination of is success at the individual level,” Information and Management, vol. 46, no. 3, pp. 159–166, 2009.
View at: Publisher Site | Google Scholar
Y. S. Wang, “Assessing e-commerce systems success: a respecification and validation of the Delone and Mclean model of is success,” Information Systems Journal, vol. 18, no. 5, pp. 529–557, 2008.
View at: Publisher Site | Google Scholar
W. H. DeLone and E. R. McLean, “The Delone and Mclean model of information systems success: a ten-year update,” Journal of Management Information Systems, vol. 19, no. 4, pp. 9–30, 2003.
View at: Publisher Site | Google Scholar
B. Tilahun and F. Fritz, “Comprehensive evaluation of electronic medical record system use and user satisfaction at five low-resource setting hospitals in ethiopia,” JMIR Medical Informatics, vol. 3, no. 2, p. e22, 2015.
View at: Publisher Site | Google Scholar
I. K. W. Lai and M. Hitchcock, “Importance–performance analysis in tourism: a framework for researchers,” Tourism Management, vol. 48, pp. 242–267, 2015.
View at: Publisher Site | Google Scholar
J. A. Martilla and J. C. James, “Importance-performance analysis,” Journal of Marketing, vol. 41, no. 1, pp. 77–79, 1977.
View at: Publisher Site | Google Scholar
S. O’Leary and J. Deegan, “Ireland’s image as a tourism destination in France: attribute importance and performance,” Journal of Travel Research, vol. 43, no. 3, pp. 247–256, 2005.
View at: Publisher Site | Google Scholar
G. G. Van Ryzin and S. Immerwahr, “Importance-performance analysis of citizen satisfaction surveys,” Public Administration, vol. 85, no. 1, pp. 215–226, 2007.
View at: Publisher Site | Google Scholar
A. Martensen and L. Grønholdt, “Improving library users’ perceived quality, satisfaction and loyalty: an integrated measurement and management system,” Journal of Academic Librarianship, vol. 29, no. 3, pp. 140–147, 2003.
View at: Publisher Site | Google Scholar
M. O. Kim, E. Coiera, and F. Magrabi, “Problems with health information technology and their effects on care delivery and patient outcomes: a systematic review,” Journal of the American Medical Informatics Association, vol. 24, no. 2, pp. 246–250, 2017.
View at: Publisher Site | Google Scholar
M. Igbaria and M. Tan, “The consequences of information technology acceptance on subsequent individual performance,” Information and Management, vol. 32, no. 3, pp. 113–121, 1997.
View at: Publisher Site | Google Scholar
S. Akter, J. D’Ambra, P. Ray, and U. Hani, “Modelling the impact of mhealth service quality on satisfaction, continuance and quality of life,” Behaviour and Information Technology, vol. 32, no. 12, pp. 1225–1241, 2013.
View at: Publisher Site | Google Scholar
P. L. T. Hoonakker, P. Carayon, R. L. Brown, R. S. Cartmill, T. B. Wetterneck, and J. M. Walker, “Changes in end-user satisfaction with computerized provider order entry over time among nurses and providers in intensive care units,” Journal of the American Medical Informatics Association, vol. 20, no. 2, pp. 252–259, 2013.
View at: Publisher Site | Google Scholar
L. Raymond, G. Paré, A. Ortiz de Guinea et al., “Improving performance in medical practices through the extended use of electronic medical record systems: a survey of Canadian family physicians,” BMC Medical Informatics and Decision Making, vol. 15, no. 1, p. 27, 2015.
View at: Publisher Site | Google Scholar
E. M. Campbell, D. F. Sittig, J. S. Ash, K. P. Guappone, and R. H. Dykstra, “Types of unintended consequences related to computerized provider order entry,” Journal of the American Medical Informatics Association, vol. 13, no. 5, pp. 547–556, 2006.
View at: Publisher Site | Google Scholar
J. Chan, K. G. Shojania, A. C. Easty, and E. E. Etchells, “Does user-centred design affect the efficiency, usability and safety of CPOE order sets?” Journal of the American Medical Informatics Association, vol. 18, no. 3, pp. 276–281, 2011.
View at: Publisher Site | Google Scholar
V. A. Zeithaml, L. L. Berry, and A. Parasuraman, “The behavioral consequences of service quality,” Journal of Marketing, vol. 60, no. 2, pp. 31–46, 1996.
View at: Publisher Site | Google Scholar
L. Nguyen, E. Bellucci, and L. T. Nguyen, “Electronic health records implementation: an evaluation of information system impact and contingency factors,” International Journal of Medical Informatics, vol. 83, no. 11, pp. 779–796, 2014.
View at: Publisher Site | Google Scholar
G. A. Churchill, “A paradigm for developing better measures of marketing constructs,” Journal of Marketing Research, vol. 16, no. 1, pp. 64–73, 1979.
View at: Publisher Site | Google Scholar
J. Xu, I. Benbasat, and R. T. Cenfetelli, “Integrating service quality with system and information quality: an empirical test in the e-service context,” MIS Quarterly, vol. 37, no. 3, pp. 777–794, 2013.
View at: Publisher Site | Google Scholar
D. W. Straub, “Validating instruments in MIS research,” MIS Quarterly, vol. 13, no. 2, pp. 147–169, 1989.
View at: Publisher Site | Google Scholar
J. F. Hair, G. T. M. Hult, C. M. Ringle, and M. Sarstedt, A Primer on Partial Least Squares Structural Equation Modeling (PLS-SEM), Sage Publications, Thousand Oaks, CA, USA, 2014.
C. M. Ringle and M. Sarstedt, “Gain more insight from your PLS-SEM results: the importance-performance map analysis,” Industrial Management and Data Systems, vol. 116, no. 9, pp. 1865–1886, 2016.
View at: Publisher Site | Google Scholar
R Core Team, R: A Language and Environment for Statistical Computing, R Foundation for Statistical Computing, Vienna, Austria, 2013.
M. Rönkkö, Matrixpls: Matrix-Based Partial Least Squares Estimation. R Package Version 1.0.4, M. Rönkkö, Otaniemi, Finland, 2017, https://cran.r-project.org/web/packages/matrixpls/index.html.
Ministry of Health and Welfare, Statistics of Healthcare Professionals 2017, Gender Equality Commit of the Executive Yuan, Taipei, Taiwan, 2018, https://www.gender.ey.gov.tw/GecDB/Common/FileDownload.ashx?sn=gmmcQ00JSDc6b6HKc%2bwdHw%3d%3d&ext=.xlsx.
J. F. Hair, W. C. Black, B. J. Babin, and R. E. Anderson, Multivariate Data Analysis–A Global Perspective, Prentice-Hall, Upper Saddle River, NJ, USA, 7th edition, 2010.
E. W. Anderson and C. Fornell, “Foundations of the American customer satisfaction index,” Total Quality Management, vol. 11, no. 7, pp. 869–882, 2000.
View at: Publisher Site | Google Scholar
C. Fornell and D. F. Larcker, “Evaluating structural equation models with unobservable variables and measurement error,” Journal of Marketing Research, vol. 18, no. 1, pp. 39–50, 1981.
View at: Publisher Site | Google Scholar
J. Cohen, Statistical Power Analysis for the Behavioral Sciences, Lawrence Erlbaum Associates, Publishers, Mahwah, NJ, USA, 2nd edition, 1988.
P. J. Hu, P. Y. K. Chau, O. R. L. Sheng, and K. Y. Tam, “Examining the technology acceptance model using physician acceptance of telemedicine technology,” Journal of Management Information Systems, vol. 16, no. 2, pp. 91–112, 1999.
View at: Publisher Site | Google Scholar

Copyright

Copyright © 2018 Kuang-Ming Kuo et al. This is an open access article distributed under the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.

PDF Download Citation

Download other formats

Order printed copies

Views

7557

Downloads

2351

Citations