Abstract

Because a consensus has yet to be reached on how to assess the success of a Public-Private Partnership (PPP) project, it is meaningful to study the multidimensional indicators to assess PPP project success. It merits attention that PPP project success indicators are totally different from PPP critical success factors (CSFs), which have attracted a lot of attention from scholars. This study conducts a questionnaire survey to identify the crucial considerations of project success from the perspectives of different PPP stakeholders. In line with previous studies, five dimensions of PPP project success that were developed from different stakeholder viewpoints and time frameworks were tested based on the empirical data. The dimensions mapped in the study not only provide a benchmark for practitioners to diagnose the extent of PPP project success but also provide a solid foundation for scholars to conduct further studies regarding PPP project success.

1. Introduction

As a new approach to gathering the skills, technologies, and resources of different stakeholders, a large-scale project procured in Public-Private Partnership (PPP) could be issued and managed as a whole rather than as small pieces that are conducted in a consecutive manner to meet the requirements of time and cost in the traditional procurement method. Therefore, PPP bridges the gap between design, construction, and facility management roles and provides better services with lower life-cycle cost. Although the project outcomes would benefit from the continuity and cooperation relationship of a PPP approach, more risks are triggered by the complexity, long-term nature, and varied stakeholders of PPP projects. Even though the PPP approach has been widely adopted in a large number of projects, promoting the application of PPP is still debatable in both the industry and academia. The source of the debate lies in the lack of a comprehensive evaluation system to assess the performance of PPP projects.

Actually, the effectiveness of the “iron triangle” criteria, which are widely adopted to measure traditional construction projects, has been widely applied to evaluate the success of PPP projects. However, the “iron triangle” criteria are too simple to reflect the expected outcomes of stakeholders for the performance of PPP projects. This is because the goal of the radical PPP approach is to procure services and/or facilities rather than assets. The goal is significantly different from that of the traditional procurement approach. A successful PPP project should also meet the objectives and anticipation of most stakeholders, including the end users. Additionally, different stakeholders would recognize project success from different viewpoints in line with their benefits. Therefore, the research question is whether it is possible to build comprehensive indicators to evaluate PPP project performance that can reflect most stakeholders’ opinions and what types of projects can be regarded as successful.

The objective of this study is to identify the key success indicators of PPP projects. To achieve this objective, a questionnaire survey was conducted to collect the empirical data from Hong Kong and investigate the crucial considerations of different PPP stakeholders into the PPP project success. The study intends to contribute to the existing literature on the key success indicators for measuring PPP and, specifically, to deliver meaningful implications for evaluating PPP project success.

2. Background

Project success indicators should be distinguished from project management success and project success factors. The main theme of project management success is assessing the potency of project management processes, while the substance of project success is investigating the extent of fulfilment of the established goals [13]. Project success can be achieved only if the project management process follows the correct direction. Although there are connections between project success indicators and project success factors, success indicators refer to standards that measure success, whereas success factors represent the events contributing to success [4, 5]. It is well known that the acknowledged early project success indicators, often referred to as the “iron triangle,” are closely associated with “hard” project objective measures, including schedule, cost, and quality standards. The “soft” subjective measures, especially the satisfaction of stakeholders, are ignored in the “iron triangle” criteria [6]. Through a combination of hard and soft measures, one common accepted criterion for evaluating project success is whether a project meets the requirements of time, cost, technical specification, and client satisfaction [7, 8]. In fact, the client is among the multiple stakeholders. It should be noted that stakeholders refer to not only the parties within the project but also the parties that could affect or be affected by project implementation [9, 10]. In other words, a project could be recognized as a successful one only if the individual or mutual expectations of the stakeholders are met [1].

As a specific construction procurement approach, PPP is usually applied in infrastructure construction projects that have significant effects on social or economic development and are closely associated with different stakeholders’ interests [11, 12]. Nisar [13] highlights the intrinsic features of the PPP approach: the private sector is responsible for financing, managing assets, and charging in accordance with the performance of assets or the extent of asset usage by either the clients or end users. Therefore, the primary correlations in PPP are shown in Figure 1.

It is clear that a PPP project is initiated and granted by a public partner resort to a concession contract, while the private partner finances, designs, builds, operates, and maintains the project strictly abiding by the concession contract. The ultimate outcomes of the project are used by the end users [14, 15]. Accordingly, the key stakeholders in this study are composed of three components, namely, a public partner, a private partner, and end users. Given the special arrangement of a PPP project, it is important to emphasize the end user as a main stakeholder as the responsibility for providing services is transferred from the public partner to the private partner. Therefore, the end users’ interests are inevitably affected. In fact, taking stakeholders’ benefits into consideration is consistent with the intrinsic essence of shareholder value (SV) management. SV management and balanced scorecard (BSC) are the two measurement systems that can be used to improve organizational performance. SV management is useful for overcoming the overinvestment problem and underinvestment problem in traditional financial performance measurement. The BSC approach is a strategic planning and management system that aligns business activities with the vision and strategy of the organization, improves internal and external communications, and monitors organization performance against strategic goals. The BSC approach has been widely adopted to evaluate the overall performance of construction firms and projects. It is notable that the essence of the PPP procurement approach is in accordance with the core contents of SV management and the BSC approach. Consequently, both financial and nonfinancial measures should be taken into account when exploring the success indicators of a PPP project.

In addition to diversified participants, the life cycle of a PPP project is much longer than that of a general construction project. In general, the concession period of a PPP project is longer than 20 years, in which the design and construction stages are relatively shorter than the operation stage. Therefore, for judging PPP project success, long-term characteristics have to be considered in the PPP project success indicators. The success indicators of PPP projects are listed in Table 1.

Shenhar et al. [16] used 13 project success measures extracted from previous studies and indicated that these measures could be categorized into four dimensions of project success in terms of time framework. The first dimension relates to the short-term objective of project efficiency. This dimension of project success could be evaluated after the design and construction of the project are completed. The second dimension of project success is associated with the medium-term objective of the effect on the stakeholders or customers. The third dimension of project success addresses the long-term objective of business success, while the fourth dimension of project success focuses on the long-term objective of preparing for the future. Dvir et al. [17] measured defence project success based on the three criteria, including the five measures related to the criterion of “meeting planning goals,” seven measures focused on the criterion of “end-user benefits,” and nine measures connected to the criterion of “contractor benefits.” In addition, Sadeh et al. [18] developed and classified project success into 4 dimensions based on time-dependent criteria and stakeholders’ viewpoints. Dimension 1 is entitled “meeting design goals,” which is similar to the classification of Shenhar et al. [16]. Dimension 2 focuses on the benefits to the end users. Dimension 3 emphasizes the benefits to the developing organization, which not only investigates the tangible profits but also explores the intangible benefits to the organization such as reputation. Dimension 4 is “benefit to the defence and national infrastructure.” Yuan et al. [19] established a conceptual KPIs framework in line with 15 performance objectives selected. The KPIs are useful to assess performance of PPP projects. Given the long-term traits and complex stakeholder relationships in a PPP project, we will explore the key success indicators of PPP projects from the perspectives of the main stakeholders using a time-dependent framework.

3. Methodology

3.1. Selection of Measures and Items

Based on a literature review of project success indicators and the specific characteristics of PPP projects, PPP project success was developed and measured in line with the five dimensions. Dimension 1 is “meeting design goals,” which refers to the short-term effects of the project and can be judged after the project design and construction are complete. Dimensions 2 through 4 (i.e., “benefits to end user,” “benefits to private partner,” and “benefits to public partner”) investigate the medium-term influences of the project because the impacts of the project on these stakeholders can be assessed only if the concession contract ends. The last dimension is “preparing for the future,” which is introduced to capture the long-term impact of the project on the entire industry and social development. The item scales used to measure “meeting design goals” are organized based on the viewpoints of Shenhar et al. [16]. The items used to measure the “benefits to end user” are developed based on the findings of Dvir et al. [17], Shenhar et al. [20], and Zhang [21]. The questionnaire items used to study on the “benefits to private partner” consist of two parts. Most items are extracted and refined in accordance with the validated item scale used in Dvir et al. [17]. Meanwhile, one item is developed in line with the findings of Zhang [21] and Li et al. [22]. The new item is highly related to PPP benefits. The items adopted to measure the “benefits to public partner” are extracted and developed based on the findings of Akintoye et al. [23] and Zhang [21]. These items are closely related to PPP superiority. To measure “preparing for the future,” items are developed based on the items used by Walker and Rowlinson [24] and modified to PPP projects according to the findings of Li et al. [22].

3.2. Methodology of Data Collection and Analysis

In this study, the quantitative approach was adopted and is briefly illustrated in Figure 2.

3.2.1. Data Collection

Because the questionnaire survey technique has been widely used to collect professional views in construction management research [2529], this study collected the data of critical success indicators of PPP projects via an empirical questionnaire survey. The questionnaire was prepared and fine-tuned through pilot study and discussions with various stakeholders in PPP projects, including government authorities, construction managers, architects, and engineers. To improve the response rate, a cover letter was attached to encourage unqualified respondents to forward the questionnaire to people who were willing and eligible to participate in the survey. The respondents were requested to rate their degree of agreement against the success indicators according to the five-point Likert scale. Specifically, values 1 through 5 were assigned from greatest disagreement to greatest agreement, respectively.

To verify the questions and gauge the likely feedback solicited from respondents, a pilot study was conducted with experienced practitioners in the industry. To carry out a full-scale survey, experienced practitioners from the public, private, and other sectors were selected as potential respondents. Because there was no publicly available database of PPP participants, the potential respondents were selected from the following sources: architectural consultants listed by the Association of Architectural Practices, members of the Hong Kong Institute of Construction Managers, building contractors recognized by the Hong Kong Housing Authority, and the listed participants of several construction conferences. To eliminate the influence of ineligible sample to the overall sample, the mails returned due to incorrect or incomplete addresses, the respondents who no longer had the recorded job, the respondents who were out of the office, and the respondents who had little PPP experience were considered as the ineligible sample and excluded from the final analysis.

3.2.2. Data Analysis

The data collected from the survey were analysed using SPSS 19.0. Missing data are analysed using SPSS 19.0 with Little’s MCAR test. The exploratory factor analysis (EFA) approach, which is widely used to handle large-scale question items and obtain useful information regarding critical factors based on the scores given by respondents [3032], was employed in this study. As the most commonly used method to extract factors, principal components analysis (PCA) was employed along with the varimax rotation method in EFA to maximize the variance of the factor loadings [33]. It should be noted that two premises have to be satisfied before using EFA. The first one is to meet the requirement of the Kaiser–Meyer–Olkin (KMO) test of sampling adequacy. The principle of the KMO measure is to compare the magnitudes of observed correlation coefficients with the magnitudes of partial correlation coefficients [34]. To pass the KMO test, the value must be higher than 0.50. If the value is above 0.70, the sample is good enough for factor analysis [35]. In addition, the appropriateness of the sample should be verified through Bartlett’s test of sphericity. The sample is appropriate for EFA only if the null hypothesis is rejected.

Using EFA, a large number of variables can be compressed into fewer factors with commonalities. Comrey and Lee [36] note that communality reflects the sum of squares of factor loadings over all of the factors. In social science research, the communality of items should be higher than 0.40 [37]. In addition to the communality check, the factor loading check is also crucial for factor extraction. In general, ±0.45 is recognized as the cut off value for factor loading [38]. In fact, even if the communality and the factor loading satisfy the requirements to extract factors, the number of principle components retained should be determined in line with the following principles: the first one is to retain the components with eigenvalues greater than 1.00. However, the number of components might be underestimated or overestimated occasionally if complying with only the first principle. To address this problem, a scree plot can be adopted as a complementary principle. The last step of factor extraction is to calculate the value of Cronbach’s alpha, which is utilized as a useful tool to test the internal consistency (or reliability) of extracted factors. Normally, if the value of Cronbach’s alpha is equal to or greater than 0.70, the internal consistency of the extracted factors is acceptable [39]. Through a strong design of research structure and an exhaustive quantitative data analysis, realistic findings can be obtained from the research.

4. Results

4.1. Results of Data Collection

The response rate by post was 7.8%, while the response rate by email was 1.32%. After deleting those questionnaires with missing data, there were 77 valid records (73 records returned by post and 4 records gathered by email).

The response rate was relatively low compared with the 20%–30% norm of most construction surveys [40]. It is not surprising that the overall response rate was low within the construction industry. Because the potential survey respondents were practitioners in the overall construction industry rather than practitioners in the PPP field, the effective response rate seems not high enough. However, all the respondents were experienced PPP practitioners, and thus the collected data can reflect the reality of PPP practice in Hong Kong. For example, 64% of the respondents are the practitioners in PPP fields with more than 15 years of working experience and can provide valuable information for references. Moreover, the sample size met the basic requirement of empirical analysis [41]. Nevertheless, the results could be more robust with a larger sample size.

Accompanied by the questions related to project success, the respondents were requested to provide a variety of background information, including role undertaken in a PPP project and the project type and PPP procurement approach adopted in the project. Table 2 presents the demographic information of the respondents.

Compared with the private partners, it was difficult for public partners to participate in this study due to the difficulty in obtaining their contact information. The public partners involved in the study were from government departments and agencies. As for the private partner, the specific roles of respondents in projects are shown in Figure 3. Transportation projects represented the highest proportion. In addition to transportation projects, PPP approaches were also used in public housing and hospital projects. Among a large number of subsets of PPP approaches, Design-Build-Finance-Operate (DBFO) and Build-Operate-Transfer (BOT) were two of the most prevalent procurement approaches.

4.2. Results of Data Analysis

The descriptive data of the replied item scales belonging to each dimension of PPP project success of this study are shown separately in Tables 37.

The results of Little’s MCAR test is χ2 = 119.643, df = 114, and . It is assumed that data are missing completely at random (MCAR) because no significant results were indicated by the missing data analysis. The result of MCAR suggests that variables of the missing data were not affected by any other variables or missing values themselves [42]. Accordingly, the missing data are imputed with the expectation maximization method.

As reported in Table 8, five factors are extracted based on the 25 question items. The value of the KMO test was 0.78, greater than 0.70, implying that the correlation matrix of the sample was good enough for EFA. In addition, the value of Bartlett’s test of sphericity was 1419.12 (), which suggested that the sample was appropriate for EFA. According to the results of EFA, five factors, rather than six factors, were extracted. Although it is suggested to extract principle components in line with the correlation matrix’s eigenvalue, a scree plot is taken into consideration with the eigenvalue [43]. According to Stern [33], only the factors with eigenvalues above the straight line should be kept if a line is drawn through the smallest eigenvalues on the scree plot.

Therefore, it is verified that 5 factors should be extracted in this study. Meanwhile, as indicated in the research Methodology section, after the sampling adequacy testing is finished and the preliminary factors are extracted, communalities should also be examined. In this study, the communalities of all the factors were higher than 0.40. Moreover, all the items had a factor loading greater than 0.50 on only one factor. Meanwhile, Cronbach’s alpha of each factor was greater than 0.70, which suggested that the extracted factors were appropriate in terms of good internal consistency. After conducting the EFA, 25 items were reduced to five principle components. Taken together, the 59.77% variance of project success could be explained by the five extracted factors, in which the underlying grouped factors account for 11.37%, 5.43%, 25.99%, 7.90%, and 9.09% of the total variances, respectively.

5. Discussions

Although the factors extracted satisfied the requirements of the KMO test, Bartlett’s test of sphericity and Cronbach’s alpha, the results of EFA were slightly different from what we expected to explore in terms of the measures of PPP project success. The two items explored to test “benefits to end user” were regrouped into the dimension of “meeting design goals.” After reviewing the meanings of items under the dimension of “meeting design goals,” this factor is interpreted as “meeting planning goals.”

The empirical findings revealed that PPP project success indicators were relatively complicated compared with the success measures of other projects. The conventional “iron triangle” measures reflect only one dimension of PPP project success. Although “the delivery time to end user” and “the charge fees for end user” are highly interrelated to the end-users’ benefits, these two issues closely associated with the “iron triangle” are not taken into consideration when the end users judge PPP project success. Obviously, if the project outcomes reach the end user on time, the designed schedule goal is certainly met. Moreover, the market-oriented economy in Hong Kong leads to the reality that the user charge is directly at the mercy of the private partner’s economic objective because cost and profit are the two components of the private partner’s economic objectives. In addition, the user charge is closely related to one aspect of the “iron triangle”: cost. As people live in a market-oriented economic system, the end users generally agree that good service deserves a higher charge. Consequently, charge fees and delivery time are not the prioritized considerations of the end users.

The dimensions explored and verified in this study were not only beneficial to overcoming the drawbacks of the “iron triangle” measures but also in line with the suggestions given by other researchers. For example, the short-term objective, the medium-term objective, and the long-term objective of implementing PPP projects should be considered when assessing the outcomes of a PPP project. Meanwhile, stakeholder satisfaction is also crucial for judging PPP project success. Among the dimensions tested in the paper, “meeting planning goals” is closely associated with the short-term objective of a PPP project. The three dimensions of “benefits to end user,” “benefits to private partner,” and “benefits to public partner” are closely related to the medium-term objective of PPP project and stakeholder satisfaction. The dimension of “preparing for the future” evaluates the long-term objective of the PPP procurement approach. In summary, the dimensions explored and divided based on the long-term and diversified participant characteristics of PPP projects are suitable for testing PPP project success.

On the contrary, significant correlations between the dimensions of project success indicators show that a successful PPP project has to meet the three objectives: the first objective is about the short-term goals of PPP project implementation, which are closely associated with the design and construction stages of a PPP project; the second objective is to satisfy the stakeholders’ objectives that can be assessed only at the end of the concession contract; and the third one is to provide benefits for the entire industry and social development.

Dvir et al. [17] noted that the end users should join a PPP project at the beginning of the project and stay through the end of the project, implying a significant relationship between “benefits to private partner” and “benefits to public partner.” However, the findings of our study revealed that neither the private partner nor the public partner benefits were significantly associated with “benefits to end user.” This implied that the end-user benefits are not the major concern throughout the implementation of a PPP project and that the end user should be involved in PPP projects during the initial period, which is crucial for improving the service provided by the private partner and achieving overall project success.

The results reflected the management reality. As the objective of a PPP project is to procure a better service, the primary issue for practitioners is “what is a better service.” For the public sector and the end users, the better service not only means that a project with higher quality and lower cost should satisfy technical and functional requirements, but also that the project should be delivered to the end users as soon as possible with lower payment. Since the criterion of a better service is known, the next thing for practitioners is how to achieve it and make a project success. As the achievement of the objectives relies on outstanding technique and management skills of the private partner, the foremost step to secure the success of a PPP project is to select a qualified private partner. Meanwhile, because the implementation of a PPP project is to provide better services for the end users rather than mitigate the financial risk of the public partner, the end user’s satisfactions are very important to appraise PPP project success. Therefore, to achieve the success of a PPP project, it merits attention that the benefits of the end users should be assigned with a priority over the interests of the private and public partners. If possible, the end users’ opinions on a PPP project should be polled before the implementation. In addition, as the benefits of the private and public partners are closely interrelated, to achieve the success of a PPP project, both parties should consider the problems that would be encountered in the project from the perspective of their counterparts. Mutual understanding is considered as the basis of contracting. Of course, renegotiation could be considered as a useful approach to maximizing the interests of both parties if the contract is not perfect [44].

6. Conclusions, Limitations, and Future Work

To ensure the economic, environmental, and social development, the demand for effective success measures has been driven in these days. This study attempts to identify and develop success measures for PPP projects based on a questionnaire survey. The questionnaire survey is carried out to solicit professional opinions about the five-dimensional success indicators which are proposed according to the literature review and pilot interviews. These results showed that the five dimensions were really covered in practice despite differences in their relative importance. Compared with the basic demand for short-term goals to maximize the benefits of the private partner, medium-term and long-term measures related to economic and social benefits were much more important.

Although the findings have realistic and profound significances in theoretical and practical aspects, some limitations still exist. Firstly, the opinions were collected based on the professional’s experience, and they relied too much on people’s subjective judgements. Although the subjectivity has been seen as a common problem in questionnaire survey, it still limits the accuracy of the findings. Secondly, cautions should be warranted because of the small sample size. Thirdly, it should be noted that, in some cases, the long-term impacts of PPP projects could not be seen at the moment until the closure of the project life cycle.

Project success measures are extremely important for stakeholders to evaluate the effects of PPP projects. The performance and success of a PPP project could not be intuitively demonstrated without explicit measures of PPP project success. Future studies would explore the dependent relationships between critical success factors and PPP project success indicators.

Data Availability

The data used to support the findings of this study are available from the corresponding author upon request.

Conflicts of Interest

The authors declare that they have no conflicts of interest regarding the publication of this paper.

Acknowledgments

This work was supported by the Humanities and Social Science Foundation of Ministry of Education of China (Grant no. 18YJC630084).