Abstract

The evaluation of the railway operation safety situation is important for managers to ensure transportation safety and make control decisions. In this study, first, six situation indicators are designed from a regional perspective based on the characteristics of railway operation accidents, and the quantitative methods of these indicators are determined. Second, an improved technique for order of preference by similarity to ideal solution (TOPSIS) method is developed to evaluate the railway operation safety situation against the situation indicators. Based on the set-pair analysis (SPA) theory and cosine similarity measure (CSM), the comprehensive evaluation values and rankings of the safety situation are first calculated from the distance and trend levels for each period. Game theory is then employed to determine a more reasonable combined weight, and the values of the parameters involved in the situation indicators are also estimated. The real-life statistical accident data in a regional area of China from 2016 to 2018 are chosen as a case study to verify the proposed method. A brief analysis is conducted, resulting in suggestions for the evaluation results. Two groups of comparative experiments are designed to demonstrate the feasibility and effectiveness of the method. Finally, the quality of the evaluation results is verified through actual conditions and expert scoring. Some extensions and potential practical applications of this work are discussed.

1. Introduction

Safety is always the first priority of railway operation. By the end of 2018, the railway operating mileage in China reached 131,000 km and the railway network density was 0.01369 km/km2. In addition, the total converted turnover was as high as 3,986.495 billion ton-km [1]. As a result, China Railway needs to undertake heavy and large-scale transportation tasks each day. At the same time, the complex operation characteristics and environment, such as continuous and dynamic train running processes, require operators to perceive and control the operational safety situation at the macroscopic level. Moreover, the goal of controlling the safety situation is to constantly strengthen railway operation safety and improve the security system. Railway operation management in China is divided by region. Therefore, railway operation safety situation evaluation (ROSSE) from a regional perspective is of great significance for operators to prevent and control operation risks, assist the decision-making process, improve the level of safety management, and ensure continuous and normal activities.

Safety situation evaluation fuses data and information at a high level, including time and space dimensions. In particular, safety situation evaluation has been widely used in military [2], road traffic [3, 4], energy [5], and other fields involving safety management [6]. With the rapid growth of passenger flow in China, the railway system, as a medium- and long-distance public transport mode, faces new challenges in safety management, especially considering the heterogeneous transportation activities and complicated network structure on a regional scale. At present, research on railway operation safety management mainly covers (i) multifactor integrated management and (ii) accident analysis and modeling. The former mostly analyzes the influencing factors of safety situations by integrating multiple elements, such as humans, equipment, organizations, and the environment. Several theoretical frameworks and indicator systems have been established [710]. By analyzing and modeling accident data, the latter are mostly used to design indicators of safety situations, determine the causes of accidents, and predict the state [1114]. With the fast-paced development of data analytic technology, railway managers in China are paying more attention to the applications of accident statistics, which provide useful, high-level views of safety situations [15]. Therefore, in this study, situation indicators were designed starting from the analysis of the multiattribute characteristics of accidents, where the ROSSE is performed based on these indicators through appropriate methods.

In terms of indicator design, many Chinese researchers select the number of accidents, the monthly average accident rates, and the accident-correlated coefficients as the main indicators to measure the safety situation [1618]. The design of these indicators is mainly aimed at a certain type of operating line. However, only a few studies simultaneously integrated high-speed lines, normal-speed lines, and passenger and freight transportation within a certain region for holistic research. Therefore, the previous studies have limited practical meaning for managers when there is need to understand the overall safety situation. The number of situation indicators also needs to be supplemented.

In terms of evaluation methods, the multicriteria decision-making (MCDM) approach is widely used. It considers the assessment and ranking of multi-indicator solutions in a certain time series [19, 20]. The most well-known method in the MCDM problem is the technique for order of preference by similarity to ideal solution (TOPSIS) method developed by Hwang and Yoon [21]. According to this technique, the relative distances close to the ideal solution can be seen as the comprehensive evaluation results. The method has achieved good results in practical applications [2224]. However, because the traditional TOPSIS method calculates Euclidean distance, it is easy to encounter a rank reversal problem (RRP) [25], which can lead to unstable evaluation results. Furthermore, most existing TOPSIS methods do not consider the similarities of trends between alternatives and the ideal solution. Therefore, the main motivation of this study is to develop an improved TOPSIS method to compensate for these two shortcomings.

The TOPSIS method needs to be applied in conjunction with the method of calculating indicator weights. The rationality of weights is critical to the validity and applicability of the evaluation results. To obtain the appropriate relative importance of indicators, the calculation of weights should combine the benefits of subjective and objective judgment methods. Hence, this is another focus of this study. In view of the above discussion, the contributions of this research can be summarized as follows:(1)Six situation indicators for evaluation were designed considering the multiattribute characteristics of railway accidents for evaluating the railway safety situation from a regional perspective. Given the situation indicators, an improved TOPSIS method is proposed based on the distance and trend levels. Meanwhile, game theory (GT) is applied to determine the combined weight.(2)A real-life case study based on the actual data of railway accidents in China is introduced, where the situation indicators and the ROSSE are calculated. Comparative experiments and analyses were performed to verify the effectiveness and evaluation quality of the proposed evaluation method.

The remainder of this paper is organized as follows. Section 2 provides a comprehensive literature review. The situation indicators for ROSSE are designed and quantified from a regional perspective, and this is discussed in Section 3. In Section 4, an improved TOPSIS method along with preliminary estimation of the values of the parameters involved are discussed. Section 5 presents a case study that is based on the real-life accident statistics in a certain regional area of the China Railway, including the calculation of indicators, results, and ranking of the ROSSE, model comparison, and performance. In Section 6, the quality of the evaluation work is further verified, and the extensions and potential practical applications of the proposed method are discussed. Conclusions and possible further research directions are provided in Section 7.

2. Literature Review

Following the research ideas of the ROSSE and the process of approach formulation, in this section, the literature is reviewed from three aspects: (i) railway safety management and evaluation, (ii) the TOPSIS method and its improvement, and (iii) combined weight.

2.1. Railway Safety Management and Evaluation

Li and Guldenmund [6] provided a broad overview of safety management systems in 2018. They studied the literature, focusing on five core aspects: definition, evolution, models, purpose, and common elements. Based on this research, the methods of safety management can be carried out using (i) organizational models and (ii) accident-related models. Moreover, railway safety organizations mainly manage multiple factors, such as human behavior and the operating environment. Therefore, a brief review of these two directions was conducted.

For multifactor integrated management, a framework was presented in Morant et al. [8] to evaluate the safety and availability of the railway operation. They quantified the probability that railway traffic is not supervised by the signalling system. Hu et al. [10] established an index system for evaluating the high-speed railway of the environmental safety situation in China by analyzing the impact mechanism of severe weather. Wang et al. [26] applied the cusp catastrophe model to describe the dynamic changing process of railway system safety, and a framework for the system risk was constructed. Crawford et al. [27] aimed to raise awareness of potential health and safety risks emerging in the railway industry. They found two mechanisms of risk that can impact the success of an integrated railway: human-automation design and progressive integration efforts.

In terms of accident analysis and modeling, Kyriakidis et al. [7] presented a framework to identify the most-significant human performance factors, which were derived from an analysis of 479 railway operational incidents. Ouyang et al. [11] applied an approach called “system-theoretic accident models and process” to analyze railway accidents and discussed the accident spreading processes. According to Li and Wang [12], the causal factors of accidents in a railway system are divided into several error types, and a model was proposed based on a complex network for risk monitoring. The risks of accident causal factors were quantified. Mirabadi and Sharifian [13] analyzed the data from past accident data of the Iranian Railway by applying association rule data-mining techniques to discover and reveal unknown relationships and patterns in data. This research considered accident conditions and relationships discovered among the most common accident factors. Using a Bayesian network model, Bearfield et al. [28] made local risk estimates from boarding and alighting incidents at railway stations, and they analyzed the data with expert judgments about causal factors. Liu et al. [29] focused on employing the fault tree analysis method combined with quantitative analysis to present a more comprehensive view of high-speed railway accidents. In China, 407 railway accident/incident reports were collected and studied by Zhou and Lei [30] using the human factor analysis and classification system framework. The results showed that the four most common errors in the railway system were “organization process,” “inadequate supervision,” “personal readiness,” and “skill-based errors.”

Accident data provide a wealth of resources to discover the causes, correlations, spreading mechanisms, and comprehensive performance to measure the railway safety situation or other characteristics. Combining big data analytics to handle this moderate size of data, one can better understand the safety situation. In terms of problems specific to China, the railway operation safety situation has attracted the attention of some researchers. For example, Wei et al. [16] divided different operational safety levels of normal-speed railway lines using the system clustering method. Railway operation safety situation indicators and their spatiotemporal distribution characteristics were analyzed by Xu et al. [17]. They proposed a safety situation prediction model for a high-speed railway based on the backpropagation neural network (BPNN). Gao et al. [18] introduced the industrial data classification method to explore the adaptability of BPNN and gray theory further in safety situation prediction.

For evaluation from a regional perspective, comprehensiveness and integrity are emphasized. Yang et al. [4] designed a series of qualitative and quantitative indices to determine the overall safety level of two highways in China. Laureshyn et al. [31] proposed a framework for organizing all traffic encounters into a severity hierarchy to describe the safety situation and trade-off between safety and efficiency in the traffic system. Relatively few studies have addressed this aspect in the field of railway safety evaluation.

2.2. TOPSIS Method and Its Improvement

In the past few decades, the application of the MCDM approach has dramatically increased in the areas of evaluation theories and assessment methodologies. According to Zavadskas et al. [19], the ROSSE of each period based on multiple situation indicators is a discrete MCDM problem. The ranking of decision-making choices can also be treated as a result of comprehensive evaluation. The TOPSIS method is a classic MCDM approach and a privileged technique in multiple solutions. It is created by selecting the best choice with the shortest Euclidean distance to the positive ideal solution (PIS) and the farthest Euclidean distance from the negative ideal solution (NIS), respectively. The traditional TOPSIS method can be used directly for evaluation purposes. For instance, Huang et al. [23] applied the entropy-TOPSIS method to evaluate urban rail transit system operation performance through an index system with eight indicators and 41 subindicators. Despite the great growth and evolution of the TOPSIS method, it has an RRP, for which De Farias Aires et al. [25] have given an interpretation. The RRP means that a decision-making preference ordering between two alternatives changes when an alternative is added or removed, which clearly contradicts the principle of independence from irrelevant alternatives. The RRP in the TOPSIS method results from the normalization procedure as well as the modifications in the PIS and NIS. Therefore, many researchers have improved the traditional TOPSIS method.

The most effective way to solve the RRP is to replace the calculation of Euclidean distance, such as the Mahalanobis distance used by Wang and Wang [32]. More studies have introduced other relatively complex theories to improve the TOPSIS method. Mousavi-Nasab and Sotoudeh-Anvari [33] integrated TOPSIS and data envelopment analysis (DEA) methods as an auxiliary tool to solve the material selection problem. Liu et al. [34] improved the risk evaluation in failure mode and effect analysis combining a cloud model with the TOPSIS method. Considering that the fuzzy-MCDM is a crucial topic in expert system and operations research, Salih et al. [22] conducted a survey on the state of the art of the fuzzy TOPSIS methods between 2007 and 2017. They further clarified the applicability and limitations of the method. Li et al. [24] applied the fuzzy TOPSIS method to evaluate the service quality of the Beijing metro system and analyzed the stability of the results by the ranking change for a line from different comparison sets of metro lines.

The frequently used normalization methods for TOPSIS were summarized [35]. It was proved that vector normalization does not change the diversity of the attribute data, and it is suitable for improving TOPSIS. This is also the reason for choosing the set-pair analysis (SPA) theory to construct the connection vectors. Herein, the application of SPA theory in the TOPSIS method is reviewed further. Kumar and Garg [36] attempted to rate the different preferences of an object using the connection number, a major component of the SPA. An extension of the TOPSIS method was developed, based on the proposed connection number, to calculate the relative closeness of sets of alternatives. They further discussed the application of the SPA-TOPSIS method under an interval-valued intuitionistic fuzzy set environment. It was verified through a real-life numerical example in the literature [37]. Yuan and Luo [38] developed a comprehensive system of evaluation criteria and an improved SPA-TOPSIS method to assess the regional energy security performance in China from 2013 to 2017.

The above research demonstrates that improvement is mainly concentrated on the relative distance. Therefore, the cosine similarity measure (CSM) is introduced in this work to study the trend characteristics of solutions. The CSM is a simple and effective metric for learning similarity, and it has been widely used in the fields of machine learning, pattern recognition, and fuzzy strategic decision-making [39]. Ye [40] proposed a weighted CSM based on intuitionistic fuzzy sets, and Wei [41] applied weighted cosine function similarity measures in picture fuzzy sets to select the optimal production strategy. The applications in fuzzy decisions guided the formulation of the approach used in this study.

2.3. Combined Weight

For the TOPSIS approach used for the ROSSE, the weights of the situation indicators reflect the relative importance of different indicators in the decision-making process. In general, there are two categories of weighting methods: subjective and objective. The subjective methods determine weights solely according to the preferences or judgments of decision-makers. By contrast, the objective methods determine weights by solving mathematical models automatically without any consideration of subjective preferences, such as natural weight and Shannon entropy weight. As discussed by Wang [42], there are certain limitations in considering a single weighting method in the MCDM approach. The subjective weight is strongly influenced by expert experiences that could also contain some prejudices. In contrast, the objective weight neglects the knowledge of the decision-makers and the actual situation. Therefore, the comprehensive weight, combining the subjective and objective weights with an effective algorithm, is more reasonable in the approach formulation.

GT, focusing on the research of strategic interaction, is used to obtain the optimal equilibrium solution. In many studies on MCDM methods, GT has been chosen as the preferred method to determine the combined weight. Lai et al. [43] applied a combined weight integrating subjective and objective weights based on GT in a fuzzy comprehensive evaluation. They also showed the rationality of the results. Analogously, the fuzzy analytic hierarchy process (AHP) weight and entropy weight were combined by Sun et al. [44]. In addition, Liu et al. [45] improved the method for determining the combination weight using GT together with the decision-making trial and evaluation laboratory approach, which adjusted the weights of criteria to make the result more reasonable. Although the combined weight has been used frequently, it is generally composed of one subjective weight and one objective weight, and less research has been performed on the combinations of multiple weights.

In order to increase the comprehensiveness and readability of the literature review, we have summarized some of the publications mentioned above and divided them into two parts: (i) publications that are relevant on the railway safety management in Table 1 and (ii) publications that are relevant on the TOPSIS method in Table 2.

3. Design and Quantification of Situation Indicators from a Regional Perspective

Forming a series of evaluation indicators after analyzing the characteristics of railway operation accidents is one of the ways to measure the safety situation effectively. For the current status of organizing railway operations by region in China, the design of indicators should have the following principles: (i) Situation indicators should be universal within an operational region, covering all routes and transportation of passengers and freight. (ii) Situation indicators should make full use of the multiattribute characteristics of the accident data to reflect the safety situation and avoid duplication between indicator functions. (iii) Situation indicators should exclude the consideration of some extreme accidents, i.e., accidents that rarely occur but can lead to serious consequences. Principle (iii) is to consider that the goal is to develop an evaluation method for the safety situation that does not consider extreme cases and is more meaningful and practical. Such extreme accidents would make it difficult to measure and take advantage of the multiattribute characteristics of accidents, except for the severity. In addition, the laws and features of these accidents have obvious uncertainties [46]. Following the above principles and based on the most commonly used methods in the literature, six situation indicators were designed for ROSSE from a regional perspective and corresponding quantitative formulas were obtained. Each indicator was designed for the same period, such as a month.

3.1. Number of Accidents:

In this study, the term “railway accidents” refers to all accidents that affect the normal operations of the locomotive and rolling stocks or other processes in a certain region. It can be directly adapted to reflect the safety situation. In China, railway accidents are divided into three categories, i.e., traffic, off-rail, and labor accidents. In particular, off-rail accidents involve collisions between trains and other types of vehicles or pedestrians. Therefore, can be quantified as follows:

3.2. Number of Accidents per Unit Turnover:

The amount of turnover is generally used to reflect the transportation workload comprehensively because railway operational products include both transportation objects and distance. It is also used by operators to create future operation plans and economic assessments. Therefore, it is important to analyze the relationship between turnover and accidents from a workload perspective. Furthermore, can be an effective supplement when the number of accidents is only accessed from the time dimension. Considering that passenger and freight transportation are both included in the regional area, the turnover here is the total weighted-sum turnover of those two modes of transportation with a unit of 100 million ton-km. The quantification method is as follows:where and represent freight and passenger turnover, respectively, and is the conversion weight and takes the value of 1 according to the current statistical system of railways in China.

3.3. Accident Grades:

For reasonable ROSSE, in addition to the number of accidents, one should also consider the severity of different accidents, the scope of influence, and the ability of operators to control the situation which are comprehensively represented as the accident grades. The Railway Accidents Investigation and Handling Rules [47] in China (hereinafter referred to as the “Handling Rules”) divide accidents into four categories: especially major accidents, major accidents, slightly major accidents, and general accidents. Among them, the especially major, major, and slightly major accidents are extreme accidents that should not be included during evaluation. For general accidents, the Handling Rules further classify them into four grades from high to low severity, abbreviated as A, B, C, and D. Therefore, only four types of general accidents are considered. The specific quantification method of is given inwhere represents the types of general accident and takes values of 1 to 4 corresponding to A to D, respectively, represents the number of accidents of type , and is the severity grade parameter of the type accident and is yet to be estimated.

3.4. Structure of Accident Properties:

Clarifying the accident properties is an important part of the investigation. Specifically, accidents with different properties can have different types of hazard sources, amounts of damage that can be caused, and identification or control measures. The severity of different accident properties can be distinguished from the efforts to control, the level of difficulty for early warning, and potentially dangerous hazards. For instance, more efforts are required to guarantee and control the safety of high-speed passenger train operations. Thus, for the structure of accident properties over a period of time, the smaller the proportion of accidents on high-speed passenger trains, the better the safety situation. Considering this, the structure of accident properties was designed as a situation indicator, combining the composition and conversion parameters of different accident properties. Equation (4) specifies the calculation method of :where represents the five categories of detailed accident properties according to the Rules, including high-speed train accidents, bullet train accidents, ordinary train accidents, operation line construction accidents, and operation standardization accidents; is the number of accidents of property ; and is the conversion parameter to be estimated, that is, influence relation of the structure of accident properties on the regional safety situation.

3.5. Degree of Accident Correlations:

Regional rail transportation can simultaneously involve several different operation activities, such as trains running at different speeds, heterogeneous operating lines, passenger boarding or alighting, and freight loading or unloading. Hence, the occurrence of one accident can easily affect normal operation activities and directly cause new accidents or become the inducement of subsequent accidents. For example, unexpected train accidents can lead to insufficient connection capacity of rolling stock, which may cause delays or disruptions if handled inappropriately [48]. Therefore, with the increase in accidents, it is possible to explore the degree of accident correlations to measure the stability and robustness of regional safety situations. The calculation method for this situation indicator is shown as follows:where represents the number of accidents that are affected by the accident .

3.6. Average Recovery Time of Accidents:

The recovery time of an accident is the time taken from the beginning to the complete elimination of the adverse effects of the accident. It reflects the ability and efficiency of the railway system to resume normal operations. On this basis, the average recovery time of all accidents can better indicate this situation from the perspective of the entire region. For instance, a shorter average recovery time means that the entire system has a more optimized maintenance scheduling or advanced incident handling technology [49]. As a result, is used as the last situation indicator of the design and its quantification method is provided as follows:where represents the time required to recover from accident .

Thus far, six situation indicators have been designed for ROSSE from a regional perspective. For intuitive display, the information of the indicators is summarized in Table 3.

4. Approach Formulation and Parameter Estimation

As discussed previously, the set of designed situation indicators was used to create the ROSSE of each time period. An appropriate MCDM approach for evaluation is proposed in this section. More than 60 MCDM methods have been applied previously [19]. However, it is challenging to choose a suitable MCDM technique for a given problem. Mousavi-Nasab and Sotoudeh-Anvari [33] conducted a detailed literature review and analysis of this selection problem. They pointed out that an increasing number of studies have used at least two MCDM methods for decision-making-related content and emphasized the straightforwardness and understandability of technique application. Mulliner et al. [50] concluded that the choice of a method should be adaptable to the goal. In the present research, each indicator can only measure one aspect of the overall safety situation, and there may be a certain trade-off relationship between indicators. For example, an increase in the number of accidents may reduce the average recovery time of accidents because of the effective rescue measures. Therefore, the comprehensiveness and integrity of the evaluation results are of the highest value.

Fortunately, TOPSIS has an easier calculation procedure than other well-known MCDM techniques, such as AHP and the analytic network process (ANP). It is also a compensatory method and provides clear trade-offs among the criteria. This good compromise between various indicators can better serve the goals of this work. However, the shortcomings of the traditional TOPSIS method cannot be ignored, as mentioned in Section 2.2. To this end, an improvement strategy combining two different methods was formulated to make the work self-contained.

On the one hand, SPA theory was used to construct the connection vectors of the situation sets in all periods. Thus, one can replace the Euclidean distance calculated in the TOPSIS method by calculating the distance of the connection vectors between the alternatives and the ideal solution. On the other hand, the CSM was employed to estimate the degree of consistency at the trend level between the situation set vectors and the ideal solution vectors. The vectors are all transformed with the NIS as the origin. It is worth mentioning that CSM is also a commonly used metric to replace the Euclidean distance and usually provides good results; however, it is not always advantageous [39]. For instance, two sets with different attribute values may have a very high similarity measure. This outcome is undesirable. Therefore, these two metrics are applied together to compensate for their shortcomings, instead of completely replacing one of them.

Mousavi-Nasab and Sotoudeh-Anvari [33] summarized the commonly used techniques, such as AHP, Shannon’s entropy, and the Simos approach, to determine the indicator weight. Expert scoring and entropy weighting are the most popular subjective and objective methods for combination. However, in determining the objective weight, the entropy weight method focuses on the discreteness or uncertainty of the information involved in an indicator, but it ignores the numerical value of the indicator. Therefore, natural weight that focuses more on the value of indicator was introduced to avoid this bias. GT was applied to determine the combined weight of the subjective, natural, and entropy weights.

The above processes of improvement were organized in steps, described as follows. The parameters involved in the situation indicators were preliminarily estimated. The implementation details are illustrated in Figure 1.

4.1. Evaluation Model Based on the Improved TOPSIS Method

Step 1. Construction and dimensionless processing of evaluation matrixFirst, an initial evaluation matrix is constructed that contains time periods according to the six situation indicators described in Section 3, where the railway operation safety situation of any period can be described by the situation set . For convenience, variable is introduced to represent the -th indicator in the -th period uniformly, where , , and :Then, the matrix is dimensionless processed by the min-max method to obtain a standardized evaluation matrix . If the indicators are of the benefit type, the calculation for standardization can be expressed as equation (9). Otherwise, if the indicators are of the cost type, the calculation for standardization can be formulated as equation (10):where represents the dimensionless form of and and and are the maximum and minimum values for the column indicators in , respectively.After dimensionless processing, the values of the PIS set and NIS set of the standardized evaluation matrix are fixed, as shown in equation (11), which can be used to simplify the construction of subsequent models:Step 2. Establishment of connection vectors using SPA theoryThe basic definition and description of the SPA theory can be found elsewhere [51]. In this study, the SPA theory mainly works on the set pairs between the situation set of each period and the corresponding PIS or NIS set, which are denoted as and . Considering the “identity,” “discrepancy,” and “contrary” features of each set pair, the connection degrees are calculated as follows:where and are the connection degrees of and in the -th period, respectively, and represents the combined weight of each indicator in the situation set, which is determined in Section 4.2. In addition, and ; and are the subconnection degrees between and the corresponding -th column indicators in and . The specific calculation rules can be further elaborated asThus, the connection vectors and of and in the -th period can be obtained using equation (14). The connection vectors between the PIS or NIS set and itself are both equal to :Step 3. Evaluation and ranking at the distance level: relative closenessThe distances and of connection vectors in the set pairs and are calculated, i.e., the distances between or and vector . Based on the TOPSIS method, the relative closeness between the situation set and ideal solution is given by equation (16):Thus, the results and ranking of railway operation safety situation can be achieved based on at the distance level. The larger the value of , the better the railway operation safety situation. It is not difficult to see that the closer the connection vector distance is between the situation set and the PIS, the farther away is the situation set from the NIS. Because the ideal solutions have fixed values, the results and ranking will not change because of the increase or decrease in the number of the situation sets to be evaluated. Therefore, it can effectively solve the RRP.Step 4. Evaluation and ranking at the trend level: cosine similarity

The idealized vector is constructed using and with the NIS as the origin. Then, the situation set in the -th period is transformed into a space vector combining the combined weight . By calculating the cosine similarity of vectors and , one can measure the consistency between the situation set and idealized target at the trend level. As part of the improved TOPSIS method, it is necessary to sort the calculation results. The larger the value of , the smaller the angle between these two vectors, which means that the trend of the railway operation safety situation is better:

The calculation process can be partially simplified by substituting the values of and as follows:

4.2. Combined Weight Based on GT

Step 1. Determine subjective weight The expert evaluation method is a common method for obtaining subjective weights. Several operators were invited to score each situation indicator according to the importance of their influence on the railway operation safety situation, and the average value of each situation indicator was calculated to obtain the weight . This weight is given directly in the case study, and it is stipulated thatStep 2. Determine natural weight The natural weight objectively reflects the importance of the index in the evaluation to a certain extent. The variable represents the attribute value of the situation indicator in the period in the standardized evaluation matrix . Therefore, the quantification method for the natural weight of each indicator is as follows:Step 3. Determine entropy weight The entropy weight method calculates the indicator weights based on the information entropy (IE) of indicators, which was originally developed by Shannon [52]. Generally, a smaller value of IE indicates a higher diversity of the corresponding data and a larger weight, and vice versa [53]. After normalizing into in the standardized evaluation matrix , the IE for each indicator can be defined as follows:Moreover, the entropy weight obtained from IE is finally expressed asStep 4. Determine combined weight

In GT, multiple types of weight can reach a compromise through a process of dispersion minimization [44]. After obtaining the above three weights, one can calculate the combined coefficient of each weight, which decides the method of linear combination of different weights. If the combination coefficients of the three weights are , , and , the optimal equilibrium weight vector of the combined weight can be expressed aswhere , , and are the weight vectors of each weight—for example, .

To minimize the deviation between and the other three weights, the optimized combined coefficients must be solved by

Therefore, the final combined weight is given in

4.3. Estimation of Parameters Involved in Situation Indicators

The parameters and involved in the situation indicators must be preliminarily estimated according to relevant laws, regulations, studies, and expert experience. Because the evaluation method adopts a sequential optimization technique based on the similarity to the ideal target, the values of the parameters focus on the ratio relationship rather than the numerical value. Therefore, the estimated parameters were normalized for the convenience of calculation. The specific estimation process of these two parameters is as follows:(1)Severity grade parameter: The value of parameter mainly depends on the relative relationship between the severity of different grades of accidents. To this end, some excerpts from the Handling Rules were selected for comparison and are listed in Table 4. There is a quantitative contrast relationship between grades A and B based on the fuzzy value range. This relationship can provide a basis for determining the ratio of the conversion factors between them. Grades C and D contain more descriptions of the accident scenarios without a clear quantitative relationship. Grade C is more inclined to passenger accidents and train operation accidents, whereas grade D is more targeted at freight accidents and shunting operation accidents. Therefore, quantitative and qualitative analyses were combined to estimate the parameters. The results are presented in

(2)Property conversion parameter: Parameter reflects the impact of different accident properties on the railway safety situation. Wang [54] conducted practical research on the accident properties in China through an association rule analysis of operation accident data. The correlation between the safety hazard and the accident properties was quantified using gray theory. The results of this study can be used as a basis for parameter estimation. It is necessary to make some appropriate adjustments to it with reference to the comments of operators. Finally, the values of are determined as follows:

5. Case Study

A real-life case study was conducted by taking a certain railway operating region in China as a background. The statistics of accidents in this region from 2016 to 2018 were taken as the initial input data, and the time unit for each period was set as one month. The original accident statistics are not presented in this study because of confidentiality reasons. Considering the limited size of the case, Microsoft Excel was used as a simple and effective tool to achieve the indicator calculations and model solving.

5.1. Calculation and Analysis of Situation Indicators

The various situation indicators for each evaluation month were calculated according to equations (1)–(6), and the calculation results are listed in Table 5. To understand and compare the trend of the situation indicators better, Figure 2 was drawn as a supplement considering only the value fluctuations.

Combining Table 4 and Figure 2 shows that (i) the fluctuations of indicators , , and are relatively stable and exhibit a downward trend as a whole, although increasing slightly in the last few months, and (ii) compared with the fluctuating trends of indicators , , and , the other three situation indicators have more obvious fluctuations, especially the indicator . In addition, the indicators and show an increasing trend, while the indicator declines steadily from the end of 2017. Judging from the changes in the above situation indicators, the frequency and severity of railway accidents in this region have been effectively controlled and the ability and efficiency of accident rescue have been improved. However, the structure of accident properties and the degree of accident correlations move in a direction that is not conducive to the safety situation. This may be related to the continuous increase in the density of high-speed trains and transportation tasks. Obviously, the scope of impacts and the degree of coeffects caused by accidents are constantly expanding. Thus, disruption management under uncertainty [55] is currently being researched extensively.

5.2. ROSSE Based on the Improved TOPSIS Method

Based on the calculation of the situation indicators, the initial evaluation matrix including 36 months was constructed. All the situation indicators were of the cost type. According to equations (19)–(26), multiple weights and the final combined weight were calculated and are presented in Table 6. The results show that the combined weights of different indicators are relatively close, with the indicator reaching a maximum value of 0.1812. However, there are still some obvious differences between different types of weight rankings. Combining the weight and parameter values, Table 7 gives the comprehensive evaluation results and their ranks at the distance and trend levels using the improved TOPSIS method.

The overall evaluation results show that the railway operation safety situation in this region was poor in the first nine months of 2016; then, it gradually improved. At the distance level, the maximum relative closeness occurred in June 2018, which means that, during that month, the safety situation was the best. The changing trend of relative closeness was generally on the rise, but it was sometimes accompanied by a sudden increase or decrease. In terms of the trend level, the maximum evaluation value was reached in March 2017. The fluctuation range of the cosine similarity was reduced after October 2016; however, it slightly deteriorated after February 2018. In addition, the values of the situation indicators show that and are the main causes of such fluctuations. The overall trends of the indicators are reflected in Figure 3.

The above results suggest that the development of railway operation safety situations in regional areas is generally benign but unstable. With continuous expansion of the operation scale, especially the rapid increase in high-speed transportation, the causes and suddenness of accidents are becoming increasingly complicated, leading to a more sensitive safety situation of the railway system and resulting in increased difficulty in safety management. Therefore, railway operators should improve the methods of risk perception and causation analysis, focus on strengthening the prevention measures of passenger accidents, and promote the daily evaluation of safety situations. For example, pattern detection methods can be used to analyze the effectiveness of safety control policies when the results of ROSSE change significantly, such as in October 2016 in the case study. More suggestions are obtained by calculating the quartile of the evaluation results to classify the safety situation levels, as shown in Table 8. This can be applied as assistance for operator decision-making in actual operations.

5.3. Comparative Experiments

Two groups of comparative experiments were designed to verify the effectiveness of the proposed method. The experiments were based on the real-life data of the case study:(1)To verify the elimination of the RRP, data for 16 months were randomly excluded, and the relative closeness was recalculated at the distance level. Table 9 compares the ranking results between the experiment and original case study. The ranking comparison includes 20 months of data, and the ranking size relationship of each month is not reversed.

(2)The traditional entropy-TOPSIS method was employed for the ROSSE as a comparison of the improved TOPSIS method proposed in this study, and the evaluation performance was analyzed. As shown in Figure 4, the improved method is the same as the traditional method for ROSSE. The changing trend of relative closeness is similar and Spearman’s rank correlation index (SRCI) [33] of ranking is 0.98, which implies that the two types are closely related. However, the difference is that the value range of the relative closeness obtained by the improved method is nearly 20% larger than that obtained by the traditional method. This indicates that the improved method is more sensitive to situation awareness. Thus, the evaluation results are more practical for detecting slight changes in the safety situation.

The above two experiments were mainly aimed at comparing the relative closeness at the distance level. In addition, the improved method in this study considers the cosine similarity at the trend level. The SRCI of ranking between these two levels is 0.74, which means that there is a certain difference between them. Therefore, it makes sense to take advantage of the cosine similarity to improve the evaluation model. Together, they make the ROSSE more comprehensive and reasonable.

6. Discussion

6.1. Quality Verification

In general, it is challenging to measure the quality of comprehensive evaluation results because access to relevant information is limited. A small number of studies achieved this verification by comparing it with the outcomes in existing or similar assessment reports [43]. However, to the best of our knowledge, this is the first time that ROSSE has been performed from a regional perspective, and therefore, the effect of comparative verification is significantly weakened. Nevertheless, the Railway Safety Announcements from 2016 to 2018 [56] were collected. These reports were analyzed, and the main conclusions are as follows:(1)The reports show that the entire safety situation of the national railway has gradually improved from 2016 through 2018, especially for the significant reduction of accident rates and accident mortality rates. This is consistent with the overall trend of the evaluation results.(2)Although the trend is optimistic, the reports indicate that the number of exposed security risks has increased exponentially. Deepening the environmental governance along the railway, especially for high-speed railways, will become the focus of transportation safety supervision. This also means that the safety situation is still unstable, which confirms the results of this study at the trend level.

In addition to the above macro conclusions, experts were invited to further score the quality of the entire evaluation work. Concerning the literature on performance evaluation [23, 38], the scoring items were designed based on mainly three aspects, that is, indicator, methodology, and result.

In terms of the indicator, the main consideration is the rationality of indicator design (RID) and the completeness of the indicator system (CIS). In addition, the effectiveness of the indicator system in reflecting the regional perspective (ERP) is required. The effectiveness of the methodology was verified. Hence, the straightforwardness (STR) and understandability (UND) described in Section 4 are important goals. Furthermore, the scalability of the method (SCM) is one of the key capabilities to respond to the changes in practical applications. Finally, the rationalities of weight calculation results (RWC), evaluation results at the distance level (RED), and evaluation results at the trend level (RET) are included in the scoring item design.

Eight experts engaged in railway operation management scored each item from 1 to 10. The higher the score, the higher the recognition of the corresponding item. All scores of eight experts and the overall average score are illustrated in Figures 5(a) and 5(b), respectively.

The scores show that experts are positive about the quality of the evaluation work. The average scores of RWC, RED, and RET are all close to 8, which indicates good performance. However, CIS and SCM still need to be strengthened. These two scoring items are both closely related to the extensions of the method; therefore, they are discussed further in the next section.

6.2. Method Extensions

The safety of railway operations depends on various factors, including infrastructure and rolling stock reliability, emergency rescue rules, natural environment monitoring, and human factors. If these elements are to be considered fully, the evaluation indicator system required must be huge and complex. Because this work is from a regional perspective, it has been restricted in the design of accident-based indicators. Hence, the extensions of the indicator system mainly consider the inclusion of other relatively independent elements, such as natural environment monitoring and human factors.

Although railway transportation is less affected by the natural environment than other modes, up to nearly 80% of passenger transport accidents are caused by bad weather or sudden natural disasters [10]. Environmental factors that are likely to cause accidents and equipment failure include heavy rainfall, lateral wind, lightning, earthquakes, ice disasters, and dramatic changes in temperature differences. However, unlike accident-based indicator design, the impact of environmental factors is uncertain. Therefore, this category of indicators should reflect the changes brought about by the environment to safety risks—especially the impact of sudden natural disasters on the safety situation.

The extension of the indicator system based on human factors is also similar to that based on environmental factors. According to Zhou et al. [30], four errors with the largest percentage of occurrence in the railway system are organization process, inadequate supervision, personal readiness, and skill-based errors. These human factors indirectly affect the overall safety situation through state changes at the supervisory and organizational levels. Therefore, human errors cannot be directly applied to the TOPSIS method, as in the accident-based approach, to deal with safety situations. Thus, a reasonable conversion process is necessary.

Considering the above possible extension direction of the indicator system, operation managers can encounter more incomplete or unobtainable information. For example, the influence on the safety situation of decision mistakes in human factors cannot be accurately evaluated using real numbers. To deal with these uncertainties and ambiguities, fuzzy theory is suitable for integration as an extension direction. This combined method can solve imprecise information by converting it into linguistic variables while retaining the effectiveness in decision making.

6.3. Practical Applications

With the increased availability of data, railway operation safety analysis and management techniques based on accidents or incidents play an important role. Operation managers can better perceive safety situations and discover potential risks through accident data mining. However, because of different purposes, different levels of managers have different needs for safety information acquisition. Generally, grassroots managers are more concerned about the specific safety performance and changes in each link of the system, whereas senior managers pay more attention to the overall safety situation of the system and the key features exposed. The comprehensive evaluation from a regional perspective in this study mainly serves the latter.

In actual safety management, railway senior managers are usually required to make comprehensive and guided decisions. For example, in China, railway managers in charge of a regional area are responsible for coordinating and supervising dozens of operation lines and hundreds of transportation resources. Faced with a large amount of available information, they need to avoid falling into data traps and make top-level decisions directly and efficiently. Therefore, the basis for making such decisions must also be regionally comprehensive and capable of judging trends.

The improved method proposed herein can appropriately meet the requirements in this regard. Distance-level evaluation results can help managers intuitively judge the performance of the safety situation throughout the entire historical period. Trend-level evaluation results can enable managers to make better judgments about decision-making directions. The TOPSIS method also has advantages in avoiding paying too much attention to the original situation sets. The division of the safety situation levels, as shown in Section 5.2, is also an effective practical application.

7. Conclusions

In this paper, six situation indicators were designed from a regional perspective and an improved TOPSIS method was proposed for the ROSSE. Starting from the multiattribute analysis of railway accidents, the safety situation can be characterized by the situation set of indicators, including the number of accidents, number of accidents per unit turnover, accident grades, structure of accident properties, degree of accident correlations, and average recovery time of accidents. Corresponding quantification methods were given, and the parameters involved in the indicators were preliminarily estimated.

In terms of the evaluation model, the traditional TOPSIS method was improved by combining SPA theory and the CSM. However, the SPA theory was used to introduce the connection vector to calculate the relative closeness as the evaluation result at the distance level. Through the vectorization of the situation set, the cosine similarity between the situation vector and ideal vector was calculated as the evaluation result at the trend level. Moreover, a combined weight based on GT was applied in the evaluation process. Then, a case study using the proposed method was performed on railway accident statistics in a certain regional area of China from 2016 to 2018. The evaluation results show that the overall safety situation has undergone benign development, but it is still unstable. It is necessary to strengthen the measures of safety management by improving the relative closeness and limiting the fluctuations of cosine similarity. The control measures of high-speed railway passenger transportation should also be considered.

The effectiveness of the improved TOPSIS method was verified through two groups of comparative experiments. The results illustrate that the RRP was effectively solved and that the improved method is more sensitive to situation awareness. The measurement of cosine similarity can also help to better understand and evaluate the railway operation safety situation. A quality verification of the entire evaluation work was conducted through the analysis of actual reports and expert scorings. Method extensions and practical applications were also discussed. In addition, evaluation work can be further combined with other transportation organization or disruption management research to formulate collaborative optimization strategies, such as robust train timetabling [57], train platforming [58], rolling stock scheduling [59], and energy saving [60]. The method of parameter estimation needs to be improved.

Data Availability

The original accident data used to support the findings of this study are currently under embargo. Requests for data, 12 months after publication of this article, will be considered by the corresponding author.

Conflicts of Interest

The authors declare that they have no conflicts of interest.

Acknowledgments

This research was supported by the National Natural Science Foundation of China (U1834209) and the National Key Research and Development Plan of China (2017YFB1200700). The authors would like to thank Editage (http://www.editage.cn) for English language editing.