Abstract
A Benefit-Cost-Deficit (BCD) model is proposed for analyzing such intentional human errors as barrier removal, the deliberate nonrespect of the rules and instructions governing use of a given system. The proposed BCD model attempts to explain and predict barrier removal in terms of the benefits, costs, and potential deficits associated with this human behaviour. The results of an experimental study conducted on a railway simulator (TRANSPAL) are used to illustrate the advantages of the BCD model. In this study, human operators were faced with barriers that they could choose to deactivate, or not. Their decisions were analyzed in an attempt to explain and predict their choices. The analysis highlights that operators make their decisions using a balance between several criteria. Though barriers are safety-related elements, the decision to remove them is not guided only by the safety criterion; it is also motivated by such criteria as productivity, workload, and quality. Results of prediction supported by the BCD demonstrate the predictability of barrier violation
1. Introduction
A risk analysis process may require studying both technical failures and human errors. According to Swain and Gutmann’s definition [1], human error occurs when human operators perform their requisite tasks incorrectly or when they perform other nonrequisite tasks that could result in degraded system performance. These other tasks may simply enhance the existing uses of a given system, and they can also be intentional deviations from requisite tasks, as is the case with violations. Existing methods of human error analysis generally assess only the unintentional errors that occur during the execution of requisite tasks. In addition, given their heterogeneous results, it would appear that these methods are difficult to use [2]. For this reason, new research is needed to define original methods for explaining and predicting human error, methods that are capable of taking into account the capacity of human operators to both avoid and correct undesirable events, as well as provoke their occurrence.
The BCD model presented in this paper is one solution to the problems mentioned earlier. It takes into account both the positive and the negative impact of violations, such as barrier removal. A barrier is a safety-related system that human operators may choose to remove in order to improve system performance. Our BCD model can explain and predict such choices in terms of the benefits, costs, and potential deficits that the action may produce.
Section 2 of this paper defines the concepts of “barrier” and “barrier removal”. Section 3 develops the principles of the BCD model. In Section 4, an experimental feasibility study that attempts to explain and predict barrier removal within the BCD model is described, and Section 5 presents the results of the study. Section 6 discusses the results obtained in terms of future research perspectives.
2. Barriers and Human Error
A barrier is a technical or procedural means designed to avoid the occurrence of undesirable events (e.g., component failure) or to protect a given system from the consequences of such events [2]. Human operators are components of the system, and, as such, can “fail” if their behaviour deviates from the behaviour stipulated in the system specifications. Clearly, human operators are able to avoid and/or correct incidents or accidents; however, they may also be the cause of such events. Many methods can be used in order to analyze the potential deviation from the rules, but, regardless of the method chosen, the designer must choose the optimal means of prevention or protection, given the nature of human error. Thus, to protect a system from potentially unsafe human behaviour, barriers are necessary.
Four barrier classes can be distinguished [3, 4]:
(i)material barriers that physically prevent an action or limit the negative consequences of a situation,(ii)functional barriers that logically or temporally link actions and situations,(iii)symbolic barriers that require interpretation, and(iv)immaterial barriers that are not physically present in the work situation.Nevertheless, some authors purpose other barrier classifications and introduce the notion of sociotechnical barrier [5]: a combination of technical, human, and organizational means that prevents or protect against an unwanted consequence. The classification is not so important. The main objective is to identify in a system hazard sources and means associated to prevent or protect from accident. Barrier analysis is a useful means of identifying the ways in which particular defenses either did or did not protect a target system from particular hazards [6].
A set of barriers is generally put in place during the design phase in order to protect the system. A basic hypothesis would hold that if all physical barriers are operational and all immaterial barriers are respected, the human activity is guaranteed to be safe. However, field observations of human operators indicate that human behaviour may interfere with this hypothesis [7, 8], particularly when the operators do not voluntarily respect these barriers. Actually, as suggested by Rasmussen [9] and Amalberti [10], systems can progressively drift from a safe and normal operating mode to an unsafe operating mode as a result of diverse pressures (Figure 1) “The safe space of performance, as expected and calculated during design, is contained within three boundaries: the individual and social regulations; the market rules; and the safety rules. When in use, the system migrates through the safety boundaries towards more performance and more individual benefits. The new resulting operational space of performance becomes largely positioned outside the initial safe space of performance. This new space is characterised by reduced margins to incidents and accidents (despite safety remains acceptable) and numerous violations and deviance. The safety trap should consist in that situation to continue a simpleminded strategy fighting violations with the development of new regulations. Cumulative regulations will then have the effect to change nothing in operations (the system is stabilised), mechanically increase violations, increase reluctance and opacity in incident reporting, and add noise in the safety monitoring strategy” [10].
The principal pressure is certainly economic. For example, in order to improve productivity, designer specifications may be ignored, resulting in an increased work pace or a change in the regularity and number of maintenance operations in order to limit production halts. Such migrations may originate in organizational needs and management decisions, with the final outcome being that some of the barriers specified by the designer are inhibited.
In addition, human operators contribute a system’s migration from safe to unsafe conditions. Locally, they must determine the operational means that will allow the new productivity constraints to be respected. Moreover, operators also take into account their own constraints, which are essentially related to workload and personal interest. Again, the final outcome is that some barriers are inhibited, resulting in a lack of protection.
Intentional deviation from the behaviour required by the system specifications, which does not have a negative effect on system performance, is called a violation [11, 12]. Barrier removal is defined as the voluntary inhibition of a barrier [13]. Thus, barrier removal, or the intentional misuse or nonrespect of a barrier under appropriate conditions, constitutes a violation.
3. The BCD Model
The Benefit-Cost-Deficit (BCD) model is an explicative and predictive model that describes a given barrier removal (BR) via three attributes.
(i)Benefit (B): barrier-removal is a goal-driven behaviour offering an immediate benefit that is seen to outweigh the cost.(ii)Cost (C): in order to remove a barrier, the human operator must sometimes modify the material structure and/or the operating mode. Such modifications have a cost, usually an increased workload and/or negative consequences on productivity, quality, or both.(iii)Deficit (D): because removing a barrier introduces a potentially dangerous situation, such actions create a potential deficit due to the related risk.With this BCD model, human operator decision-making becomes a process of balancing the advantages and disadvantages of removing a barrier [14]. In order to describe this balancing process more precisely, it is necessary to adopt a multicriteria viewpoint. Clearly, the behaviour of human operators has consequences on safety, and also on other criteria, such as productivity, quality, and workload, to name a few. These criteria depend on the nature of the activity.
The benefit and cost factors take into account the consequences of barrier-removal if it happens normally, with no deviate events, while the deficit factor evaluates the consequences of the barrier-removal if deviate events occur (cf. Figure 2). The deficit factor is related to the potential risk associated with barrier-removal. Determining benefit, cost, and deficit requires evaluating the consequences of operator behaviour. When evaluating the consequences of a given behaviour, either a designer-specified behaviour or a barrier removal, two cases are possible. The first considers the success of the behaviour, and the result of the evaluation is denoted CS. The second considers the failure of the behaviour, and the result is denoted CF. These evaluations are done by human operators and constitute a sort of risk evaluation procedure. Since the evaluation takes several criteria into account, CS and CF are vectors. The analysis of a human operator’s behaviour can integrate or ignore the possible interference of other behaviours by the same human operator or by other human operators. To calculate the benefit B(BR), cost C(BR), and deficit D(BR), two behaviours must be evaluated: the designer-specified behavior-evaluated using CS(P) and CF(P)—and the barrier-removal behaviour—evaluated using CS(BR) and CF(BR). Given CS(P), CF(P), CS(BR), and CF(BR), the vectors B(BR), C(BR), and D(BR) can be calculated as follows (cf. Figure 2 and (1)):
where is the criterion considered.
The decision to remove a barrier, or not to remove it, is guided by the results of the Benefit, Cost and Deficit evaluations, which are used to estimate the usefulness of both choices (cf. Figure 3).
The CS vector is calculated for each evaluation criterion. This calculation follows from the intermediate calculation , where is the instantaneous performance and is a predefined threshold with regards to a criterion :
CF is similar vector determined using
In order to illustrate this decision-making process, let us consider a road traffic situation. A driver is on a road whose speed is limited to 90 km/h over a distance of 5 km. This driver is following a vehicle driving at 60 km/h. According to the Highway Code, the driver cannot overtake because of the continuous line in the center of the road. Nevertheless, the driver has two options: to follow the slow-moving vehicle, thus conforming his/her behaviour to the Highway Code, or to overtake the vehicle, thus engaging in barrier removal. An analysis of the driver’s behaviour may or may not take the possible interference of other vehicles’ normal and/or abnormal behaviours into account. If the driver respects the rules, driving time will be 5 minutes with a fairly low probability of an accident occurring, given no other interfering events. If the driver decides to overtake, the driving time will be reduced since the vehicle will be able to proceed at the lawful speed of 90 km/h. However, even given no other interfering events, the probability of an accident will be higher, since another vehicle could arrive on the opposite side of the road during the overtaking maneuver.
The benefit accrued by barrier removal in this case is only 1.7 minutes. Though there is no cost, there is a potential deficit in that an accident is possible (cf. Table 1).
Copying with a new task, human operators will normally evaluate the potential advantages of removing a barrier. After considering the benefit, cost, and deficit factors, they then decide whether or not to procede. This decision is supported by the operators’ knowledge, which is updated based on operator observations of the results of the chosen action. (cf. Figure 3). Since the human operator’s point of view may differ from the designer’s point of view, the BCD model of barrier removal tries to take this divergence into account. In order to illustrate this divergence, an experimental study was conducted in collaboration with the Technical University of Delft (TU Delft, The Netherlands).
4. Experimental Study
The TRANSPAL experimental platform, developed in the LAMIH research laboratory at the University of Valenciennes (France) in collaboration with the Delft University of Technology (The Netherlands), was used to study risk perception in human operators, the acceptable degree of the risk inherent to barrier removal, and the interdependencies between the various risk evaluation criteria. This platform simulates train movement from depot to depot, via several transformation stations at which human operators load/unload the products located in a train stopped at the station platform. A human operator controls the flow of train traffic. In order to limit the relative risks and to control the traffic flow, several barriers were defined, both material and immaterial. Figure 4 presents a sample of the TRANSPAL Human-machine interface.
In the experiment, seven barrier classes were set according a preliminary risk analysis. In this paper, only the four classes of materiel barriers are discussed:
(i)BR1: signals for input/output movements at the depots,(ii)BR2: signals for input/output movements at the transformation stations,(iii)BR3: signals before and after the shunting device,(iv)BR4: stop signals at the transformation stations during the loading/unloading of goods.Twenty human operators participated in this study, and their barrier-removal choices were analyzed. The participants were voluntary students (Ph.D.) or researchers from 2 laboratories (between 22 and 40 years old): the Man-Machine System team for TU-Delft (The Netherlands, 10 participants) and from the LAMIH (France, 1à participant). The human operators mission was to supervise the train traffic process and to control the different signals and switches that manage train movement. Operator performance was computed dynamically in terms of productivity (e.g., rate of jobs at the transformation stations), quality (e.g., schedule delay), safety (e.g., train collisions or absence of announcements concerning train movement in transformation areas), and workload (e.g., the number of actions executed on the HMI). For each criterion, CS and CF were computed using formulas given in (2) and (3) (cf. Figure 5).
After a training period, the human operators were presented with two experimental configurations of the TRANSPAL platform. The first one was a full-barrier configuration. After running the first configuration, the human operator was allowed to define the second configuration by selecting the barriers to be included. This operator-designed configuration led to the removal of some signals. The human operators then controlled the barrier-removal situations. In order to get an idea of their reasons for selecting or rejecting the various barriers, a questionnaire based on the BCD model was presented to the operators after each phase of experimentation. The questionnaire contained four subquestionnaires related to TRANSPAL’s four classes of material barriers. For each barrier class (BR1 to BR4), the operators were asked to give their own estimation of the usefulness of the barrier and the benefit, cost, and deficit related to removing these barriers (Barrier-removal). These factors were estimated with regard to four criteria: productivity, quality, safety, and workload (cf. Table 2). In addition, the operators were asked to estimate their own performance in terms of these four criteria following each phase of experimentation.
5. Results
The two principal objectives of the study were to determine the reason for barrier removal and to analyze whether human operators correctly estimate the consequences of their intentionally deviate behaviour. The statistical analysis of barrier-removal yields useful preliminary results.
5.1. An Explanation for Barrier Removal
The questionnaire responses were analyzed in order to determine the individual motivations for removing barriers. Human operators felt that the utility of the barriers in classes BR1, BR2, and BR3 was related to safety, whereas they thought that the utility of the BR4 barriers was primarily related to productivity. They deemed the benefit of barrier removal important in terms of quality (time saved) and workload (with regard to interaction with the interface) for BR1, BR2, and BR3 barriers, despite the cost and potential deficit in terms of safety (cf. Figure 5). However, the operators thought that removing BR4 barriers produced no benefit, but induced both cost and potential deficit in terms of productivity (cf. Figure 6). This second result combined with the first allows the four criteria (quality, productivity, safety, and workload) to be hierarchized. In this study, the principal criterion on which human operators appeared to focus the most attention was productivity. If the human operator thought a barrier removal could have negative consequences on productivity, the corresponding barrier was not removed. The secondary criteria were quality and workload. If the human operator judged that barrier removal would have no negative consequences on productivity but might improve quality and/or workload, the barrier was removed.
The analysis of the questionnaire also highlighted the similarity between the operators’ evaluation of barrier usefulness and the potential deficit resulting from barrier removal, which would appear to indicate that human operators are aware of the barrier’s protective role. Barrier-removal led to a decrease in the workload though this result may be more related to the characteristics of the TRANSPAL platform than anything else. In the group of human operators who removed more than 10 barriers, 70% decreased their safety performance, as compared to only 40% for the group of operators who removed less than 10 barriers. Throughout the experimentation, barrier-removal had no impact on productivity, which was not true for the quality criterion. In fact, in the group of operators who removed more than 10 barriers, 60% decreased their quality performance, while 100% of those who removed less than 10 barriers increased their quality performance. Some results were minor. For example, the more barriers removed, the lower the workload and the lower the safety level. However, combined with the results of the analysis of operator motivations, the effects of barrier-removal demonstrate an acceptable balance between workload and safety. Overall, those who removed a limited number of barriers experienced relatively limited negative consequences. Those who removed a large number of barriers were more likely to produce negative consequences. However, the human operators’ perceptions of the consequences of their decisions were inaccurate. They tended to focus their attention principally on the benefits, while minimizing the potential negative consequences of their behaviour.
5.2. Prediction of Barrier Removal
Prediction consists of determining the number of barriers that will be removed by human operators. A case-based reasoning system (developed by us for other applications) was used in order to study the feasibility of using the BCD model for predicting barrier removal (cf. Figure 8). Faced with a new case described in terms of the benefits, costs, and deficits of barrier removal, the case-based reasoning engine searches similar cases in order to propose a solution, and then determines the number of likely barrier removals for each barrier class. This case base contains the BCD evaluations and the barrier-removal decisions of a predefined number of human operators for each barrier class. Having access to additional subjective BCD consequences of possible human behaviours, the system is able to predict barrier removal decisions.
The case base was initialized using the data and decisions of the 15 first human operators. Prediction was done for the last five operators (cf. Table 3). The results of the barrier removal predictions were quite acceptable. The system was able to correctly predict barrier removal for a given class 90% of the time. In 70% of these predictions, the system yields the exact number of barrier removals. Only 10% of the predictions were erroneous. These erroneous predictions related to Operator 16 for whom two errors were made:
6. Discussion
The BCD model thus appears to help explain and predict such human decisions as barrier removals. Nonetheless, despite these positive results, three important areas require more research: the variability of human behaviour, the possible errors of subjective BCD assessment and the possibility of criteria aggregation.
Each human operator had the option of removing 45 barriers, and as shown in Figure 8, their barrier-removal decisions are widely dispersed (from 3 to 36). Barriers in classes BR1 and BR2 were the most often removed, followed by those in class BR3. Barrier class BR4 was subject to only one removal. These initial results are very important, illustrating as they do that the barrier-removal decision does not depend only on the barrier’s intrinsic characteristics but also on the human operator’s preferences.
Human operators were asked to estimate the evolution in their own performance between the two phases of the experiment. Table 4 compares their estimations with the actual results. Overall, human operators accurately estimated the positive evolution of their performance:
Workload and quality were the most accurately perceived criteria. Productivity, on the other hand, is usually underestimated, while safety is consistently overestimated. In addition, though the benefits of barrier removal are usually accurately estimated, the cost is frequently underestimated.
Since there are several criteria, it is necessary to aggregate the total benefit, cost and deficit related to each criterion. Clearly, the operators consider some criteria (i.e., productivity) to be more relevant. In addition, benefit, cost, and deficit are not given the same priority. A utility function would allow the results to be aggregated while also taking user priorities into account. Theoretically, the decision-making process could be formalized using the following utility function:
where is the utility of the barrier-removal, is a criterion, is the number of criteria considered, is the weight of the criterion, is the weight of the criterion in terms of benefit, is the weight of the criterion in terms of deficit, is the weight of the criterion in terms of cost, is the benefit associated with the barrier-removal with regard to criterion , is the cost associated with the barrier-removal with regard to criterion , is the deficit associated with the barrier-removal with regard to criterion , is the probability of success, is the probability of failure.
The above formalization is able to take into account the different priorities that users assign to the criteria as well as the probability of erroneous human perceptions. Because each criterion does not have the same importance in the decision-making process and is, in fact, perceived differently, the weight can be used to take this kind of subjectivity into account. Since some decision-makers consider that the expected benefit is more important than the potential deficit, the model also weights the benefit, cost, and potential deficit (resp., , and ) so that this reality can be taken into account.
Results of the experimental study presented in this paper are confirmed by studies realized in real field. For example, works of Morel and Chauvin related to fishing vessels collision illustrate resilient mechanism dealing with barrier violations [15]. Violation of barriers may be seen as a resilient mechanism [16]. In order to satisfy some operational constraints, human operators, and also the organization, accept to violate some barriers [17]. The BCD approach gives some explanation of this constraints relaxation. Moreover, the flexibility of a system is necessary to manage unforeseen situations. Barriers violations may be a mean to be more flexible. However, only a part of barriers violations have to be considered as resilient mechanisms. Some barriers violations lead to potential hazardous situations. The comparison of subjective BCD and objective BCD may be used to distinguish resilient behaviours between other ones. The main question is “which are the acceptable violations and under which conditions?” The challenge in the future is to design Man-Machine Systems more resilient [18] in order to avoid negative barrier deviations. These systems have to be more flexible and able to adapt their behaviour and task allocation regarding the current situation.
7. Conclusion
This paper has developed an original model to explain and predict decision of barrier deviations. This kind of deviation may lead to an error. This model assesses the positive and the negative consequences of a given action by evaluating the benefits, costs, and potential deficits associated with a barrier-removal decision. Validated through experimentation, this BCD model confirms that human operators consider barriers to be constraints whose removal relaxes the constraints, resulting in improved system performance. Although human operators are aware of the barriers’ protective role, other criteria often supercede this awareness in the decision to remove a barrier. Clearly, though barriers are safety-related elements, the decision to remove them is not guided only by the safety criterion; it is also motivated by such criteria as productivity, workload, and quality.
The results of our study demonstrate that the BCD approach is useful for analyzing barrier removal and for building a mental model of the human operator. Our long-term research goal is to improve risk analysis and safety design. Currently, risk analysis is guided by a single objective: safety. However, on site, human operators must manage several goals at the same time, for instance, the desires to improve productivity and quality, reduce operator workload, and avoid accidents. Confronted with this multicriteria problem, their solution is often to remove some barriers in order to satisfy the majority of constraints. The resulting migration is not a simple combination of erroneous acts, but an intentionally designed activity based on violations.
Since one of the primary objectives of risk analysis is to evaluate and/or validate system design, risk analyses must begin to take these violations into account by adopting a multicriteria approach to avoid counter-productive decisions being made during the design phase. For example, new barriers could be added to the design in an effort to increase the level of safety, incidentally making it more difficult to achieve productivity goals. The risk analysis needs to evaluate the impact that new barriers will have on the other criteria. In the case of negative impact, the BCD model can be used to evaluate barrier-removal decisions.