Using the Functional Resonance Analysis Method (FRAM) in Aviation Safety: A Systematic Review

Tian, Wulin; Caponecchia, Carlo

doi:https://doi.org/10.1155/2020/8898903

Journal of Advanced Transportation

On this page

Abstract Introduction Results Discussion Conclusion Conflicts of Interest References Copyright Related Articles

Special Issue

Methodologies for the Analysis and Prevention of Aviation Safety Occurrences

View this Special Issue

Review Article | Open Access

Volume 2020 | Article ID 8898903 | https://doi.org/10.1155/2020/8898903

Using the Functional Resonance Analysis Method (FRAM) in Aviation Safety: A Systematic Review

Wulin Tian¹and Carlo Caponecchia¹

Academic Editor: Kyriakos Kourousis

Received07 Aug 2020

Revised01 Oct 2020

Accepted06 Oct 2020

Published21 Oct 2020

Abstract

The functional resonance analysis method (FRAM) is a system-based method to understand highly complex sociotechnical systems. Besides learning from safety occurrences or undesirable states, FRAM can be used to understand how things go well in a system, by identifying gaps between “work as imagined” (WAI) and “work as done” (WAD). FRAM is increasingly used in many domains and can enhance our understanding of a complex system and proposes strategies to refine the work design. This systematic review identified 108 FRAM research papers from 2006–2019. Most of these papers were conducted by European researchers and employed qualitative methods such as document analysis, interviews, and focus groups with subject matter experts (SMEs) and observations to develop WAI and WAD. Despite being used in healthcare, construction, and maritime sectors among others, aviation was the most commonly explored domain in FRAM studies. The 26 FRAM studies in aviation explored many aspects of the aviation industry, including Air Traffic Control (ATC) systems, cockpit operation, ground handling, maintenance, and a range of past safety incidents, like runway incursions. This paper also characterises the FRAM studies focused on aviation in terms of the common methods and steps used to build FRAM and the available software tools to build FRAM nets. Current FRAM illustrates its advantages in capturing the dynamic and nonlinear nature of complex systems and facilitates our understanding and continual improvement of complex systems. However, there are some critical issues in FRAM use and interpretation, such as the consistency of methods and complexity and reliability of data collection methods, which should be considered by researchers and FRAM users in industry.

1. Introduction

Complex systems comprise different groups of humans, technologies, and organisations that may interact with each other in many industrial domains. Ladyman et al. [1] argued that a complex system has the following features: nonlinearity, feedback, robustness and lack of central control, emergence, spontaneous order, hierarchical organisation, and numerosity. The essential characteristic of complex systems is nonlinearity; that is, the presence of factor A does not necessarily lead to outcome B, and vice versa [1]. A complex system consists of numerous interacting components. Components affect and are affected by one another. Normally, this kind of feedback does not affect the overall system’s operational status. The system itself is robust enough and able to absorb minor variabilities. Nevertheless, under particular situations, the same set of variabilities may make the system fail. In other words, interactions between different components are dynamic and emergent, rather than static and ancillary. The dynamic conditions and interactions make the system behaviour difficult to predict. The traditional simple linear relationship cannot explain the entire complex sociotechnical systems comprehensively (e.g., [2, 3]).

One increasingly common method for analysing complex systems is the functional resonance analysis method, known as “FRAM.” FRAM is a system-based analysis method, which considers the whole system and focuses on its functioning rather than the structure of its components. Originally known as the “functional resonance accident model,” FRAM was initially established by Hollnagel in 2004 [4], to investigate accidents and incidents in complex social-technical systems. However, learning only from accidents/incidents is not sufficient to understand the interactions between technologies, humans, and organisations that make up a complex sociotechnical system. This is especially the case when it comes to ultrasafe systems, such as in the aviation and nuclear industries. Safe operation of complex systems requires a better understanding of how the work is actually carried out (“work as done,” WAD) [5]. FRAM’s scope has been expanded from an accident model to a more general analysis method, termed the functional resonance analysis method. The gaps between WAD and how the work is supposed to be done (“work as imagined,” WAI) generate variabilities in the daily working procedure. Any single instance of variability alone is not able to lead to the accident/incident. However, under specific conditions, these variabilities may lead to functional resonance, causing an undesired outcome or even crashing down the entire system.

Since its establishment, experts from different disciplines have applied FRAM to a range of industries or contributed to developing FRAM theory and methods. The advantages of FRAM as a methodology have been suggested by several studies. Applying FRAM can facilitate a better and more in-depth understanding of interactions between complex system functions. For example, Woltjer and Hollnagel [2] applied FRAM to analyse the Alaska Airlines Flight 261 accident. The analysis suggested that FRAM could capture the dynamic and nonlinear nature of this complex system failure. Later on, Hollnagel and colleagues [6] again used FRAM to reanalyse the Comair Flight 5191 accident. The FRAM analysis results suggested a number of additional countermeasures compared to those suggested in the official NTSB report. By monitoring how component variabilities resonate, FRAM modelling can also identify the critical path of variabilities that emerged in the dynamic system [2, 7]. Using FRAM to detect the gap between WAI and WAD helps us improve safety and work design [8]. Compared to currently employed investigation methods and other systematic methods, such as the Domino model, Swiss Cheese Model, and the Bow-Tie model, FRAM is able to analyse complex systems and provide more comprehensive results [6, 9–11]. For example, Hollnagel and colleagues [6] reanalysed the Comair Flight 5191 using FRAM. The National Transportation Safety Board (NTSB) report mainly identified crew members’ mistakes and flaws in the Federal Aviation Administration (FAA) requirements for Air Traffic Control (ATC) clearance authorization. By taking the whole context into consideration, FRAM unveiled further details of why the accident occurred than those presented in the NTSB official report. These included that information regarding runway construction and closure was missing in both Notice to Airmen (NOTAMs) documents and the Automatic Terminal Information Service (ATIS) and that the first officer was too busy to monitor the aircraft’s position on the runway.

Given the increasing use of FRAM to better understand complex systems and its potential for use retrospectively and prospectively, this paper aims to systematically review FRAM studies, with an emphasis on how it has been applied in aviation. We first outline the principles of FRAM and how the method should be conducted, before reviewing the literature and analysing the methods used, the locations of FRAM studies, and the systems to which FRAM has been applied.

2. Principles of FRAM

The following section outlines the theoretical principles that guide the implementation of FRAM. The first principle is that of the equivalence of success and failures [5]. Traditional safety theories emphasised learning from system failures, such as incidents and accidents [5]. However, learning from failures is not enough for keeping current complex sociotechnical systems safe. FRAM can be applied to analyse either system incident/accident or the normal operational procedure. According to FRAM, to understand what goes right when the daily work is carried out is as important as understanding what failed in the system.

The second principle is that of approximate adjustments [5]. Human performance can be influenced by many factors, both internal and external, such as fatigue, stress, emotions and mood, vigilance, task demands, and deadlines. Sometimes organisational factors such as the effectiveness of communication or unclear guidelines can make workers’ tasks more difficult. The complex working context may make the work task more challenging and require workers to make their own decisions. Workers have to adjust their behaviour accordingly to meet the system’s requirements to produce the desired outcome. They usually need to make some trade-offs between being efficient and to make sure the work can be completed as precisely as possible. Hollnagel [4] termed these kinds of adjustments as efficiency-thoroughness trade-offs (ETTOs). These adjustments are necessary and understandable; however, any changed system behaviour may raise variabilities in the system.

The third principle is that of emergence [5]. Under each analysed case, the context and combination of variabilities in the system are unique. As the interactions within a complex system are dynamic and nonlinear, the occurrence of an outcome is emergent. To be more specific, minor variabilities always exist in normal system operations and do not affect system safety. However, the particular external environment may integrate variabilities and magnify their influence to generate an undesired outcome.

The fourth principle is that of functional resonance [5]. From the FRAM perspective, variabilities exist in normal daily operations. These small variabilities may not be able to crash the system alone, but aggregated with other variabilities in the system may cause resonance, which generates a negative outcome. The whole system should be taken into account, instead of focusing on one segment of the system.

In performing a FRAM analysis, several steps are involved, which then takes these principles into account. The following describes the main steps of conducting a FRAM analysis [5]: Step 0: define the purpose and scope of analysis. Before the analysis starts, analysts should clarify whether the analysis would be conducted in relation to an incident scenario or the context of normal operations. As there is no clear boundary to conduct the FRAM and the results of FRAM would be complicated, setting the scope of the proposed analysis can prevent the results from being too detailed and too complicated. For example, to examine the catering delivery procedure during a flight turnaround, it would be good to start from the function “preparing food in the catering department” and end up with “crew confirms catering delivery.” Otherwise, the food preparation process could be traced to very early stage functions, like “purchase ingredients,” “design menu,” or even “grow vegetables.” Without a clear boundary, the FRAM analysis might generate a lot of data and appear comprehensive but be ultimately unhelpful in understanding the system of interest. Step 1: identify and describe the essential system functions. FRAM deconstructs the complex sociotechnical system into “functions.” A function is a task or an activity that is required to produce a certain outcome. According to Hollnagel [5], there are three types of functions: technological functions, human functions, and organisational functions. Each identified function can be described by six aspects (see Figure 1): Input (I): input is what activates or starts a function and/or that is used or transformed by the function to produce the output. Preconditions (P): preconditions refer to the conditions that must be satisfied before a function is carried out. Preconditions alone cannot activate a function. Resource (R): resource is something that is needed or consumed when the function is active. Resources will be consumed up as the function is executive. Output (O): output is the outcome of a function. Control (C): control is that which supervises, regulates, or monitors the function such as guidelines, regulations, or even social expectations. Time (T): time refers to the temporal constraints on the function such as duration and starting point. Function identification can start from anywhere within a complex system. Documents such as an operation manual or task procedures are useful resources for identifying system functions. Each function interacts with other functions via one or more of their aspects. Interactions connect functions together to form a FRAM net. Figure 2 shows a simple FRAM net related to the cabin crew’s work procedure before take-off. Each hexagon represents a function and its six aspects. For example, the output of the function “Greet passenger” is that all passengers are welcomed on board. This output can transform to be the input of the following function “Confirm the number of passengers.” Functions that operate before others, and have a potential effect on others, are termed “upstream” functions. Functions impacted by others are “downstream” functions. For example, in Figure 2, the function “Confirm the number of passengers” is upstream of the function “Close headlock” and downstream of the function “Greet passengers.” Step 2: identify the actual or potential variabilities between functions. According to Hollnagel [5], performance variabilities can be categorized in terms of their origins, internal (endogenous) variability, and external (exogenous) variability. Internal variability refers to the variation caused by the function itself, such as software pitfalls or operator’s experience. By contrast, the external variability refers to the influence of other functions, such as weather conditions and organisation’s culture. Hollnagel [4] classified the external variabilities as eleven common performance conditions (CPCs): availability of resource, adequacy of training and experience, communication, adequacy of interface and operational support, availability of procedures or plans, working conditions, number of simultaneous goals, available time, crew collaboration quality, and adequacy of organisation. In 2012 [5], Hollnagel again suggested an elaborate solution and a simple solution to consider the performance variabilities. The elaborate solution identifies variabilities in terms of speed, distance, sequence, object, force, duration, direction, and timing. Meanwhile, the simple solution considers function output variability in terms of timing and precision. According to the simple solution, the output of a function would generate too early, on time, too late, or not at all and can be precise, acceptable, or imprecise. In our example in Figure 2, for instance, the function “Run safety check” could be completed too late and carelessly/imprecisely, which brings variabilities into the system. Step 3: analyse the aggregation of variability. In the FRAM net, the output of a function interacts or “couples” with other functions which are represented in the net by lines connecting the functions (known as “couplings”). As illustrated in Figure 2, the function “Conduct safety briefing” couples with function “Run safety check.” In this way, the output of an upstream function may vary and then transfer the variability to its downstream function(s). All upstream-downstream couplings can be analysed in terms of timing and precision. Taking one of the couplings from Figure 2, for example, if the output of the upstream function “Conduct safety briefing” comes too late, the downstream function “Run safety check” would experience a delay. The aggregation of these variabilities may cause resonance in the system, which leads to an undesired outcome. In the present case, the flight may not be able to take-off on time. Step 4: propose ways to manage variability. The previous steps help to identify the variabilities and their potential aggregation within the system. The last step should be proposing strategies for managing variability. The FRAM considers system success and failures equivalently. We should consider different management strategies for variabilities that lead to positive outcomes and those contribute to an undesired result. Clarke et al. [12] assessed the potential risks of transferring cargo between two floating harbour transhippers (FHTs) using FRAM. The results showed the manner in which variabilities are added together to influence the system. Some recommendations were developed to improve system design. For example, the number of crew should be sufficient on both vessels, and equipment should be regularly checked and placed in designated places [12]. Despite the growth in the frequency with which it is used (e.g., [13–15]), there are few systematic reviews that examined FRAM and its implementation. Some reviews have included FRAM papers, but in the context of examining other system-based analysis tools. For example, Hulme et al. [16] reviewed peer-reviewed articles that applied systemic accident analysis methods to understand contributing factors between 1990 and 2018. They chose four groups of system-based accident analysis methods: AcciMap, the Human Factor Analysis and Classification System (HFACS), the System Theoretic Accident Model and Processes-Causal Analysis based on STAMP (STAMP-CAST), and FRAM. Only four FRAM studies were included in their analysis. The authors examined accident contexts, the number of identified functions, source of accidents, the nature of accidents, and features of eligible articles. All these reviewed analysis methods resulted in multiple contributing factors, couplings, and functions. However, they concluded that the results of FRAM would be highly complex and difficult to interpret [16]. Little is therefore known about the pattern of use of this emerging systems safety tool. Accordingly, the present systematic review aims to outline how, where, and for what purpose FRAM has been used, with a particular focus on how FRAM has been applied in the aviation industry.

3. Methodology

FRAM was initially developed to investigate accidents and known as the functional resonance accident model. By adopting the Safety-II perspective [5], FRAM transformed to the functional resonance analysis method and expanded its analysis scope to system normal operation (e.g., [17]). Preliminary searches showed that searching for “functional resonance analysis method” alone returned thousands of papers concerned with “functional magnetic resonance imaging” (fMRI) from medical journals and other methods or theories. Accordingly, the functional resonance analysis method was joined with “FRAM” in subsequent searches. In order to cover all eligible FRAM research studies in English, we used the search item “functional resonance analysis” OR “functional resonance accident” and “functional resonance analysis method” AND “FRAM” in the title and abstract of as a keyword across five online databases: ProQuest, PubMed, ScienceDirect, Scopus, and Web of Science. The search timeframe was unlimited although it is recognised that most FRAM studies have occurred since 2012 when the key resource on the method was released. EndNote X9 for Mac was used to organise all references.

At the screening stage of the review, all records were assessed manually. Records were excluded if they focused on other systematic analysis methods rather than FRAM, such as STAMP (without a focus on FRAM). Book chapters, theses, commentaries, newspaper articles, and corrigenda were excluded. Documents where the full text was unavailable (after library databases and web searches) or the full text is in a language other than English were also excluded.

A taxonomy of current FRAM studies was developed to organise the resulting papers. Our analysis examined the distribution of the number of published papers across years, the regions of current FRAM studies, and the contexts of current FRAM instantiations.

4. Results

Figure 3 is a PRISMA (Preferred Reporting Items for Systematic reviews and Meta-Analyses) [18] flowchart diagram showing the search process. The initial search across five databases identified 2212 records. After removing duplicates, the remaining 1651 records were then screened manually by reviewing titles and abstracts. As mentioned above, the functional resonance analysis method is highly similar to fMRI in medical research, and 1481 records were excluded. Furthermore, when the remaining 170 records were assessed for eligibility, 75 records that did not meet inclusion criteria, such as full text being unavailable or the full text was not in English, were removed. In addition to the 95 eligible papers, 13 additional papers were identified from the reference lists of papers that had been found in the search. Finally, 108 papers were included in the analysis (see Figure 3).

Figure 4 shows the number of published papers over time. Before 2012, 10 out of 16 FRAM studies used accidents or incidents as instantiations. Later on, by adopting the perspective of Safety-II, FRAM expanded its analysis scope to normal operational conditions. In 2012, Hollnagel published the first book about FRAM, FRAM: The Functional Resonance Analysis Method: Modelling Complex Socio-Technical System. This book provides systematic background information, steps, and principles to use FRAM. The number of FRAM-related papers increased around two years later. However, since 2017, published FRAM papers reduced to 19 and 16 in the following two years, respectively.

The FRAM has attracted researchers’ attention worldwide. Among the eligible 108 papers, over half (56.48%, n = 61) were conducted by researchers from European countries, such as Denmark, Sweden, and Italy. Asian researchers from China, Japan, and other countries published 22 papers, accounting for 20.37%. South American researchers contributed 10.19% (n = 11) FRAM papers, while North American researchers published seven studies (6.48%). Oceanian researchers published seven studies (6.48%) related to FRAM. Studies on FRAM also facilitated international cooperation; however, the analysis above was based on the lead author’s location. For instance, Damen et al. [19] analysed preoperative anticoagulation management (PAM) in normal operations using FRAM. The study was conducted in surgery departments in both Australia and the Netherlands.

As a general safety analysis method, FRAM could be applied to a variety of complex sociotechnical systems. Researchers from different areas expanded and illustrated the application of FRAM in a wide range of instantiation contexts. Figure 5 shows that aviation, healthcare, construction, and maritime contexts are the most popular domains of FRAM application. Those contexts with fewer than three papers were categorized as “others” and included applications of FRAM that focused on the environment, policy-making, sport and recreation (hunting), and natural disasters (flood). As some researchers illustrated their proposals by analysing several cases, the total number of selected papers in Figure 2 is beyond 108. For example, Amorim and Pereira [20] applied FRAM to understand the accidents resulting from improvisation in workplaces and used aircraft maintenance, construction, and shoe manufacturing as case studies. Similarly, Moškon and colleagues [21] demonstrated their proposed method in five cases, including emergency room triage, fire prevention, construction management, aircraft take-off, and flight operations.

In terms of existing research methods, the results indicated that current FRAM is still predominately a qualitative method. Eighty-five papers (78.7%) employed qualitative approaches, such as interviews, documentary reviews, focus groups, or observations to develop WAI and WAD. More specifically, a majority of the papers that mentioned their method to map WAI indicated that they used document review and analysis [14, 19, 22–24]. In order to get an insight regarding WAI, the most popular methods are semistructured interviews, direct observations, and workshops with operators and regulators [3, 10, 23, 25–33].

The authors of the remaining 23 (21.3%) papers applied FRAM by using quantitative or semiquantitative methods, such as Monte Carlo simulation and modelling. Furthermore, 19 out of these 23 studies integrated FRAM with other methods to quantify their proposed models, including the Cognitive Reliability and Error Analysis Method (CREAM) (e.g., [34]), the Analytic Hierarchy Process (AHP) (e.g., [35, 36]), and finite state machine (FSM) (e.g., [37]). Among these quantitative and semiquantitative studies, it was much more common practice to characterise the performance variability in terms of timing and precision, rather than the 11 CPCs as originally identified by Hollnagel [5].

5. FRAM in Aviation

As reported above, FRAM has been used most frequently in the aviation industry compared to other domains. Hence, we further analysed the 26 papers which applied FRAM to the aviation industry (see Table 1). The pattern of aviation FRAM studies is consistent with what we found in relation to characteristic of FRAM studies in other industries in terms of locations and methodologies.

In most of the 26 aviation FRAM studies, 19 (73.08%) were completed by researchers from European countries, while researchers based in Asia, South America, and Oceania published 4 (15.38%), 2 (7.69%), and 1 (4.35%) paper(s), respectively.

FRAM studies in aviation were also dominated by qualitative methodologies, with 20 papers (76.92%) using qualitative methods. Building the FRAM network to understand WAI and WAD, the common methods were literature reviews (such as examining previous relevant accident reports and operational manuals), direct observations, workshops, and interviews with SMEs, such as pilots and air traffic controllers. After completing the initial FRAM network, researchers invited SMEs to check the completeness and validate the FRAM network. Only 6 (23.08%) of the aviation papers attempted to quantify FRAM. Hirose et al. [34] adapted CREAM to use FRAM in a systematic and quantitative way. Yang et al. [14] used Simple Promela Interpreter (SPIN) to demonstrate the functional resonance in system. Patriarca and colleagues [32] discussed using the Resilience Analysis Matrix (RAM) to enhance FRAM-based accident analysis. Moreover, Patriarca et al. [13] proposed a method to quantify function variabilities in relation to the ATM system based on Monte Carlo simulation.

Hollnagel [5] suggested that variabilities can be defined from several perspectives. Except for five studies that did not clarify how they categorized identified variabilities, most of the existing research used the simple solution, considering variabilities in terms of only time and precision (n = 9). Seven studies used CPCs to identify variabilities. Some researchers proposed new ways of defining variabilities. For example, Duan et al. [9] considered variability “within” functions and “between” functions. Variability within a function refers to how an output could be influenced by variabilities from the other five aspects of the same function. Variability between functions refers to how the output variability of upstream functions could influence aspects of downstream functions. Frost and Mo [45] suggested using two sets of guidewords to identify potential variabilities. Firstly, the variability in each of the five aspects (input, time, control, precondition, and resource) is rated as early, delayed, absent, wrong rate, underspecified, or overspecified. Secondly, for the same function aspects, variability from the influence of the specific conditions (such as time pressure, goal conflicts, communication quality, and organisation culture) is assessed.

Half of the analysed papers (n = 13, 50%) followed the four-step FRAM analysis method. Few papers (n = 2, 7.69%) indicated that their FRAM analysis consists of five steps. However, some of them started the first step from determining the scope of the proposed analysis (see [3]). Some research only focused on the first three FRAM steps. For example, Macchi et al. [40] employed FRAM analysis to assess the risks of the Minimum Safety Altitude Warning system (MSAW). To explore more possibilities of FRAM, some researchers modified the original FRAM steps to satisfy their research goal. For example, Frost and colleagues [45] consulted expert groups to refine and confirm the baseline FRAM model. The validated FRAM model was used to develop instantiations for later analysis in a hazard identification and analysis (HAZID) workshop. The modified FRAM was then used with System Hazard Analysis (SHA) to identify system hazards. Yang and Tian [47] extended the original FRAM to a 7-step method. In addition to the original first three steps, they defined safety requirements and transferred FRAM models into the MuSMV model checker. The proposed approach was evaluated by analysing the landing process from the flight crew’s perspective. The analysis identified a series of variabilities which may violate safety requirements.

Topics of FRAM studies in aviation covered different aspects of aviation systems, including the overall air accident/incident analysis (n = 8, 30.77%), ATM/ATC system (n = 8, 30.77%), cockpit operation (n = 3, 11.54%), landing approach (n = 3, 11.54%), Airline Operation Control System (OCC) (n = 1, 3.85%), helicopter operation (n = 1, 3.85%), ground handling procedure (n = 1, 3.85%), and aircraft maintenance (n = 1, 3.85%).

Among these 26 aviation FRAM studies, 13 of them used FRAM in a prospective way to assess risks, while the remaining 13 studies used FRAM retrospectively to analyse what happened in aviation incidents. Meanwhile, some studies also used FRAM to assess the target system’s resilience characteristics. For example, Woltjer [39] reanalysed the Alaska Airline Flight 261 accident using FRAM and evaluated resilience characteristics, such as buffering capacity, flexibility, margin, tolerance, and cross-scale interactions, based on FRAM analysis.

Currently, there are two software tools developed specifically for FRAM: the FRAM Model Visualiser (FMV) and myFRAM. Both of them are useful tools to build a FRAM model net.

FMV works using Adobe Air, while myFRAM can be used in Microsoft Excel. The output of myFRAM can be exported to FMV. Only 7 out of 26 studies indicated the software they used to build and visualize their FRAM. All of these seven used FMV, while two of them also used myFRAM.

6. Discussion

The present study aimed to review existing FRAM studies to understand how, where, and for what purpose FRAM has been used, with a special interest in its application in the aviation industry. Our analysis provides a taxonomy of current FRAM studies with respect to their origins, contexts, and research methodologies. It can assist users of the method to interpret the assumptions in the implementation of FRAM and evaluate recommendations for system improvements.

Another recent review (see [16]) considered a range of systems safety analysis tools over a similar time period yet had a limited inclusion of FRAM, finding only 4 FRAM papers. By contrast, our systematic review found 108 FRAM papers from 2006–2019. Most FRAM research was conducted in Europe. While there was a wide distribution of focal domains including healthcare, construction, and maritime, most papers focused on systems in aviation such as ATC and aviation incidents. This suggests that FRAM is an emerging methodology in aviation safety, which may influence its adoption in other domains. Most of the aviation papers adopted a qualitative approach to gather information to build the FRAM net (e.g., using focus groups to collect data) and were evenly distributed in terms analysing past incidents and current systems operations.

Despite there being similarities in the general approach adopted (such as qualitative methods of interviews and focus groups and qualitative methods including Monte Carlo simulation) across all papers selected in the review, there was no standard method for completing FRAM. Hollnagel [5] indicated particular steps to conduct FRAM analysis in the first book about FRAM. Nevertheless, there was a significant variation in the number of FRAM steps used in papers (e.g., [34, 47, 50]).

Similarly, this diversity was reflected in the manner in which variability is indexed within FRAM. Several researchers used the “simple solution” (e.g., [7, 13, 19, 33, 51–55]), while others use more detailed indices of variability, such as the 11 CPCs (e.g., [23, 56–60]).

Given that FRAM is focused on mapping and understanding variability within complex systems, the divergent approaches to indexing variability could be a concern in relation to consistency and reliability. FRAM analysis is used to understand a sociotechnical system under specific conditions [5]. The result from one analysis cannot be generalized to another context [22, 61], and typically, it is not intended to be generalized. However, the issue of consistency of implementation of the method is relevant to evaluate the method itself. That is, like all systems analysis tools, while we may not be able to compare the outcomes because we are analysing different systems, it is still important to assess and compare the methods used to generate these outcomes. Doing so can result in refinement of the method and identification of practice that violates it assumptions.

Variations in methods when implementing FRAM has implications for users of FRAM and those interpreting FRAM results in industry. It is important to consider how these different methods could be used and compared: why they are used and whether they are appropriate for the system and users in question and how they may have influenced the outcomes of tool. In addition, it may be that particular methods for indexing variability are useful for particular purposes. For example, the simple solution of indexing variability only by time and precision may be most effective for FRAM users in industry. In this context, more streamlined versions of the method may be desirable so that it is easier to collect data in the field and to interpret and use the method in practice. Alternatively, other systems for which highly precise data are already available may lend themselves to collecting and analysing additional metrics of variability and to more complex analysis methods.

As reflected in our analysis, the majority of existing FRAM studies employed qualitative methods. While FRAM has been shown to be a very useful tool for analysing complex systems (e.g., [2, 19, 33, 53, 62, 63]), it has also been suggested that its qualitative nature means it cannot provide an accurate calculation of risks [64, 65]. Furthermore, due to such qualitative methods, FRAM can be very time-consuming and complex to learn and to interpret the results [56, 63, 64, 66]. Ways of simplifying FRAM while maintaining its rich analysis of complex systems require further exploration.

A consequence of the reliance on qualitative methods is that current FRAM results largely rely on the knowledge of subject matter experts to inform WAI and validate WAD. The typical SMEs in existing FRAM studies include accident investigators, ATC officers, safety experts, and pilots. Compared to novice users of the system, SMEs have more of an idea of how different system components work together. At the same time, the experience of (nonnovice) system users is desirable for FRAM. While it is difficult to train day-to-day system users in the use of FRAM, methods for collecting data from their direct experience are needed to inform WAD with sharp-end operational performance.

An alternative to qualitative analysis is to apply FRAM to simulated systems. Based on our analysis of the overall FRAM papers, the majority of quantitative research uses simulation data rather than data from real-working scenarios [8, 13, 45, 48, 60]. Simulation makes it possible to get a vast amount of data quickly and at low cost, while also avoiding the difficulty in data collection in the real world. Both approaches have their advantages and disadvantages. As mentioned above, qualitative methods are problematic because the time involved and the lack of quantification while quantitative methods do not take account of data from real-working scenarios.

One solution might be a hybrid model, whereby in addition to using SMEs’ knowledge in constructing WAI, the FRAM net is refined by data collected from system users, who are not experts in FRAM but are experts in the system use. Examples would include flight and ground crew, front-line maintenance engineers, and ramp workers. These data can be generated from real processes and collected automatically or from system user inputs and ratings.

A further potential criticism of FRAM is that it is usually applied to a relatively small part of a complex system rather than the entire system, while it attempts to understand complex systems. For example, aviation is a complex system that consists of numerous complex components, ATC being just one of them. Existing FRAM studies on ATC systems are more likely to analyse only one phase, such as the aircraft landing phase (e.g., [40]). FRAM results from the selected phase are already complex and difficult to interpret. Using the same method to analyse the whole system would be impractical in terms of time and resources. Some research has used FRAM to examine a more extensive system, such as accident reanalysis studies (e.g., [2, 6]). The functions were identified at macrolevels, where macrolevel functions could be examined further in detail to understand microlevel functions within them. In this way, FRAM could be used at different levels within a system, to build a “FRAM of FRAMs.” For example, a higher-level function may be a summary of a group of functions (a FRAM net of a lower level set of functions).

Interestingly, most aviation FRAM studies did not specify which software products they used to generate FRAM nets. The analysis software should be reported in the future to facilitate replication of results, as well as an understanding of the particular constraints under which FRAM nets are constructed and analysed. This is especially the case given that there are platforms other than FMV and myFRAM with which to build FRAM nets. Microsoft Visio and Power BI are two options that provide data visualisation solutions and are compatible with other system information. Using these new tools to visualize FRAM may provide more detailed, dynamic, and interactive information. These platforms are consistent with those currently used in industry for displaying a range of other data, such as finance and marketing [67], and should be further explored for FRAM and related complex systems analysis tools.

One of the limitations of this paper is that grey literature was not examined. There may be instances of FRAM use that exist in the grey literature or that have not been published. The focus of this review was to assess publicly available and published accounts of FRAM use. It would be interesting to analyse all instances of FRAM; however, companies may wish to maintain the confidentiality of their systems.

7. Conclusion

This study introduced and described the FRAM perspective, methods, and steps. We identified 108 existing FRAM studies from 2006 to 2019 and further analysed those applied in the aviation industry. Our findings added to our understanding of how, where, and in what systems FRAM has been used so far and its potential for understanding aviation safety occurrences. The present analysis also identified critical issues in FRAM use, which need be considered by researchers and FRAM users in industry, such as consistency, data collection methods, ease of use, and expanding the range of subsystems analysed and the analysis scope.

These results highlighted a range of issues for future research. While FRAM has been applied to various aviation areas, there are many other unexplored aviation systems that may benefit from FRAM analysis. Examples include cabin crew’s operations and catering. As discussed previously, the scope of the FRAM analysis could be expanded, to cover broader systems and understand how components interact. Nevertheless, while considerable effort has been made to implement FRAM in industry, some obstacles limit their application in practice. Future studies need to adapt FRAM implementation by simplifying methods for training in how to implement and interpret FRAM and methods for collecting system user data. The potential for using new tools to represent and analyse FRAM nets in a manner consistent with existing platforms already embedded in industry needs to be considered.

Conflicts of Interest

The authors declare that they have no conflicts of interest.

References

J. Ladyman, J. Lambert, and K. Wiesner, “What is a complex system?” European Journal for Philosophy of Science, vol. 3, no. 1, pp. 33–67, 2013.
View at: Publisher Site | Google Scholar
R. Woltjer and E. Hollnagel, “The Alaska Airlines Flight 261 accident: a systemic analysis of functional resonance,” in Proceedings of the International Symposium on Aviation Psychology, pp. 763–768, Dayton, OH, USA, April 2007.
View at: Google Scholar
M. Studic, A. Majumdar, W. Schuster, and W. Y. Ochieng, “A systemic modelling of ground handling services using the functional resonance analysis method,” Transportation Research Part C: Emerging Technologies, vol. 74, pp. 245–260, 2017.
View at: Publisher Site | Google Scholar
E. Hollnagel, Barriers and Accident Prevention, CRC Press, Boca Raton, FL, USA, 2004.
E. Hollnagel, FRAM: The Functional Resonance Analysis Method–Modelling Complex Socio-Technical Systems, Ashgate Publishing Limited, Farnham, UK, 2012.
E. Hollnagel, S. Pruchnicki, R. Woltjer, and S. Etcher, “Analysis of comair flight 5191 with the functional resonance accident model,” in Proceedings of the 8th International Symposium of the Australian Aviation Psychology Association, Sydney, Australia, April 2008.
View at: Google Scholar
F. Costantino, G. D. Gravio, and M. Tronci, “Environmental audit improvements in industrial systems through FRAM,” IFAC-PapersOnLine, vol. 51, no. 11, pp. 1155–1161, 2018.
View at: Publisher Site | Google Scholar
P. Wachs, A. W. Righi, and T. A. Saurin, “The functional resonance analysis method as a debriefing tool in scenario-based-training,” Advances in Intelligent Systems and Computing, vol. 819, pp. 132–138, 2019.
View at: Publisher Site | Google Scholar
G. Duan, J. Tian, and J. Wu, “Extended FRAM by integrating with model checking to effectively explore hazard evolution,” Mathematical Problems in Engineering, vol. 2015, Article ID 196107, 11 pages, 2015.
View at: Publisher Site | Google Scholar
R. Patriarca and J. Bergström, “Modelling complexity in everyday operations: functional resonance in maritime mooring at quay,” Cognition, Technology & Work, vol. 19, no. 4, pp. 711–729, 2017.
View at: Publisher Site | Google Scholar
M. Stogsdill and P. Ulfvengren, “Mapping risk models/methods onto a complexity spectrum,” Transportation Research Procedia, vol. 28, pp. 133–140, 2017.
View at: Publisher Site | Google Scholar
L. J. Clarke, G. J. Macfarlane, I. Penesis et al., “A risk assessment of a novel bulk cargo ship-to-ship transfer operation using the functional resonance analysis method,” in Proceedings of the 36th International Conference on Ocean, Offshore, and Artic Engineering, Trondheim, Norway, June 2017.
View at: Google Scholar
R. Patriarca, G. Di Gravio, and F. Costantino, “A Monte Carlo evolution of the functional resonance analysis method (FRAM) to assess performance variability in complex systems,” Safety Science, vol. 91, pp. 49–60, 2017.
View at: Publisher Site | Google Scholar
Q. Yang, J. Tian, and T. Zhao, “Safety is an emergent property: illustrating functional resonance in air traffic management with formal verification,” Safety Science, vol. 93, pp. 162–177, 2017.
View at: Publisher Site | Google Scholar
A. Adriaensen, R. Patriarca, A. Smoker, and J. Bergström, “A socio-technical analysis of functional properties in a joint cognitive system: a case study in an aircraft cockpit,” Ergonomics, vol. 62, no. 12, pp. 1598–1616, 2019.
View at: Publisher Site | Google Scholar
A. Hulme, N. A. Stanton, G. H. Walker, P. Waterson, and P. M. Salmon, “What do applications of systems thinking accident analysis methods tell us about accident causation? a systematic review of applications between 1990 and 2018,” Safety Science, vol. 117, pp. 164–183, 2019.
View at: Publisher Site | Google Scholar
R. Woltjer and E. Hollnagel, “Functional modelling for risk assessment of automation in a changing air traffic management environment,” in Proceedings of the 4th International Conference Working on Safety, Crete, Greece, 2008.
View at: Google Scholar
A. Liberati, D. G. Altman, J. Tetzlaff et al., “The PRISMA statement for reporting systematic reviews and meta-analyses of studies that evaluate health care interventions: explanation and elaboration,” Journal of Clinical Epidemiology, vol. 62, no. 10, pp. e1–e34, 2009.
View at: Publisher Site | Google Scholar
N. L. Damen, M. S. de Vos, M. J. Moesker et al., “Preoperative anticoagulation management in everyday clinical practice: an international comparative analysis of work-as-done using the functional resonance analysis method,” Journal of Patient Safety, 2018.
View at: Google Scholar
A. G. Amorim and C. M. N. A. Pereira, “Improvisation at workplace and accident causation-an exploratory study,” Procedia Manufacturing, vol. 3, pp. 1804–1811, 2015.
View at: Publisher Site | Google Scholar
M. Moškon, M. Tkalec, N. Zimic, and M. Mraz, “Towards the declaration of inter-functional protocol for FRAM,” in Proceedings of the Annual Reliability and Maintainability Symposium, Orlando, FL, USA, 2019.
View at: Google Scholar
R. Clay-Williams, J. Hounsgaard, and E. Hollnagel, “Where the rubber meets the road: using FRAM to align work-as-imagined with work-as-done when implementing clinical guidelines,” Implementation Science, vol. 10, 2015.
View at: Publisher Site | Google Scholar
S. Myneni, D. McGinnis, K. Almoosa, T. Cohen, B. Patel, and V. L. Patel, “Effective use of clinical decision support in critical care: using risk assessment framework for evaluation of a computerized weaning protocol,” Advances in Healthcare Informatics and Analytics, vol. 19, pp. 217–232, 2016.
View at: Publisher Site | Google Scholar
M. V. C. Aguilera, B. B. da Fonseca, T. K. Ferris, M. C. R. Vidal, and P. V. R. Carvalho, “Modelling performance variabilities in oil spill response to improve system resilience,” Journal of Loss Prevention in the Process Industries, vol. 41, pp. 18–30, 2016.
View at: Google Scholar
I. A. Herrera, E. Hollnagel, and S. Håbrekke, “Proposing safety performance indicators for helicopter offshore on the Norwegian continental shelf,” in Proceedings of the International Conference on Probabilistic Safety Assessment and Management, Seattle, WA, USA, June 2011.
View at: Google Scholar
K. Laugaland, K. Aase, and J. Waring, “Hospital discharge of the elderly-an observational case study of functions, variability and performance-shaping factors,” BMC Health Services Research, vol. 14, 2014.
View at: Publisher Site | Google Scholar
M. V. Cabrera, M. C. R. Vidal, T. K. Ferris, and P. V. R. de Carvalho, “Modeling oil spill defense system using Functional Resonance Analysis Method,” in Proceedings of the 10th Global Congress on Process Safety, New Orleans, LA, USA, January 2014.
View at: Google Scholar
G. Praetorius, E. Hollnagel, and J. Dahlman, “Modelling vessel traffic service to understand resilience in everyday operations,” Reliability Engineering & System Safety, vol. 141, pp. 10–21, 2015.
View at: Publisher Site | Google Scholar
A. Melanson and S. Nadeau, “Managing OHS in complex and unpredictable manufacturing systems: can FRAM bring agility?” Advances in Ergonomics of Manufacturing: Managing the Enterprise of the Future, vol. 490, pp. 341–348, 2016.
View at: Publisher Site | Google Scholar
S. Albery, D. Borys, and S. Tepe, “Advantages for risk assessment: evaluating learnings from question sets inspired by the FRAM and the risk matrix in a manufacturing environment,” Safety Science, vol. 89, pp. 180–189, 2016.
View at: Publisher Site | Google Scholar
L. Pickup, S. Atkinson, E. Hollnagel et al., “Blood sampling—two sides to the story,” Applied Ergonomics, vol. 59, pp. 234–242, 2017.
View at: Publisher Site | Google Scholar
R. Patriarca, G. Del Pinto, G. Di Gravio, and F. Costantino, “FRAM for systemic accident analysis: a matrix representation of functional resonance,” International Journal of Reliability Quality, and Safety Engineering, vol. 25, no. 1, 2018.
View at: Google Scholar
D. C. Raben, B. Viskum, K. L. Mikkelsen, J. Hounsgaard, S. B. Bogh, and E. Hollnagel, “Application of a non-linear model to understand healthcare processes: using the functional resonance analysis method on a case study of the early detection of sepsis,” Reliability Engineering & System Safety, vol. 177, pp. 1–11, 2018.
View at: Publisher Site | Google Scholar
T. Hirose, T. Sawaragi, and Y. Horiguchi, “Safety analysis of aviation flight-deck procedures using systemic accident model,” IFAC-PapersOnLine, vol. 49, no. 19, pp. 19–24, 2016.
View at: Publisher Site | Google Scholar
L. V. Rosa, A. N. Haddad, and P. V. R. de Carvalho, “Assessing risk in sustainable construction using the functional resonance analysis method (FRAM),” Cognition, Technology & Work, vol. 17, no. 4, pp. 559–573, 2015.
View at: Publisher Site | Google Scholar
A. N. Haddad and L. V. Rosa, “Construction sustainability evaluation using AHP and FRAM methods,” in Proceedings of the Industry and System Engineering Research Conference, Nashville, TN, USA, May 2015.
View at: Google Scholar
Z. X. Zheng and J. Tian, “Bridging the gap between FRAM and safety practice by applying FSM and model checking,” in Proceedings of the First International Conference on Reliability Systems Engineering, Beijing, China, October 2015.
View at: Google Scholar
T. Sawaragi, Y. Horiguchi, and A. Hina, “Safety analysis of systemic accidents triggered by performance deviation,” in Proceedings of the SICE-ICASE International Joint Conference, Busan, South Korea, October 2006.
View at: Google Scholar
R. Woltjer, “Resilience assessment based on models of functional resonance,” in Proceedings of the 3rd Symposium on Resilience Engineering, vol. 9, Juan-les-Pins, France, October 2008.
View at: Google Scholar
L. Macchi, E. Hollnagel, and J. Leonhard, “Resilience engineering approach to safety assessment: an application of FRAM for the MASW system,” in Proceedings of the Eurocontrol Safety R&D Seminar, Munich, France, 2009.
View at: Google Scholar
I. A. Herrera and R. Woltjer, “Comparing a multi-linear (STEP) and systemic (FRAM) method for accident analysis,” Reliability Engineering & System Safety, vol. 95, no. 12, pp. 1269–1275, 2010.
View at: Publisher Site | Google Scholar
P. V. R. de Carvalho, “The use of functional resonance analysis method (FRAM) in a mid-air collision to understand some characteristics of the air traffic management system resilience,” Reliability Engineering & System Safety, vol. 96, no. 11, pp. 1482–1498, 2011.
View at: Publisher Site | Google Scholar
D. Nouvel, S. Travadel, and E. Hollnagel, “Introduction of the concept of functional resonance in the analysis of a near-accident in aviation,” in Proceedings of the 33rd ESReDA Seminar, Ispra, Italy, August 2011.
View at: Google Scholar
C. Martinie, P. Palanque, M. Ragosta et al., “Understanding functional resonance through a federation of models: preliminary findings of an avionics case study,” Computer Safety, Reliability, and Security, vol. 8153, pp. 216–227, 2013.
View at: Google Scholar
B. Frost and J. P. T. Mo, “System hazard analysis of a complex socio-technical system: the functional resonance analysis method in hazard identification,” in Proceedings of the Australia System Safety Conference, pp. 28–30, Melbourne, Australia, 2014.
View at: Google Scholar
M. Ragosta, C. Martinie, P. Palamque, D. Navarre, and M. A. Sujan, “Concept map for integrating modelling techniques for the analysis and re-design of partly-autonomous interactive system,” in Proceedings of the 5th International Conference on Application and Theory of Automation in Command and Control System, pp. 41–52, Toulouse, France, September 2015.
View at: Google Scholar
Q. Yang and J. Tian, “Model-based safety assessment using FRAM for complex systems,” in Proceedings of the 25th European Safety and Reliability Conference, Zurich, Switzerland, September 2015.
View at: Google Scholar
P. Rutkowska and M. Krzyzanowski, “FRAM modelling of the transfer of control over aircraft,” Zeszyty Naukowe, Transport/Politechnika Śląska, vol. 101, 2018.
View at: Google Scholar
P. N. P. Ferreira and J. J. Cañas, “Assessing operational impacts of automation using functional resonance analysis method,” Cognition, Technology & Work, vol. 21, no. 3, pp. 535–552, 2019.
View at: Publisher Site | Google Scholar
K. Bjørnsen, A. Jensen, and T. Aven, “Using qualitative types of risk assessments in conjunction with FRAM to strengthen the resilience of systems,” Journal of Risk Research, vol. 23, no. 2, pp. 153–166, 2018.
View at: Google Scholar
Y. Gao, Y. Fan, J. Wang, and Z. Duan, “Evaluation of governmental safety regulatory functions in preventing major accidents in China,” Safety Science, vol. 120, pp. 299–311, 2019.
View at: Publisher Site | Google Scholar
G. K. Kaya, H. F. Ovali, and F. Ozturk, “Using the functional resonance analysis method on the drug administration process to assess performance variability,” Safety Science, vol. 118, pp. 835–840, 2019.
View at: Publisher Site | Google Scholar
D. Smith, B. Veitch, F. Khan, and R. Taylor, “Understanding industrial safety: comparing fault tree, bayesian network, and FRAM approaches,” Journal of Loss Prevention in the Process Industries, vol. 45, pp. 88–101, 2017.
View at: Publisher Site | Google Scholar
F. de Felice, F. Zomparelli, and A. Petrillo, “Functional human reliability analysis: a systems engineering perspective,” in Proceedings of the CEUR Workshop Proceedings, Naples, Italy, 2017.
View at: Google Scholar
J. Tian, J. Wu, Q. Yang, and T. Zhao, “FRAMA: a safety assessment approach based on Functional Resonance Analysis Method,” Safety Science, vol. 85, pp. 41–52, 2016.
View at: Publisher Site | Google Scholar
W. Qiao, X. Li, and Q. Liu, “Systemic approaches to incident analysis in coal mines: comparison of the STAMP, FRAM and “2-4” Models,” Resources Policy, vol. 63, 2019.
View at: Publisher Site | Google Scholar
K. Fukuda, T. Sawaragi, Y. Horiguchi, and H. Nakanishi, “Applying systemic accident model to learn from near-miss incidents of train maneuvering and operation,” IFAC-PapersOnLine, vol. 49, no. 19, pp. 543–548, 2016.
View at: Publisher Site | Google Scholar
M. A. B. Alvarenga, P. F. Frutuoso e Melo, and R. A. Fonseca, “A critical review of methods and models for evaluating organizational factors in human reliability analysis,” Progress in Nuclear Energy, vol. 75, pp. 25–41, 2014.
View at: Publisher Site | Google Scholar
G. Praetorius, M. Lumdh, and M. Lützhöft, “Learning from the past for pro-activity-a reanalysis of the accident of the MV Herald of Free Enterprise,” in Proceedings of the 4th Resilience Engineering Symposium, pp. 217–225, Sophia Antipolis, France, June 2011.
View at: Google Scholar
F. Belmonte, W. Schön, L. Heurley, and R. Capel, “Interdisciplinary safety analysis of complex socio-technological systems based on the functional resonance accident model: an application to railway trafficsupervision,” Reliability Engineering & System Safety, vol. 96, no. 2, pp. 237–249, 2011.
View at: Publisher Site | Google Scholar
B. C. F. M. Schutijser, I. P. Jongerden, J. E. Klopotowska, S. Portegijs, M. C. de Bruijne, and C. Wagner, “Double checking injectable medication administration: does the protocol fit clinical practice?” Safety Science, vol. 118, pp. 853–860, 2019.
View at: Publisher Site | Google Scholar
D. McNab, J. Freestone, C. Black, A. Carson-Stevens, and P. Bowie, “Participatory design of an improvement intervention for the primary care management of possible sepsis using the functional resonance analysis method,” BMC Medicine, vol. 16, 2018.
View at: Publisher Site | Google Scholar
A. G. A. A. Pereira, “Introduction to the use of FRAM on the effectiveness assessment of a radiopharmaceutical dispatched process,” in Proceedings of the International Nuclear Atlantic Conference-INAC 2013, Recife, Brazil, November 2013.
View at: Google Scholar
L. Tan, B. Liu, X. Li, and S. Yang, “Modeling the software development process as a socio-technical system based on FRAM to facilitate the risk analysis and software defects prevention,” in Proceedings of the 17th AIAA Aviation Technology, Integration, and Operations Conference, Denver, CO, USA, June 2017.
View at: Google Scholar
R. Mock, L. Lopez, C. Zipper, and M. Schonenberger, “Resilience assessment of internet of things: a case study on smart buildings,” in Proceedings of the 26th European Safety and Reliability Conference (ESREL 2016), pp. 2260–2267, Glasgow, UK, 2017.
View at: Google Scholar
R. Patriarca, A. Falegnami, F. Costantino, and F. Bilotta, “Resilience engineering for socio-technical risk analysis: application in neuro-surgery,” Reliability Engineering & System Safety, vol. 180, p. 336, 2018.
View at: Publisher Site | Google Scholar
Microsoft power platform, explore power BI, 2020, https://clouddamcdnprodep.azureedge.net/gdc/gdc9NVYSN/original.

Copyright

Copyright © 2020 Wulin Tian and Carlo Caponecchia. This is an open access article distributed under the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.

PDF Download Citation

Download other formats

Order printed copies

Views

6768

Downloads

1828

Citations