Abstract
The purpose of this systematic review is to highlight the salient elements of learning from incidents in the aircraft maintenance and continuing airworthiness management area. This involved the review of more than 1,000 publications reflecting practice in different domains. The cache was eventually distilled to 18 publications of relevance to learning from incidents. The systematic review of the literature was not intended to be exhaustive, but it was deliberately bound by the parameters of predefined search terms. A robust analysis was performed on the 18 distilled publications with the use of the NVivo software. A critical and systematic examination of this body of literature further supported the development of the five codification themes. The analysis of the literature revealed the benefits of a just culture as an enabler of reporting and learning from incidents. Moreover, it identified limitations inherent in the current body of knowledge. The most evident being a paucity of literature relevant to the featured industry segment. Some impediments to learning from incidents are also highlighted. Central to this is the prevalence of lack of effective focus and practice on satisfactory causation of events. Currently, the efforts applied across many featured domains appear to be based upon ineffective legacy linear practices. However, emerging investigative philosophies that look beyond direct cause and effect contain opportunities for practitioners to consider causation through dawning axioms. This systematic review could be used in the European aviation regulatory activities associated with improving learning from incident in aircraft maintenance and continuing airworthiness management.
1. Introduction
Freeman Dyson, the notable theoretical physicist and mathematician, once said, “aviation is a branch of engineering that is least forgiving of mistakes” [1]. It is true that such high reliability domains can pose a great degree of risk that may in turn contribute to mistakes being made. However, a guiding principle of continuously improving aviation safety is our ability to learn from events such as incidents. In the world of aviation safety, standards and recommended practices tend to be biased towards translating the experiences from such events into tangible outcomes aimed at preventing similar reoccurrences.
A review of safety in aviation from the perspective of maintenance and continuing airworthiness management staff is the key to understanding the relationship between safety and the concept of learning from incidents [2]. Despite the efforts of fallible humans and the ever-increasing complex systems they moderate, achieving a utopian reality where there are no risks or hazards present is clearly an unreasonable expectation [3]. Safety in aviation has evolved along a continuum from the early 1900s where aircraft mechanical and design issues were the primary contributors to aircraft accidents, according to the International Civil Aviation Organisation (ICAO) [4]. Improvements in these technical factors reached a plateau in the 1970s and the challenges realised then were centred around human performance and limitations [5]. Notwithstanding efforts and investment in human factor initiatives, accidents and incidents continued to occur. In the 1990s, there was a clear recognition that, as the aviation industry continued to develop, there were a number of factors outside the human at play with a potential to affect safety behaviour [6]. This paradigm-shift informed today’s systematic approach to safety and, in particular, the approach to learning from incidents [7].
Most people relate safety to freedom from risk and danger [8]. Unfortunately, risk and danger are often ubiquitous in the presence of high reliability activities. Managing sources of risk and danger are a tall order for some organisations. The ICAO Doc 9859 [4] recognises that “aviation systems cannot be completely free of hazards and associated risks.” However, the guidance does acknowledge that, as long as the appropriate measures are in place to control these risks, a satisfactory balance between “production and protection” can be achieved. Perrow [3] acknowledges that “we load our complex systems with safety devices in the form of buffers, redundancies, circuit breakers, alarms, bells, and whistles” because no system is perfect.
When one thinks of the word “incident,” it conjures up the notion of an action that may have grave consequences. Similarly, the word “accident” is often used in the context of an unplanned event or a particular circumstance. In many industrial sectors and business domains, these descriptors are used with a degree of interchangeability when the words are applied to describe events. In the world of aviation, there are clear high-level definitions for both event categories, and these are based on potential for harm. Throughout aviation, learning from incidents is often considered to be one means of augmenting what Perrow [3] terms “safety devices.” “Experience is the best teacher” according to Kleiner and Roth [9] as they claim that the causes of the mistakes are often not featured and continue to be present in the absence of learning. In general terms, Nonaka [10] suggests that creating new knowledge extends past a mechanistic approach and is strongly related to employees’ insights. An effective enabler of learning in this area is the collation of information on incidents. Details of the related processes, environment, procedures, competencies, and implementing timely corrective actions all have a positive impact on learning and help prevent recurrence in the future. Learning from incidents is therefore mainly associated with post-incident learning.
Detecting and identifying hazards highlighted through incident reporting systems is recommended by International Civil Aviation Organisation (ICAO) standards and recommended practices as an effective means of achieving practicable levels of safe operations. Therefore, objective data mined from a reporting system offers the potential to enlighten aviation stakeholders and to illuminate weakness that may be present. Such information can assist with a better understanding of events and augment mitigating measures against the potential effects of these hazards. When incidents occur, this can be an indication of a failure in an organisation’s process and/or practice. Due to continuous challenges faced by the organisations in the aviation industry there is potential to learn from resulting incidents and precursors. The learning is based on the potential new knowledge available from the associated collection, analysis, and interventions of these events. Effective learning can be considered as a successful translation of safety information into knowledge that actively improves the operating environment and helps prevent recurrence of events we can potentially learn from. Learning in this context can often be experienced as modifying or implementing new knowledge where cultural, technical, or procedural elements are integrated. Therefore, when learning in this context is transformed into measures to prevent reoccurrence, an organisation often has a reasonable means of mitigating future similar events.
The objective of this systematic review is to examine how learning from incidents occurs in aircraft maintenance and continuing airworthiness management and other sectors and what issues impact learning in those areas. It also intends to identify the contributing and constraining factors to learning from incidents. A qualitative review approach was selected as it has the advantage of providing a deeper contextual understanding of the literature and can assist with better research integration. Applying a degree of rigour and comprehensiveness can assist with advancing knowledge and identifying research gaps and aspects for further research in this particular area.
The publication’s systematic literature review covered primary publications up until 2017. As the subject of learning from incidents is a valid topic with potential to augment safety, a brief review of a cross-section of the latest publications was performed to see if a “delta” in the knowledge exists. Insley and Turkoglu [11] reaffirm aircraft maintenance is still a key point of concern within many areas of aviation. Their work highlights frequently recorded maintenance related consequences, naming runway excursions and air turn-backs in the highest percentile. The study identified factors relating directly to these events naming inadequate and incorrect procedures, poorly executed inspection tasks, and incorrect installation as common causal factors ascribed to the event categories named. These issues are not unique to Europe. Habib and Turkoglu [12] review a dataset of maintenance-related incidents originating outside of Europe (Nigeria). Their analysis revealed causal factors such as poor aircraft husbandry, deficiencies in inspection and testing, and inadequate safety oversight (organisation and regulator). Habib and Turkoglu [12] also consider the consequential impact of errors as causal elements in subsequent events. They also highlight the increase in incidents recorded and attribute this to a recent increase in air movements. Batuwangala et al. [13] present the idea that forecasted growth in air traffic requires a strong effort to ensure aviation incidents continue to be progressively reduced. They recognise a novel approach to safety improvements will need to be propagated in support of this. Although the authors point out some of the benefits of implementing a safety management system (SMS), they reaffirm the notion that not all areas of aviation operations are mandated to comply with SMS requirements. Some of the implementing constraints recorded by Batuwangala et al. [13] include protection of safety data/reporters, lack of just culture and reporting, and reporting system deficiencies, to name a few.
The review of the sample examining a cross-section of current research in the area of aircraft maintenance and continuing airworthiness does not identify any significant new knowledge in support of this publication. The additional exercise reaffirms the concept that some organisations are continuing to ineffectively embrace a desire to learn from incidents.
2. Materials and Methods
In order to conduct an efficient and effective review, a structured approach was deemed necessary. Okoli and Schabram [14] state that “a dedicated methodological approach is necessary in any kind of literature review.” An initial search of literature highlighted a scarcity of best-practice guidelines for conducting systematic literature reviews in the subject domain. This situation is also experienced in other sectors as Levy and Ellis [15] and Webster and Watson [16] confirm. Qualitative research involves handling considerable volumes of data and a degree of discipline is required so that search results and decisions regarding subject inclusions and exclusions are recorded and references are well managed. Endnote was used in support of the literature review during this research. An electronic database is useful for supporting a search strategy, arranging publications, and storing references [17]. The qualitative data analysis software NVivo [18] was used to augment the data management, storage, and analysis associated with the literature review. NVivo possesses many functions that are capable of facilitating the synthesis of a review [19]. However, the software does not have the capability of understanding text and the analytical skills of a researcher cannot be replaced in this respect.
2.1. Search with Predefined Terms
Bandara et al. [19] suggest two main criteria to consider before a search to identify papers for extraction and review begins: the source and search strategy. The source considers which outlets and databases to target, and the search strategy refers to the search terms and discipline to be exercised during the manuscript extraction process. A systematic search of the literature was performed in the following databases:(i)Web of Science [20](ii)Scopus [21](iii)IEEE Xplore [22](iv)ProQuest [23](v)EBSCO [24]
The following set of predefined terms associated with the thematic of the systematic review was selected to search in these sources:(i)“learning from incidents”(ii)“learning from experience”(iii)“aircraft maintenance”(iv)“aircraft management”(v)“safety management systems”
This step concluded with the creation of an initial set of publications, which would further be filtered in next steps.
2.2. Practical Screen of Title and Abstract
In this step, each title and each abstract were reviewed (practical screen). This part of the process not only had to be broad enough to create a sufficient number of applicable publications but also had to be practically manageable. The following criteria were laid down for the practical screen of the source bibliographic details, title, and abstract:(i)Subject: related to learning from incidents and past experiences(ii)Setting: any high reliability industry or sector where learning from incidents is critical.(iii)Publication: journal or peer reviewed conference proceedings(iv)Date range: published post 1992
The output of the practical screen step produces a list of publications denoted as the screened set of publications. An Endnote library was created to store and manage the full text of the retrieved publications.
2.3. Classification to Primary and Secondary Publications
This step involved the filtering (classification) of publications in the following two categories:(i)Primary publications: any research publication based on original data collected by the publications’ author(s)(ii)Secondary publications: those publications based on data generated by somebody other than the author(s), e.g., a review and use of existing literature/data developed by another party
Effectively, the screened set of publications was split over to a subset of primary publications and subset of secondary publications. Of those, in the next step, only the subset of primary publications was used.
2.4. Application of Inclusion and Exclusion Criteria
Brunton et al. [25] suggest there needs to be explicit inclusion and exclusion criteria in order for the reviewer to screen titles and abstracts for topical, population, temporal, and methodological relevance. Having a set of criteria helps to reduce any researcher bias in the screening system. A set of inclusion and exclusion criteria was developed considering the below objectives and in accordance with the guidelines included in [26, 27]:(i)To review current literature and to identify factors related to learning from incidents(ii)To identify obstacles and to learn from incidents(iii)To make recommendations how learning from incidents might be improved in the aircraft maintenance and continuing airworthiness management sector
In this context, the inclusion and exclusion criteria presented in Table 1 were used for the filtering of the subset of primary publications. The output of this step leads to the creation of the final set of publications.
2.5. NVivo Analysis and Codification with Themes
In this step, the Endnote library containing the final set of publications is imported to NVivo for further analysis. The following approaches, previously suggested by Bandara et al. [19], were used for the selection of the codification themes:(i)Deductive: themes reported on are predetermined to some extent. In this case, these predetermined themes were the output of a focus group process. The present review paper does not report details on the focus group, as this is within the scope of a future research paper of the authors.(ii)Inductive: themes reported are derived from analysis of the literature.
In addition to the three inductive themes (learning from incidents, just culture, and precursors) arising during the literature review, two additional themes (root cause and reporting) were deduced from conducting focus group activities concurrently with the review. The aggregate of both of these efforts resulted in five themes being developed. According to Kitzinger [29], “focus groups are group discussions organised to explore a specific set of issues such as people’s views and experiences.” The idea of conducting group interviews is not a new one. Bogardus [30] is an early example of a reference to utilizing the group interview. Frey and Fontana [31] say that group interviews can be formally structured for a specific purpose or can be performed in a more informal setting where a researcher can “stimulate a group discussion.” A total framework of five nodes eventually representing the themes was constructed in the NVivo database and used in support of completing the systematic literature review. These five nodes were also later used as the main framework for the semistructured interview template.
The description and origin (focus group or literature analysis) of the themes identified are described in Table 2.
Using the codification themes, the final set of publication was searched using the NVivo software to extract and code the passages identified to any of the coding categories. NVivo only provides thematic classifications of data based on the occurrence of key words. This merely assisted in identifying common prescribed keywords in publications, enabling classification into categories or clusters of words and examination of relationships within these publications. As NVivo does not perform analysis, the researcher must search the outputs and extract meaning for themselves. Thus, each of the publications were physically reviewed inductively by the researchers. Effectively, the final set of publications was searched and coded to Table 2 which has five themes. The coding process consisted of selecting relevant passages of text that were captured in one or several of the framework nodes. The overall document screening process and associated steps described in the previous sections are illustrated in the flowchart of Figure 1.
Maykut and Morehouse [32] define a propositional statement as “a statement of fact the researcher tentatively proposes, based on the data.” Memos were used to draft these summary statements which form part of Section 3 of this paper.
3. Results and Discussion
In the first step of the process described in Section 2 of this paper, the search with predefined returned in excess of 1,000 publications (initial set of publications). From this tranche, a total of 239 publications were retrieved in the practical screen phase (constituting the screen set of publications), which were then classified to a subset of 53 primary publications and a subset of 186 secondary publications. The final set of publications was derived by applying the inclusion and exclusion criteria of Table 1, leading to a total of 18 publications. The progressive filtering process is presented in the flowchart of Figure 2.
The 18 publications are summarised in Table 3, where the utilised methodology (qualitative and quantitative of mixed) and the application domain (different industries) are also provided.
In the next step, this final set of 18 publications was analysed and codified with NVivo, using the five codification themes described in Table 2. This has led to the distribution of publications per codification theme shown in the flowchart of Figure 3.
One can observe from this distribution that publications share some common codification themes. This is presented in Table 4, which provides the results of the mapping exercise of the 18 publication against each of the five codification themes.
Memos were used to draft the literature summary statements, which formed the final narrative for the synthesis. NVivo facilitated collation of the summary statements and enabled a transparent audit trail in support of the literature review exercise presented separately in sections under the five codification themes.
3.1. Root Cause
An overview of the Jacobsson et al.’s [43] study findings that relate to poor causation identification can be consolidated as follows: fewer event aspects recorded, often only operator error and technical failure recorded, and shallow root causation. It was found that when limited analysis of underlying event causes is performed, only limited effective actions are possible. This is evident when poor root cause analysis only contributes to minor procedural, and cosmetic changes are aimed at preventing recurrence. Such deficiencies were considered to have a limited impact upon the potential lessons available as a result of ineffective root cause establishment.
Pickthall [44] considers root cause through the lens of an individual’s competence when a technical and human factors-related impediment is present. The research examined the prevalence of these factors when aircraft maintenance staffs perform fault diagnosis on complex aircraft systems. The researcher found that often maintenance staffs are unable to diagnose faults in an accurate and timely manner. The results of the study indicated that events are often caused by poorly resourced supports, such as system diagnostics and test equipment. On a practical level, these contributing factors are believed to have a negative influence on the inability to establish adequate root causes and prevent the recurrence of faults.
The Hobbs and Williamson [42] research study explored patterns of potentially unsafe acts often perpetuated by aircraft maintenance staff. Violations (routine and exceptional) and mistakes were found to be closely related to deteriorating maintenance standards. A potential relationship reinforces a link between violations and less than optimal safety standards. According to the researchers, root cause of such violations can often be traced back to the prevailing culture within the organisation itself.
3.2. Reporting
In their work, Gray and Williams [40] examined whether culture surrounding learning from incidents can be compounded by “strategic defence routines,” resulting in recurrence of the event or similar ones. Their study was conducted through questionnaire in health services’ domains. They found that real learning from incidents can take place as a result of a transformation effort facilitated by a holistic approach. The authors refer to “reframed learning approach;” however, the publication contains little practical exemplars which would expand more on the details and the applicability of a similar approach to learning from incidents.
Gartmeier et al. [38] examined if reporting can be used as a strategy for workplace learning in a health service setting. They have considered error reporting attitudes and behaviours in a two-stage study performed via a longitudinal survey. The results suggest that organisations should highlight benefits of error reporting, ease of use and accessibility of reporting systems are important, and barriers can be modified to encourage reporting.
Bjerg Hall-Andersen and Broberg [41] conducted a “natural experiment” in an engineering consultancy firm. Following implementation of an information transfer database, discreet learning processes found to be interconnected within some domain elements. However, there is no evidence of collective interdomain learning across functions. The lessons learned are not through potential negative consequences and respective actions arising from a reporting system input but brokered through a moderated database. A single “embedded” case study may not support the generalizability of the results in other domains. However, for those who wish to develop a better understanding of learning processes across knowledge boundaries, the “implications for practitioners” contained in the study are considered applicable.
Steiner [46] conducted a qualitative study set in a workshop environment with data collected through semistructured interviews, participant observations, document analysis, and note taking. The theoretical shortcomings defined by the literature that relate to barriers to organisational learning are discussed in the work. One may note that a consolidating feature of organisational learning, such as reporting of issues and data capture, are not adequately discussed in the study.
Atak and Kingma [33] conducted an ethnographic-based case study in an aircraft maintenance environment, augmented by field notes, document reviews, and interviews. Tensions between quality assurance and maintenance management were identified and the prevailing safety culture examined in the context of “integration, differentiation, and fragmentation.” The study offers a comprehensive picture of the applied challenges experienced by aviation safety staff from an “embedded” perspective. However, the measures to prevent bias and understanding the issues are not well-defined in the publication.
Pickthall [44] examined the mixed methods approach using a structured interview devised from an academic format. This study examined issues that arose when aircraft maintenance staff interacted with complex aircraft systems for defect rectification. Occasionally a “no fault found” determination has been found to be made. However, the fault-finding inputs in that case were ineffective, and the fault returned soon afterwards. The research considered the management-influenced behaviours such as time pressures, poor communication, failure to adopt and share best practice, inadequate training, and reluctance to change. The work uncovered that indispensable resources, such as aircraft test equipment, integrated onboard diagnostic systems, and maintenance manuals, often fail to support maintenance staff when undertaking diagnosis tasks. The results suggest that these elements can actively constrain maintenance staff when they attempt to consistently manage effective and timely defect rectification. Moreover, the results are well presented and worthy of consideration when developing training material in support of learning from incidents.
Storseth and Tinmannsvik [47] performed a qualitative study, using semistructured interviews in marine and rail industries domains, to examine how individuals retrospectively look back and consider learning from events. Learning indicators for the study were developed by the authors in an earlier related study. The research methods were augmented by theoretical studies and document analysis. They have found that learning within organisations takes place within the parameters of “actor-context constellations” where there are no defined start and finish points. This assumption is not sufficiently balanced against the need to formally consider the exigency for structure when developing learning from incident outcomes.
In their research study, Zwetsloot et al. [49] endorse the importance of learning when implementing a “zero-accident vision” in nonaviation-related domains. The work also highlights safety commitment, communication, and safety culture as learning enablers. Research design was a mixed method approach using a quantitative survey supported by interviews and workshops. The qualitative component of the research verified that learning was evident throughout the featured organisations. “Learning by doing” was considered a more effective approach in support of learning from incidents where employees are motivated to fully engage in the process, and supervisors can moderate theme-based safety dialogue. An extensive survey was performed across 27 organisations. The qualitative methods (interviews and workshops) were applied although they were not formally analysed, and their synopses were used to validate the survey results. The survey component of the research records high scores relating to learning action; however, there were differences noted between staff’s perception (and management) of learning action in approximately 25% of cases. Moreover, there was less diversity recorded across the learning condition dimension. The researchers considered this analogous to organisational commitment to safety. Safety commitment, communication, culture, and learning were examined as individual aspects of implementing a zero-accident environment. However, their cumulative relationship was not fully examined and the impact is not discussed sufficiently.
Hobbs and Williamson [42] conducted a mixed method study examining the application of a previously developed “three-way distinction” of unsafe acts questionnaire in an aircraft maintenance context. An initial questionnaire was developed through the application of a disciplined confidential critical interview technique with 72 aircraft maintenance mechanics. The results yielded 48 elements (validated by air accident experts) and transposed into a maintenance behaviour questionnaire distributed to 4,600 licensed and 300 unlicensed aircraft maintenance mechanics (1359 questionnaires were returned). The principle component analysis was the method used to reduce the number of variables in the dataset for analysis by extracting those considered important to the study. The authors’ choice of analysis does not appear to consider the competence in the context of skill-based errors and complex situations such as automation. However, the focus the publication brings on the need for aircraft maintenance staff to be aware of the cumulative effect of “seemingly insignificant” incidents fortifies the need to be proactive when it comes to learning from incidents.
3.3. Learning from Incidents
The objective of Lukic et al. [2] study was to highlight factors considered to be important for effective learning, e.g., participants, process, incident, and knowledge. Staff involvement and trust were positive attributes capable of supporting learning. Attributing blame and poorly developed root causation were found to detract from learning. The research also examined impact of formal and informal learning initiatives. Informal learning was found to be more difficult to record and codify, and potential for learning could be limited in some cases. In their paper, Lukic et al. [2] highlighted that the “over-simplification” of incidents and contend id, often the reason of incidents, are misunderstood when attempting to translate incident and accident data into knowledge and learning. It is noted there is an absence of information on the structure applied to the quantitative analysis and how rigour was applied to the process. However, the authors do clarify the analysis was both data and participant driven.
The Gerede [39] study considered some of the challenges associated with the successful implementation of safety management systems (SMS) in aircraft maintenance organisations. The SMS structure is comprised of “safety policy and objectives, safety risk management, safety assurance, and safety promotion.” Safety risk management and safety assurance were found to be important elements underscoring the effectiveness of day to day activities. Failure to create a just culture and fear of punishment for reporting shares a common cultural association. The situation is attributed to a potential combination of lack of trust and negative perceptions associated with organisational culture. Moreover, Gerede [39] identified that the absence of communication and trust may present implementation challenges within the maintenance organisations. If a just culture does not exist at national aviation authority state level, then it is questionable if the implementation of an SMS would be effective. It is unclear if the four structural elements of safety management were fully considered during the training or the data gathering phase of the study. This may account for the absence of any direct reference to learning from incidents in the study’s findings.
Drupsteen et al. [36] conducted case studies with selected individuals in various domains, including transportation. Their survey considered the following elements: steps in the process where learning is lost, formal organisation of steps, efficiency of steps on a daily basis, difference between espoused and actual performance of steps, and differences amongst featured areas. In their work, they also state that “many incidents occur because organisations fail to learn from past lessons” because the traditional approach often stops short of preventing future incidents. The research paper presented a model that examines the investigating and analysing incidents, planning and prevention, and intervening and evaluating steps in a learning process. The evaluation stage was found to be a primary learning bottleneck and reporting of incidents being next. Results indicated daily practice of learning was good, but follow-up steps in the process are often neglected in comparison to incident analysis. There was a significant difference between how well the investigation and incident analysis stage and the evaluation stage were performed and organised.
In their work, Ward et al. [48] offer a concise overview of key aspects of aircraft maintenance practice and present an accurate snapshot of the development and architecture of pertinent regulation. Understanding the aircraft maintenance system complexities is an essential precursor to implementing improvements. Organisational processes cannot be explained in terms of a linear approach due to the nonlinear characteristics of flexibility and variability of comprising elements. It was found that the resulting relationship between the individuals and the systems have a direct impact upon the system and prevailing environment. Their model comprised of the following elements: system level, process activity, dependencies, and stakeholders. Four reporting veins were uncovered focusing on unique aspects of product airworthiness and system performance, i.e., data inaccuracy, quality assurance, personal injury, and occurrence reporting and suggested changes were highlighted. The researchers found that regardless of how an issue presented, the staff continue to experience performance constraints if communication remains poor.
Jacobsson et al. [43] acknowledge the degree of interest invested in learning from incidents but question the efficiency of learning from incidents in some organisations. They found that event investigations often stop short and only partially deal with some of the elements affecting the event. Although unwelcome events are less prevalent, less severe events provide learning opportunities. Analysis of the learning cycle is valuable and such an approach can offer an insight into inherent precursors to accident conditions. They present a model featuring: reporting, analysis, decisions, implementation, and follow-up in an incident learning cycle format. Assessing effectiveness of an incident learning cycle was designed from analysing each individual step against the following dimensions: scope, quality, time, and information of the first cycle loop. A general assessment of the second learning loop was performed using participant interviews. Subject matter experts applied their judgement in support of developing weighting factors for each of the model elements. The paper refers to the analysis of incident learning systems but the purpose of conducting the safety audit is not specified. The relationship (if any) between the outcome of the safety audits and the efficiency of the learning systems does not appear to be fully articulated.
Silva et al. [45] examine how organisations use accident information to reduce the occurrence of unwelcome events. They suggest it is necessary to achieve a balance between adequately resourcing safety initiatives and maintaining acceptable levels of safety. They suggest that factors such as organisational culture, just culture, and event data, if managed, can contribute to a reduction in events. Learning within organisations should address effective information processing and interpretation. Combining technical and social strategies resulted in uncovering four patterns of practice that corresponded to different levels of learning.
In their work, Drupsteen and Wybo [35] conclude that organisations use experience gained from past events in order to improve safety. They introduce the term “propensity to learn” which refers to an organisation’s predisposition to learning and suggest an organisation can apply lessons from past events such as warning signals, mistakes, incidents, and accidents. They found that hindsight can determine if an organisation did learn from an event, but there are no models to assist with gauging the “propensity” of an organisation to learn. The object of the study was to expound two sets of indicators that would contribute to gauging an organisation’s inclination to learn. Using a previously validated questionnaire, the participants’ perception was assessed on learning indicators. They deduced from the review of literature that organisations displaying high learning propensity were also successful with learning from experience and sharing lessons amongst staff. Indicators based on three categories (attitudes and organisational conditions and systems) utilizing six indicators were developed to gauge organisational learning. A second set of indicators was developed in support of assessing individual propensity to learn from experience, specifically measuring attitude towards each of the stages of a generic learning process, i.e., detection, analysis, follow-up, evaluation, and sharing information. However, as the study was based solely on the perception of staff, it is unclear if the presented indicators alone would be satisfactory to elicit enough potentially subjective data to reinforce the results.
Furniss et al. [37] examined Hollnagel et al.’s [50] Functional Resonance Analysis Method (FRAM) which explores how functional variability resonates within systems, i.e., how well elements work together in a system. They also discuss how FRAM can be modified to support complex socio technical system improvements. This is presented in the context of four principles that encase the main assumptions (equivalence of success and failure, approximate adjustments, emergence, and functional resonance) from a FRAM practitioner perspective. Their study considered how human factor methods “are functionally coupled to a broader system of human factors practice” [37]. The four steps of the FRAM analysis were augmented by two additional steps: the purpose of FRAM analysis and respondent validation.
Drupsteen and Hasle [34] examined if organisations can learn more effectively from past incidents, and future incidents could be prevented. They suggest that learning can be improved if limiting factors are addressed. The learning process in different companies was analysed and discussed. The researchers used a topic list to assess if human, technical, or organisational aspects were being addressed and in which elements were related to specific learning phases. They found that some of the main causes of the constraints to learning can be related to lack of knowledge, unwillingness to report, causation not established, and uncertainty regarding follow-up action. Some conditions that enable these deficiencies are centred around misplaced cultural issues, over-focus on direct causation, and poorly defined safety management procedures for example. The benefits of considering all active and latent failures as direct and indirect causes, respectively, are unclear. The study concentrated on the latency of causation. The authors state learning from incident initiatives should exercise a more generic effort to support prevention. However, one of the limitations stated was the lack of homogeneity amongst the participating organisations.
3.4. Just Culture
Ward et al. [48] endorse the perception that aircraft maintenance is a “highly regulated, safety critical, complex, and competitive industry.” They also state that to positively perpetuate the above attributes, it is necessary to further develop an operational model that can account for “what is meant to happen and what actually happens.” A just culture is defined as “where people feel they can report mistakes made without fear of punishment (deliberate acts of damage or violations are different).” The researchers proffer that a just culture can be considered as an effective enabler of good quality incident reporting.
Gerede [39] examines some of the challenges associated with the implementation of the ICAO SMS standards and recommended practices which support the aviation industry and regulators to transition from prescriptive oversight methods to those based on performance metrics. These challenges relate to the successful propagation of a just culture which is considered as a basic principle of successful SMS implementation. The study strongly suggests that a failure to foster a just culture would be considered to have a negative impact upon effective data collection (reporting), organisational learning, and the subsequent ability to learn from incidents.
Silva et al. [45] put forward the value of information gleaned from incidents in support of learning and future event prevention. They examine how organisations utilise information and the strategies that assist with the propagation of lessons. They also highlight the need for organisations to encourage a learning culture and suggest the positive contribution made by reporting. It was found that a seminal element of organisational learning is a just culture, where errors and mistakes can be reported, and violations are managed fairly. In parallel, it is suggested that proportionate organisational responses are required to balance safety and accountability.
In their work, Drupsteen and Hasle [34] proffer that learning from past incidents can assist with understanding potential future events and possibly reduce their consequences. The study examines the causes associated with organisations failure to learn from previous events. Trust and openness were identified as key elements necessary for organisational learning. In the absence of these values, under-reporting is often evident. The researchers point out that the presence of what they term a “blame culture” also inhibits learning as potential reporters fear of being treated unjustly for their actions.
3.5. Precursors
Ward et al. [48] suggest improvements can be gained when organisational factors with a potential to contribute to incidents are understood. They consider these elements in the context of the reason [8] taxonomy (immediate, workplace, and organisational) of factors as systemic precursors. An improved understanding of these elements can also shift the focus of unwarranted blame from “the individual” within the system. Aviation maintenance management systems are increasingly adopting an approach where identifying systemic precursors contribute to a just outcome.
The main purpose of the Drupsteen and Wybo [35] study was to develop a set of indicators capable of determining an organisation’s “propensity to learn.” The researchers argue that the most effective set of indicators are those that could be proactively considered as “leading indicators.” Precursors that represent activity-based inputs can signal early degradation of safety systems.
One of the main aims of the Hobbs and Williamson [42] study was to ascertain if unsafe acts could be predicted as a result of analysing self-reported unsafe acts. Their analysis of demographic variables suggested that the occurrence of routine and exceptional violations was associated with a participants’ age. Higher levels of associated behaviours were linked with younger participants. The researchers were able to identify potential precursors to aircraft quality issues by association with less than optimal performance of aircraft maintenance staff. The analysis implied a distinction exists between what are termed routine and exceptional violations. The former tends to be more frequent and can be associated with shortcuts linked to routine tasks. The latter group is of a high-risk nature but occurs less frequently.
3.6. Common Limitations Identified in the Reviewed Publications
Although there was a distinctive scarcity of information across the reviewed literature relating to the domain under primary investigation, enablers, and challenges to learning in the featured preserves, which were well noted, learning from incidents across all domains shares a kindred desired outcome of delivering lessons that help prevent recurrence of similar incidents in the future. However, throughout the review, a few common limitations were discovered in the literature and summarised as follows:(i)All research papers do not follow the same discipline of section title and content.(ii)Few of the reviewed publications feature enough detail in the methodology sections to aid with the exact replication of the featured study.(iii)Details of piloting and testing data gathering instruments such as semistructured templates were scarce.(iv)The robustness of some analyses was difficult to determine.(v)The study featured participant perceptions, gauging the efficiency of lessons learned was not well supported in the text.(vi)Safety culture and just culture are mentioned as pivotal to learning. However, there is no solid mechanism featured in support of objectively measuring either cultural component in an aircraft maintenance and continuing airworthiness management environment.(vii)The literature review uncovered many instances of formal learning. It was noted that informal learning practices were not well represented.
4. Conclusions
The primary aim of learning from incidents is to support actions that contribute to preventing recurrence of unwelcome events. The literature review revealed the existence of a solid formal architecture capable of delivering lessons within the featured domain activities. However, learning from incidents is not specifically articulated as a requirement and therefore presently not all elements required are explicitly articulated with the regulatory code. Although some domain requirements mandate formal training, informal learning initiatives are not required to be capitalised upon. Additionally, inadequate incident causation can deflect from potential learning opportunities arising from reporting. Poorly resourced efforts to establish appropriate causation are recorded as a central impediment to learning. The importance of reporting (incidents) and enabling facilitators such as the presence of a just culture cannot be overstated. Encouraging a reporting culture also reflects positively on the potential to learn from reported incidents.
The literature review also revealed the prevalence of similar constraints to learning in other industries. Lukic et al. [2] highlight the increasing focus on learning from incidents in the health, safety, and environmental areas of the energy industry. They put forward factors they consider to be important for effective learning which bring a focus on; the participants of learning, types of incident, types of knowledge, and learning process. Drupsteen et al.’s [36] industrial research (chemical, construction, energy, governmental metal, and transport) states that “many incidents occur because organisations fail to learn from past lessons.” They point out that the traditional approach to learning often features only a careful analysis and formulation of lessons in the hope future incidents will be prevented. They suggest that, in addition to focusing on prevention of reoccurrence, the learning process should be improved which in turn can contribute to making an organisation safer. Others such as Jacobsson et al. [43] question the efficiency of learning from incidents in some organisations (petrochemical, food and drug, and energy) but suggest there is value in the analysis of the learning cycle. Such an approach can offer an insight into inherent weakness that often enables accidents. Silva et al. [45] examine how organisations (manufacturing, construction, production, and distribution of energy) use accident information to reduce the occurrence of unwelcome events. They acknowledge there is a need to achieve a balance between adequately resourcing safety initiatives and maintaining acceptable levels of safety. In healthcare, Drupsteen and Wybo [35] suggest an organisation can apply lessons arising from past events such as warning signals, mistakes, incidents, and accidents. Hindsight can assist with determining if an organisation did actually learn from an unwelcome event, and their study expounds two sets of indicators that could contribute to gauging an organisation’s inclination to learn. By considering the outputs of research in domains parallel to continuing airworthiness, the benefits of proven approaches in other industries could be leveraged and applied without further delay.
Many aspects of current literature are developed from a linear or sequential view of how an accident/incident occurs. This of course might be an appropriate place to start to examine the retrospective aspects of learning that an unwelcome event can provide. However, more proactive models such as Hollnagel et al.’s [51] FRAM model, as highlighted by Furniss et al. [37], are very capable of delivering more sustainable lessons. Nevertheless, it is evident from the literature search and review that research in the aircraft maintenance and continuing airworthiness management arena are yet not well represented in respect of learning from incidents.
One potential benefit of digressing from the traditional view of causation is that models such as FRAM can be applied in support of specific analysis frameworks capable of deciphering: what went wrong, hazards that may have not been previously considered, and the feasibility of potential solutions to prevent recurrence. As human systems and artificial intelligence continue to occupy shared workspaces, an appreciation of exactly how the system works is essential in order to deliver effective lessons when unwelcome events do occur. Further research in the continuing airworthiness area utilizing forward looking frameworks such as FRAM will have a positive impact on better understanding event causation. It will also present a need to examine and augment legislative requirements to support the needs of regulatory and ethical oversight of systems that employ a blend of human and autonomous functionality.
It is believed that the systematic review could be used to refine terms of reference for a European legislative working group tasked with improving the content of the implementing regulations in the area of learning of incidents within the context of SMSs in aircraft maintenance and continuing airworthiness management organisations.
Data Availability
The data supporting this systematic review are from previously reported studies and datasets, which have been cited. The processed data are available from the corresponding author upon request.
Conflicts of Interest
The authors declare that they have no conflicts of interest.