Abstract
The product characteristics and maintenance characteristics of space complex repairable facilities were analyzed to aim at the maintenance requirements with a large number of on-orbit replaceable units, complex systems, and high operation accuracy. An intelligent induced maintenance system for space application facilities was developed, and unique evaluation criteria were established. The system, key technologies, and application effects are verified through typical application cases and comparative experiments. The results show that the implementation of the system can effectively reduce the workload of the astronauts in orbit and improve the work efficiency and provide new cooperation means for the astronauts to solve the urgent and unexpected tasks in orbit.
1. Introduction
At present, China’s space industry is in a critical period of development. Complex space application facilities carry heavy science experiment tasks [1], which require astronauts to efficiently, quickly, and accurately complete various science experiment operations and maintenance operations during orbit. Currently, most of the on-orbit missions planned by astronauts are the first to be executed. They mainly rely on electronic manuals to obtain tools and operation information. Under the microgravity environment, the astronauts’ psychological and physical quality is decreased [2], and text descriptions in electronic manuals are often difficult to understand. The astronauts have to switch frequently between the iPad and the operation object during the mission [3], which undoubtedly brings a particular psychological burden to the astronauts’ mission.
There are precedents for applying induced maintenance technology in aerospace [4]. To find a new user interface that can replace the traditional astronaut command station and make various maintenance tasks on the International Space Station easier for astronauts, NASA and ESA [5, 6] have researched the use of maintenance induction technology on the International Space Station. Many domestic institutions have also studied induced maintenance technology in railway [7], aviation [8, 9], military [10, 11], and other industries. To sum up, the industry’s application of induced maintenance technology is in its infancy. In contrast, using this technology in scientific experiments and maintenance and installation tasks in space application facilities can assist astronauts in completing in-orbit tasks quickly and efficiently, save maintenance time, and reduce maintenance errors. In addition, the maintenance plan for significant and emergencies can be quickly edited into an induction maintenance application program and injected into the interactive perception system to guide astronauts in dealing with problems. The realization of an intelligent induced maintenance system for space application facilities is conducive to the popularization and application of induced maintenance technology in aerospace, military, and advanced fields, which is the general trend of technological development.
2. System Analysis and Design
2.1. System Analysis
2.1.1. System Application Objectives
Interact through natural means perception around the scene and maintenance object state changes, timely and accurately will need information about digital maintenance induced to the physical object [12] (the maintenance object, experimental operation, etc.), makes the astronauts able to get the natural way of perception such as visual, auditory operation guidance, and warning.(1)Support astronauts to carry out complex on-orbit maintenance, installation and adjustment, experimental operation, and other tasks which are convenient, fast, real-time, and efficient, saving maintenance resources, reducing the operating load of astronauts, improving efficiency, and giving full play to benefits.(2)Support astronauts to carry out on-orbit maintenance, installation and adjustment, experimental operation, and other tasks training and training on the ground [13].
2.1.2. System Requirement Analysis
(1) Analysis of Maintenance Objects. According to the structural design requirements of space application facilities, space application facilities are generally divided into three categories, the whole cabinet level, single machine level, and module/board level. The whole cabinet-level facilities include 16 categories: high micro cabinets and containerless cabinets. Single-machine facilities include 12 types, such as experimental cabinet controllers and universal heat-control drawers. The module/board level is a clear division of the whole cabinet level and the single machine level, including the fast breaker, electrical connectors, industrial cameras, and other on-orbit replaceable units.
(2) Maintenance Scenario Analysis. According to the maintenance design requirements of space application facilities, there are two basic methods: in situ maintenance and out-of-position maintenance. Among them, in situ maintenance refers to the maintenance operation carried out by astronauts. The maintenance object is not separated from the main structure, such as the overall replacement of the science experiment module, the replacement of the filter cylinder, and other maintenance tasks. Out-of-position maintenance refers to the maintenance tasks that astronauts need to move and fix on the maintenance platform or other vacant space for operation due to space constraints, such as in-orbit assembly and state setting of plug-ins and maintenance platform unlocking.
(3) Analysis of System Application Scenarios. The application target of the system determines that the application scenario of the system should be divided into two parts, namely, a space-based induced maintenance system and a ground-based test system. Space-based induced maintenance system is the core of the system application goal realization, which mainly supports real-time interaction with astronauts to obtain maintenance operation instruction information and data management and transmission. The ground-based test system is the foundation of the successful realization of a space-based induced maintenance system, which mainly supports the production of induced information, ground test, data management, and transmission.
2.1.3. System Functional Requirements
Based on the above analysis and research on maintenance objects, maintenance scenarios, and system application scenarios, the following functions are proposed for system functions: ① Process control function, including tracking registration, gesture, or voice awareness. ② Information dynamic virtual and real superposition and visual display [14]. ③ Data collection and transmission during operation. ④ Feature extraction, feature modeling, and object recognition of operation objects [15]. ⑤ operation-induced scene modeling [16], including 3D prototype generation and rendering, physical attribute modeling, and motion behavior modeling. ⑥ Maintenance induction information production [17, 18], including the prototype, video, sound, symbols, and text, which reflects the induction process of the association and establishment. ⑦ Induction maintenance application demonstration and generation. ⑧ Induction maintenance application management and deployment. ⑨ Has power supply interface with equipment cabinet and has data communication interface function with the notebook.
2.2. System Design
2.2.1. System Composition
By analyzing the application scenarios and functional requirements of the system, the intelligent induction maintenance system can be divided into three subsystems: induction information production system, data management system, and perceptual interaction system. ① The induction information production system is the production platform of the intelligent induction maintenance system. It provides product information management, the virtual and actual integration of interactive control, and a standardized production process. It is a platform for the efficient production of intelligent induction maintenance applications. ② The data management system is the information relay center of the intelligent induced maintenance system, storing and releasing induced maintenance applications, and managing induced maintenance video recording data. It is called a space-based data management system operated on the orbiting notebook and a ground-based data management system installed on the ground workstation. ③ The interactive perceptual system is the execution terminal of the intelligent induced maintenance system. The astronaut’s voice, gesture, gaze, and other interactive [19] means to control the accurate expression of the induced information. The organizational relationship of the intelligent induced maintenance system is shown in Figure 1.
2.2.2. System Architecture Design
From setting out, the application of the system environment and object characteristics puts the corresponding functional requirement to the system. To meet the requirements, this paper designed an intelligent induction maintenance system architecture for space application facilities, as shown in Figure 2.
3. System Implementation and Key Technologies
The intelligent induced maintenance system has been developed to solve the maintenance needs of space application facilities such as many sizeable in-orbit replaceable units, complex systems, and high-precision operation, reduce the astronaut workload and improve work efficiency. The key technologies such as agile production of maintenance operation process, 3D target recognition and tracking registration [20], multi-dimensional induction information fusion, and timely guidance mechanism are broken through. The whole process is realized from inducing information management and process making, data management, and application publishing to inducing maintenance application task making and application.
3.1. Induction Information Production System
3.1.1. Inducing Information Production System Architecture
The induced information production system is the production platform of the whole intelligent induced maintenance system. The system has designed eight material libraries, including the induced information base, the target identification and tracking registration method library, and the model library. Through the agile production technology of the maintenance operation process, the induced information of facility maintenance requirements can be quickly extracted from the material library and edited into the induced maintenance operation process. An induced maintenance application data package is generated. The system architecture diagram of induced information production is shown in Figure 3.
3.1.2. Maintenance Operation Process Agile Production Technology
The maintenance process of agile manufacture technology in the model library, photo gallery, and animation library database based on visual management into the space application facilities of maintainability demands, ergonomics, and other elements, has realized the maintenance process editor based on the model of the induction and induction process template production. Maintenance work processes agile production, as shown in Figure 4.
3.1.3. Realization of the Induction Information Production System
The induction information production system considers the maintenance characteristics, key technologies, functional layout, and other requirements of space science experimental facilities. It is transformed into the template [21] of the induction maintenance operation process. The operation process template dramatically improves the production efficiency of the induced maintenance application program and breaks the professional barrier of induced maintenance program production to process designers and provides technical support for the intelligent induction maintenance system to solve major and emergency events. The interface layout of the induction information production system is shown in Figure 5.
3.2. Data Management System
3.2.1. Data Management System Architecture
The data management system is the data center of an intelligent induced maintenance system. Its primary function is visual management of induced maintenance application data packets and interface interaction between in-orbit notebooks and augmented reality glasses. It is divided into a space-based data management system and a ground-based configuration file generator.
The space-based data management system runs on the in-orbit notebook, the data transfer center of the ground application production end, and the in-orbit astronaut use end. Its primary function is to support data transmission, management, and editing between the local induced maintenance application library and the equipment induced maintenance application library, and the local multimedia library and the equipment multimedia library. The remote installation function for the induction maintenance task application is completed. The architecture diagram of the space-based data management system is shown in Figure 6.
The profile generator runs on the ground workstation. The generated configuration file is necessary for space-based data management software to classify and store uplink-induced maintenance applications. At the same time, details are added to the configuration file generator, such as maintenance task code, special tools, maintenance time, and the main operation points. It is the primary file for space-based data management software to implement application classification, search, detailed view, and other functions. Induced maintenance application data packets generated by induced information production software must be added into configuration files before the ground operational control terminal can use them through the IP tunnel up to space-based data management software.
3.2.2. Data Management System Implementation
The induced maintenance application generated by the induced information production system needs to add the detailed information of the induced maintenance application in the configuration file generator, generate the configuration file, package it together with the induced maintenance application program and transmit it to the space-based data management software through the heaven and Earth link from the operation control center. After receiving the uplink application package, the system connects the augmented reality glasses with the on-orbit notebook through a cable. The system responds instantly to open the interface and automatically refreshes the induction maintenance application. According to the maintenance task list or maintenance task name query and the corresponding induced maintenance application deployment, in augmented reality glasses, astronauts can perform according to the maintenance task list or maintenance task name query and the corresponding induced maintenance application deployment in augmented reality glasses. Figure 7 shows the interface of data management software deploying induced maintenance applications to augmented reality glasses.
3.3. Perceptual Interactive System
3.3.1. Perceptual Interactive System Architecture
The perceptive interactive system uses the head tracking and depth camera of the binocular penetrable augmented reality device to obtain scene information and guides astronauts in maintenance operations through the visual display of 3D graphics and indication data, as well as the interaction of voice or gesture [22]. The behavior perception module of the system receives the astronaut’s gesture or voice command. The system combined 3D target recognition and tracking registration technology, multi-dimensional induction information fusion, and a timely guidance mechanism to express the required induction information conveniently. The operation logic diagram of the perceptual interaction system is shown in Figure 8.
3.3.2. 3D Target Recognition and Tracking Registration Technology
3D target recognition and tracking registration technology realize accurate target recognition and high-precision real-time guidance for high similarity application facilities, which is the premise of an intelligent induction maintenance system to provide accurate induction information for astronauts. The verification of 3D target recognition and tracking registration technology is shown in Figure 9.
3.3.3. Multi-Dimensional Induction Information Fusion and Timely Guidance Mechanism
Multi-dimensional induced information fusion and timely guidance mechanisms are divided into multi-dimensional induced information fusion technology and multi-dimensional induced information timely guidance technology. According to the specific maintenance mode of space application facilities, multi-dimensional induced information and fusion technology can be divided into three categories: ① Maintenance induced by the complete superposition of the virtual and the real. ② Maintenance induction of mirror of the maintenance object. ③ Maintenance induction of the virtual display. The multi-dimensional induced information timely guidance technology realizes the dynamic control of the induced maintenance operation process. Deeply analyzes the unique operating environment of astronauts facing space application facilities and achieves the goal of ultimately liberating astronauts’ hands by using voice recognition technology based on the design principle of “astronaut as the center.”
3.3.4. Realization of the Perceptual Interactive System
As the functional core of an intelligent induced maintenance system, an interactive perceptual system is a multi-dimensional expression and interactive terminal of induced maintenance information of space application facilities. The comprehensive system is composed of induced maintenance applications and augmented reality equipment. When the system runs, it obtains multi-dimensional operation guidance or reminders through interactive means such as behavioral perception gestures or voice and natural visual and auditory perception [24]. To realize the “Conversational” human-computer interaction effect, guide the astronauts complete complex system maintenance tasks [25]. The effect of typical application scenarios of perceiving interaction is shown in Figures 10 and 11.
4. System Evaluation Criterion System Design and Verification
4.1. System Evaluation Criteria Design
The induced maintenance application is generated in the induced information production system and runs in the perceptual interaction system through the data transfer center. The three are closely related and indispensable. The intelligent induced maintenance system directly serves astronauts, so this paper designs a set of particular evaluation criteria system based on the actual application scenarios of the intelligent induced maintenance system. The evaluation system is divided into three types of comprehensive evaluation factors [26]: the appropriateness of induced maintenance time, the friendliness of the induced maintenance method, and the accuracy of induced maintenance information. In the current maintainability national standard and the national military standard system, these three factors belong to the qualitative factors that are difficult to quantify in the maintainability evaluation standard, so the language variables and fuzzy numbers can be used to deal with them in order to facilitate the evaluators to give qualitative estimation and judgment of these factors.
The evaluation criteria of the intelligent induction maintenance system proposed in this paper are in the form of score values for evaluators to choose and score. The range of score values was specified as 1 to 10. The smaller the score is, the better the corresponding evaluation content is, as shown in Table 1.
Various evaluation factors in the evaluation criteria of the intelligent induction maintenance system are described in detail in Tables 2, 3, and 4.
4.2. System Effect Verification
According to the above evaluation system of the intelligent induction maintenance system, combined with the astronaut selection rules of the upper and lower limits of height and other indicators, this paper invited five aerospace ergonomics experts to evaluate the intelligent induced maintenance system. The technical status of the experts involved in the evaluation is shown in Table 5.
In the verification process, the experts use the system to carry out complete maintenance operations on the same space science application facilities, and the timing, methods, and information of induction maintenance were evaluated by subjective judgment. Table 6 shows the average results obtained by different experts after scoring each evaluation factor of the intelligent induction maintenance system.
Meanwhile, to verify the intelligent induced maintenance system’s effectiveness in reducing astronauts’ workload and improving work efficiency, 20 trainees were invited to replace the filter of the fluid loop assembly. Trainees from 1 to 10 completed the test with the help of an intelligent induction maintenance system, and trainees from 11 to 20 completed the operation with the electronic manual based on the tablet. All trainees were skilled in operating the intelligent induction maintenance system and tablet, and this was the first time for them to perform the experimental operation. The time taken by the two groups of trainees to complete the test is shown in Table 7. The comparison of the time taken by the two groups of trainees to achieve the same operation using different guidance methods is shown in Figure 12.
It can be seen from the evaluation results that the intelligent induced maintenance system has gained high recognition among experts. Furthermore, the efficiency of using the intelligent induced maintenance system is improved by 35.73% compared with using an electronic manual [27] to complete the same test task. The results show that the intelligent induction maintenance system remarkably affects the convenience of operation, reducing the operating load and improving the work efficiency of astronauts.
5. Conclusion
This paper discusses the design and implementation methods of the intelligent guidance maintenance system for space application facilities, verifies the main functions and key technologies of the system through the whole system and aspects of an example and establishes an evaluation criterion system with ergonomics experts and volunteer students in the field of aerospace as test objects. It shows that the intelligent induction maintenance system is an effective way to reduce the workload of astronauts, reduce the operation error rate and improve the work efficiency in unique environments such as complex aerospace systems, high operation requirements, and emergencies.
With the continuous development of Chinese space technology and the endless selection of payload experts, an intelligent induced maintenance system is of great significance in daily training on the ground and on-orbit auxiliary payload experts to complete various tasks. At the same time, under the vigorous promotion of new technology, intelligent induced maintenance technology has vast development potential, and mainly has the following points [23].
More advanced communication technology is combined with induced maintenance technology
With the continuous breakthrough of 5G, 6G, and other advanced communication technologies, the data transmission efficiency is greatly improved, and the data delay can be ignored. The large-scale model and high-fidelity rendering work are transferred to the cloud server, breaking through the limitation of hardware performance. Not only that but low latency data transfer can also bring a better experience for augmented reality-based remote experts.
Artificial intelligence technology is combined with induced maintenance technology.
The continuous development of artificial intelligence technology such as computer vision and machine learning will enable induced maintenance technology to be used in automatic scene perception, equipment state data acquisition, and induced maintenance process autonomous decision making. The combination of artificial intelligence technology and induced maintenance technology will make induced maintenance more intelligent.
Data Availability
The data are available from the corresponding author upon reasonable request.
Conflicts of Interest
The authors declare that there are no conflicts of interest.
Acknowledgments
This work was supported by the Key Laboratory of Space Utilization, Chinese Academy of Sciences (Y7140211XN).