- About this Journal
- Abstracting and Indexing
- Aims and Scope
- Annual Issues
- Article Processing Charges
- Articles in Press
- Author Guidelines
- Bibliographic Information
- Citations to this Journal
- Contact Information
- Editorial Board
- Editorial Workflow
- Free eTOC Alerts
- Publication Ethics
- Reviewers Acknowledgment
- Submit a Manuscript
- Subscription Information
- Table of Contents
Advances in Mechanical Engineering
Volume 2013 (2013), Article ID 730925, 10 pages
Development of a Virtual Maintenance System for Complex Mechanical Product
1School of Mechanical and Electrical Engineering, China University of Mining & Technology, Xuzhou 221116, China
2Sports School, China University of Mining & Technology, Xuzhou 221116, China
Received 20 June 2013; Accepted 19 August 2013
Academic Editor: Moran Wang
Copyright © 2013 Xin-hua Liu et al. This is an open access article distributed under the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.
In order to improve the maintenance training effect of complex mechanical product, a virtual maintenance system was developed. The system framework was proposed, and the main functional modules were elaborated. A multilevel information representation model for complex mechanical product was put forward, and the flowchart of model transformation technology was designed. Moreover, a collision detection method based on hierarchical bounding volume was proposed, and the maintainability analysis and evaluation solution based on maintenance knowledge was presented. Finally, a prototype system was developed, and the proposed system was proved to be efficient through an example of hydraulic winch.
Nowadays, for the complex mechanical product with integration of mechanical, electronic and computer technology, and so forth, the disassembly/assembly process is more and more complicated in structure. It is hard to carry out the maintenance training tasks through a physical prototype not only due to the great limitation of work environment, but also due to the long maintenance time and high maintenance cost. In order to achieve the maintenance training effect through physical prototype, virtual maintenance system based on virtual reality technology (VR technology) is applied to the maintenance training tasks for complex mechanical product. Due to advantages such as no space limitation, short maintenance time, low maintenance cost, and risk [1–3], virtual maintenance system is widely applied in many engineer fields such as aerospace [4, 5], weapon equipments , and medical equipments [7, 8].
At present, many researchers have worked on the problem and proposed different solutions in order to realize high efficiency and visualization of product maintenance process. In [9–11], disassembly sequence planning approach was investigated to simulate machine assembly maintenance process. In , a framework capable of performing complex tasks depending on the user’s order was proposed, and this framework enabled VR applications to animate from dozens of characters in real time for complex constraints. In , a framework for synthesizing new character motions from existing motion capture data based on a frame-based solver was presented, and this ensured animating hundreds of humanoids with different morphologies in real time. In , Xie et al. proposed a virtual maintenance training system (VMTS) based on multiagent system, and the system is composed of seven collaborative agents. VMTS can be transformed to the development of multi-agent system, thus the robustness, flexibility, and reusability of VMTS were improved. In , Liu et al. presented a virtual assembly oriented multilayer exact collision detection algorithm based on the analysis of the problem in the virtual assembly environment. In , a novel collision evaluation algorithm based on generalized penetration depth, approximation information, and tolerance information was proposed to solve the problem in which collisions between polygonal models may not reflect collisions between real objects in reality. In , a bounding volume hierarchy based on tetrahedron and spheres for collision detection between complex polyhedra was presented. The method drastically reduced the number of intersection tests between the features of both polyhedra. In , a new scene graph structure was presented to remove the limitations of traditional scene graph structure in the development of virtual maintenance training system (VMTS). In , Liu et al. presented a multilayer component information expression structure model in virtual assembly. This method solved the data transformation between CAD system and virtual assembly system very well. In , the therblig characteristics required by virtual maintenance simulation were analyzed, and the maintenance therbligs were classified and defined, respectively. The modeling method of virtual human was studied, such as fatigue model of virtual human upper limb , regulation of human locomotion , and modeling of skeletal muscles coupled with fatigue . In , the implementation of basic body, retrieval body, and control body was studied, and the architecture and theory of multibody structure maintenance information modeling for multilayer representation were proposed. In , the information model and directed graph expression were studied, and the collaborative manufacturing task decomposition process covering the design of LMU and LMP was put forward. Moreover, the design methods for LMU and LMP were provided. In , the problem of how to build a virtual prototype (VP) used for virtual maintenance simulation was investigated, and a maintenance-task-oriented virtual maintenance prototype (VMP) modeling was addressed to represent geometry modeling and human-object interaction modeling. Moreover, other systems were established based on virtual reality [27–30].
Bearing the above observations in mind, we present a virtual maintenance system for complex mechanical product, and the rest of this paper is organized as follows. Section 2 presents the system framework and elaborates the main functional modules. Section 3 describes the key technologies such as model transformation technology, collision detection method, and a maintainability analysis and evaluation solution based on maintenance knowledge. Section 4 develops a prototype system based on the above-mentioned key technologies. Section 5 summarizes our conclusions and future work.
2. Framework of the Proposed System
According to the structure and maintenance training demand of complex mechanical product, a virtual maintenance system should meet the following requirements.(1)A virtual maintenance training scene with high credibility should be modeled.(2)The integration between CAD system and virtual maintenance system should be realized, and as much as possible CAD model information should be reserved.(3)Natural, simple input/output interfaces should be provided.(4)Various maintenance tools can be applied in the virtual maintenance system.(5)Maintenance knowledge management and maintainability analysis should be provided to improve the automation and intelligence of maintenance process.
According to the above requirements, a framework composed of several functional modules was proposed as shown in Figure 1.
There are several functional modules in the virtual maintenance system, and the functionalities of main functional modules are given as follows.(1)System modeling: it includes virtual maintenance scene modeling, virtual prototype modeling, and virtual human/virtual hand modeling. The virtual prototype models for complex mechanical product are modeled in CAD system and transformed into the virtual maintenance system with a multilevel information representation model.(2)Virtual maintenance simulation operation: it mainly includes interactive operation, collision detection, and disassembly sequence planning. User sends the maintenance operation orders through the data glove, and the virtual human carries out the maintenance tasks. Meanwhile, the virtual maintenance system executes the collision detection operation.(3)Maintainability analysis and evaluation solution: it includes the maintenance inspection, maintainability analysis, and maintenance data management. The virtual maintenance system judges the correctness of user operations timely and prompts users the right guidance based on the maintenance knowledge.
3. Key Technologies
3.1. Model Transformation Technology
A complete product model should contain the following aspects of information: geometrical information, topological information, hierarchical structure information, and assembly constraint information. Usually, the virtual prototype models are transformed from CAD models, and the geometry information of CAD models, are expressed by accurate mathematical form. Because of the large amounts of time to display CAD geometric model, it is very difficult to achieve the complex graphic's real-time display in virtual maintenance system. Generally, CAD models can be converted into triangular patch models which can be applied in virtual maintenance system. However, during the transformation process, virtual prototype models also lost the topology information and geometry information. In order to solve this problem, a multilevel information representation model for complex mechanical product is put forward. According to the product information of different levels of abstraction, product information can be divided into five layers which are product layer, component layer, part layer, geometric surface layer, and display layer as shown in Figure 2.
Product layer can be described as follows: where is product nodes. The information of product nodes includes name information, function information, quality information, fault information, materials information, and other management information.
Component layer can be described as follows: where are component nodes, is the hierarchical mapping relationship between component and product, and is the relationship of assembly constraints and relative position among the components.
Part layer can be described as following: where are part nodes. The part nodes mainly include part name information, physical properties information, technical information, and management attributes. is the hierarchical mapping relationship between part and component. is the relationship of assembly constraint and coupling among parts such as connection relationship of bolt and splines.
Geometric surface layer can be described as follows: where are geometric surface nodes, is the hierarchical mapping relationship between geometric surface layer and part layer, and is the constraint relationship of geometric surface.
Part is composed of the geometric surface which can be accurate to express the geometric shape and size of a part though keeping internal geometric constraint. The part position can be identified precisely based on the geometric constraint in a virtual maintenance system.
Display layer can be described as follows: where are triangular patch nodes. The triangular patch nodes mainly include geometric properties of triangular patch such as coordinates and color. is the hierarchical mapping relationship between display layer and geometric surface layer. It was applied in model visualization and collision detection in a virtual maintenance system.
The information of product layer, component layer, part layer, and geometric surface layer can be obtained through the second development of CAD system, and the information of display layer can be obtained through the neutral format files such as STEP and STL format which are output from the CAD system through the model transformation interface. The model transformation process can be shown in Figure 3.
3.2. Collision Detection Method
Collision detection is not only the basic elements of a virtual maintenance system but also the premise for virtual maintenance operation. Virtual maintenance system should possess the function of accurate real-time collision detection. For complex mechanical product, the model coupling degree is very high which would increase the amount of calculation for collision detection greatly. Thus, a quick and efficient collision detection method should be provided to realize the authenticity of a virtual maintenance system.
According to the requirements of collision detection in a virtual maintenance system, a collision detection method based on hierarchical bounding volume is proposed, and the overall process of the proposed method is shown as in Figure 4.
In virtual maintenance system, the part models are generally composed of triangles, and a collision can be recognized through judging whether the triangles have intersection points. The collision detection algorithm based on triangles can be elaborated as follows.
Suppose pair triangles exist.
Where, . , and are the vertexes of triangles , and , and are the vertexes of triangles . , are the planes of pair triangles. , are the normal vector of planes.
If , , and should be arranged clockwise, and if , , and should be arranged counterclockwise.
, , and are the vectors from the vertex to the three vertexes of triangle , respectively, and can be described as follows:
, , and are dot product of and , , and , respectively, and can be described as follows:
Judgment of the relative place for two triangles in the three-dimensional space can be described as follows.(a) If, , and have the same sign, triangles are disjoint.(b) If only one of , , and is zero and the other two have the same sign, only one vertex of the is in the plane . For this situation, the judgment of relative place for two triangles should refer to the situation . If the other two have different signs, the judgment of relative place for two triangles should refer to the situation .(c) If only two of, , and are zero, the judgment of relative place for two triangles should also refer to the situation .(d) If all of , , and are zero, and are in the same plane. The judgment of relative place for two triangles should also refer to the situation .(e) If , , and have different signs, the line segment which is one intersection of and to the other intersection of and is figured out, and the line segment which is one intersection of and to the other intersection of and is also figured out. Triangles have the same joint in the condition of the line segment and the line segment having overlap. Otherwise, triangles do not have joint.(f) Judgment of the relative place for point and triangle . Set one point in the plane ; , , and are the vectors of three edges of triangle and can be described as follows: is the normal vector of vector . expresses the linear equation of vector , and is the value of plugging point into this equation and can be described as follows: If , is in the triangle in the case of , . If , is in the triangle in the case of , .(g) Judgment of the relative place for two triangles in the same plane . Two triangles in the same plane are disjoint only if there is an edge of a triangle making the two triangle vertexes on the opposite side. In other words, two triangles are disjoint in two conditions, and one condition can be described as follows.
There is an edge of the triangle , and , . The two triangles are disjoint while , , or , , .
The other condition can be described as follows.
There is an edge of the triangle , and , . is the normal vector of vector . The two triangles are disjoint while , , , or , , .
Otherwise, they are intersection.
3.3. Maintainability Analysis and Evaluation Solution
For a complex mechanical product, maintenance tasks are complicated and difficult to be carried out by a common engineer. Usually, the task of maintainability analysis should be carried out, and the evaluation results and modification proposals including the factors of object accessibility, object visibility, maintenance time, and maintenance environment should be provided to guide the maintenance operations. In order to improve the intelligence and automation of maintenance process, a maintainability analysis and evaluation solution based on maintenance knowledge are proposed as shown in Figure 5.
The maintainability analysis and evaluation solution for complex mechanical product are composed of reachability analysis, maintenance time estimation, error analysis, maintenance safety analysis, disassembly sequence automatic planning, and dismantling path automatic detection. Maintenance knowledge can be acquired from product information, relative standard, and expert experience knowledge. According to product model information features, maintenance knowledge can be divided into three domains: structure domain, maintenance domain, and analysis domain.
Structure domain mainly expresses product structure information such as size, color, weight, and assembly relationship. For a more reality training effect, it is important to make the size and color of the model as far as possible as same as the reality object. Assembly relationship induces topological relationship, tolerance information, and spatial position among parts and is prepared for maintenance simulation.
Maintenance domain mainly includes maintenance process for product operation knowledge such as dismantling path planning, testing, therblig, and selection of maintenance tools. In order to realize maintenance simulation in nonimmersion virtual maintenance system, the virtual human movement must be decomposed. Maintenance activities are decomposed into therbligs based on motion analysis method for engineering, and common maintenance activity is composed of a series of therbligs. The therbligs are divided into three classes: moving class, static class, and operation class. The detail information of therblig classes can be described in Table 1.
Analysis domain mainly focuses on the maintenance analysis and evaluation such as maintenance fault analysis, maintenance time estimation, and maintenance reachability analysis. Maintenance fault analysis acquires test data in the maintenance process to judge failure reason, failure type, the failure effects, and so forth. The test data includes the collision detection, distance detection, and tool change frequency. Maintenance time estimation mainly determines spending the real time of a therblig in reality maintenance process which is based on the time of a therblig in the virtual maintenance process and the zoom factor based on the moving element maintenance difficulty level and the human posture. Maintenance reachability analysis mainly analyses maintenance personnel space, maintenance position, and visual range in the maintenance process.
4. An Example
Based on EON studio platform, a prototype of the virtual maintenance system for complex mechanical product was developed, and an application in the mechanical product of hydraulic winch was provided as shown in Figure 6.
The 3D models in this example were set up by the CAD software UG and were converted to the virtual maintenance system through the model transformation interface as shown in Figure 6(a). In the proposed system, the bill of material for hydraulic winch can be organized according to the assembly constraints and the multilevel information representation model as shown in Figure 6(b). The system carried out real-time detection of collision whether or not it occurred. If collision occurred between two parts, the color of two parts was changed to a different color, meanwhile, the system issued a warning in the form of sounds, as shown in Figure 6(c) and enlarged Figure 6(d). The maintenance activities were composed of therbligs, and the maintenance activities of gear-shaft installation that consist of therbligs were shown as in Table 2. The components put on the table, as that shown in Figure 6(e). Virtual human was adjusting the position of gear shaft as shown in Figure 6(f). The system can automatically analyze maintainability such as tool change frequencies, maintenance space, and maintenance visibility, as shown in Figure 6(g). Finally, the disassembly results and maintainability analysis report were output as shown in Figure 6(h).
5. Conclusions and Future Work
A virtual maintenance system for complex mechanical product was designed and implemented to improve the maintenance training effect. The framework and its key technologies of the proposed system were elaborated. Finally, an example was provided, and the proposed system was proved to be feasible and efficient.
Ongoing and future work will focus on the improvement and extension of the proposed system. Furthermore, the applications in other complex mechanical products are also an important research for the authors.
Conflict of Interests
The authors declare that they have no conflict of interests.
The support of the National Natural Science Foundation of China (nos. 51005231, 51209203), the National Natural Science Foundation of Jiangsu Province (no. BK2012131), and the Priority Academic Program Development of Jiangsu Higher Education Institutions in carrying out this research is gratefully acknowledged.
- L. Ma and C. Lu, “Study on virtual maintenance technology,” Journal of Computer-Aided Design and Computer Graphics, vol. 17, no. 12, pp. 2729–2733, 2005.
- K. Le, Z. Dong, and L. Chuan, “A new maintenance time measurement method by virtual reality,” Journal of Theoretical and Applied Information Technology, vol. 43, no. 1, pp. 74–81, 2012.
- X. Liu, G. Peng, X. Liu, and Y. Hou, “Development of a collaborative virtual maintenance environment with agent technology,” Journal of Manufacturing Systems, vol. 29, no. 4, pp. 173–181, 2010.
- A. F. Abate, M. Guida, P. Leoncini, M. Nappi, and S. Ricciardi, “A haptic-based approach to virtual training for aerospace industry,” Journal of Visual Languages and Computing, vol. 20, no. 5, pp. 318–325, 2009.
- Y. Gao, X. Zhong, Z.-G. Xu, and T. Zheng, “Virtual teaching experiment system of aero-engine based on VR,” Journal of System Simulation, vol. 20, no. 11, pp. 2925–2930, 2008.
- X.-F. Cui, W.-B. Feng, C.-L. Xiang, Z.-J. Wang, and H.-B. Hao, “Research of maintainability and validation methods in armored equipment based on virtual maintenance,” Acta Armamentarii, vol. 30, no. 11, pp. 1430–1434, 2009.
- S. Yang, W.-H. Hwang, Y.-C. Tsai, F.-K. Liu, L.-F. Hsieh, and J.-S. Chern, “Improving balance skills in patients who had stroke through virtual reality treadmill training,” American Journal of Physical Medicine and Rehabilitation, vol. 90, no. 12, pp. 969–978, 2011.
- C. A. Ràbago and J. M. Wilken, “Application of a mild traumatic brain injury rehabilitation program in a virtual realty environment: a case study,” Journal of Neurologic Physical Therapy, vol. 35, no. 4, pp. 185–193, 2011.
- X. Liu, G. Peng, X. Liu, and Y. Hou, “Disassembly sequence planning approach for product virtual maintenance based on improved max-min ant system,” International Journal of Advanced Manufacturing Technology, vol. 59, pp. 829–839, 2012.
- R. D. Yang, X. Fan, D. Wu, and J. Yan, “Virtual assembly technologies based on constraint and DOF analysis,” Robotics and Computer-Integrated Manufacturing, vol. 23, no. 4, pp. 447–456, 2007.
- A. C. K. Choi, D. S. K. Chan, and A. M. F. Yuen, “Application of virtual assembly tools for improving product design,” International Journal of Advanced Manufacturing Technology, vol. 19, no. 5, pp. 377–383, 2002.
- F. Multon, R. Kulpa, and B. Bideau, “MKM: a global framework for animating humans in virtual reality applications,” Presence: Teleoperators and Virtual Environments, vol. 17, no. 1, pp. 17–28, 2008.
- F. Multon, R. Kulpa, L. Hoyet, and T. Komura, “Interactive animation of virtual humans based on motion capture data,” Computer Animation and Virtual Worlds, vol. 20, no. 5-6, pp. 491–500, 2009.
- P. Xie, Q.-X. Su, and H.-Q. Gu, “Research on development and design method of virtual maintenance training system of equipment,” Journal of System Simulation, vol. 18, no. 8, pp. 2195–2198, 2006.
- J. H. Liu, J. Yao, and R. X. Ning, “Research and realization of collision detection algorithm in virtual assembly environment,” Journal of System Simulation, vol. 16, no. 8, pp. 1775–1779, 2004.
- J. Shi, J. Liu, R. Ning, and W. Hou, “A collisions evaluation method in virtual environment for collaborative assembly,” Journal of Network and Computer Applications, 2013.
- J. J. Jiménez and R. J. Segura, “Collision detection between complex polyhedra,” Computers and Graphics, vol. 32, no. 4, pp. 402–411, 2008.
- P. Xie, Q.-X. Su, and H.-Q. Gu, “Research on scene graph of virtual maintenance training system of armament,” Acta Armamentarii, vol. 27, no. 4, pp. 741–744, 2006.
- J.-H. Liu, J. Yao, and R.-X. Ning, “Research on information integration technology between CAD system and virtual assembly system,” Computer Integrated Manufacturing Systems, vol. 11, no. 1, pp. 44–67, 2005.
- X. Li, J. Hao, and H. Liu, “Design and realization of maintenance therblig model in virtual maintenance simulation,” China Mechanical Engineering, vol. 16, no. 2, pp. 156–160, 2005.
- Q.-X. Zhou, G. Cai, B.-Q. Zhang, K. Mi, and J.-X. Yang, “Study on fatigue model of virtual human upper limb while pulling/stretching,” Journal of System Simulation, vol. 21, no. 15, pp. 4823–4826, 2009.
- M. Buekers, G. Montagne, A. de Rugy, and M. Laurent, “The regulation of externally paced human locomotion in virtual reality,” Neuroscience Letters, vol. 275, no. 3, pp. 171–174, 1999.
- C. Y. Tang, C. P. Tsui, B. Stojanovic, and M. Kojic, “Finite element modelling of skeletal muscles coupled with fatigue,” International Journal of Mechanical Sciences, vol. 49, no. 10, pp. 1179–1191, 2007.
- P. Wang, Z.-J. Liu, and Z.-M. Xie, “Research on the technology of maintenance information modeling supporting product lifecycle in virtual environment,” Journal of Hunan University Natural Sciences, vol. 38, no. 8, pp. 28–33, 2011.
- C.-F. Yao, D.-H. Zhang, K. Bu, and Z.-Q. Wang, “Collaborative manufacturing task information model and its model building method for complex parts,” Computer Integrated Manufacturing Systems, vol. 15, no. 1, pp. 47–52, 2009.
- S.-S. Wang and J.-P. Hao, “System of virtual maintenance prototype and modeling technology,” Journal of System Simulation, vol. 17, no. 5, pp. 1132–1136, 2005.
- S.-P. Yu, R.-X. Lü, X.-F. Pang, B.-L. Zheng, and T.-Y. Chai, “Simulation system of scheduling for steelmaking and continuous casting based on virtual reality,” Journal of Central South University (Science and Technology), vol. 40, no. 1, pp. 277–283, 2009.
- B.-B. Liu, L. Tian, Y.-H. Yang, and W.-R. Duan, “Design and implementation of aviation virtual maintenance system,” Computer Integrated Manufacturing Systems, vol. 17, no. 11, pp. 2324–2332, 2011.
- C. Noon, R. Zhang, E. Winer, J. Oliver, B. Gilmore, and J. Duncan, “A system for rapid creation and assessment of conceptual large vehicle designs using immersive virtual reality,” Computers in Industry, vol. 5, no. 63, pp. 500–512, 2012.
- X. Zhang, A. Wang, and J. Li, “Design and application of virtual reality system in fully mechanized mining face,” Procedia Engineering, vol. 26, pp. 2165–2172, 2011.
- H. Jin, Research on the collision detection algorithms in virtual environment [Ph.D. thesis], Huazhong University of Science & Technology, 2006.
- X. Liu, An intelligent algorithm of collision interference detection for 5-axis NC machining based on triangle-facet intersection [Ph.D. thesis], Guangdong University of Technology, 2011.