Research Article | Open Access
Achmad Widodo, Djoeli Satrijo, Toni Prahasto, Gang-Min Lim, Byeong-Keun Choi, "Confirmation of Thermal Images and Vibration Signals for Intelligent Machine Fault Diagnostics", International Journal of Rotating Machinery, vol. 2012, Article ID 847203, 10 pages, 2012. https://doi.org/10.1155/2012/847203
Confirmation of Thermal Images and Vibration Signals for Intelligent Machine Fault Diagnostics
This paper deals with the maintenance technique for industrial machinery using the artificial neural network so-called self-organizing map (SOM). The aim of this work is to develop intelligent maintenance system for machinery based on an alternative way, namely, thermal images instead of vibration signals. SOM is selected due to its simplicity and is categorized as an unsupervised algorithm. Following the SOM training, machine fault diagnostics is performed by using the pattern recognition technique of machine conditions. The data used in this work are thermal images and vibration signals, which were acquired from machine fault simulator (MFS). It is a reliable tool and is able to simulate several conditions of faulty machine such as unbalance, misalignment, looseness, and rolling element bearing faults (outer race, inner race, ball, and cage defects). Data acquisition were conducted simultaneously by infrared thermography camera and vibration sensors installed in the MFS. The experimental data are presented as thermal image and vibration signal in the time domain. Feature extraction was carried out to obtain salient features sensitive to machine conditions from thermal images and vibration signals. These features are then used to train the SOM for intelligent machine diagnostics process. The results show that SOM can perform intelligent fault diagnostics with plausible accuracies.
Machine condition monitoring and fault diagnostics are very important for machineries in the industry to guarantee high performance of their functionality. High reliability is needed to reduce product and profit loss because of unwanted downtime. Therefore, machine condition monitoring and fault diagnostics become a critical issue in maintenance activity to ensure availability, minimize operator hazard, and reduce economic losses . For this purpose, several sensors such as vibration sensors, acoustic emission sensors, and temperature sensors are used to fulfill the requirement for machine condition monitoring procedure . The assessment of machine components of critical rotating machines, such as bearings (journal or rolling element), using such sensors is significant for early detection of machine condition.
The application of vibration sensors as fundamental tools for machine condition monitoring has been used extensively over a period of approximately four decades . The reason to use these sensors was their effectiveness of measurement process and data analysis that can represent the machine conditions. Vibration signal monitoring of installed bearings in rotating machines is still effective to obtain the machine’s behavior. Such monitoring could extract the machine conditions through signals analysis in time and frequency domains. Machine fault diagnostics technique using vibration sensors has been reported by many researchers including application of intelligent system in their proposed method [4–7].
Machinery monitoring and diagnostics by means of acoustic emission (AE) sensor have also been conducted by experts with their own proposed techniques [8–10]. Their works were mostly applied to slow speed bearings monitoring and fault diagnostics. According to published results, AE sensor was more sensitive than vibration sensor in monitoring low speed bearings. The use of AE sensor, along with intelligent technique, that is, relevance vector machine (RVM), for help in machine diagnostics process using such sensor was reported by Widodo et al. . Another research in this area was reported by El-Ghamry et al.  in which he used the automated pattern-recognition procedure to perform fault diagnostics of reciprocating machines by AE sensor.
The use of temperature monitoring for journal bearings has been reported by Glavatskih et al.  and Glavatskih . Their work did not employ any intelligent systems for monitoring and diagnostics. Temperature monitoring by means of thermocouple sensor is usually used as a secondary sensors instead of vibration and AE sensor. Therefore, the reports which employ intelligent diagnostics using such sensor are rarely found.
Another method that is recently popular in machine condition monitoring and fault diagnostics is the thermography technique. The good performance and simplicity to use have made this technique highly popular in the engineering maintenance area. This technique has been used in wide range area such as the mechanical, electrical, petrochemical, material, medical, and structure and building areas, and so forth . In the case of machine condition monitoring and diagnostics, the use of such method has been reported by Mazioud et al.  in which fault detection of rolling element bearing degradation using infrared thermography was performed. The implementation of intelligent system use thermography has been proposed by Fidali . He proposed a continuous monitoring using thermography camera and fault-diagnostics based statistical texture analysis. His work employed Fisher criterion and minimization of error classification for diagnostics procedure. Younus et.al.  reported evaluation on thermal data image for machine diagnostics. Image features are extracted by principal component and independent component analysis. Their proposed system was augmented by support vector machine (SVM) for diagnostics process. A recent publication by Younus and Yang  implemented feature extraction of images by two-dimensional discrete wavelet transform. The intelligent system used the SVM for machine diagnostics.
In this paper, confirmation of thermal images and vibration signals for intelligent machine fault diagnostics is studied. The aim of this work is to find a new alternative source feature and a machine diagnostics technique by utilizing thermal images and SOM. Thermal images are processed by the thresholding method for image segmentation and clustered by means of -means algorithm. Salient features are then extracted from binary images after images processing. The extracted features used for SOM training are area, perimeter, and central moment of region of interest (ROI). After SOM diagnostics, the result will be compared with the vibration-based method for validation purposes.
2. Basic Theory
2.1. Thermal Image
Infrared camera does not measure directly a temperature of surface. The detectors of the camera are sensitive to the luminance emitted by the object being studied. This luminance is naturally dependent on the temperature of the surface. Thermal image is an image which is captured by infrared thermography camera. This device can capture electromagnetic spectrum within infrared bands, 0.78–1000 μm. A thermal image is a function of radiated energy on an inspected object . This image can be digitized and represented in matrix form for computational processing as follows:
2.2. Image Thresholding
Image thresholding is mostly used for image segmentation [21, 22]. Thresholding is also common method for defect detection of material by IRT image. Thresholding is a process to separate objects from its background in a digital image. Histogram is the main tool in this separation process. Suppose that the grey level corresponds to an image that is composed of lights objects on a dark background, in such a way that object and background pixels have grey levels grouped into two dominant modes. Extracting the object from the background is performed by selecting a threshold that separates these modes. A thresholded image is defined as The method of global threshold is selected in this paper where the grey-level histogram is calculated from the entire image .
2.3. Feature Extraction
2.3.1. Image Features
The perimeter and area are meaningful for binary image. Consider a discrete binary image containing one more object, where if a pixel is part of the object and for all nonobject or background pixels. The bit-quads  method is adopted to determine the perimeter and area of ROI. The moment of order of an image , denoted as , is defined as where and are coordinates of the pixel, and is the grey level of the pixel. The higher order moments represent spatial distribution of the objects. The previous two characteristics make moments a reasonable and proper feature for the classification task. However, the above equation may be greatly affected by the objects shifting. Therefore, to overcome this problem central moment is used and defined as where and .
The relationship between moments and central moments up to order 3 are listed as follows: Then, central moments could be further normalized by the following formulas: where
2.3.2. Vibration Features
2.4. Self Organizing Map (SOM)
The SOM is a neural network model that implements a characteristic non-linear projection from a high dimensional space of sensory or other input signals onto a low-dimensional array of neurons. The SOM is an unsupervised learning algorithm which produces a map pattern features on its output layer . Input patterns with similar features are mapped onto neighbouring output nodes. The network consists of an input layer with neurons and an output layer with neurons as shown in Figure 1.
The image of the signal space tends to manifest the clusters of input information and their relationship on the feature map. With every node , a parametric reference vector is associated. In an abstract scheme it may be imagined that at time the input , by means of some parallel computing mechanisms, is compared with all , and the location of best match in some metric is defined as the location of the response. However, in many practical applications, the Euclidean distances can be made to define the best matching node (neuron) which is signified by the index , where ; ; represents the input vectors and represents the weight vectors, of the th output vectors. is the error which is given by .
3. Method and Material
The flowchart of diagnostics process is presented in Figure 2. After image acquisition from the object, thermal image is cropped to select the ROI, that is, rolling element bearing image. Next step, image clustering will be performed by using -means algorithm to cluster the hotspot area based on color palette. Image segmentation by thresholding is conducted by using the a Otsu method . Image thresholding will produce binary image which is useful for image feature extraction. Perimeter, area, and central moments are then calculated based on resulted binary images. After feature extraction, extracted features from normal and faulty condition are separated for SOM training purpose. The feature of normal condition will be used as baseline of the diagnostics process. Finally, SOM will make decision of whether the machine condition is normal or faulty.
Data acquisition was conducted on MFS which is able to simulate abnormality in rotating machine, that is, rolling element bearing fault, unbalance, misalignment, and looseness. Thermal images and vibration signals were simultaneously acquired from the machine that has been operated for 30 minutes to generate initial heat in the bearings. The image acquisition produced 30 images per second, and the machine was operated at 1800 rpm. Figures 3 and 4 show the data acquisition process in the laboratory using infrared thermography camera and accelerometer sensors.
4. Results and Discussion
Data acquisition of thermal image and vibration signal of faulty bearing is presented in Figures 5 and 6. FLIR SC5000 camera was used to capture thermal images and LMS Test. Xpress was used to vibration signal acquisition from MFS. The acquired images were stored in RGB format with size of 256 × 256 pixels, and vibration signals were acquired in 5 kHz sampling frequency.
After cropping, thermal images captured from right and left bearings are presented in Figures 7 and 8. The ROI was determined by red or pink colors that represent the hottest temperature area generated by bearings. Such colors were then extracted and clustered by -means algorithm with . Figure 9 depicts the clustering result by means of -means algorithm.
Figure 9 also shows the colors in ROI from which feature extraction will be performed. The selected ROI will be transformed into binary image by thresholding method. Next step is feature extraction of perimeter, area, and central moments from ROI. This process is depicted in Figure 10.
Figures 11–13 show image features of area, perimeter, and central moment , respectively. The labels U, M, L, and B represent the faults in the machine namely unbalance, misalignment, looseness, and bearing faults, respectively. Figure 11 shows random patterns for the hot area of ROI which are not good patterns because the conditions of the machine cannot be distinguished clearly. The feature representation of Figure 12 shows better patterns than the feature of area, even though some data are out from the cluster of conditions. Observing this figure, temperature is increased between normal condition and faulty condition. This plot could be a reference for determination of bearing condition. Figure 13 also shows random structure of patterns but is better than feature of area. These patterns will be then input data for SOM training for fault diagnostics process.
Feature extraction was also performed on time domain vibration signal by calculating its statistical features. The results of feature extraction are presented in Figures 14, 15, 16. It is clearly observed that the machine conditions are well clustered. Comparing with features from thermal images, vibration features show better performance in clustering of machine conditions.
The fault diagnostics process is started by training SOM algorithm using features from thermal images and vibration signals. SOM is trained by 20 training data which consist of five machine conditions. The results of training process are presented by using U-matrix with hexagon topology to visualize distances between neighboring map units and help in seeing the cluster structure of the map. Map units indicate cluster border between conditions in the machine which are visualized by different color in the map. The U-matrices of SOM training are shown in Figures 17 and 18. These maps will be references from which fault diagnostics procedure is performed.
After SOM training, the performance of SOM diagnostics was evaluated by testing process using 10 data from thermal images of bearing fault and unbalance case (BU label), and 8 data from vibration signals of bearing fault case (B label) respectively. In the case of thermal images data, 8 data successfully activated cluster of BU, and 2 data were misclassified in BU cluster. It means that the accuracy is 80%. In addition, testing process using vibration data gives accuracy of 100% which means that all data tests successfully activate the cluster of B. From the experiment, the performance of testing process using vibration data is better than thermal images data. It was well known that vibration data can produce good accuracy in machine fault diagnostics. However, it could be said that the results of testing process by using thermal images and vibration data show good performance. Testing process of SOM using thermal images and vibration data are visualized by presenting the topography of best matching unit (BMU) as depicted in Figures 19 and 20, respectively.
This paper deals with the confirmation of thermal images and vibration signals for intelligent machine fault diagnostics using SOM network. Thermal images and vibration signals were acquired from MFS that is a reliable tool for machinery simulation. Image segmentation technique was performed on thermal images to obtain binary images for image feature extraction process. The features extracted from thermal images were area, perimeter, and central moments. Statistical features, namely, RMS, means, skewness and kurtosis, were also extracted from time-domain vibration signals. SOM were then trained by using extracted features from both thermal images and vibration signals. After training, SOM was tested by feeding some data to evaluate the performance of fault diagnostics. The results show that good accuracy was reached by using both source features thermography and vibration. Moreover, it showed that the thermography method could be an alternative way for conducting intelligent machine fault diagnostics with plausible results.
This work was supported by the Faculty of Engineering, Diponegoro University, Indonesia, through Research Grant for International Research Collaboration with Contract no. 3203/UN7.3.3/PG/2011. This work is also dedicated to the late Professor Bo-Suk Yang, the founder of the Intelligent Mechanics Laboratory at Pukyong National University Republic of Korea, where the first author spent almost five years with him in research area of machine fault diagnostics and prognostics. The authors gratefully acknowledge the support from the Brain Korea (BK21) project on this study.
- Y. Lei, Z. He, and Y. Zi, “Application of an intelligent classification method to mechanical fault diagnosis,” Expert Systems with Applications, vol. 36, no. 6, pp. 9941–9948, 2009.
- B. Eftekharnejad and D. Mba, “Seeded fault detection on helical gears with acoustic emission,” Applied Acoustics, vol. 70, no. 4, pp. 547–555, 2009.
- A. Barber, Handbook of Noise and Vibration Control, Elsevier Advanced Technology Publications, London, UK, 6th edition, 1992.
- B. S. Yang, D. S. Lim, and J. L. An, “Vibration diagnostic system of rotating machinery using artificial neural network and wavelet transform,” in Proceeding of 13th International Congress on COMADEM, pp. 12–20, Houston, Tex, USA, 2000.
- B. Samanta, “Gear fault detection using artificial neural networks and support vector machines with genetic algorithms,” Mechanical Systems and Signal Processing, vol. 18, no. 3, pp. 625–644, 2004.
- A. K. S. Jardine, D. Lin, and D. Banjevic, “A review on machinery diagnostics and prognostics implementing condition-based maintenance,” Mechanical Systems and Signal Processing, vol. 20, no. 7, pp. 1483–1510, 2006.
- A. Widodo and B. S. Yang, “Support vector machine in machine condition monitoring and fault diagnosis,” Mechanical Systems and Signal Processing, vol. 21, no. 6, pp. 2560–2574, 2007.
- T. Yoshioka and T. Fujiwara, “Application of acoustic emission o detection of rolling bearing failure,” ASME. Production Engineering Division Publication, vol. 14, pp. 55–76, 1984.
- N. Jamaludin, D. Mba, and R. H. Bannister, “Condition monitoring of slow speed rolling element bearings using stress waves,” Journal of Process Mechanical Engineering, vol. 215, no. 4, pp. 245–271, 2001.
- A. Morhain and D. Mba, “Bearing defect diagnosis and acoustic emission,” Proceedings of the Institution of Mechanical Engineers, Part J, vol. 217, no. 4, pp. 257–272, 2003.
- A. Widodo, B. S. Yang, E. Y. Kim, A. C. C. Tan, and J. Mathew, “Fault diagnosis of low speed bearing based on acoustic emission signal and multi-class relevance vector machine,” Nondestructive Testing and Evaluation, vol. 24, no. 4, pp. 313–328, 2009.
- M. H. El-Ghamry, R. L. Reuben, and J. A. Steel, “The development of automated pattern recognition and statistical feature isolation techniques for the diagnosis of reciprocating machinery faults using acoustic emission,” Mechanical Systems and Signal Processing, vol. 17, no. 4, pp. 805–823, 2003.
- S. B. Glavatskih, O. Uusitalo, and D. J. Spohn, “Simultaneous monitoring of oil film thickness and temperature in fluid film bearings,” Tribology International, vol. 34, no. 12, pp. 853–857, 2001.
- S. B. Glavatskih, “A method of temperature monitoring in fluid film bearings,” Tribology International, vol. 37, no. 2, pp. 143–148, 2004.
- H. Kaplan, Practical Applications of Infrared Thermal Sensing and Imaging Equipment, SPIE Publication, 3rd edition, 2007.
- A. Mazioud, L. Ibos, A. Khlaif, and J. F. Durastant, “Detection of rolling bearing degradation using infrared thermography,” in Proceeding of the 9th International Conference on Quanitative Infrared Thermography, Krakow, Poland, July 2008.
- M. Fidali, “An idea of continuous thermographic monitoring of machinery,” in Proceeding of the 9th International Conference on Quantitative Infrared Thermography, Krakow, Poland, July 2008.
- A. M. Younus, A. Widodo, and B. S. Yang, “Evaluation of thermography image data for machine fault diagnosis,” Nondestructive Testing and Evaluation, vol. 25, no. 3, pp. 231–247, 2010.
- A. M. Younus and B. S. Yang, “Intelligent fault diagnosis of rotating machinery using infrared thermal image,” Expert System with Applications, vol. 39, no. 2, pp. 2082–2091, 2011.
- T. Kohonen, Self-Organizing Maps, Springer, Berlin, Germany, 1995.
- N. Otsu, “A threshold selection method from gray-level histograms,” IEEE Transactions on Systems, Man, and Cybernetics, vol. 9, no. 1, pp. 62–66, 1979.
- R. C. Gonzalez and R. E. Woods, Digital Image Processing, Prentice-Hall, Upper Saddle River, NJ, USA, 2nd edition, 2002.
- B. Chanda and D. D. Majumder, Digital Image Processing and Analysis, Prentice-Hall of India Private, New Delhi, India, 2000.
- J. Shao, H. Xin, and J. D. Harmon, “Comparison of image feature extraction for classification of swine thermal comfort behavior,” Computers and Electronics in Agriculture, vol. 19, no. 3, pp. 223–232, 1998.
Copyright © 2012 Achmad Widodo et al. This is an open access article distributed under the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.