Abstract
There are different applications of computer vision and digital image processing in various applied domains and automated production process. In textile industry, fabric defect detection is considered as a challenging task as the quality and the price of any textile product are dependent on the efficiency and effectiveness of the automatic defect detection. Previously, manual human efforts are applied in textile industry to detect the defects in the fabric production process. Lack of concentration, human fatigue, and time consumption are the main drawbacks associated with the manual fabric defect detection process. Applications based on computer vision and digital image processing can address the abovementioned limitations and drawbacks. Since the last two decades, various computer vision-based applications are proposed in various research articles to address these limitations. In this review article, we aim to present a detailed study about various computer vision-based approaches with application in textile industry to detect fabric defects. The proposed study presents a detailed overview of histogram-based approaches, color-based approaches, image segmentation-based approaches, frequency domain operations, texture-based defect detection, sparse feature-based operation, image morphology operations, and recent trends of deep learning. The performance evaluation criteria for automatic fabric defect detection is also presented and discussed. The drawbacks and limitations associated with the existing published research are discussed in detail, and possible future research directions are also mentioned. This research study provides comprehensive details about computer vision and digital image processing applications to detect different types of fabric defects.
1. Introduction
Computer vision and image classification-based models are used in various applied domains including industry-based problems [1–4]. Clothing is considered as one of the basic requirements for human life, and the history of textile industry is as old as human civilization. Fabric is considered as a main element for human clothing and is also used in many industrial products [5–8]. Natural elements such as wool, cotton, a composite of polyester, or nylon can be used to create textile fabric bib9 [8, 9]. Sophisticated machines are used in textile industry to create this fabric, and defects are located through the inspection process. Traditionally, inspection process is completed by using manual human efforts to ensure the quality of fabric [6, 10]. The price of fabric that is sent to the market depends on the number of co-occurrence of defects and price increase with the increase in the number of defects [9, 11, 12]. As a defect is detected, the production process is stopped and the details about the occurred defect are recorded with its location by the machine operator. The main drawbacks associated during manual inspections are as follows: (1) training of individuals is required to make them fabric inspector; (2) major defects can be detected while small defects can be ignored due to human carelessness; (3) lot of human effort is required to locate fabric defects; and (4) it is very difficult for fabric inspectors to keep focus on the production process for a time that is more than 10 minutes and all of this can lead to a low efficiency of production. According to research [3], 60–75% human accuracy is reported to detect fabric defects and the wastage due to fabric defects leads to the high price of product in market.
Due to these reasons, it is recommended to apply some automated processes to detect the fabric production defects and this process can save labor cost. Computer simulations are used for this purpose and textile products are refined through this process and can provide higher inspection quality. Computer vision and digital image processing plays a vital role in textile industry for this inspection process bib9[2, 9]. The inspection of fabric is carried out during the production process that is considered as a real-time application. Automated inspection in textile industry is performed while using computer vision and digital image processing techniques. Figure 1 represents the automatic fabric defect system. Figure 1(a) shows the scheme based on manual efforts, and defect in this case will be detected by humans. Figure 1(b) shows an automated process for fabric defect detection. According to the published research, more than 70 different possible fabric defects are reported and some common defects are shown in Figure 2. These defects are caused by different reasons and can be classified as major, minor, or critical defect (depending on the severity of the defects [13]. Some of the frequently occurring major defects are broken pick, ends out, float, holes, stitches, knots, loose threads, starting mark, oil stains and marks, bad selvedge, double pick, snarls, cracks, and smash. Hole is classified as a major defect which is caused by many reasons such as by broken needle or due to defective machine. Oil stain is mainly caused due to excessive oil from the machinery. Multiple netting are minor defects caused when multiple broken threads are combined together. Broken end appears in the fabric when warp yarns break during weaving. Thick and thin bar defects appear when variations occur in yarn. Broken weft appears when in weaving filling yarn is broken. Wrong weft defect appears when the process weaving does not follow a schematic pattern during weft insertion. There are various categories of fabric defects and many ways to classify them such as common fabric defects or classification based on fabric color. In this article, mostly grayscale fabric defect detection algorithms and classification models are discussed in detail. Most of the computer vision algorithms are designed for grayscale images and use different feature extraction approaches to create discriminative image representations. Other than this, the grayscale image processing can extract descriptors more easily as compared to the color images and the grayscale image processing can reduce the computational complexity. The colored fabric defects have their own importance and detection models for example misarranged warp yarns fabric defects [14–16].
(a)
(b)
Generally, a set number of yarns are used for the formation of fabrics, and different techniques such as knitting, weaving, and felting are used for producing fabric. The type of the fabric varies based on the fibers used, the fabric formation technique, machinery used for their production, and the finishing techniques. There are three main challenges related to the fabric defect detection. (1) There exists a broad range of fabrics exhibiting different characteristics. Application of general algorithms to varied texture types is difficult and may lead to instability in the traditional defect detection methods. (2) The varied characteristics and categories of the fabric defects contribute to the difficulties. (3) It is extremely difficult to collect large numbers of defect samples particularly of some rare types, which results in data imbalance or even complete failure of traditional supervised methods. To deal with these challenges, numerous researchers have made substantial efforts. According to the literature, the fabric defect detection methods are broadly categorized as motif-based approaches and non-motif-based approaches [6, 8, 17]. More research is focused on non-motif-based group; therefore, this article mainly covers non-motif-based methods. In this review article, we aim to present an overview about fabric defect detection for textile industry. The proposed study presents a detailed overview about approaches for fabric defect detection. The performance evaluation criteria for automatic fabric defect detection is also presented and discussed in detail. The drawbacks and limitations associated with the existing published research are discussed in detail, and possible future research directions are also mentioned. This research study provides comprehensive details about computer vision and digital image processing applications to detect different types of fabric defects.
2. Histogram-Based Approaches
A histogram is a display of statistical information computed on the basis of the number of co-occurrence of gray levels in an image [18, 19]. According to the literature [20], the spatial approaches have the advantage of being computationally simple but have weak performance in the detection of small defects. Wavelet transform-based approaches outperform the spatial methods in terms of computational efficiency and performance (particularly for detecting small fabric defects). Ye et al. [20] applied fuzzy inductive reasoning (FIR) for the detection of defects in fabrics. The histogram statistics extracted from the images are used to distinguish between the faulty and the faultless fabric images. Figure 3 shows the RGB (red, green, and blue) channels and combined histogram of defective images. Since faultless fabric exhibits a regular repetitive global pattern, FIR can help to detect fabric defects by analyzing the fabric structure. To analyze the fabric texture, the histogram projections of the images are considered a good choice since the grayscale variations occur in a certain manner along a warp or weft. Four classes of fabric defects, i.e., wrong draw, oil warp, slubby yarn, and harness skip are examined by using this approach [20]. The difference between the predicted and the real class values triggers an alarm vector that indicates the presence of detect. Besides defect occurrence, the proposed approach also identifies the location of defects.
Based on the characteristics of cord fabric, Zhang et al. [21] proposed a multiple windows gray ratio (MWGR) for the detection of cord fabric defects. The main motivation behind MWGR is that the normal cord fabric grayscale image exhibits an alternating gray and white pattern. A threshold is used to split the image foreground and the background regions. The image is partitioned into several regions and the gray ratio change is analyzed to determine the defect in window. The proposed approach [21] achieved satisfactory results for the detection of cord fabric defects and enhanced the automated quality control for fabric defect detection (FDD). Li et al. [22] proposed an algorithm based on visual saliency for defect detection in both patterned and nonpatterned fabrics. The features computed from the saliency maps are used for the detection of fabric defects. At the first step, the algorithm generates the saliency maps to create a distinction between defective and defect-free regions. The second step involves the extraction and selection of saliency histogram features for effective discrimination between faulty and faultless fabric images. Lastly, classification is done using binary support vector machine (SVM) that was trained by using defective and nondefective fabric samples. The proposed approach resulted in accurate detections as compared to the state-of-the-art research and can be applied for defect detection in both patterned and nonpatterned fabric images. Table 1 represents a summary about various histogram-based approaches proposed for fabric defect detection. Histogram-based methods are noise sensitive, and in nonregular textures, it has low detection rate.
3. Color-Based Approaches
In computer vision and digital image processing, color is used in various applications that are using visible spectrum [23–25]. Color is extracted as an important visual feature in various fabric defect detection-based approaches [26]. Numerous research efforts have been made to improve the accuracy and efficiency of FDD with varying viewpoints; however, FDD still remains a challenging task for complicated textures [26]. Zhang et al. [26] claimed the visual saliency of defective fabric image regions, by arguing that the defective and the nondefective local regions of fabric images have visual differences in color. In addition to this, the defective regions are not widespread in the image; rather, they are grouped in a small region. Based on these observations, the authors proposed to determine the defective value of a pixel by an integrated measure of color dissimilarity and positional similarity. To enhance the reliability and robustness of results, the contrast of both the defective and the nondefective regions is enhanced by applying multiscale dissimilarity analysis. The overview of the proposed method for FDD is shown in Figure 4. Experiments are conducted using MATLAB R2014a on Intel Core processor with 6 GB RAM and 3.2 GHz frequency. Experimental results conducted with diverse fabric images achieved a better performance as compared to the state-of-the-art results for complicated patterns. However, for box-patterned and motif-based images, the effectiveness of the proposed saliency method could not be established. Table 2 represents a summary about color-based approaches proposed for fabric defect detection.
4. Segmentation-Based Approaches
Segmentation is used to divide the image into subregions, and the success of any computer vision algorithm depends on the effectiveness of the applied segmentation technique [27, 28]. The high computational cost associated with image segmentation is considered as one of the limitations of approaches that are relying on image segmentation [29, 30].
To detect defects in plain woven fabrics, Guan et al. [31] investigated an approach based on the segmentation of regions of interest (ROI) from defected images. The processing is done by using images in grayscale mode, and an image enhancement technique is applied to highlight the regions with defects. To further enhance the accuracy and to reduce the algorithm complexity, the noise is removed by applying the low-pass filtering which highlights the defected regions. The defected regions are then segmented with edge detection based on first-order derivatives. The Roberts operator was observed to efficiently detect the edge of defect regions and resulted in better accuracy. The ROIs are then obtained by isolating the defect regions based on boundary. The experimental results demonstrated that segmentation of defective regions was done successfully which enhanced the defect detection rate [31]. Table 2 represents a summary about segmentation-based approaches proposed for fabric defect detection.
5. Dictionary Learning-Based Approaches
Dictionary learning-based approaches have shown good results for fabric defect detection [3]. Li et al. [32] proposed a learned dictionary-based visual saliency approach for FDD. This approach partitioned the image into blocks, and a dictionary is then constructed for both normal and defective image samples based on sorted local binary features correlation. The N largest correlation coefficients are computed between each test image block and the dictionary, and the remaining correlation coefficients are set to zero. A saliency map is then generated based on the sum of nonzero coefficients related to the defective image samples. The saliency maps created by the proposed approach are compared with the state-of-the-art methods. The proposed approach successfully localized defective regions for complex fabric images.
In another research, Zhou et al. [33] proposed a novel texture feature extraction technique for the detection and classification of fabric defects based on multiscale dictionary learning. A linear summation is applied to the multiscale dictionaries for accurate representation of image features. To create a classification and detection model, an algorithm is proposed that can deal with the limitation of parameter setting and low accuracy in the original differential evolution algorithm. The detection results obtained using the proposed method are compared with the state-of-the-art methods such as morphological operation, Gabor filter, and local binary pattern. According to experimental results, the proposed method achieved higher precision and detection efficiency as compared with the traditional methods. The overview for dictionary learning phase and defect classification is shown in Figure 5. Table 3 represents a summary about dictionary learning-based approaches proposed for fabric defect detection.
6. Texture-Based Approaches
Texture analysis refers to the characterization of regions in an image by their texture content and is widely used in different computer vision application domains [34, 35]. Liang et al. [36] proposed an intelligent integrated system for the evaluation of yarn surface appearance. Three methods are used for feature extraction: attention-driven method is applied using saliency maps for fault detection and wavelet texture features and various statistical measures are applied to formulate appearance description of yarn surface. The fuzzy ARTMAP neural network (NN) is employed to learn features for grading and classification of yarn surface quality. A comparative analysis is performed among the fuzzy ARTMAP, backpropagation NN, and SVM to obtain the optimal classification performance (Figure 6 shows working of SVM). The experimental results demonstrated that the proposed approach while using fuzzy ARTMAP achieved superior results to classify yarn surfaces.
According to [37], fabric defects are basically abrupt features and the wavelet response for background and abrupt features is totally different, and hence, the wavelet can accurately detect the defects. The methods used for FDD using wavelets are categorized as nonadaptive and adaptive; the former methods use existing bases in combination with some feature extraction method for detection. However, the existing wavelet bases have their drawbacks, and the nonadaptive methods do not attain satisfactory results for all different fabric defects types. Wen et al. [37] proposed an approach for the detection of different image fabric defects by applying adaptive wavelet bases. The adaptive wavelet bases were designed to deal with fabric defects and they generally comprise a training stage and a detection stage. They used a hybrid approach by the combination of wavelet bases and an optimized method to deal with different images. A cost function model is used to obtain a filter coefficient for the detection of fabric defects. A limitation of the proposed approach is that the regular images are required to train the model for the detection of patterned fabric defects. Experimental results demonstrated that the proposed approach effectively achieved accurate detection of fabric defects.
According to Karlekar et al. [38], efficiency and real time are important factors that should be considered for fabric defect detection systems. Karlekar et al. [38] used wavelet analysis since it detects line features efficiently with low complexity. The images are first converted to grayscale, histogram equalization, and median filter are applied for the reduction of noise from fabric images. The wavelet decomposition is applied at different scales to extract significant features for defect classification. Thresholding is then applied for the segmentation of image defects, and morphology filter is applied for removal of the remaining noise. The proposed method obtained satisfactory performance for thresholding and segmentation of fabric defects caused by warp knitting machine. Yapi et al. [39] introduced a generalized learning-based approach based on the statistical signatures to distinguish between defective and defect-free fabrics. The proposed system is able to deal with textile fabrics of varying types, ranging from simple to more complex ones. The system is comprised of three steps: preprocessing, classifier training, and testing. The descriptors representing the fabric are robust to noise and changes in illumination and are invariant to translation and scale transformation. It was concluded that the proposed approach has provided better performance as compared to the state-of-the-art research.
Hamdi et al. [40] proposed an algorithm for unsupervised defect detection in patterned fabric images. The proposed algorithm comprises three steps: preprocessing, a texture filtering step, and defect classification step. The preprocessing step involves improving the contrast of the image and the extraction of fabric periodic pattern. During texture filtering, the image is segmented into different regions based on texture and is filtered to remove the effect of noise. For defect classification, the filtered image is partitioned into blocks and the statistics aggregated from the median of all blocks are used with k-means to distinguish the defected blocks with high difference value. Wu et al. [41] presented a dictionary-based method to enhance the texture fabric representation and boost the computation speed that can assist in the detection of fabric flaws. Discrete cosine transform is used to learn dictionary by applying the alternating least-squares method. The proposed approach provided remarkable results in terms of consistent results, computation speed, and error rate. The proposed approach can be applied to investigate the impact of fabric density on fabric texture representation and for the detection of fabric flaws.
In another recent research, Hanbay et al. [42] proposed a vision-based system for inspection of fabric defects for circular knitting machine. The proposed system is comprised of two modules, i.e., feature extraction and defect classification. Initially, a fabric defect detection database is constructed. For feature extraction from fabric images, shearlet transform is applied on the normalized images acquired continuously from one line scan. The means and variances computed from all subbands are concatenated to create a high-dimensional feature vector. For defect detection, a three-layered artificial neural network (ANN) is used and shown in Figure 7. The performance evaluation is done on a single jersey knitted fabrics on a circular knitting machine in a textile factory. For quantitative evaluation, five different indexes are used. The proposed approach attained better outcomes as compared to the related research on fabric defect detection. Table 4 represents a summary about texture-based approaches proposed for fabric defect detection. Image acquisition is done by using line-scan CCD camera and Intel Core i7 (3.5 GHz) with 16 GB RAM. It is concluded that the wavelet transformation-based approaches are mainly best suited for real-time application due to its computational performance.
7. Frequency Domain Operation
For frequency domain operations, an image is converted from spatial domain to frequency domain [19]. Fast Fourier transformation (FFT) can be used to convert an image from spatial domain to frequency domain [19]. According to Ismail et al. [43], the originality and the examination of fabric by using human eyes cannot guarantee that fabric is 100% original and also it is difficult to process in case of jersey fabric. Therefore, Ismail et al. [43] proposed a method based on FFT to investigate the quality of the jersey fabric (especially sport jersey fabric). For this purpose, they build a sample or prototype to examine the quality of the fabric and to analyze their method based on their structure pattern of the fabric. On the base of Fourier transformation spectrum, the authors [43] used the magnitude and phase graph to compare its results of test samples with the original sample. To differentiate both images, absolute error and mean error are also computed. The results for magnitude and phase graph of two samples (original and test) are compared, and if there is a difference between samples, then the test sample is fake, and if there is no difference, then the sample is original.
For defect detection in fabric with periodic surface texture, Hu et al. [44] proposed an unsupervised method that is based on a combination of discrete Fourier transform (DFT) and discrete wavelet transform (DWT). This method only depends on the periodic surface texture to reduce some problems between the sample under examination and used for comparison (reference sample), i.e., orientation, scale, and unbalanced-illumination problems. It neither depends on any feature extraction approach nor it relies on any sample for judgement [44]. As it is known that Fourier domain is suitable for the periodicity of an object, so this defining characteristic has been used to eliminate the background periodic pattern by zero-masking their dominant frequency in each sample under examination. The image obtained by Fourier-restored residual is then subjected by wavelet shrinkage, and lastly thresholding is applied to get binary images which are then combined for a single output. Hu et al. [44] performed experiments on the number of real textured surfaces where all image samples were composed of 8-bit gray level and pixels. For performance evaluation, two groups, c1r1 and c1r3, from the TILDA textile dataset have been used and false-positive rate (FPR), true-positive rate (TPR), and accuracy (Acc) have been measured.
Sakhare et al. [45] proposed two domain techniques that are spatial and spectral for the detection and classification of defects. The first approach contains morphological and statistical operations which have been used to find out the intensity values. The second approach contains wavelet and Gabor filter, FFT, and DCT (discrete cosine transform) to obtain transformed image which is further analyzed for the detection of defects. Performance evaluation is performed for four types of defects: missing warp, missing weft, hole, and tom out in terms of percentage accuracy. Sakhare et al. [45] detected the fabric defects and proposed a combination of spatial and spectral techniques and performed comparison with various methods.
According to the literature [3], defect detection in the Yarn-Dyed fabric is still limited due to its three-layered structure of patterns, fabric textures, and defects. Zhang et al. [46] proposed a method based on frequency domain filtering, similarity measurement, and distance matching function for the detection of defects in the Yarn-Dyed fabric. The established algorithm does not completely separate the three different structures (patterns, fabric textures, and defects), and basically, this multilayered structure is the major reason for complexity in yarn-dyed fabric for finding defects as separation of these layers is a difficult task [46]. In this method [46], frequency filtering is used to separate the two layers (patterns and fabric textures) of the fabric, and distance matching function to measure the pattern period and similarity measure has been used to recognize the defective and defect-free units. For the verification of proposed method, the analysis has been performed on the following types of defects: warp or weft-lacking, holes, and on manually (artificial) added defect, and reliable and fast result has been achieved, but the results are not mentioned explicitly. Table 5 represents a summary about frequency domain-based approaches proposed for fabric defect detection.
8. GLCM-Based Defect Detection
In textile industry, the quality of products and name of the brand are maintained by the production of defect-free products. Detection of defects can be done by using manual human efforts and this process can be automatic. Raheja et al. [47] proposed a fast defect detection system (FDDS) embedded on DSP (which is digital signal processor electronic board used for experiments). It is a GLCM-based method and implemented on a Texas Instruments Evaluation Module. Here the sliding window analysis is used for the computation of textural energy of the fabric image from the GLCM. For implementation, Texas TMS320DM642 instrument evaluation module is used with two decoders, one encoder with three on-board video ports. The analysis is performed on every frame, and GLCM feature correlation, energy, homogeneity, and contrast are extracted from defective image samples. But energy feature has been used to compute results and to find out defects in texture images by taking the advantage of features that they give the most noticeable peaks in faulty area.
Arnia et al. [48] proposed a method based on gray-level co-occurrence matrix (GLCM) using its two features contrast and energy. Discrete cosine transform (DCT) is used with the objective to evaluate and comply with their method on MPEG (Moving Picture Experts Group) and MJPEG (Motion Joint Photographic Experts Group) video compression standards to reduce space and cost savings by replacing the frame. Before this, GLCM has been used with many other methods for the detection of defects. In the proposed method, DCT images are used to preserve the appearance and details of the image in low- and high-frequency bands, respectively. Moreover, for the effectiveness of this approach using DCTb-Is many compression rates and for defect detection lowest possible rate in DCTb-Is have been evaluated. The results showed that the compression rate greater than 25% can only able to detect and locate the stains, drops, holes, and drop stitches from DCTb-Is.
For the classification of defects in textile fabric, Zhang et al. [49] proposed a hybrid approach based on LBP (local binary patterns) and GLCM (gray-level co-occurrence matrix) to differentiate and to classify defects. The production process can be improved by producers if defects in fabric are correctly differentiated [49]. Zhang et al. used LBP to extract the local features of defective images and GLCM to extract the information of texture features. For the verification of their proposed hybrid approach, authors performed experiments on 6 types of defects where each defect includes 100 images of 256 256 from the TILDA database to get results for classification average accuracy. The results showed that the classification accuracy remarkably increased by the proposed hybrid approach (LBP and GLCM), and with the increase in values of dimension of features, accuracy can be increased.
Defect detection in yarn-dyed fabric is complex due to its multilayered structure, so Zhu et al. [50] proposed an approach for defect detection in yarn-dyed fabric which is based on autocorrelation function and GLCM and defect detection is done by using GLCM window and texture features. In their approach, Zhu et al. [50] selected a detection window based on the periodic pattern of yarn-dyed fabric and used Euclidean distance to differentiate between two images. Defects such as oil stain, holes, broken weft, stretched warp, and wrong weft are considered to conduct experiments on two- and three-colored yarn-dyed fabric. The results have shown that the proposed method has high accuracy. Fabric defect detection based on the GLCM technique is proposed by Zhang et al. [51] where the distance between angular relation (theta) and pixel pair is calculated. Experiments have been conducted on defective and nondefective images gathered from different sources. Different window size of 10 10, 20 20, 50 50, and 60 60, d = 1, and theta = 0° are the used experimental factors. Zhang et al. [51] compared results in terms of percentage detection rate and the results indicate that the proposed method is effective for the detection of fabric defects. Table 6 represents a summary about GLCM-based approaches proposed for fabric defect detection.
GLCM-based methods depend on the matrix of co-occurrence that is substantially influenced due to noise. The limitations of GLCM-based approaches are as follows: it has high computational cost, and in high-resolution images, it gives less performance. Other than these findings, optimal displacement factor is difficult and it depends on the scaling and rotation and requires a procedure for feature selection. For texture having large size primitive cannot analyze texture, and due to undirected distance, it may loss valuable information.
9. Feature Fusion-Based Defect Detection Techniques
According to Zhu et al. [52], the fabric is pressed in the dyeing device during the heating roller calendar process to enhance the aesthetic of the fabric for soft glow on the fabric surface. Sometimes, the heating roller becomes defective due to the fabric seam, and then, fabric defects will also appear when the defective roller will be used in the continuous calendaring process and it is important to detect the defect in this case. Zhu et al. [52] proposed two methods to detect seam using images, textural analysis by wavelet energy method (say M1: method 1) and by feature extraction in CIELAB color space (say M2: method 2) for texture analysis in the image. Energy characteristics for M1 of the fabric image are calculated, and for fabric seam detection, characteristic parameters such as variation coefficient mean and standard deviation are used for M2 in CIELAB color space, and then, data obtained from these two techniques were used to determine the threshold value used for seam detection. The results of both proposed methods have been compared in terms of accuracy and computation time. The system with Core i7 processor (2.2 GHz frequency), MATLAB 2020b, and Windows 7 is used to perform this research [52]. The results have shown that the proposed method M2 (textural analysis in CIELAB color space) is better in terms of accuracy and time.
Jing et al. [53] proposed a new technique for real-time automated defect detection in textile cloth. The proposed research is based on the multiple Gabor filters (MGFs), which are used for feature extraction, and kernel principal component analysis (KPCA) is used for the reduction of high-dimension data to identify many uniform and structural detects. OTSU thresholding approach is used to indicate defects, and this method is effective for the identification of holes, thread errors, oil spot, and object on surface of weaving structures. For the effectiveness of the proposed method, the experiments are conducted on defective and defect-free fabric images taken from TILDA textile dataset, and the true detection rate of the proposed method is calculated. TILDA is a standardized textile texture database developed as a part of the research programme “Automatic Visual Inspection of Technical Objects” by the working group Texture Analysis of the DFG (Deutsche Forschungsgemeinschaft). The database comprised of 8 groups of representative textile kinds. Each class comprised of 50 TIF grayscale images of size 786 512 pixels. The entire textile texture database contains 3200 TIF images with a total size of 1.2 Gbytes [54]. The test samples defective and nondefective with their binary and fused images of class C1 from TILDA textile dataset are represented in Figure 8. Intel (R) i5 (2400M) with 3.10 GHz, 4 GB RAM, and Windows 7 operating system is used as implementation platform. Thus, the result shows that the proposed algorithm has less computational cost for real-time defect detection and has high detection rate.
According to the literature [55], presence of detects in fabric reduces its price from 45% to 65%. To improve the quality of the fabric and for automatic defect detection, Li et al. [55] proposed a method that relies on multiscale wavelet transform and Gaussian mixture model based on expectation-maximization (EM). Multiscale wavelet transform is used to extract features from images, and Gaussian mixture model’s main idea is to explain the feature’s probability space distribution. Six types of defect segmentation results are discussed. Experiments are conducted to evaluate the results where 12 samples of texture images with 256 grayscales are used from TILDA texture database.
According to Li et al. [56], there are so many types of defects, and common linear defects of warp knitting fabrics are due to broken end of warp yarns which will become significantly large if the warp knitting machine is not stopped immediately. So, to detect defects (warp knitted fabric defects of vertical orientation) automatically, Li et al. [56] proposed an automated vision examination system based on Gabor filters and pulse coupled neural network (PCNN). Li et al. enhanced the image contrast using Gabor filters and they used simplified PCNN for segmentation purpose with some adaptive parameters to reduce the computational complexity. To remove noise, morphological operations are performed and Figure 9 represents the architecture of the automated inspection system. They conducted experiments for the evaluation of system and used two images as test images which have been captured by a smart camera. Comparison has been performed with Gabor and wavelet methods in terms of true-positive rate (TPR), positive predictive value (PPV), and accuracy (Acc). MATLAB R2012B is used for implementation with the following hardware specifications: AMD Athlon 3.01 GHz, 3.25 GB RAM. The results of actual experiments have shown that the proposed automated vision examination system is effective and 98.6% is the detection accuracy.
Peng et al. [57] proposed a quick and efficient defect detection method for the detection of hole, stain, crease, hook wire, broken weft, and other defects by installing a camera to capture a fabric image in a weaving circle system. Their proposed method relies on Blob, Canny, and Rotating Integral algorithm which are used for feature point detection, contour identification, and grayscale integration approach. Results are compared with other techniques used to detect the flaws. The false, missing, and recognition rates of the proposed algorithm are 1%, 1%, and 98% which shows the effectiveness of the proposed approach. The system with Intel Core i5-337um Hynix DDR3L, 8 GB RAM, and Seagate ST750LM022 HN-M7502MBB disk is used to evaluate the research [57].
In manufacturing industries such as fabric or steel, quality control is an important factor for surface defect detection. Huangpeng et al. [58] proposed an unsupervised approach for automatic defect detection of structured or nearly structured textured surface using W-LRR (weighted low-rank reconstruction) method based on texture prior. Texture prior map of image is calculated first and then W-LRR has been used to detect defects. The limitations of the proposed method are as follows: (1) the performance depends on the texture prior map to some extent where the results for prior map are not best and the detection result will be affected and (2) the proposed method takes defect as a foreground. Synthetic and real experiments have been performed for the effectiveness of the proposed method. Synthetic experiments are performed with different patch sizes. Computational accuracy has been measured with the objective to find out the best patch size for further experiment and 10 synthetic defective images have been tested. In experiments, benchmark fabric defect dataset (which contains five types of defects) consisting of 25 defective and 25 nondefective images from each pattern belonging to dot, star, and box is used. Huangpeng et al. performed quantitative comparison for detection accuracy in terms of F-measure, true-positive rate (TPR), accuracy (Acc), false-positive rate (FPR), and positive predictive value (PPV). The result of the proposed method has been compared with that of the five state-of-the-art methods and it shows that it has better detection accuracy and computational efficiency. The experiments are performed on Intel (r) Core (TM) 2.00 GHz and MATLAB is used for implementation. Figure 10 shows the architecture of PNN (probabilistic neural network) model and Figure 11 shows the classification model based on PNN. Table 7 represents a summary about feature fusion-based approaches proposed for fabric defect detection.
10. Sparse-Based Operation
The use of sparse image representation has shown good results in various automation processes [59–61]. Due to this benefit of sparse representation, Zhou et al. [62] proposed a sparse dictionary reconstruction technique to detect defects in fabric. To detect anomalies, support vector data description (SVDD) has been used as SVDD is more versatile in defining decision boundaries. As a first step, the sample of the normal image is subdivided into patches, and then with a defined parameter, the dictionary learning process is completed. In the second step, these patches of defect-free images are used as training, and from these images, two features (correlation coefficient and standard deviation) have been extracted and normalized feature vectors serve as an input to SVDD. In the last step, for testing, new samples have been calculated and labeled with classifier (SVDD). According to Zhou et al. [62], as there is no standard dataset for experiment and benchmark comparison, two basic types of fabric, twill fabric and plain fabric images with 256 gray levels of 512 512 size, have been used. In these experiments, 10 different and common textures types with the aim to use three defect types structural, intensity, and combination of both are used. For evaluation purpose, 12 defect-free and 3 defective images from each type have been used and false detection rate (FDR) and correct detection rate (CDR) have been measured. These experiments are performed on OS Linux and MATLAB R2009b is used for the implementation of this research [62].
For the quality assurance of a textile fabric, auto inspection of defects is mandatory. Lizarraga et al. [63] proposed a rough-set-based textile analysis (RTA) for defect detection in pattern-based textile fabric while using local binary features. In this approach, for feature extraction, local binary features (LBPs) and coordinated clusters representation (CCR) are separately extracted from the same image. For the evaluation of the proposed system, defective and defect-free images are used and the process is performed using two phases: testing phase and training phase. The standard UHK dataset provided by the University of Hong Kong has been used and it has images for the most common defects in textile which are hole (H), thin bar (TnB), broken end (BE), thick bar (TkB), and multiple netting (MN)). For training [63], 500 mixed images are used, and from each pattern, 25 defective images are used for testing. DSR (detection success rate) and average success rate (ASR) for different patterned fabrics are calculated and the result has been compared with the state-of-the-art methods. Analyzing time for one image using WEKA/MATLAB is 60 ms on an ordinary Core i3 Intel processor.
For automatic defect detection, Feng et al. [64] presented a model based on Small-Scale Over-Completed Dictionary (SSOCD) using sparse coding (SC) which is basically hardware accelerated system. From the industrial environment, fabric images are acquired and then training SSOCD of SC of defect-free fabric images takes place (in training, images are preprocessed using Gabor filter and patches are extracted). Then the features are extracted by measuring coding co-coefficients for inspection of defects. For analysis and experiments, TILDA textile dataset of twill fabric has been used and results are compared with other methods. Detection ratio has been used as an evaluation index. For performance evaluation, results have been tested while using different number of cores and on platforms. The result shows that the proposed method can inspect the defects with industrial detection speed and with effective parallel efficiency. Intel Core 2 Duo CPU E7300 is used as hardware with a clock frequency of 2.27 GHz. The proposed method is transplanted on TMS320C6678 with single-core and multicore series. Table 8 represents a summary about sparse feature-based approaches proposed for fabric defect detection.
11. Morphology-Based Approaches
Morphological image processing is a collection of nonlinear operations related to the shape or morphology of features in an image [65]. To detect cord fabric defects from an image, Wang et al. [66] presented a method based on morphological operations and discussed some traditional methods of edge detection. To enhance the effectiveness of the proposed research [66], the original image of cord fabric serum spot defects is used and the results of all kinds of edge detection are discussed (differential, Laplace, Sobel, morphological dilation, morphological erosion, and improved morphological erosion detection methods). Thus, result comparison showed that proposed improved shape erosion method is fast, accurate, and good for preserving edge details and noise elimination effective function.
A novel defect detection technique using morphological filters (MF) has been proposed by Mak et al. [67] for the inspection of woven fabrics. In this proposed technique, pretrained GWN (Gabor wavelet network) has been used to extract features of the textile fabric and then these extracted features have been used in the structuring element which is used further to perform some morphological operations to differentiate the defects and for the removal of background. Morphological operations have been used as it is clear from the previous research study that false alarm rate can be reduced by using MF [67]. Extensive off-line experiments have been performed on 78 images gained from the Manual of Standard Fabric Defects in the Textile Industry for the evaluation of the proposed method which contain many fabric defects and the following parameters for evaluation are measured: true detection (TD), false alarm (FA), and missed detection (MD). Moreover, this (technique) has been further evaluated for real-time analysis on prototyped defect inspection system (which is a low-cost system and has been developed) where twill weaving fabric containing many defects has been used for the experiment and 276 image frames have been used and analyzed. Both frameworks are installed on a Pentium four PC with 2.9 GHz and 1 GB RAM.
According to Jayashree et al. [68], manual and electronic inspection of defects in gray plain fabric fails to find out micronatured defects. So for defect detection in fine gray plain weave fabric, Jayashree et al. [68] proposed a hybrid approach (HA) based on correlation approach (CA) followed by morphological approach (MA). Using this hybrid approach, false alarm rate (FAR) has been reduced. Jayashree et al. [68] performed experiments on GFDD for HA and CA and MA separately to see the effect of this approach and the result has been compared in terms of overall detection accuracy (ODA) using DSA (defect search algorithm). The improvement of HA can be seen from the comparison although this method identifies different defects and has improved results, but the drawback of this HA approach is the selection of the correct defect model and the repetitive selection of element. Table 9 represents a summary about morphology-based approaches proposed for fabric defect detection. Morphological-based operations are suitable for nonperiodic fabric defect type and they are shape-sensitive and best suited for segmentation.
12. Deep Learning-Based Approaches
Deep neural networks are considered as a new framework for modelling biological vision [69]. The range of the vision tasks tackled by the deep nets is expanding rapidly and they do represent a quantum leap as compared to earlier computer vision systems. Deep networks can approximate functions and dynamics by learning from examples [70]. Most of the deep learning models such as convolutional neural networks, recurrent neural networks, and deep belief networks are dominating since the last decade of research [71]. These models have evolutionary paths lasting more than half a century and have a diversity of directions. For example, convolutional neural network (CNN) is evolved from the prior knowledge of biological vision systems [72]. The initial development of neural networks fulfills the thrust to simulate human brains. The maturity of these shallow networks (biological vision system) into deep architectures significantly reduces the requirement of resources while retaining the representation power. The recent focus of research is on the use of deep learning models and they have good results in various fields of computer vision including textile image analysis [73–75].
According to Mei et al. [17], traditional computer vision techniques are not robust to handle the existing requirements of textile industry. Inspired from the recent trends about the use of deep learning in various computer vision-based problems, they applied an automated unsupervised deep network approach to detect and localize the fabric defects without manual intervention.
Wei et al. [76] proposed a modified faster regional-based convolutional neural network (faster RCNN) that is based on the structure of VGG net. The network is modified to enhance the performance while using the fabric defect benchmark. The fabric dataset is expanded to reduce the effect of input data. ReLU activation function is used with pooling and convolution layers to extract the image feature map. Region proposal network (RPN) is used for the generation of bounding box regression and foreground anchors to compute proposals. In the last step, the computed proposals are fed to the ROI pooling layers so that they can be used by the network of softmax layer to classify the images. To reduce the overfitting, vertical and horizontal flips are used to augment the textile dataset. The fabric defects detected are broken pick, felter, drawback, sundries, broken end, and oil stains. The authors have created their own dataset with defects to evaluate the proposed model. The created dataset consists of 06 classes with 135 fabric defects per class. The proposed research is compared with traditional computer vision approaches, and experimental results show the effectiveness of the proposed faster RCNN model. High training cost is one of the drawbacks of the proposed faster RCNN model [76].
Hu et al. [77] proposed unsupervised learning approach based on deep convolutional generative adversarial network (DCGAN) to locate the surface defects for texture. The standard DCGAN is extended by using a new encoder block that can reconstruct query image with no defect and normal texture. The reconstructed images are then subtracted from the image with defect to find the defected regions. The defects are located in the image by using probability of occurrence on the basis of likelihood map. A fusion map is created by using likelihood and residual map. The resultant map creates uniform gray levels for the defect-free regions while deviations over the defected area. The proposed research is evaluated by using a variety of real fabric defect samples.
According to Liu et al. [78], the high computational cost and training time are main drawbacks associated with the use of deep learning-based approaches. These drawbacks limit the applications of deep learning-based models for embedded and handheld devices. To reduce the computational cost, authors [78] proposed a computational efficient CNN-based framework (named as YOLO-LFD) that is using a reduction in feature dimensions. K-means dimension reduction is applied for the computation of ground trues and initial anchor boxes. The proposed research provides a compression of YOLO-v2 model 241 MB to 8 MB with an increase in detection speed with a multiplicative factor of 2.6x. COCO dataset is used in this research to pretrain the deep network, and fine-tuning of network is performed by using a fabric defect dataset that addressed the problem of limited training samples. Fabric image benchmarks with 3000 samples and five classes are used for the evaluation of proposed research and results are compared with SSD, YOLO-v3, and Tiny-Yolo-v3 models using the same environment and parameters. 2000 images were used for training while 1000 were selected for testing purpose. The proposed research has outperformed the state of the art in terms of input parameters and defect detection speed. On the basis of the experimental result, it will be safer to conclude that the proposed research can be used in an embedded device to detect fabric defects in industry [78]. Figure 12 shows an example of classification by VGG-16 model. Wu et al. [79] proposed an automated approach for fabric defect detection that can meet industrial requirement. To achieve the main objective, a stacked feature pyramid is used with deformable convolution filter to sort defect patterns. Approach is evaluated by using fabric defects (FBDFs), and it consists of 2575 optical fabric images covered by 20 categories and 4193 defect instances with a spatial resolution of 2446 1000 pixels.
Wei et al. [80] proposed a method for defect classification using CNN with compressing sensing based on small sample sizes. According to Wei et al. [80], CNN yet not achieved good classification results on small sample sizes; that is why the compressive sensing approach is being used with CNN. Fabric defect image (FDI 500) dataset is used for the experiment and the images are acquired from industrial monitors of textile factory [80]. The sample images contain normal, slub, oil stain, broken end, felter, double flat, and mispick images. For the implementation of method and for evaluation purpose, the operating system Ubuntu 14.04 with TensorFlow 0.8.0, 128 GB RAM, and 4 NVIDIA GeForce 1080 GPUs is used. Classification accuracy is calculated and has been compared with many other models. The reported classification accuracy of the proposed model is 97.9%.
Jeyaraj et al. [81] proposed an innovative method for the detection of defects based on advance deep learning approach. ResNet512-based CNN is used to learn the features from images, and according to Jeyaraj et al., the proposed method also localizes the minute defects. For experiments, i7 with 8 GHz (frequency) 1 TB disk capacity and NVIDIA GPU is used. TILDA textile dataset is used for experiments, and many quantitative values are computed. The classification accuracy is used to compute and to validate the accuracy of the method and results have been compared with some other classifiers also as SVM and Bayesian [81]. Table 10 represents a summary about deep learning-based approaches proposed for fabric defect detection.
For minor defect detection, Wen et al. [82] presented a CNN-based method patch extractor triple matric (PETM) which is self-similar estimation algorithm, and to preprocess it, most likely defective patches are extracted from the original image. The minor fabric defect images (MFDI) that contain 100 normal and 100 minor defective images dataset is built. The size of patched Np = 10, and n = 4 with 40 × 40 is used. From 100 defective fabric images, 901 normal patches and 99 defective patches are extracted. The true-positive rate (TPR), true-negative rate (TNR), and accuracy (Acc) are used as evaluation parameters, and accuracy has been compared with some other methods. The reported accuracy for each element of the proposed method is 79.32%.
Jin et al. [83] proposed Mobile-Unet method based on CNN for fabric defect detection. The proposed method is efficient as it achieves pixel-level classification of defects and best suited for online FDD. To improve the detection process and accuracy, median frequency balancing loss function is used. Intel Core i7-7700K 4.2 GHz, with 128 GB RAM, NVIDIA GeForce 1080Ti, and PyTorch deep learning framework on Windows 10 OS is used as the platform for this research. The benchmarks named as Fabric Images (FI) and Yarn-dyed Fabric Images (YFI) are used to train and test the proposed method. Recall, precision, F1-measure, and intersection over union (IoU) are used as evaluation matrices.
Learning-based approaches require intensive computation for feature vectors having large size. They serve as black box character, with abundance of feature, and there is difficulty in copying and they are not suited for the detection of small defects.
13. Performance Evaluation Criteria
For fabric defect detection, the performance can be evaluated in many ways as there is no standard way or method to measure the defect detection performance. Commonly, there are two such ways that are being mostly used which are (a) sensitivity and specificity and (b) detection success rate. Some other metrics used are false alarm rate, detection rate, and positive predictive value. Many authors used these terms/metrics to evaluate their performance, but the selection of the “evaluation criteria” mainly depends on the nature of the approach, application domain, and environment and on other factors. Before going into depth of these criteria, we must know some basic terms that are used to measure these terms such as true positive (TP), true negative (TN), false positive (FP), and false negative (FN) that are discussed below and are shown in Figure 13.(1)TP: defective images which are detected or identified as defective(2)TN: defect-free images which are detected or identified as defect-free(3)FP: defect-free images which are detected or identified as defective(4)FN: defective images which are detected or identified as defect-free
Thus, Sensitivity: it is defined as the correct identification of defective samples and represented as follows: It is also known as recall. Specificity: it is defined as correct detection of nondefective samples and expressed as follows: Detection success rate (DSR): it can be calculated as follows: It is known as detection accuracy (DA) and can be expressed as follows: False alarm rate (FAR): it can be calculated as Detection rate (DR): it can be calculated as follows: Positive predictive value (PPV): it can be calculated as It is also known as precision.
Li et al. [22] used accuracy, precision, and recall and Zhang et al. [26] used detection rate. Hanbay et al. [42] used five different indexes for quantitative evaluation, i.e., specificity, sensitivity, accuracy, negative predictive value, and positive predictive value. Hamdi et al. [40] used precision, recall, and detection success rates. Hu et al. [44] used false-positive rate (FPR), true-positive rate (TPR), and accuracy (Acc). Zhang et al. [49] used percentage detection rate. Accuracy (Acc), F-measure, true-positive rate, false-positive rate, and positive predictive value have been used to quantify the detection accuracy [58]. Hu et al. [77] used receiver operating characteristic (ROC) graphs analysis based on the true-positive rate (TPR, detection rate), the false-negative rate (FNR, missed detection), the false-positive rate (FPR, false alarm rate), the positive predictive value (PPV, precision), and the accuracy (Acc, comparison of the time, detection performance).
The imbalanced detection is determined by average recognition precision and variance fluctuation between different categories [79]. Precision for a single class predicts a model’s ability to classify only the relevant objects. However, recall gives a measure of the ability of the model to identify all of the relevant cases. The imbalance between inter- and intraclass fabric defects results in different values of average precision (AP) among categories. Firstly, mean AP (mAP) is determined for overall performance evaluation of detectors which refers to the mean of AP of all categories. Secondly, since the data distribution varies significantly among different classes termed as interclass imbalance, ROC curve is considered a better choice as compared to the PR curve as it takes into account both and examples. To compute the imbalanced detection between different categories, variance precision (VP) is used as shown in equation (8), which illustrates the interclass accuracy stability:
Thirdly, the intraclass imbalance is computed that relies on size variance and the AP for large, medium, and small objects is divided by cross scale and .
14. Limitations and Future Work
Traditionally, the quality of textile fabrics is determined by human-oriented analysis of textile fabric defects. However, the manual inspection leads to lower productivity and higher market losses. In this study, automated fabric defect detection methods are discussed in eleven groups. Since there are many different types of fabrics and defects, a single method that can run on all fabric types and contain these defects has not been found. Each method has its strengths and weaknesses that are discussed in subsequent sections. In the vast majority of the methods examined, the authors have created their own database under different lighting conditions. Some studies have used the TILDA fabric database, but it is not easily accessible for all researchers as it is paid. Anonymously accessible fabric databases are needed to develop objective and reliable methods. Since a large number of fabric defect detection methods exist in the literature, their comparison is helpful for researchers to find the optimal method depending on fabric type and defect. However, it should be considered that the studies are conducted using different databases, different parameters, and varied imaging systems, hence making the validity and reliability of methods far from objectivity. This review article covers the state-of-the-art research on textile fabric defect detection methods, and it is thought that comprehensive studies similar to this one will contribute significantly to the textile industry. For future research, there is a critical need for some common publicly available fabric image databases that will open up new directions for the researchers in textile industry. Hybrid approaches with a combination of deep features with other visual features are also considered as a promising direction for future research.
15. Conclusion
In this review article, we have presented an overview about fabric defect detection for textile industry. According to the literature, the process based on manual human inspection is not sufficient and effective to meet the current need of textile industry. To enhance the quality and to decrease the production cost of the final textile product, it is the current requirement that the inspection process must be done by using some industrial automation. Computer vision and digital image processing can provide a base that can cover this industrial gap. We have discussed the techniques based on eleven different subgroups with performance criteria for fabric defect detection. The basic theme of each approach is discussed with advantages and disadvantages while the tabular representation is used to summarize the research. The pictorial representation is used in the review article to provide an overview of the popular research models for fabric defect detection. The image benchmarks used in each research model are discussed in detail.
According to this review, significant research has been reported to detect by using texture, frequency domain, GLCM, feature fusion, sparse feature representation, image morphology, and deep learning-based approaches. For any defect model, the main requirement is to sort the defective region of textile fabric. The current trends are shifted to the use of deep learning models that require high computational cost for training and lot of training data. The use of feature fusion such as color and texture with deep learning can be a possible future direction. Real-time defect detection is still an open research area in this domain, and only few research models are reported for real-time applications. In future, defect detection using small handheld devices with less computational cost will attract textile industry. According to this review, the research models based on fabric defect detection are evaluated by using different image benchmarks that are created by the users according to their requirements. One of the limitations of existing research is the number of publicly available datasets except TILDA textile dataset. It will be good if some common image benchmarks will be created and shared at pubic repertoires so that researchers can use them as standard for further research. This will provide an opportunity in a way that researchers will be able to compare their research to sort out the best approach. All of these are the possible future research directions for automated fabric defect detection.
Data Availability
All the details about data availability is mentioned within this manuscript.
Conflicts of Interest
The authors declare no conflicts of interest.