Abstract
With the rapid development of China’s printed circuit board industry, bare-board defect detection has high research and application values as an important factor in improving production quality. In this paper, a new detection method based on YOLOv5 is proposed to solve the balance problem of efficiency and performance in the task of circuit board defect detection. First, the -means++ method is used to improve the location matching of the prior anchor boxes. Second, the Focal-EIOU loss function is used instead of GIOU to address the former’s degeneration issue. Finally, the ECA-Net module is added to enhance the sensitivity of the model to channel features. Experiments were conducted on a public defect dataset, and superior performance was achieved. The proposed method achieves 99.1% mean average precision at 86 frames per second. Compared with other advanced methods, the proposed method achieves the highest mean average precision value, and the detection speed allows real-time applications.
1. Introduction
With more than 100 years of history, printed circuit boards (PCBs) are one of the key concepts in electronics, as they form the substrate for the electrical connections between the different electronic components attached. The main advantage of circuit boards is that they reduce the probability of wiring and assembly errors and their manufacturing can be highly automated. With the advancing development of semiconductor design and manufacturing technology, PCBs are becoming ultrathin, high-density, and multilayer to accommodate high-performance components. The quality of a PCB depends on the quality of each line and hole, and problems in any one of the thousands of lines and holes on a board can affect the quality of the final product.
In the early days, manual inspection of PCB defects was performed. The operator used a magnifying glass or a calibrated microscope to determine whether the PCB was qualified. The disadvantages of this method are high cost, discontinuous defect discovery, and difficulty in data collection. With the increasing production of PCB boards, this method is no longer economical or practical.
In recent years, much research has been conducted on detecting PCB defects. With the development of machine vision technology, automated visual PCB defect detection has been gradually accepted by the industry and is being widely used. Different vision-based defect detection methods have been studied and introduced in the literature for automatic PCB inspection [1–3]. In recent years, an automated optical inspection (AOI) technique has been developed to test and check PCBs for potential defects [4, 5], such as surface, dimension defects, or component placement defects. As PCBs are becoming more and more complicated, manual checks for defects are becoming difficult and time-consuming, so AOI plays an important role in verifying the quality of the printed circuit board after assembly and significantly affects the line’s production quality and efficiency.
On the other hand, with the development of deep learning, many researchers have proved that deep learning technology can be used to detect PCB defects with higher accuracy, which results in higher productivity [6, 7].
In this paper, an improved YOLOv5 network is proposed, which ensures that the model size can meet the requirements of embedded deployment with an improved ability to detect defects in real time. The main contributions are summarized as follows: (1)The -means++ clustering method is used to obtain suitable bounding boxes on the training dataset(2)The existing loss function Generalized Intersection over Union (GIOU) in YOLOv5 is replaced by the Focal-EIOU loss, which accelerates convergence faster and solves the sample imbalance problem in regression(3)An efficient channel attention mechanism is embedded to improve the feature extraction ability
The remainder of this article is organized as follows: in Section 2, work related to PCB defect detection is introduced. In Section 3, the focus shifts to the description of the YOLOv5 model, and then, the details of the proposed improved framework are introduced. In Section 4, the experiments and analyses of the results are presented, while the conclusions are summarized in Section 5.
2. Related Work
Current leakage and open circuits are caused by defects like shorts, spurs, spurious copper, missing holes, and mouse bites, which can lead to degraded performance or even render the PCB inoperable. In recent years, researchers have used many different techniques, including image processing, machine learning, and deep learning, to streamline and automate the detection of defects in bare PCBs.
In 2018, Bonello et al. [8] presented an image subtraction technique to detect defects in bare PCBs. Tsai and Huang [9] presented an automatic defect detection method based on template matching and used a global Fourier image reconstruction method to detect and localize minor defects in images with nonperiodic patterns. However, these detection methods are limited to specific types of PCB defects. Wavelet-based methods for defect detection have also been implemented. Ibrahim and Rahman Al-Attas [10] proposed an image difference method based on the wavelet transform to locate defective areas. The method uses the Haar wavelet and maps the defective areas found in a coarse difference image to the fine resolution of the tested image. Its major limitation is its poor generalization ability.
Classical machine learning methods have been employed in several studies to classify defect types. For example, Zhang et al. [11] used an optimized support vector machine classifier to recognize and classify defects using gray-scale histograms and geometric features. Hassanin et al. [12] presented an approach for automatic defect detection from PCBs based on sped-up robust feature extraction and morphological operations. This technique is used for two purposes: registration of the PCB to be checked on a reference PCB and detection of feature points of missing components. These methods require extensive feature engineering to process the raw data and identify meaningful features for the model. Annaby et al. [13] proposed a modified low-complexity scheme, which converts 2D images in the matching process into 1D feature descriptions augmented with a spatial statistical metric and then creates the feature description using the discrete cosine transform. The key to the success of detection methods based on feature matching is to select appropriate feature information; however, many problems in extracting image features occur when the target images are fuzzy or the background is complex.
In traditional techniques, the initial classification is performed by an AOI machine and then a quality inspection engineer verifies the classification. Despite the adoption of specialized AOI machines for quality inspection, there are still some defects that cannot be detected using these traditional methods. With the rapid development of computers and information technology, deep neural networks have become the mainstream image classification framework. Considering the imbalanced class distribution as well as the costs of real and pseudo defects in the PCB industry, some scholars have used advanced feature preprocessing methods to improve the classification accuracy of deep learning methods by removing irrelevant and/or redundant features [14–16]. On the other hand, Hu and Wang [17] constructed a deep learning network based on Faster R-CNN, using ResNet50 with a Feature Pyramid Network as the backbone, which can better detect minor PCB defects. Kim et al. [18] proposed an advanced PCB inspection method using skip-connected convolutional autoencoders. The deep autoencoder model can decode the original nondefective images and compare them with the input images to identify the defect location. Bhattacharya and Cloutier [19] proposed an end-to-end framework for PCB manufacturing defect detection. The framework uses a combination of contemporary deep learning techniques, such as transformers, multilevel feature fusion, data augmentation, and object detection.
The YOLO series models are typical representatives of single-stage models in deep learning. They are accurate, fast, and suitable for industrial applications. In PCB defect detection applications [20, 21], many researchers have used YOLO series methods with different common backbones and achieved excellent accuracy. For example, Mamidi et al. [22] proposed a fast PCB defect detection network. The network uses an improved MobileNetV2 model as its backbone and also introduces the spatial pyramid pooling structure to increase the image receptive field. Adibhatla et al. [23] introduced the most advanced YOLOv5 into PCB defect detection applications and carried out comparative experiments with other YOLO series methods to prove its effectiveness. However, the defects on the PCB surface are irregular in shape, random in location, and different in size; moreover, there are often large numbers of targets with smaller scales. Under these circumstances, direct application of these methods for PCB defect detection does not lead to satisfactory results.
3. Proposed Method
3.1. Improved YOLOv5s Network Framework
The YOLOv5 method adopted YOLOv4’s CSPDarkNet53 as the backbone feature extraction network and added some new features [24, 25]. The superior flexibility of YOLOv5 makes it convenient for edge computing. Depending on the numbers of feature extraction modules and convolution kernels at specific locations of the network, the YOLOv5 family comprises four network models, namely, YOLOv5s, YOLOv5m, YOLOv5l, and YOLOv5x. The corresponding numbers of network parameters are , , , and , and the model sizes are 16.5 Mb, 49 Mb, 109.1 Mb, and 205.7 Mb, respectively [26]. As the number of network model parameters increases, the complexity of the basic network model increases, and the accuracy of the model is improved, but more computational and hardware resources are required, and inference speed decreases.
The basic YOLOv5 framework can be divided into four parts: input, backbone, neck, and prediction. Mosaic data enhancement, adaptive anchor box operation, and picture scaling are used to process the input dataset. The backbone mainly comprises CBS, C3, and SPPF, which increase the local receptive field range, improve the network speed by slicing the input image, and enhance the learning ability and ensure the accuracy of the network while reducing the number of operations. Compared with the backbone, the components of the neck are relatively simple and basically consist of CBS, upsample, Concat, and C3. The FPN+PAN structure is used in the neck to integrate different levels of features. FPN layers convey semantic features from top to bottom, and PAN uses up- and downsampling operations to combine the low- and high-level features. The prediction part includes bounding box loss and nonmaximum inhibition. YOLOv5 uses GIOU_Loss [27] as its loss function. Compared with IOU_Loss [28], GIOU_Loss solves the problem of nonoverlapping bounding boxes.
In this paper, an efficient channel attention (ECA-Net) [29] mechanism and the Focal-EIOU loss [30] function are introduced to improve the PCB defect detection performance. Moreover, the -means++ clustering [31] algorithm is adopted to optimize the selection of the initial clustering centre to obtain suitable anchor boxes. The improved YOLOv5s network structure is shown in Figure 1.
3.2. -Means++ Optimization
YOLOv5 uses -means [32] to cluster the anchor boxes, but this method has certain limitations. -means is a heuristic method, i.e., it cannot guarantee convergence to a global optimum, as the selection of the initial centre will directly impact the clustering results. -means randomly selects sample points as clustering centres, which can easily lead to local convergence or require an increased number of iterations before convergence is achieved. On the other hand, the initial a priori anchor frame parameters of YOLOv5 are obtained using -means clustering on the universal target detection datasets. However, PCB defect types only have six categories, and the size of the a priori frame needs to be redesigned.
Therefore, the -means++ method is used to optimize the selection of the initial clustering centres and obtain suitable anchor boxes for PCB defect detection. -means++ converts the selection of initial points into a probability problem, which facilitates the estimation of better initial clustering centres and accelerates the method’s convergence. The specific steps of -means++ are as follows: (a)A central point is chosen randomly from dataset (b)For each data point, the Euclidean square distance from the nearest previously chosen centroid is calculated(c)The probability of each sampling point selected as a new cluster centre is calculated separately using the following equation: (d)Steps b and c are repeated until centroids have been sampled(e)The classical -means algorithm is applied until there are no more clustering centre changes
3.3. Boundary Loss Function
The loss function measures the distance between the predicted and the expected output of the neural network [33, 34]. The closer the predicted output is to the expected one, the smaller the value of the loss function. YOLOv5 uses the GIOU loss function to calculate the box loss. However, when the height and width of the two prediction boxes are the same and on the same level, the GIOU will degenerate into an IOU, which will not reflect the coincidence between the two boxes accurately. In addition, the convergence speed of GIOU is slow, and the regression is not accurate enough. In this paper, the Focal-EIOU loss function was used instead of GIOU.
Focal-EIOU measures the discrepancies of three geometric factors in bounding box regression. Moreover, focal loss for regression is designed to solve the problem that the loss value fluctuates significantly due to low sample quality. The Focal-EIOU loss function is shown in where is a parameter that controls the degree of outlier inhibition, is the IOU loss, is the distance loss, and is the aspect loss. and indicate the width and height of the minimum bounding box covering the predicted and ground-truth boxes.
3.4. Attention Mechanism Network
PCB defect detection through deep learning requires effective feature extraction. However, extracting and identifying features using generic methods is difficult because the defects of bare PCBs are too small relative to the whole image. When high-resolution images are used, the detection speed of convolutional networks is slow. Hence, it is challenging to meet the real-time detection requirement of practical applications. The attention module can learn where the feature information that needs to be focused on is in the feature space and can guide the network to extract defect features [35]. For example, in the SE-NET mechanism attention module [36], the input features go through the global mean pooling channel by channel, then through two fully connected layers, and finally through a sigmoid nonlinearity to generate the weights of each channel. These two fully connected layers capture nonlinear interactions across channels, which reduces dimensionality effectively. Although this strategy has achieved good application results, dimensionality reduction inevitably causes side effects and is not conducive to capturing the dependencies between channels.
In this work, an efficient channel attention mechanism network, namely, ECA-Net, is introduced to improve cross-channel attention. ECA-Net is a convolutional neural network with a novel channel attention mechanism that improves the channel attention module of SE-NET and follows a local cross-channel interaction strategy without dimensionality reduction, which can be implemented effectively using one-dimensional convolution. Further, ECA-Net proposes a method to adaptively select the size of the 1D convolution kernels to determine the coverage of local cross-channel interactions, thereby reducing model complexity while improving performance. Figure 2 shows that the ECA module performs global average pooling [37] to obtain features without dimensionality reduction. Then, the ECA module uses fast 1D convolution to capture local cross-channel interactions, and the 1D convolution parameter is generated according to the size of the input channel. The sigmoid function generates the weight ratio of each channel. Finally, all feature channels are combined using an element-wise product. As a result, the effective features are enhanced and the ineffective features are suppressed.
The morphological features are contained in feature maps of different scales, which are extracted by the backbone network. For small targets, the foreground information in the feature map is sparser. To focus on useful defect targets, the ECA module was embedded into the backbone network, connected in series to the C3 module, and named as C3-ECA. Then, the C3 module of the original YOLOv5 backbone was replaced.
4. Experiments and Analysis
4.1. Dataset and Environment
In this study, the popular public PCB defect dataset proposed by Ding et al. [38] was used. Examples of the dataset are shown in Figure 3. The PCB defects in the dataset are classified as missing holes, spurious copper, short circuits, burrs, mouse bites, and open circuits. The PCB dataset distribution statistics are shown in Table 1.
(a) Missing_hole
(b) Mouse_bite
(c) Open_circuit
(d) Short
(e) Spur
(f) Spurious_copper
The resolution of each raw image in the dataset was . The amount of data in the original dataset was relatively small, which easily leads to overfitting during training, so data augmentation techniques were adopted before training. The images were then cropped into subimages to form the training, validation, and test sets with 6720, 2881, and 1067 images, respectively. The initial value of the learning rate is 0.01 and was reduced using the cosine annealing strategy. The batch size was set to 16, and a total of 300 epochs were used for training. In addition, image-level data augmentation methods were also used, such as the augment HSV, random affine, mosaic, image scale, and other transforms. Our experiments were performed on a Linux 4.15.0-142-generic Ubuntu 18.04 with Intel quad-core i7-7700HQ, 2.80 GHz CPU, 16 GB memory, and NVIDIA GeForce GTX1080Ti (11 GB).
4.2. Evaluation Method
The evaluations were carried out to verify the effectiveness of the proposed model. The main indicators are selected: precision (), recall (), average precision (), and mean average precision (). The corresponding equations are as follows: where represents the number of defects correctly recognized by the detection model, represents the number of incorrect or unrecognized defects, expresses the number of mistakenly detected targets, is the precision, and is the recall. The average precision represents the integral of the precision rate to the recall rate. The average of of all categories is the mean average precision , which measures the performance of the entire model.
4.3. Experimental Results
Ablation experiments were carried out to verify the effectiveness of the presented method using three groups of networks. The objective evaluation and comparison results of the test model are shown in Table 2.
The original YOLOv5 defect detection model has a more comprehensive receptive field design range and achieved more general detection results on the PCB defect dataset compared to the other highly performing defect detection models. Improved choices were obtained for the anchor boxes using the -means++ method, which lead to an increase of the mAP value by 0.32 percentage points. To explain the reason for this, YOLOv5 adopts the -means method to cluster the anchor boxes generated using the COCO dataset and adopts a generic method to adjust the anchor boxes during the training process. However, the first thing to be completed before -means clustering is to initialize the cluster centres. At the same time, it is precisely this reason that acts as a shortcoming of the -means clustering algorithm, i.e., that convergence depends heavily on the initialization of the cluster centres. The adoption of the -means++ can alleviate this problem effectively and improves the detection accuracy and effect to some extent. In addition, the effects of difference loss functions such as EIOU [30], SIOU [39], and Focal-EIOU were investigated experimentally, and it was found that the effect of Focal-EIOU was the best, as it increased mAP by 1.36 percentage points. Due to the enhanced contribution of the most probably anchor box in the model’s optimization, Focus-EIOU has stronger robustness to small datasets and noisy boxes and yielded the best mAP results compared with other loss functions such as GIOU, EIOU, and SIOU. The attention mechanism module also improved the performance of the model. Similar to adding an attention module such as SE or ECA in front of the SPPF layer in the backbone, ECA was embedded directly in the C3 module, and the combined module was named C3-ECA. The results of the ablation experiment showed that C3-ECA led to the most obvious improvement of the mAP value, at 2.13 percentage points.
Figure 4 shows that the performance of the model is same across training and validation, which suggests that the model does not overfit the data. It can be seen from the results that the loss function value had a downward trend during the training process. Before the training batch reached 20, the loss function value dropped rapidly, while the precision, recall rate, and average accuracy rapidly improved. On the other hand, the model has a high precision and a high recall, which suggests that the model has low false negatives.
Figure 5 shows the precision, recall, , and curves of the proposed model. It can be seen that the improved model achieved good recognition results for all types of defects. The curves of the corresponding categories for the six defect types in the dataset are depicted. It can be seen from the figure that the average value of the method reaches 99.1%, which shows excellent detection performance.
(a) Precision-confidence curve
(b) Recall-confidence curve
(c) -confidence curve
(d) Precision-recall curve
To further verify the detection effect, the proposed method was compared on the PCB dataset with advanced object detection methods, including Faster R-CNN [40], SSD [41], RetinaNet [42], YOLOv4 [43], and YOLOv5s. Faster R-CNN is a representative two-stage deep learning object detector. SSD draws on the anchor mechanism of Faster R-CNN and the end-to-end one-step structure of the YOLO method, in which object classification and location regression are performed directly in the convolution stage. RetinaNet is a unified network consisting of a backbone CNN network and two task-specific subnets, namely, a classification subnet and a box regression subnet. YOLOv4 is a real-time object detection model published in April 2020 that achieved state-of-the-art performance on the COCO dataset. The comparison results for the accuracy, speed, and model size are shown in Table 3.
It can be seen in Table 3 that, compared with the traditional methods, the proposed model has obvious advantages and achieves a higher FPS rate. First, the -means++ clustering method is used to determine the anchor box size of the PCB defect dataset and improve the accuracy of the model for small object defect positioning. Second, the Focal-EIOU loss function is adopted to replace the original confidence loss and compensate for the imbalance between positive and negative sample classes. Finally, the C3-ECA module is added to the optimized YOLOv5 network to improve the weights of the more important feature channels. Through the above improvements, the optimized model’s mAP increased by 3.78, 14.43, 9.4, 1.47, and 0.38 percentage points compared to Faster R-CNN, SDD, RetinaNet, YOLOv4, and YOLOv5s, respectively, and the detection FPS meets the real-time requirements of engineering applications.
5. Conclusion and Future Work
In the paper, an improved YOLOv5 model is presented for defect detection in PCBs. First, the -means++ method is used to obtain relatively reasonable anchor positions, which improves the subsequent positioning. Second, Focal-EIOU is used to replace the original GIOU loss function of YOLOv5, which improves the network’s performance. Finally, the ECA-Net attention module is embedded in the C3 model of the backbone to extract reasonable features, contributing to the higher efficiency and accuracy of defect detection results. Compared with different state-of-the-art methods, the proposed method achieved the highest mAP of 99.1%. It is worth noting that the network can be extended to other fields, such as fruit ripeness identification and workpiece surface defect detection.
In the future, more defect samples will be collected to expand the training set and fine-tune the network to adapt to the detection of more types of defects.
Data Availability
Data supporting the results of this study may be obtained from the corresponding authors upon reasonable request.
Conflicts of Interest
The authors declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.
Acknowledgments
This work was supported by the National Natural Science Foundation of China (61573101), Natural Science Foundation of Jiangsu Province (BK20201264), Natural Science Foundation of Changzhou College of Information Technology (CXZK202206Q), and Qinglan Project of Jiangsu Province of China.