Abstract

This paper proposed a method to realize inclination measurement for the industrial Internet of Things (IIoT) assembly platform by utilizing deep learning technology. Because the existence of inclination angle of platform will introduce slight damages to the products during the assembly process, our research group designed a system to tackle this problem by replacing the motion estimation algorithm with a classification mechanism. In other words, by classifying the image of the installation platform, it is capable to determine which inclination angle (image class) an input image belongs to. From the analysis of evaluation experiments, the proposed system for inclination measurement mechanism is proved effective when tackling the problems mentioned above. In addition, the proposed IoT system costs less computation power and achieves relatively higher measurement accuracy.

1. Introduction

The industrial group, which is in charge of assembly transmission module for heavy trucks, mainly produces various parts as well as completes the assembly work of transmissions. The transmission consists of an aluminum casing, many gears, and bearings. Nearly, a hundred of different parts, as described, are installed individually in the metal casing. Nowadays, factories mainly employ the automated robot arms to work for the assembly of truck transmissions. This enhancement not only benefits the efficiency of assembly work but also cuts the burdens of engineers and improves the quality of products.

Although the quality of products has been greatly improved, some problems still exist in the manufacturing process. One of the most significant problems is a small number of products were found to be excessively worn. Through analysis, engineers finally found that the existence of inclination angle between assembly platform and the ground contributes to the problem. Therefore, it is essential to measure the inclination angle before adjusting the pose of the assembly platform. Our research group proposed a method to tackle the problem of inclination measurement, as mentioned above. Furthermore, a few contributions of this research will be listed as follows:(1)On the basis of the original industrial field dataset, perform data enhancement and establish a new dataset to meet the training dataset requirements of the neural network model and the machine learning classifier model.(2)Propose a mechanism based on feature clustering. Experiments have verified that compared with the traditional feature matching mechanism, the calculation of feature vectors for classification through the feature clustering mechanism can reduce the amount of calculation by 45.3%. In other words, it is able to improve time efficiency of inclination measurement and helps to save computing resources.(3)Based on the obtained K value, a shallow neural network structure is proposed. Through analyzing the experimental results, it revealed that the classification accuracy of the neural network structure exceeds other machine learning classifiers, including random forest classifier. And the classification accuracy rate reaches 95%, which is indeed a pretty good performance.

The organization of this study is as follows. Section 2 presents some of the related works that mainly adopt the traditional methods and technologies. Section 3 gives out an overview of the workflow about the proposed method. Sections 4 and 5 explain the feature clustering method, which helps to reduce computation complexity of the entire system. The proposed neural network structure is demonstrated in Section 6. Section 7 introduces the evaluation experiments, which is followed by conclusions in Section 8.

Wang developed a system to solve the inclination measurement [1, 2]. In their work, ORB algorithm is utilized because of its excellent performance and good time efficiency [3, 4]. In addition, an inclination measurement PnP algorithm based on trust domain optimization is proposed to measure inclination [5]. However, due to the dependence of the traditional feature matching mechanism, the system has higher requirements for computing power. For this reason, we proposed a feature clustering-based mechanism in this research to further reduce the computational complexity of system. On the contrary, inclination measurement is achieved through the use of artificial neural networks and other machine learning classifiers.

3. Overview of the Proposal

The traditional feature matching method has the advantages of excellent performance, stable matching results, and high reliability of the algorithm, but this method requires a lot of computing resources. In order to further diminish the computing cost of the inclination measurement system, our group developed feature clustering-based method, which shrinks the computational complexity of the whole algorithm without affecting the classification accuracy. This is mainly because the dimensionality of image features has been compressed. After that, according to the number of optimized clustering centers obtained, our group proposed a shallow neural networks structure and employed the feature vector which is designed by us to train the neural networks. Finally, the trained model and testing dataset are utilized to verify the effectiveness of the inclination measurement system. The entire workflow is shown in Figure 1.

4. Image Feature Algorithm

Nowadays, scale-invariant feature transform (SIFT) has been widely employed in the research field of computer vision [68]. On the basis of the comparison experiments about the robustness of image feature, our group finally decided to adopt SIFT to extract image features. This is mainly because the strict requirements of algorithms applied in the project of inclination measurement.

In contrast of the traditional feature matching approach, it requires relatively high computing power. Therefore, our group intends to propose a solution for the platforms with constrained computing resources. Experiments show that proposed method effectively reduces the computational burden of the system, while maintaining good measurement accuracy and stability. The workflow of our solution is depicted in Figure 2.

5. Feature Clustering and Representation

K-means algorithm [9] is a widely used solution to deal with the clustering of data. Due to its good performance and other features which are easy to implement, our group finally decided to employ it. Also, we adjusted the original algorithm to make it perform better when dealing with the task of inclination measurement. About the termination condition to stop clustering, whenever mean square errors (MSE) are becoming minimum, the algorithm will stop. With tremendous times of testing, our group achieved the best clustering performance under the circumstance that K equals to 70.

With the help of clustering algorithm, the computation burden of feature representation regarding our dataset decreases greatly, which benefits for our systems with constrained computing resources. About representation of image features, our group utilizes feature vectors obtained from clustering to represent the original image data. Specifically, through generating frequency histograms of image features, our group realized feature representation of image data.

6. Proposed Classification Algorithms

In order to verify the classification performance of the proposed methods, our group implemented a few classic machine learning algorithms as the reference substances, which involves support vector machine (SVM) [10], K-nearest neighbor (KNN) [11], Naive Bayes (NB) [12], and random forest (RF) [13].

In the research division of machine learning, artificial neural networks (NN) [14, 15] are realized through connecting a tremendous amount of artificial neurons. Theoretically speaking, NN is a kind of adaptive system; thus, flexibly adjustment of its internal structure makes it appropriate to tackle the classification task of inclination measurement. In this work, considering the restricted resources as well as the characteristics of image dataset, our group constructed shallow neural networks to solve the classification task of feature vectors. The basic structure of NN is depicted in Table 1.

7. Implementation and Evaluation

Here, we would like to demonstrate the realization of the proposed method. Furthermore, evaluation experiments and comparison works will also be illustrated in this section. Specifically, the prototype work is implemented by using the IDE named “Anaconda” with Python programming language and libraries including Tensorflow and Keras. The neural networks are constructed by utilizing Keras library. Moreover, configuration of the hardware employed in evaluation experiments is as follows: CPU (i7-8700), GPU (GTX1080Ti), and memory (64 GB). As for the image dataset, it consists of real industrial scenes captured by camera and augmented images. The image augmentation is made by using OpenCV library.

In the real industrial application, the inclination angle of assembly platform is limited within 10 degrees. Thus, our group supposes the inclination angle varies from 0 degree to 10 degrees, that is, 11 classes of images in total. However, the capacity of the original dataset is limited, which will make it hard to train our neural networks. To overcome the difficulties, we applied data augmentation and built a dataset that contains 1100 images in 11 categories. In detail, each category represents the case that the assembly platform is inclined with tilt angles ranging from 0 degree to 10 degrees. The dataset consists of the training set and the test set. Three-fold validation is applied in the validation experiments. Figure 3 presents an instance of the dataset mentioned above.

As mentioned earlier, the principle task of this research will be measurement of inclination angle using a vision system. Nowadays, several types of vision systems are widely employed, which could be roughly divided in two configurations, that is, eye-to-hand and eye-in-hand configuration. In this work, we adopt the latter system settings. Basically, the eye-to-hand system consists of the robot arm and the camera. Its camera component is placed at a fixed position which can clearly record the assembly process of robot arms. In specific, in this case, the configuration makes it easy to capture the image of the assembly platform at any time.

The evaluation experiments take several aspects into accounts, including time consumption, computational complexity, time cost of the prediction process, and classification accuracy. To verify the ability of reducing computation complexity, a comparative experiment between the traditional mechanism and our proposal has been carried out. The comparison results about computational complexity are summarized in Figure 4.

As for classification performance, our proposed mechanism is superior to other algorithms without introducing additional computational burden, as summarized in Figures 4 and 5. Comparison result on classification accuracy is shown in Figures 5 and 6 which depict a summary of the prediction time among different algorithms.

With a comprehensive consideration about the algorithm performance, we could draw a conclusion that our proposed algorithm which is based on feature clustering combined with the proposed neural network achieves the best overall performance when dealing with inclination measurement task. The advantages involve but not limited to low computational complexity and high performance.

Experimental result shown in Figure 5 proves the effectiveness of our proposed neutral working structure, which is superior to the other algorithms and achieves 100% prediction accuracy under the condition of K = 70.

Before proposing our own algorithms, some prevalent networks have been implemented to deal with the inclination classification task. In the evaluation process, we implemented MobileNetV2 [16], Xception [17], ResNet [18], and LeNet [19] to classify the inclination angle. These popular convolutional neural networks are implemented using TensorFlow and Keras libraries. Furthermore, image dataset which is demonstrated in Section 8 was employed to train and test the model. Three-fold cross validation is also applied to improve the reliability of evaluation results. The classification performance of different networks is summarized in Figures 7 to 12 as well.

From the classification performance of MobileNetV2, as shown in Figure 7, we found that MobileNetV2 failed to converge well within 50 training epochs, since the validation loss gradually increased as the training process continued.

Performance of Xception in regard of inclination classification is summarized in Figure 8. Thus, we can conclude that Xception failed to perform well in the inclination classification task. After training for 39 epochs, the validation loss increased which proved the divergence of the networks. On the contrary, the overall validation accuracy remained under 25% through the training process, which indicates the disability of dealing with inclination classification.

The cases of ResNet are portrayed in Figure 9. It seems that ResNet failed to converge in the training process, and the overall performance keeps at a poor state, which proved the dissatisfaction towards dealing with the industrial task in this work.

Experimental performance regarding LeNet is depicted in Figure 10. It is easy to find that LeNet reached 94.2% about validation accuracy. However, to a certain extent, the validation loss of LeNet increases sharply, which is a disadvantage of LeNet to solve the image classification problem, as shown in Figure 10.

Regarding the neural network proposed in this paper, Figure 11 summarizes its performance. Obviously, the proposed neural network converges well in the training process. Specifically, the proposed method converges within 12 training epochs. For this reason, we depict the training process diagram from 1 to 12 epochs in Figure 12, which shows more details of the first 12 training epochs of the proposed method. In epoch 12, not only the validation accuracy rate reached 100% but also the verification loss also got the minimum value. In short, the proposed neural network is superior to other neural networks. The experimental results prove that the proposed neural network is suitable for solving the problem of inclination angle classification.

As for the dissatisfaction of MobileNetV2, Xception, and ResNet when tackling the classification task of this work, it is mainly because the high complexity of network structure. On the contrary, the input of the convolutional neural networks is images of our dataset. In other words, the features of these images in the dataset are not clustered in advance. Inputs of our proposed neural network are feature vectors obtained after the feature clustering process. In addition, the proposed network structure is associated with the tuned K value, which will also affect the classification performance of the image dataset established in this work. To sum up, the abovementioned reasons lead to a good performance in the classification of inclination angles.

8. Conclusions

To some degree, the inclination measurement of bearing assembly platform remains a challenge. This is mainly because of the relatively high computation cost of the algorithms. Therefore, in our research, we proposed a method to tackle it. Specifically, the idea of image classification is adopted to deal with inclination measurement by classifying the images of assembly platform. First of all, we detect the feature points of an input image. Then, the clustering-based method is applied to construct feature vectors for the image of assembly scenery. Secondly, those feature vectors will be utilized to train the model of neural networks. In this way, inclination measurement is transformed to an image classification task. The output of the neural network gives the exact category to which an input image belongs. An improved clustering algorithm is realized, and the tuned parameter K was finally found after many iterations of testing experiment. In contrast to those traditional methods, the proposed method achieves the best performance while reducing computational complexity by 45.31%. In addition, validation accuracy of the presented neural networks with tuned parameters reaches 95%, which outperforms other base-line models. Evaluation results prove the effectiveness of our method for inclination measurement. As for future works, we will make effort to improve the performance of algorithm. On the contrary, boosting methods such as AdaBoost will also be considered to help enhance the performance of inclination measurement.

Data Availability

The datasets used and/or analyzed during the current study are available from the corresponding author upon reasonable request.

Conflicts of Interest

The authors declare no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.