Abstract
An automatic recognition algorithm based on the feature extraction of working parameters to recognize each state in the loading cycle process of an electric mining shovel was proposed. The working parameters were collected using the electric shovel’s online monitoring system. The swing angle of the shovel boom and motor operating signals were used as key recognition objects; the waveform features of each stage were extracted as recognition marks in the loading cycle of an electric shovel using the time domain characteristic analysis method. The algorithm was developed to recognize the loading cycles in real time. Moreover, the dynamic time warping (DTW) algorithm was used to detect and classify the preliminary recognition results by optimizing its distance threshold parameters, reducing the error rate of the model. The method was validated by comparing synchronous video-recordings with the results of the algorithm. Results showed that the proposed recognition method of the shovel loading cycle process exhibited real-time performance and high accuracy in understanding the different work tasks, providing effective data support for mining and the analysis of shovel working parameters, helping to improve the energy efficiency of electric mining shovel.
1. Introduction
Electric mining shovel is large and complex engineering machine, which can directly perceive the blast-pile excavating difficulties and indirectly evaluate the blast-pile crushing effect during the loading materials in an open-pit mine [1–4]. The real-time recognition of the electric shovel’s loading process is the key to customer who needs a significant improvement in fuel efficiency and cost. The on-board monitoring system provides real-time feedback to the driver. The drivers adjust their loading operation to current working condition to achieve the highest productivity. Real-time identification of shovel working states can be used for construction site supervision and management, resource allocation, and task scheduling.
At present, the methods used to recognize the loading equipment’s operation status include the multisensor device [5, 6], machine vision technology [7], and the device’s own parameters, for example, the displacement of the equipment mechanism [8, 9], the hydraulic oil pump pressure [10], the motor voltage and current [11, 12], and the vibration spectrum of the chassis [13, 14].
In related work on online working excavators, Oloufa et al. [15] tracked the location information of equipment to observe its operation status using GPS. Zhang et al. [16] used a high-precision positioning and direction-finding control system to measure the relative rotation angle of an excavator’s boom and chassis to detect the attitude of the excavator's working devices and recognize its loading state. Arsalan et al. [17], Gong, and Caldas [18] used image processing and computer vision methods to recognize various equipment activity patterns and measure the idle time to improve equipment utilization. However, the above research only focused on the overall excavator operating mode and did not further examine the single operation cycle process. References [19, 20] matched images collected in real time with those of standard operating conditions, proposing a multiclass support-vector-machine method to recognize excavator movements. However, the accuracy of this method can be greatly affected by the camera resolution, lighting conditions, and environmental variables. Hao et al. used an inclination sensor to detect the excavator joint angle, combining it with the moving direction and speed of the boom, to determine whether the excavator had entered the full bucket rotation stage from the excavation stage or returned from the empty bucket stage during the unloading stage [21]. Huang et al. realized the intelligent identification of the operating cycle stage by monitoring the pressure of a hydraulic excavator’s two main pumps and using the pressure bands of each stage of the shovel operation as a distinguishing mark [22]. Branscombe [23] proposed a multiparameter threshold automatic segmentation method to recognize the loading cycle using the minimum, maximum, and amplitude of the motor signal as judgment conditions for each action state of the shovel. However, due to the inconsistency of actions during the electric shovel’s operation, it could be easily affected by the shovel working environment and the skill of the operator, resulting in false recognition.
To successfully apply a recognition method, it must be possible to run the algorithm in real time on an operational electric shovel. The main challenges are to first define which parameters are relevant and then to perform online usage identification that is sufficiently robust in the face of usage disturbances.
This paper integrates the high-precision positioning data and the working data of motor to develop a recognition method for the automatic recognition of shovel operations. It begins by defining a single shovel loading operation and then offers a state analysis vis-à-vis the ideal loading cycle processes. The following sections describe the method used to extract the characteristic parameters of each stage of shovel operations and the results of real-time monitoring of an electric mining shovel. A discussion on using the dynamic time warping (DTW) algorithm to detect and classify results is presented.
2. Model of Electric Shovel Operation
2.1. Electric Shovel Working Parameters’ Collection
To recognize the loading cycles of an electric shovel, one must first understand how the shovel works and what resistive forces may be acting on it. A WK-35 electric shovel was tested in an open-pit metal mine in China, with a nominal dipper capacity of approximately 80 metric tonnes or approximately 35 cubic meters of material volume. The test monitored the primary working mechanism of an electric shovel excavating rock from a muck-pile, before loading it onto a mining dump truck. The load change characteristics of each actuator were coupled to the electrical performance parameters (voltage and current) during digging and hoisting operations. For example, the loading cycle includes the crowd motor, which governs the penetration depth of the bucket in the muck-pile, the hoist motor, which governs the raising of the bucket after the bucket filling process, and the swing motor, which moves the bucket. These signals were accessible from the shovel’s programmable logic controller (PLC) cabinet in the control room below the operator cab. In addition, the action information of key components of the shovel can be obtained by installing a high-precision Global Navigation Satellite System. The information fusion of position data and the motor armature parameters can truly provide feedback for most of the working states of electric shovel. The shovel monitoring parameters are shown in Table 1.
As shown in Figure 1, a real-time online monitoring system can be used to realize data collection, including PLC, industrial computers, data acquisition equipment, and cloud server. The data acquisition and control system is installed on the electric shovel for collecting the motor parameters and GPS data, sorting and sending them to the cloud server. The cloud server parses and stores the received data packets, realizes the real-time display of monitoring data, and supports the historical data query function. To ensure the stability of real-time transmission, the monitoring data are recorded synchronously with a sampling frequency of 1 Hz.
2.2. Electric Shovel Loading Operation
The shovel is designed to move in a three-dimensional space, which in this case is consistent with that shown in Figure 2. For example, the bucket movement of a shovel loading cycle can then be divided into five stages: digging stage (A—B), load swing stage (B—C), dump (C), empty bucket return stage (C—A), and digging preparation (A). In ideal working conditions, the main actions of the digging and filling process include bucket pushing and hoist ropes retraction or a combination of the two. After digging, the bucket moves from the muck-pile to a position above the dump truck via rotation of the rotary shaft and the lifting of the hoist ropes or a combination of the two. And then, the rotating shaft rotates in the opposite direction, so that the upper body can rotate from the dump position to the next digging position.
Each motor’s power can be calculated from its voltage and current. Combined with Figure 3, we can then analyze the variation characteristics of the swing angle, swing power, hoist power, and crowd power at each stage of the shovel loading cycle:(i)Load swing stage (yellow): the swing angle gradually increases relatively to the initial value. The swing power exhibits sine wave fluctuations. Since the swing torque increases at the beginning of the swing, the swing power is positive, accelerating the swing. When the swing is close to the dump position, the swing stops slowly due to the large inertia of the load; the braking torque increases; the swing power is negative and symmetrical. During the swing, the hoist power and the crowd power are relatively constant due to the minimal load changes.(ii)Dump stage (blue): during this process, the swing angle reaches its maximum value relatively to the initial position.(iii)Empty bucket return stage (green): the swing angle gradually decreases, symmetrically with the waveform of the load swing stage. The hoist power is reduced to its minimum value at the bucket lowering stage.(iv)Digging preparation (white): there is no prominent feature of the swing angle during the adjustment process between two work cycles. When preparing to dig, the hoist power quickly changes from negative value to positive value and the crowd power has a negative value.
(a)
(b)
(c)
(d)
It can be seen from Figure 3 that each stage of the work cycle can be distinguished by the swing angle and motor power parameters. A single operating cycle comprises the actions of digging preparation, digging, load swinging, dump, and empty bucket return in a fixed sequence. The typical data features of each stage can be extracted to establish an automatic identification model of the shovel loading cycle operation.
3. Methodology
3.1. Recognition of Loading Cycle Operations
The time series data of a single shovel operation can be denoted by , the subsequence of swing angles can be denoted by , the subsequence of hoist power can be denoted by , the subsequence of crowd power can be denoted by , and then time subsequence can be denoted by . To characterize the essential characteristics of the signs that may occur in digging, load swinging, dump, empty bucket return, and digging preparation stages, parameters such as the minimum, maximum, zero, and gradient can be selected as waveform feature vectors to recognize the corresponding swing angle, hoist power, and crowd power of each stage, each stage of the loading cycle being a process event, which should meet the shortest time requirements of the actual physical process.
If the incremental swing angle is 0 and if the hoist power and crowd power are constant during the digging stage, then the digging time should be longer than the shortest digging time. Formally, there exists an interval (, the parameter being the digging time), whereby a digging event is generated if
A load swing is generated if there exists an interval (, the parameter being the load swing time), where the incremental swing angle is positive, the swing angle is greater than constant value, both the hoist power and the crowd power are within the value threshold, and the swing time is longer than the shortest swing time. A corresponding condition for the load swing can be expressed as follows:
A dump event is generated when the swing angle remains near the maximum value , and the dump time is longer than the shortest dump time. Therefore, there exists an interval (, the parameter being the dump time), such that
The empty bucket return event has similar characteristics with the load swing event, but its direction is opposite to that of the load swing. If the incremental swing angle is negative, the swing angle is greater than the constant value, the hoist power is negative in the end of the interval, and the empty bucket return time is longer than the shortest return time; an empty bucket return event is detected. There exists an interval , the parameter being the empty bucket return time), such that
The digging preparation is a complex event. It can be assumed to have been generated if the hoist power is positive and the crowd power is negative at a discrete time. Formally, there exists an interval , such that
Note that the parameters , , , , , , and will be discussed in Section 4.1.
The working parameters of the shovel are collected in real time in a continuous time series. The end of one shovel operation is followed by the start of the next. To realize the recognition of a loading cycle, the digging preparation characteristic condition is used as a starting (end) point identification mark; the single loading operation characteristic is being used as a segmentation mark. The segment flag is associated with a specific operation stage, only meeting the parameter characteristic conditions of (1a) to (5b) to complete a prescribed action.
For the processing and analysis of the real-time data, we developed a model for the automatic recognition of loading cycles using Python programming language. As shown in Figure 4, the key parameters such as the swing angle, hoist power, and crowd power are input first; the value at the current moment is being used as the initial value. Based on the characteristics of a single loading operation, the feature vector recognition can be carried out using the continuous real-time data stream. If all eigenvalues satisfy the judgment conditions, the loading cycle recognition task is completed.
It is worth noting that the shovel walking and loading cycles are two independent operating modes. When the shovel is walking, the hoist and the crowd motors are dormant, and the data is null in the real-time data stream. Therefore, the raw data is cleaned, integrated, and normalized using big-data preprocessing techniques. The data in a nonshovel operational state are filtered before the shovel loading state is recognized. In addition, since the shovel loading operation of the electric shovel can be affected by subjective operator actions, the results still need to be tested further to improve the recognition accuracy.
3.2. Dynamic Time Warping
The DTW is an algorithm for aligning two time series which are similar, but out of synchronization and generally not of the same length exactly [24]. It aligns two time series through measuring and minimizing the distance between each point of the two series sequences [25]. DTW is applied in speech recognition [26], feature matching [27], and diagnostic monitoring [28] applications to recognize the signals with similar shape features in the time domain. Since then, it has been employed for classification in electric signal [29] and process monitoring [30].
For example, set two time series is an ordered length of real value and is an ordered length of real value. To measure the minimizing distance between the and , an distance bs is constructed, where is the local distance between and . Suppose that two time series are of equal length (I = J); the EU-distance is given by
Suppose that two time series are of different length ; the distance is calculated by a warping path . The warping path is restricted by the continuity, monotonicity, and endpoint constraints [31]. is calculated from matrix which consists of a set of table elements that defines a mapping and alignment between and , k as path length meets , the k element are expressed as , is an index of the time series and meets , and is an index of the time series T and meets .If the above three conditions are met, the distance D(C,T) between time series C(i) and T(j) is found by the warping path W minimizes in equation (7). The symbol dis(...) denotes a suitable element-wise distance measure:
The DTW algorithm solves the optimal solution of a path using dynamic programming methods. The cumulative length of the optimal regular path can be obtained by calculating the minimum cumulative distance of three adjacent elements using recursive algorithm [32], as follows:
The similarity between signals can be judged by calculating the distance or concentration of distribution. The DTW algorithm is a local distance measurement tool. It calculates the regular path of the matrix to determine the distance between a sample sequence and a test sequence [32]. The minimum DTW distance is the optimal similarity.
DTW(I, J) can be used to compare the similarity of the data of two classes and normalize them within the range [0, 1]. During the loading cycle, the swing angle has considerable periodicity, so the DTW distance between the test sequence and the typical class sequence is calculated to determine whether it exceeds the threshold, and the test sequence can be classified.
4. Results
4.1. Finding Loading Cycles
Based on the test sample data, the performance indicators of the loading cycles were measured; the digging time interval was more than 5 s, the swing speed was 0∼7°/s, the swing angle was more than 35°, the bucket unloading time was more than 2 s, each truck was loaded with 4∼5 shovels, and the amplitude of the motor power in each stage was monitored. The values of the constants in this case are listed in Table 2.
The results of online recognition of the loading cycles within a time period based on multiparameters feature fusion are shown in Figure 5, which illustrates 18 shovel loading cycles; mark 13, however, is an incorrect recognition result, although the rest of the results are correct. Based on the correct recognition results, from a comparison of the operation cycle and the overall waveform changes, it can be seen that the loading cycle operation can be divided into long loading cycles (1, 5, 9, and 17) and short loading cycles (2, 3, 4, 6, 7, 8, 10, 11, 12, 14, 15, 16, and 18); the difference between the two categories is that the swing angle increases discontinuously and forms a step-like waveform during the load swing period, reflecting the process of the shovel waiting for the next transport truck to reverse into position actual production.
4.2. Detection
The test sequence is detected by determining whether the DTW distance between the test sequence and the class sequence exceeds the threshold. Because the accuracy of detection results is related to the selection of threshold parameters, if the threshold is set too small, much of the shovel loading waveform may be missed. Moreover, if the threshold is set too large, the waveform recognition may not belong to the shovel loading operation.
The test compares the waveform of the monitoring data to generate statistics of the on-site loading cycles and nonloading operation and calculates the missed recognition and overrecognition rate under different threshold conditions.
As shown in Figure 6, the error rate refers to the error statistics of the shovel loading cycles not fully identified in the field test sample data, which shows the fitting curve of the missing and overrecognition rates’ statistical results; the missing recognition rate decreases with increasing threshold; the overrecognition rate increases with increasing threshold. The analysis shows that the error rate is the lowest when the normalized DTW distance threshold is 0.54.
4.3. Classification
The purpose of classifying the loading cycles is to create labels that assign a class to each test subsequence. The experiment used four datasets to record the loading cycles of the operator loading different rocks. A total of 161 shovel loading cycles were recognized using the algorithm; the DTW algorithm is being used to calculate the DTW distance score, which was calculated twice for comparison; the smallest DTW distance value is being classified as the label to which the sequence belonged. A time series of the swing angles with typical long (short) loading cycles was selected from the test samples. Figure 7 shows the loading model data with long (short) loading cycles.
(a)
(b)
The algorithm found and classified all the loading cycles present in the database correctly, an example of which can be seen in Figure 8, which summarizes the online identification of 22 short loading cycles (blue) and 8 long loading cycles (red). The starting point of each loading cycle is denoted by a circle; a dotted line indicates that the cleaning blast-pile operation remains unrecognized. Three short loading cycles at an interval generate a long loading cycle, suggesting that it takes four loading operations to complete the loading volume of each truck, in a total time of approximately 180 s.
4.4. Evaluation of Cycle Recognition Algorithm
The recognition accuracy of the method was evaluated by examining the video records. The results are shown in Table 3. It may be noted that the goal of developing the loading cycle recognition algorithm was not to recognize all loading cycles but to recognize the cycles corresponding to full face only. The loading time based on algorithmic recognition result is slightly longer than the actually recorded time. One reason is that the positioning time of bucket is short, which is recognized as the starting time of shovel loading by algorithmic program; another reason is the delay error in manual recording.
In order to verify the accuracy of the algorithm, Table 4 lists the four datasets collected in the test for verification purposes. The video recorded 180 shovel loading operations; the algorithm identified 161 of them, with a recognition rate of 89.4%.
5. Conclusions
In this paper, a waveform at the beginning or end of each stage of electric loading cycles was used as a segment mark to realize the division of operation cycle and the identification of each stage by shovel monitoring parameters.
Through the construction of a shovel operation model and field tests, a recognition algorithm based on multiparameter feature fusion could recognize the start (end) times of each shovel operation of an electric shovel in real time and count the number of shovel loading cycles.
The DTW detection and classification algorithm was used; the effective distance threshold parameter was set to 0.54; this value improved the accuracy of the shovel loading operation recognition, accurately dividing the data into long and short loading cycles. The on-site video verification demonstrated that the algorithm was robust under different working conditions, showing that the algorithm's characteristic parameters and thresholds selected were reasonable. It had a recognition accuracy of 89.4%.
The automatic loading cycle recognition method took both real-time requirements and accuracy into account, realizing real-time statistics of operator loading efficiency and loading cycles. It has great significance for mine management personnel to better understand mining and loading production, control loading quality, and standardize operation skills.
Data Availability
The authors provided details regarding where data supporting reported results can be found, including links to publicly archived datasets analyzed and written Python programming code during the study (https://github.com/wangbonan11/Real-Time-Recognition-of-Loading-Cycles-Process).
Conflicts of Interest
The authors declare that they have no conflicts of interest.
Acknowledgments
This research was funded by the Major Scientific Program of Jiangxi Copper Co., Ltd., Grant no. DTYJ 2018001, and the Scientific Program of Baotou Steel Union Co., Ltd., Grant no. 202107.