Abstract

Inertial measurement unit (IMU) (an IMU usually contains three gyroscopes and accelerometers) is the key sensor to construct a self-contained inertial navigation system (INS). IMU manufactured through the Micromechanics Electronics Manufacturing System (MEMS) technology becomes more popular, due to its smaller column, lower cost, and gradually improved accuracy. However, limited by the manufacturing technology, the MEMS IMU raw measurement signals experience complicated noises, which cause the INS navigation solution errors diverge dramatically over time. For addressing this problem, an advanced Neural Architecture Search Recurrent Neural Network (NAS-RNN) was employed in the MEMS gyroscope noise suppressing. NAS-RNN was the recently invented artificial intelligence method for time series problems in data science community. Different from conventional method, NAS-RNN was able to search a more feasible architecture for selected application. In this paper, a popular MEMS IMU STIM300 was employed in the testing experiment, and the sampling frequency was 125 Hz. The experiment results showed that the NAS-RNN was effective for MEMS gyroscope denoising; the standard deviation values of denoised three-axis gyroscope measurements decreased by 44.0%, 34.1%, and 39.3%, respectively. Compared with the Long Short-Term Memory Recurrent Neural Network (LSTM-RNN), the NAS-RNN obtained further decreases by 28.6%, 3.7%, and 8.8% in standard deviation (STD) values of the signals. In addition, the attitude errors decreased by 26.5%, 20.8%, and 16.4% while substituting the LSTM-RNN with the NAS-RNN.

1. Introduction

With the booming of the location-based service (LBS), the demand for position, velocity, and time (PVT) information has gained a significant increment [15]. Global Navigation Satellite System (GNSS) receiver has been the indispensable equipment for various vehicles, carriers, and smart devices, for instance, unmanned ground vehicles (UGV), unmanned aerial vehicle (UAV), smartphone, and so on [15]. With a GNSS receiver, these users are able to obtain accurate PVT information under an open-sky environment [6, 7].

According to the GNSS working principle, GNSS is the radio wireless signal-based navigation system. The receivers operate positioning function relying on receiving the wireless signals from the navigation satellites [810]. For GNSS, the navigation satellites in orbit emit the signals to the earth, and the receivers get the signal for obtaining distance between the user and satellite through measuring the transmitting time of the navigation signal. However, while the signal reaches the earth, it is quite weak and easy to be disturbed or interfered [810]. As illustrated in past publications, the following two reasons account for this phenomena: (1) the transmitting signal power is limited by the energy of the navigation satellite, saving the energy and for keeping the satellite life span, it is hard to enlarge the navigation signal strength will consume more energy; (2) for saving the cost of constructing the GNSS, while meeting the demand of covering the earth with fewer satellites, the orbit is far away from the earth and the signal transmits a long distance before reaching the earth [811]. Due to the above drawbacks, a standalone GNSS is usually unable to output seamless and ubiquitous navigation solutions. Thus, it is of significance for enhancing the performance of the GNSS in signal challenging environments.

In the past decades, researchers are devoted to improving the GNSS receiver performance under weak signals. The first approach is to extend the integration time, which is effective for weak signal tracking [1214]. The second method is to utilize advanced signal tracking architecture, for instance, vector tracking loop (VTL) [15, 16]. However, apart from the weak signals, the receiver cannot receive all the available satellite signals due to the signal blockage. Under this condition, extending the integration time is usually ineffective. New approaches are proposed to address this situation. According to past investigations, integrating GNSS with other available sensors or navigation system for improving the availability of navigation solutions is the most popular solution [1517]. Among them, the inertial navigation system (INS)/GNSS integrated navigation system is the most popular, due to the highly complementary characteristics between GNSS and INS [1821].

The INS is a famous self-contained navigation system, which produces navigation solutions from processing data from inertial measurement unit (IMU). Commonly, an IMU is composed of three orthogonal gyroscopes and accelerometers [1821]. Gyroscopes measure angular velocity; the accelerometers sense the specific force. Measurements from the IMU are processed together for outputting navigation solution (position, velocity and attitude, PVA) continuously. Due to the random noise contained in the raw signals, the INS navigation solutions diverge dramatically over time. GNSS has precise navigation solutions, which could be integrated with INS for providing more reliable navigation solutions [1821]. As mentioned above, the INS and GNSS are complementary. Thus, GNSS/INS integration system is always the preferable selection for long-term accurate navigation solutions. While GNSS signal is normal, the GNSS can calibrate the INS and compensate the INS errors. While the signal is abnormal, the INS could provide short-term navigation solutions [1821]. But, the errors diverge over time, which is determined by the noises contained in the IMU measurements. To further improve the performance of the GNSS/SINS integrated navigation system, it is critical to reducing the IMU measurement noise. Especially, for volume and cost-sensitive applications, Micromechanics Electronics Manufacturing System (MEMS) IMU is usually employed to construct the INS. In fact, there are several kinds of IMUs; among them, MEMS IMU is popular due to low cost and small size. MEMS IMU also performs the lowest accuracy. It is of great significance for reducing or suppressing MEMS IMU noises [1821].

Firstly, inspired by the time sequence processing in data science community, Allan variance (AV) was employed to analyze the MEMS IMU error components, and then ARMA models are employed for modeling and representing the noise [2231]. After this, some machine learning methods are also employed in this application, for instance, neural networks and support-vector machine (SVM). With the rapid development of the semiconductor technology and computing capacity, recently, deep learning (DL) gained a boom in data science community [2231]. Artificial intelligence (AI) methods were employed in sequence data processing and obtained great advances while compared with the conventional machine learning methods [2231].

In this paper, inspired by the excellent capability of deep neural networks in sequence data processing, the advanced Neural Architecture Search Recurrent Neural Network (NAS-RNN) was employed and tested for reducing the MEMS gyroscope noise (only the gyroscope was tested using the proposed method, and the accelerometers processing was similar to that of gyroscope). A commercial MEMS IMU STIM300 was employed in the experiments [2231]. The sampling frequency was 125 Hz, and the testing time was approximately 800 seconds.

Specially, Long Short-Term Memory Recurrent Neural Network (LSTM-RNN) and Simple Reduced Unit Recurrent Neural Network (SRU-RNN) were employed in MEMS gyroscope noise suppressing. Both LSTM-RNN and SRU-RNN were popular variants of RNN, and they both performed better than conventional machine learning or regression method [32, 33]. However, it is interesting to explore a more feasible RNN structure in this application. NAS-RNN was able to automatically search for a more appropriate neural architecture for certain application. A popular commercial method was employed in this experiment for testing the NAS-RNN performance, and the LSTM-RNN was selected as the reference for demonstrating the NAS-RNN superiority compared with the LSTM-RNN.

Apart from the introduction section, the rest of the paper is organized as follows: (1) Section 2 introduces the model and the workflow of the employed NAS-RNN model; (2) Section 3 illustrates the NAS implementation, training, results, and the comparison details with the LSTM-RNN; (3) Section 4 concludes the paper and the discussion is added following this section.

2. Methods

In this section, we first introduce the basic architecture and the working flowchart of a popular LSTM-RNN, and then, an RNN generative method using NAS is presented. This section is divided into two subsections: (1) LSTM unit basic structure, relative mathematical equations, and workflow; (2) NAS-RNN mechanics and setting up, especially how NAS-RNN is trained and the logic.

2.1. Long Short-Term Memory Unit

Before introducing the NAS-RNN, Figure 1 gives the basic architecture of the LSTM unit, which is composed of three “gates” including “forget gate,” “input gate,” and “output gate.” Basically, the three different gates have its unique functions, and the details are as following:(1)“Forget gate” is the first gate, and its function is to decide what information will be thrown away from the previous cell state. As illustrated in Figure 1, the gate takes the variable and as the inputs of the function, which will generate values representing the forgetting degree of each number in the cell state. The operation equation is as follows:where is a sigmoid function, is the updating weights, is the bias, is the hidden state, and is the input vector.(2)The next is the “input gate,” which controls what new information should be inputted to the module. The relative equations are as follows:here is a sigmoid function, is the updating weights, is the bias in the input gate, is the hidden state at time , is the updating weights, is the bias, is the input vector, and is the new cell candidate values. This gate has two functions.(3)The last part is the “output gate,” and by its nature, this gate is designed for managing the outputs of the unit. Two functions are employed in this gate, and the details are given in equations (3) and (4).where is the updating weights, is the bias in the output gate, and is the cell state at time .

2.2. Neural Architecture Searching Recurrent Neural Network

The last subsection gives the basic LSTM unit computation in detail. In fact, it requires expert knowledge and takes time to design such basic recurrent unit. Recently, a neural architecture searching is proposed for generating neural networks with reinforcement learning [3437]. Figure 2(a) is the overview of the NAS; the work is based on the observation that the structure and connectivity of a neural network can be typically specified by a variable-length string. It is therefore possible to use a recurrent network, the controller, to generate such string. Training the network specified by the string, the “child network,” on the real data will result in accuracy on validation set. Using this accuracy as the renewal signal, we can compute the policy gradient to update the controller. As a result, in the next iteration, the controller will give higher probabilities to architectures that receive high accuracies.

In fact, the basic LSRN-RNN computations could be extended or illustrated as the tree of steps. Figure 3 shows an example of this tree structure, which is composed of two leaf nodes and one internal node. The leaf nodes are indexed by 0 and 1, and the internal node is indexed by 2. In this architecture, the controller RNN needs to predict 3 blocks first, each block specifying a combination method and activation function for each tree index. After this, it needs to predict the last 2 blocks that specify how to connect and to temporary variables inside the tree. Specifically, according to the predictions of the controller RNN in this example, the following computation steps will occur:(1)The controller predicts Add and tanh for tree index 0, and this means that we need to compute ;(2)The controller predicts ElemMult and ReLU for tree index 1, and this means that we need to compute ;(3)The controller predicts 0 for the second element of the “Cell Index”and Add and ReLU for elements in “Cell Inject,” which means that we need to compute . Notice that we do not have any learnable parameters for the internal nodes of the tree;(4)The controller RNN predicts 1 for the first element for the “Cell Index,” and this means that we should set to the input of the tree at index 1 before the activation, .

3. Experiments and Results

In this section, field tests were carried out for evaluating the performance of the proposed method. Moreover, a common LSTM-RNN was employed and compared with the proposed NAS-RNN in terms of the standard deviation of the gyroscope signals and the attitude errors. In this experiment, a popular MEMS IMU STIM300 was adopted, which was manufactured by Sensors AS Company from Norway. The IMU was presented in Figure 4, the gyroscope’s full measurement range was 400/s, the gyroscope’s bias instability was 0.3°/h, the angle random walk was 0.15°/, and the sampling frequency was 125 Hz [37].

Overall, this section is organized as follows:(1)First subpart presented the setting up of the NAS-RNN, and the results of the NAS-RNN, the training loss, the standard deviation values, and attitude errors were employed as indicators for evaluating the performance of the NAS-RNN denoising(2)In the second subpart, LSTM-RNN was compared with the proposed method for analyzing the difference between them

3.1. NAS-RNN Results

As aforementioned, Section 2 presented the diagram and the architecture of the NAS-RNN in detail. The NAS-RNN was implemented using the Python programming language (version 3.4) and the popular Google open source deep learning library TensorFlow (version 1.12.0) in this experiment. Among the NAS-RNN parameters, the learning rate was set to 0.01 and the training step was set 100. Input date length was 15, which means using the past 15 gyroscope measurements to predict the current gyroscope outputs. After this, the predicted outputs were employed for compensating the random noise contained in the raw signals. Before applying the NAS-RNN to predict noise, a training procedure was carried out for determining appropriate parameters and generating proper neural network architecture in NAS-RNN.

Figures 3, 5, and 6 list the training loss for the three-axis gyroscope outputs. It could be seen that the NAS-RNN training loss converged during the training procedure. In Figures 5(b), 3(b), and 6(b), the red line represents the NAS-RNN denoised gyroscope signals and the green line represents the raw gyroscope signals. Furthermore, Table 1 lists the comparison of standard deviation values (SDV) between NAS-RNN and the raw gyroscope signals. Compared with the raw signals, the SDV of the NAS-RNN results decreased by 44.0%, 34.1%, and 39.3%, respectively. This result demonstrated the effectiveness of the NAS-RNN in this MEMS gyroscope noise suppressing application.

3.2. NAS-RNN vs. LSTM-RNN

For further analyzing the NAS-RNN performance in MEMS gyroscope denoising, and in this subsection, the LSTM-RNN was compared with NAS-RNN for exploring whether there was a more feasible architecture for this application. Table 2 lists the comparison results between raw signals and the NAS-RNN. As listed in Table 2, the SDV decreased by 60.0%, 36.6%, and 44.6% compared with the raw signals. Furthermore, Table 3 lists the comparison between LSTM-RNN and NAS-RNN. Compared with LSTM-RNN, the NAS-RNN realized a further decrease in SDV. More specifically, the three-axis SDV decreased by 28.6%, 3.7%, and 8.8% compared with LSTM-RNN.

In addition, Figures 79 present the training loss and testing loss comparison between LSTM-RNN and NAS-RNN. It could be seen that NAS-RNN converged to smaller values, but NAS-RNN performed at slower convergence speed. Actually, the NAS-RNN was more complicated than LSTM-RNN, and NAS-RNN had a more heavy computation load. More specifications were introduced in Section 2.2. Table 4 lists the comparison of attitude errors; the attitude errors obtained 26.5%, 20.8%, and 16.4% improvement in the three-axis attitude errors compared with LSTM-RNN.

4. Discussion

The motivation of this paper was to explore more feasible RNN for MEMS gyroscope noise modeling and compensation, and we think the following problems need more discussion:(1)In this paper, the training procedure or processing data was limited by the computation capacity, and this might influence the generation capacity of the neural networks.(2)Only a STIM300 was employed in testing the proposed method; it was of significance for evaluating the performance of the NAS-RNN in different MEMS IMUs since different IMUs performed different noise characteristics.

5. Conclusions

In this paper, an advanced variant of RNN was employed in MEMS gyroscope noise modeling and compensating. A commercial MEMS IMU product STIM300 was employed in this experiment for evaluating the performance of the proposed NAS-RNN-based method. Compared with the raw signals, the three-axis NAS-RNN denoised signals performed 60.0%, 36.6%, and 44.6%, decreasing in standard deviation individually. In addition, compared with popular LSTM-RNN, the NAS-RNN obtained 28.6%, 3.7%, and 8.8% further decreasing for the three-axis gyroscope signals. Basically, the NAS-RNN was effective for this application, and it performed better than LSTM-RNN. We hope these results might provide reference for embedding a deep learning module in MEMS gyroscope for improving the accuracy of the MEMS gyroscope.

Data Availability

The source codes used to support the findings of this study are available from the corresponding author upon request.

Conflicts of Interest

The authors declare no conflicts of interest.

Acknowledgments

The authors gratefully acknowledge the financial support from China Scholarship Council (CSC) and Excellent Doctor Training Fund of Nanjing University of Science and Technology (NJUST). This study was supported by the Fundamental Research Funds for the Central Universities (grant no. 30917011105), Jiangsu Planned Projects for Postdoctoral Research Funds (grant no. 1501050B), and China Postdoctoral Science Foundation (grant no. 2015M580434). Also, this study received the special grade of the financial support from the China Postdoctoral Science Foundation (grant no. 2016T90461).