Advanced Infrastructure Systems Integrating Hardware and Software PlatformsView this Special Issue
Research Article | Open Access
Shen Li, Hailong Zhang, Huachun Tan, Zhiyu Zhong, Zhuxi Jiang, "An Attention-Based Model for Travel Energy Consumption of Electric Vehicle with Traffic Information", Advances in Civil Engineering, vol. 2021, Article ID 5571271, 10 pages, 2021. https://doi.org/10.1155/2021/5571271
An Attention-Based Model for Travel Energy Consumption of Electric Vehicle with Traffic Information
Mileage anxiety is one of the most important factors that affect the driving experience due to the limitation of battery capacity. Robust and accurate prediction of the energy consumption of the journey of the electric vehicle can guide the driver to allocate the power rationally and relieve the anxiety of the mileage. Since vehicle sharing is the biggest application scenario of electric vehicles, it is a critical challenge in share mobility research area. In this paper, a travel energy consumption prediction model of electric vehicles is proposed in order to improve the mobility of shared cars and reduce the anxiety of drivers because they are worried about insufficient power. A recurrent neural network with attention mechanism and deep neural network is used to build the model. To validate the proposed model, a simulation is demonstrated based on both traffic and vehicle information. After the simulation, experimental results show that the proposed model has high prediction accuracy, and we also show through visualization how the model finds high relevant road segments of the road network while dealing with corresponding traffic state input.
In recent years, people have paid sufficient attention to environmental pollution problems and the depletion of oil resources caused by excessive use of fossil fuel. The development of pure electric vehicles to replace the traditional energy vehicles is one of the important ways to solve these problems. The current commercialized application of electric vehicles is car sharing, which has many benefits such as being environmentally friendly, saving energy, and reducing emission . However, the imperfection of the current battery technology and the battery capacity constraint still leads to car sharing user experience mileage anxiety while driving a shared electric car.
There are many existing applications for car sharing system using electric vehicles showing the strong correlation between electric vehicles and car sharing technologies, but with the lack of discussion on range anxiety caused by limitation of the current battery technology and the battery capacity constraint [2, 3]. Neubauer and Wood published a paper to examine the sensitivity of BEV utility to range anxiety and different charging infrastructure scenarios, including variable time schedules, power levels, and locations (home, work, and public installations). The results indicated that the effects of range anxiety can be significant but are reduced with access to additional charging infrastructure . Some state-of-the-art works have made effort to address the aforementioned problem by improving the battery performances during the battery manufacturing production chain. In , a random forest based feature analyses and modelling method is proposed for battery manufacturing, and the coupling effects of the battery features are mined and quantified by using the ML technologies. Further in , a Gaussian process regression model is used to analyze the influence of battery electrode mass load under the underlying correlation of four intermediate production feature variables. Meanwhile, applying advanced battery management technology is the other approach to reduce mileage anxiety. For instance, King et al. used on-demand vehicle access method to alleviate electric vehicle range anxiety, and with the help of battery management strategy, the driving range of EVs get is further improved . In , a multiobjective constrained nonlinear optimization is applied to adjust batteries charging patterns, which effectively improve the energy-saving performance of EV. In , a battery changing management strategy is proposed to cooptimize the energy consumption and battery degradation. In , battery charging process considering internal temperature control is addressed by generalized predictive control (GPC). Energy consumption prediction is a core technology for battery management of EVs; besides the driver, accurate prediction of energy consumption can effectively alleviate the driver’s anxiety and help them concentrate on driving the vehicle. However, the energy consumption of an electric car is affected by many factors, such as traffic status, vehicle status, and weather. For this reason, travel energy consumption prediction has long been considered as a difficult problem.
There has been much research on this issue. Cartenì et al. proved that the performance of electric vehicle has huge impact on car sharing models. The results indicated that the most statically significant attributes for those models are travel cost and travel time . Ralf made a research on the estimation on energy consumption of the navigation route. A rule-based method was used in this work, which was conditioned on historical traffic and speed limit. However, the design of rules and the setting of coefficients needed experts, which should be changed under different situations . De Cauwer et al. constructed energy consumption calculation models based on traffic information, vehicle parameters, road topology, and other information. The models are constructed by multiple linear regression, which was based on vehicle dynamics equation as underlying physical model. The model is complex and requires extensive feature preprocessing of the data . Moreover, there are many similar works on the vehicle’s energy consumption prediction [14, 15]. However, there is a complex and coupling relationship between driving condition and energy consumption of EVs. The tradition method cannot handle the nonlinear regression with spatiotemporal features especially considering traffic information. Since 2006, deep learning has been applied in many fields and has achieved very successful results , such as object recognition [17, 18], machine translation , and speech recognition . Deep learning creates abstract high-level features by combining lower-level features to discover the distributed representation of the data, and in the high-level feature space, we can perform some operations that cannot be done in lower-level space. A deep convolutional neural network based energy consumption prediction method was proposed considering the influence of vehicle speed, tractive effort, and road elevation . Recurrent Neural Networks are adapted to estimate the energy consumption of EVs during the specific route . However, these methods cannot guarantee the prediction accuracy and robustness because they do not take the impact of traffic flow on vehicle operating conditions into account, where the traffic information on the driving route is merely considered in the existing ones at best. In fact, the traffic states of the whole road networks, especially the neighbor roads of the driving route, would make great influence on the energy consumption of EVs. However, the information of the whole road networks is tremendous to capture and distill the useful knowledge for deep neural networks. However, working with such high-dimensional information necessitates a powerful mechanism to steer the model to information important to the task at hand. Thus, an adaptive weighting mechanism should be applied when exacting features of road networks. Fortunately, the attention mechanism has a great promotion effect in sequence learning task and has many successful applications in the field of image caption , NLP  graph transformers , and other areas [26, 27], which is a kind of encoder that imitates the behavior of human attention. However, the attention mechanism based energy consumption prediction method has not been constructed; further, as a result of this, the traffic information of the road networks is still not fully unutilized to further improve the prediction accuracy for energy consumption of EVs. Motivated by that, we proposed an attention based prediction model to improve the robustness and accuracy by extracting traffic state information in different sections of the road network during the forecasting process.
Accordingly, a deep learning method is used to construct an electric vehicle travel energy consumption prediction model based on traffic conditions and vehicle states in this paper, which aims to make a robust and accurate prediction of the vehicle energy consumption during the journey. In the proposed model, we utilize traffic information and vehicle status information, which are two different types of information. Since traffic conditions of the road network are continuously changing during the trip, traffic conditions during the trip as a sequential input were considered. And the recurrent neural network is used, which is suitable for dealing with sequential input, to extract the features of the traffic status of the whole trip. Then, the deep neural network was applied to fuse the vehicle state features with the traffic state features and output the prediction results of energy consumption of the trip. For the traffic feature extraction, the traffic information of the road network is used as the input of the model. The traffic conditions of the road network can help provide information on the traffic state changes during the driving of the vehicle in the task of predicting travel energy consumption. More specifically, the congestion, the morning peak hour, and the evening peak hour are major scenarios that would be tested in the paper. Besides, an attention mechanism was employed to help the model extract the information that has a great influence on the current task from the traffic state of road network. Therefore, we extract the feature of these two kinds of information separately and fuse the extracted feature information in the hidden space. Finally, we use the fused feature to predict the energy consumption of the trip. Furthermore, we show how attention mechanism can help the energy consumption forecasting model extract traffic state information and how its attention degree changes in different sections of the road network during the forecasting process.
In the following sections of the article, first, we described the construction of the model. Then, we conducted a comparative experiment and analyzed the influence of input of different information and the model structure on the accuracy of the model. Finally, we visualize the attentional mechanisms of the model and illustrate the characteristics of attentional mechanisms in extracting traffic characteristics.
The travel energy consumption of a vehicle is mainly determined by the driving cycles of the vehicle, and the operating conditions of the vehicle are affected by traffic conditions and the status of the vehicle. Therefore, in order to accurately predict the energy consumption of electric vehicle travel, we need to use these two kinds of information. In this section, a prediction method of travel energy consumption of electric vehicles based on deep learning model combined with attention mechanism is proposed. Figure 1 gives the framework of the proposed deep learning architecture. The details are given as follows.
RNN is a kind of neural network, which is suitable for dealing with sequential input data. The connection between hidden layers makes RNN have memory of preceding input data . Road network is composed of numerous road segments, and the vehicle travel is a sequential composition of several road segments. We use traffic states of the road segments, in which the car during the travel is the input of RNN. In this paper, we use traffic flow, velocity, density, and the length of road segment as traffic state input. The traffic states input of a travel is represented as .where is the number of road segments of the travel and is the size of input vector. In this paper, is equal to 4 as we described above.
Vehicle travel energy consumption is directly affected by vehicle driving cycles, and vehicle driving cycles are mainly affected by traffic states. For the travel energy consumption prediction task, the future traffic state information corresponding to the traveling route of the vehicle is the key information. However, future traffic states of each road have a complex correlation with the current traffic states of road network. In order to make a reasonable and precise explanation of the total traffic state of vehicle travel, we utilize not only the traffic states of the route, but also the traffic states of road network as input. We represent current time traffic states of road network aswhere is the number of roads of road network and is the traffic state vector of i-th road segment of the road network.
For each trip, the current traffic conditions of some sections of the road network may affect the traffic conditions on the vehicle’s path in the future. However, it is difficult to determine which roads we should pay more attention to. In order to help the model find out high relevant roads to the prediction task, we employ attention mechanism in our model. Attention is implemented as a hidden layer, which computes a categorical distribution (or hierarchy of categorical distributions) to make a soft-selection over source elements . For each road segments in the road network, the attention mechanism produces -dimensional positive weight . Each dimension of , shown as , can be interpreted as the relative importance to give to road segment in blending the ’s together.
The is computed by an attention model for which we use a multilayer perceptron (MLP) conditioned on and previous hidden states .
Once weights are computed, the traffic state representation feature can be computed bywhere is the learned weights and is the concatenation of two vectors.
The initial hidden state is computed through one MLP conditioned on .
The traffic states feature vector of the whole trip can be computed in a recursive way.
We use the final hidden states as the output of traffic information extractor.
The other important factor of travel energy consumption is vehicle state. For electric vehicle, State of Charge (SOC), cell temperature, and cell voltage are key influence factors of vehicle energy consumption. In order to combine vehicle states into prediction model, we employ a MLP as vehicle states extractor. The vehicle state is a dimensional vector, which contains all key factor of vehicle energy system, which can be collected. The final output of is computed through a MLP conditioned on .where and are the learned weights and bias and  is the activation function.
The task of energy consumption predictor is to make the prediction conditioned on the output of traffic information extractor and vehicle states extractor. We concatenate traffic information extractor’s output and vehicle states extractor’s output as and use it as the input of the predictor. We use a deep neural network (DNN) to make the prediction, and 3 hidden layers should be enough in this task. The output of energy consumption predictor is the prediction result of the task. The model can be trained jointly by minimizing the residual between model’s prediction result and ground truth, and the loss function iswhere is the number of training sets.
In the training process, the sequenced traffic information is input into RNN, and is outputted by the last layer, then the feature vector mapped to the vehicle state vector is fused with as a input vector of DNN, and the final energy consumption prediction result is obtained by DNN. Thus, the loss function can be calculated with the ground truth . Though backpropagation, the neural network parameters can be optimized with the optimizer RMSprop , where the learning rate is set as 0.00001 before the 10th epoch and 0.000001 after that. After 100 epochs, the training process will be terminated, and the trained neural network will be tested under the test dataset.
3. Experiments Design
In this section, we introduce the process of the construction of dataset and implement detail of the model.
3.1. Data Description
Our model is a data-driven model and needs two kinds of data, traffic information of road network and the vehicle state. However, it is difficult for us to obtain traffic status data and vehicle status data corresponding to each other. We have built a simulation platform for electric vehicle operation data collection and travel energy consumption calculation.
We utilize Quadstone Paramics traffic simulation software to construct a traffic simulation model, from which we can collect traffic states of all road segments of road network and driving cycles of vehicles. Moreover, we build a vehicle simulation model by Simulink/MATLAB, from which we can collect vehicle states and compute travel energy consumption. The logic diagram of the simulation platform is shown in Figure 2.
3.2. Traffic Simulation Model
Quadstone Paramics is a traffic microsimulation software, which is widely used in many aspects of transportation. A Paramics model is represented by a combination of “nodes, links, and traffic zones” to construct the road network in the real world. Through the set of OD matrices, users can adjust the traffic flow on different roads. Users can also change the individual vehicle movements by adding plug-ins in the model.
As shown in Figure 3, we construct the road network of traffic simulation model based on the road network of Beijing Northwest Third Ring Road. Each road segment of the road network has an index from 1 to n, where n is the amount of road segments in the road network. In this paper, we construct a road network with 330 road segments and 27 travel regions. From traffic simulation model, we get traffic information of road network and driving cycles of vehicles, which run on the road. Traffic information contains the average traffic flow, velocity, and density of every road segment in the road network every minute. And, we randomly choose numerous vehicles running on road from the traffic simulation model and record its driving cycles by granularity of 0.5 s and the index of the road segments the vehicles have passed.
When making a prediction, we should choose a possible route at first and construct traffic states input based on this route. We also add vehicle location information to traffic state input and one-hot encoding vector is employed to represent location information (e.g., in the case road, the network has 5 road segments, for the index of road segment of the route is No. 1).
3.3. Vehicle Simulation Model
As shown in Figure 4, we build a vehicle simulation model based on electric vehicle in Simulink/MATLAB.
3.3.1. Driver Simulation Model
The driver controls vehicle vertical dynamic by the accelerator or brake pedal based on the difference between the actual speed of the feedback and the expected speed of the vehicle, thereby determining the driving or braking torque, and real-time following the speed. Commonly using the PI controller to simulate the driver’s operation, the driver’s PI controller principle is shown in the following formula:where is the reference speed, is the real speed, and is the error between and , is the angel of the accelerator or brake pedal normalized into [−1, 1], in which [0, 1] represents acceleration and [−1, 0] represents deceleration, A is the full range of pedal angel, is the proportional factor, is the integral factor, and is the Integral antisaturation coefficient. The driver model adjusts the output driving or braking torque as follows:
is the motor torque, where represents the driving torque, and represents the brake torque.
3.3.2. Motor Model
The drive motor uses a quasi-steady state method to simulate the motor and its controller, and the quasi-steady state equation is shown in the following equation:
In the formula, is the rotation speed of the drive motor, is the torque of the drive motor, and is the efficiency of the main drive motor. Its efficiency MAP is shown in Figure 4:
3.3.3. Battery Model
The battery model uses a Rint model, which is an equivalent circuit model characterized by open circuit voltage and resistance. The calculation formulas of current and power under this model are as follows:
Among them, Voc is the open circuit voltage, and Rint is the internal resistance of the battery; they are the function of battery SOC and temperature T. The energy consumption of the battery can be calculated by the following formula:
3.3.4. Vehicle Dynamic Model
The dynamic model of the vehicle can calculate the acceleration and speed of the vehicle according to the driving resistance balance equation:
In the formula, f is the rolling resistance, α is the road ramp angle (the default value is 0), m is the curb mass, is the acceleration of gravity, Cd is the air resistance coefficient, and A is the windward area of the vehicle.
We randomly set the initial SOC and collect the status of various key components in the running process of the vehicle and calculate the travel energy consumption. The travel energy consumption results of the vehicle are used as the ground truth of prediction model. We used the data collected by real vehicles to verify the accuracy of the vehicle simulation model. The test results show that the error in the calculation of travel energy consumption of our vehicle simulation model is within ±3%. Therefore, we can treat the simulation data as a replacement of real data.
We made a four-hour traffic simulation and collected 31175 vehicle driving cycles using the traffic simulation model. Furthermore, we compute the energy consumption of these driving cycles by utilizing the vehicle simulation model. For stabilizing the training process, we normalize the data by rescaling data to [0, 1].
4. Results Analysis
We divide the dataset into numerous batches with a size of 30. We make the dataset by combining traffic states information and vehicle states information together. The model can realize an end to end training by minimizing the loss function, and we use RMSProp to optimize the model.
We employ the deterministic “soft” attention  in our model, which computes the weighted expectation of road network’s traffic states. Ideally, the attention mechanism should be able to find out high relevant roads to the prediction task, by setting high value to the corresponding road of the weight vector. The visualization of attention mechanism is shown in Figure 5.
As we can see in Figure 6, during the whole trip, the attention mechanism promotes the prediction model, paying more attention to the traffic state of currently located and roads, where the vehicle on it can reach the road in a short time. Furthermore, at the initial stage of the trip (before 5 min in Figure 6), the model pays more attention to roads near to the location of the vehicle. As the vehicle travels on the preset route (10∼15 min in Figure 6), attention is attached to the mainline far away from the vehicle location. The pattern learned by attention mechanism agrees with the principle of traffic evolution process. As the traffic flow of the road network changes dynamically, the near future traffic state of a road is mainly affected by the current traffic state of the road itself and roads, where the vehicle on it can reach the road in a short time, where the far future traffic state of a road is further affected by the traffic state of a wider range of road network. Besides, the attention mechanism also helps the RNN based prediction model with recursively feeding traffic states input into the model.
In order to explore whether the attention mechanism helps improve the model’s performance on prediction, we make a comparison experiment by removing attention mechanism from our model but using the same dataset for training and testing. Furthermore, we explore the importance of traffic information and vehicle information to our model by making a simply modification to our model and training the model by using traffic information and vehicle states separately. For verification, the results are shown in Table 1 and Figure 7.
The bold values represent the algorithm with the least error; that is, the prediction effect of the algorithm is the best.
The experiment result shows that the model with attention mechanism has a better performance than the model without attention. Combined with the visualization of the attention, we can know that the attention mechanism exactly helps the model extract the feature of traffic state. We verify the importance of vehicle state information by removing vehicle states extractor from the model, and we train the model using traffic state and distance data. For the verification of the importance of traffic states information, we use vehicle states and distance to finish the prediction task. For the selection of model, we choose a DNN with three hidden layers. From the results, we know that traffic states information is the key information of the travel energy consumption prediction, and the model has a poor performance without traffic states information. The performance of the model trained without vehicle states information also has a decline. In this paper, the impact of removing traffic information is greater than removing vehicle state information from the model.
In this paper, a method of traffic state feature extraction was proposed based on attention mechanism, which can extract effective key features from high-dimensional traffic network state data and reduce the interference of irrelevant features on travel energy consumption prediction model. Furthermore, a deep learning prediction method for energy consumption of electric vehicles based on traffic information is demonstrated. The method integrates vehicle status information and traffic network status features. The experimental result shows that the travel energy consumption prediction proposed in this paper has a high prediction accuracy. The idea of this paper can help the future work in travel energy prediction and traffic state prediction. Besides pure electric vehicle, how other vehicle models affect the proposed method needs to be further investigated in future studies.
However, there are still some limitations of this paper. The proposed prediction model is trained with data produced form simulation because the match data of traffic networks and electric vehicles is hard to be obtained in the real world. For future research, the model will be further researched and tested with real data. Besides, the effects of air condition usage and driver styles will be taken into consideration for a more accurate energy consumption prediction.
The EV data collected from ROEWE used to support the findings of this study were supplied by SAIC under license, and so these cannot be made freely available. Requests for data, (6/12 months) after publication of this article, will be considered by the corresponding author.
Conflicts of Interest
The authors declare that they have no conflicts of interest.
This work was financed by the National Science Foundation of China (no. 61620106002) and the National Key R&D Program of China (no. 2019YFB1600100).
- A. Luè, A. Colorni, R. Nocerino, and V. Paruscio, “Green move: an innovative electric vehicle-sharing system,” Procedia—, vol. 48, pp. 2978–2987, 2012.
- S. Carrese, T. Giacchetti, M. Nigro, and S. M. Patella, “An innovative car sharing electric vehicle system: an Italian experience,” Urban Transport XXIII, vol. XXIII, no. 176, p. 245, 2017.
- R. Mounce and J. D. Nelson, “On the potential for one-way electric vehicle car-sharing in future mobility systems,” Transportation Research Part A: Policy and Practice, vol. 120, pp. 17–30, 2019.
- J. Neubauer and E. Wood, “The impact of range anxiety and home, workplace, and public charging infrastructure on simulated battery electric vehicle lifetime utility,” Journal of Power Sources, vol. 257, pp. 12–20, 2014.
- K. Liu, X. Hu, H. Zhou, L. Tong, and J. Marco, “Feature analyses and modelling of lithium-ion batteries manufacturing based on random forest classification,” IEEE, no. 99, 2021.
- K. Liu, Z. Wei, Z. Yang, and K. Li, “Mass load prediction for lithium-ion battery electrode clean production: a machine learning approach,” Journal of Cleaner Production, vol. 289, no. 1, 2020.
- C. King, W. Griggs, F. Wirth, K. Quinn, and R. Shorten, “Alleviating a form of electric vehicle range anxiety through on-demand vehicle access,” International Journal of Control, vol. 88, no. 4, pp. 717–728, 2015.
- K. Liu, C. Zou, K. Li, and T. Wik, “Charging pattern optimization for lithium-ion batteries with an electrothermal-aging model,” IEEE transactions on industrial informatics, vol. 14, 2018.
- K. Liu, X. Hu, Z. Yang, Y. Xie, and S. Feng, “Lithium-ion battery charging management considering economic costs of electrical energy loss and battery degradation,” Energy Conversion and Management, vol. 195, pp. 167–179, 2019.
- K. Liu, K. Li, and C. Zhang, “Constrained generalized predictive control of battery charging process based on a coupled thermoelectric model,” Journal of Power Sources, vol. 347, pp. 145–158, 2017.
- A. Cartenì, E. Cascetta, and S. De Luca, “A random utility model for park & carsharing services and the pure preference for electric vehicles,” Transport Policy, vol. 48, pp. 49–59, 2016.
- A. Ralf, “Apparatus of calculating a navigation route based on estimated energy consumption,” 2009, US Patent 20090254266.
- C. De Cauwer, J. Van Mierlo, and T. Coosemans, “Energy consumption prediction for electric vehicles based on real-world data,” Energies, vol. 8, no. 8, pp. 8573–8593, 2015.
- C. De CauwerW. Verbeke, T. Coosemans, S. Faid, and J. Van Mierlo, ““A data-driven method for energy consumption prediction and energy-efficient routing of electric vehicles in real-world conditions,” Energies, vol. 10, no. 5, p. 608, 2017.
- Z. Yi and H. Bauer Peter, “Adaptive multi-resolution energy consumption prediction for electric vehicles,” IEEE Transactions on Vehicular Technology, vol. 66, no. 99, p. 1, 2017.
- B. Yoshua, “Learning deep architectures for AI,” Foundations and trends® in Machine Learning, vol. 2, no. 1, pp. 1–127, 2009.
- G. Ross, “Fast R-CNN,” in Proceedings of the IEEE International Conference on Computer Vision, Santiago, Chile, December 2015.
- S. Ren, K. He, R. Girshick, and J. Sun, “Faster R-CNN: towards real-time object detection with region proposal networks,” IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 39, no. 6, pp. 1137–1149, 2017.
- R. Collobert and J. Westonet, “A unified architecture for natural language processing: deep neural networks with multitask learning,” in Proceedings of the 25th International Conference on Machine Learning, pp. 160–167, Helsinki, Finland, July 2008.
- H. Geoffrey, L. Deng, D. Yu et al., “Deep neural networks for acoustic modeling in speech recognition: the shared views of four research groups,” IEEE Signal Processing Magazine, vol. 29, no. 6, pp. 82–97, 2012.
- S. Modi, J. Bhattacharya, and P. Basak, “Estimation of energy consumption of electric vehicles using deep convolutional neural network to reduce driver’s range anxiety,” ISA Transactions, vol. 98, pp. 454–470, 2020.
- M. Laroui, A. Dridi, H. Afifi, H. Moungla, M. Marot, and M. A. Cherif, “Energy management for electric vehicles in smart cities: a deep learning approach,” in 2019 15th International Wireless Communications & Mobile Computing Conference (IWCMC), pp. 2080–2085, IEEE, Tangier, Morocco, June 2019.
- K. Xu, J. Ba, R. Kiros et al., “Show, attend and tell: Neural image caption generation with visual attention,” in Proceedings of the International Conference on Machine Learning, pp. 2048–2057, Lille, France, July 2015.
- C. Ronan, W. Jason, K. Michael, K. Kavukcuoglu, and P. Kuksa, “Natural language processing (almost) from scratch,” Journal of Machine Learning Research, vol. 12, no. 1, pp. 2493–2537, 2011.
- D. Bahdanau, K. Cho, and Y. Bengio, “Neural machine translation by jointly learning to align and translate,” 2014, https://arxiv.org/abs/1409.0473.
- Y. Lecun, L. Bottou, Y. Bengio, and P. Haffner, “Gradient-based learning applied to document recognition,” Proceedings of the IEEE, vol. 86, no. 11, pp. 2278–2324, 1998.
- K. Yoon, D. Carl, L. Hoang, and A. M. Rush, Structured attention networks, Springer, Berlin, Germany, 2017.
- D. C. Plaut and T. Shallice, “Deep dyslexia: a case study of connectionist neuropsychology,” Cognitive Neuropsychology, vol. 10, no. 5, pp. 377–500, 1993.
- K. He, X. Zhang, S. Ren, and J. Sun, “Delving deep into rectifiers: surpassing human-level performance on ImageNet classification,” pp. 1026–1034, 2015, https://arxiv.org/abs/1502.01852.
- J. Max, S. Karen, V. Andrea, and A. Zisserman, “Deep structured output learning for unconstrained text recognition,” Eprint Arxiv, vol. 24, no. 6, pp. 603–611, 2014.
Copyright © 2021 Shen Li et al. This is an open access article distributed under the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.