Design and Implementation of Digital Twin-Assisted Simulation Method for Autonomous Vehicle in Car-Following Scenario

Xiong, Huiyuan; Wang, Zhijun; Wu, Guohui; Pan, Yuelong

doi:https://doi.org/10.1155/2022/4879490

Journal of Sensors

On this page

Abstract Introduction Conclusion Data Availability Conflicts of Interest Acknowledgments References Copyright Related Articles

Special Issue

Sensors of Digital Twin Systems

View this Special Issue

Research Article | Open Access

Volume 2022 | Article ID 4879490 | https://doi.org/10.1155/2022/4879490

Design and Implementation of Digital Twin-Assisted Simulation Method for Autonomous Vehicle in Car-Following Scenario

Huiyuan Xiong,¹Zhijun Wang,¹Guohui Wu,²and Yuelong Pan²

Academic Editor: Houssem Jerbi

Received29 Mar 2022

Accepted29 Apr 2022

Published25 May 2022

Abstract

The automated system replaces the driver, which makes autonomous vehicle to improve safety and convenience, so the market of autonomous vehicle is huge. However, the real-world application of autonomous vehicles faces many challenges due to the immaturity of automated systems. As a consequence, simulation verification plays an irreplaceable role in the application of autonomous vehicle (AV). Car-following is the most common driving scenario in mixed traffic flows, so it is essential to develop an appropriate and effective simulation method for AV. Combined with the existing AV simulation methods and digital twin (DT) technology, this paper proposes a DT-assisted method for AV simulation in a car-following scenario. The method makes the physical vehicle interact with the DT vehicle, and the DT vehicle can dynamically regulate the physical entities through real-time simulation data; the simulation verification can be displayed in the DT scenario to ensure the security of the simulation. Meanwhile, a DT-assisted simulation framework of AV is proposed, the framework includes physical entity components, DT components, and data processing and evaluation components. Besides, a DT-assisted simulation platform is developed base on Unity engine. Finally, the DT-assisted simulation of AV in the car-following scenario is implemented in field experiment. The experimental results show that the proposed method can be effectively conducted AV simulation in car-following, and the average of communication latency is 52.3 ms, which is smaller than the update frequency 15 Hz (66.6 ms) between DT-assisted platform and AV. The DT-assisted simulation method of AV proposed in this paper is applied in the car-following scenario, which effectively solves the challenges of car-following scenario simulation through virtual-real interaction.

1. Introduction

In recent years, AV has entered a stage of rapid development both in theory and technology [1]. Establishing safety of AV is a challenging endeavor due to the variety of conditions an AV has to operate in and the complexity of their system implementation, which is one of the important reasons why AV has not been applied in a large scale [2]. Hence, in the process of AV development, simulation verification plays an irreplaceable role. In particular, car-following is the most common driving scenario in mixed traffic flows, and it is very crucial to select an appropriate and effective simulation method for the car-following scenario. At present, the main methods of autonomous vehicle simulation include software-in-the-loop (SIL) [3], hardware-in-the-loop (HIL) [4], and vehicle-in-the-loop (VIL) [5]. SIL focuses on algorithm simulation verification, HIL focuses on algorithm and partial subsystem simulation verification, and VIL takes into account vehicle system and algorithm simulation. However, the vehicle can only move within finite degrees of freedom in the VIL, due to the physical vehicle on the wheel-hub. In addition, the interaction between the vehicle and the environment (e.g., wheels and the road surface) is all in the simulation platform. Closed road testing of AV is risky and costly.

With the advent of industry 4.0 era, the emergence of DT technology brings new applications in the field of AV simulation [6]. At first, Dr. Michael Grieves introduced the concept of DT equivalents of physical products in the product life cycle management executive course at the University of Michigan in 2003 [7], since DT can synchronize the running state of physical entities and realize real-time simulation test in the DT space. Otherwise, DT can dynamically regulate the physical entities through real-time simulation data form a closed-loop simulation test, so many researchers around the world begin to pay attention to the DT technology. Rasslkin et al. [8] constructed DT connecting physical entities and virtual models in the real world and assigned a special unsupervised prediction and control platform for the performance evaluation of the energy system for AV. Szalai et al. [9] proposed a hybrid reality simulation environment that integrates real vehicles into virtual environments and uses the hybrid reality environment to create a diversified test environment around real vehicles. Ge et al. [10] proposed a method of using DT technology to carry out AV test in extreme environment. In the simulation test environment of autonomous driving, the test of real AV under the virtual complex road scenarios can be realized by using the DT mapping. da Sliva et al. [11] introduced a novel concept called Cyber DT, which transfers the idea of the DT to automotive software for the purpose of security analysis, and DT is used to evaluate automotive security requirements through automated security requirement verification using policy enforcement checks and detection of security vulnerabilities.

The car-following scenario is one of the most common driving scenarios of AV. There are also many previous achievements in the simulation of AV in the car-following scenario. Wang et al. [12] proposed a new car-following model by using vehicle-to-everything technology. In the end, a case study was conducted in VISSIM to verify the proposed model. Osorio and Punzo [13] developed a simulation-based optimization methodology for the efficient calibration of car-following models of large-scale stochastic networks, and the methodology was verified by using AIMSUN simulator. These efforts focus on the validation of the algorithm and ignore the influence of other parts, so the results of these works may differ greatly from the actual scenario. Ma et al. [14] proposed an optimized control approach for cooperative vehicular platoon, and a HIL test was performed with Carsim to simulate a high-fidelity vehicle dynamic model. Zhou et al. [15] developed a hierarchical control strategy for vehicle stability; the HIL simulation test show that the effectiveness of proposed strategy. Ard et al. [16] experimentally demonstrates the effectiveness of an anticipative car-following algorithm by a VIL experiments. In the HIL simulation, virtual vehicle kinematics and dynamics models are used. In the VIL simulation, the physical vehicle is embedded into a virtual environment. However, this is quite different from vehicle-road interaction in the physical world, so the validity of the simulation results is questionable.

The DT technology and existing simulation method of AV in car-following scenario are combined, and we propose a DT-assisted simulation method for AV in car-following scenario in this paper. The main insights and contributions of this paper are as follows: (1)A DT-assisted simulation method for AV in car-following scenario is proposed. This method uses physical vehicle to interact with the DT vehicle, which dynamically regulates the physical entities through real-time simulation data(2)A DT-assisted simulation framework of AV is proposed, which is composed of three parts: physical entity components, DT simulation components, and simulation evaluation components(3)The DT-assisted simulation platform is implemented based on Unity3D engine, including DT scenario, DT vehicle, and network communication module. Finally, a field experiment is implemented and verified the effectiveness of DT-assisted simulation method

The rest of this paper is organized as follows: Section 2 introduces the DT-assisted simulation framework. Section 3 details the DT-assisted simulation platform. Section 4 introduces the AV simulation in car-following by DT-assisted method. Section 5 details the real DT-assisted field implementation, and its results are analyzed in Section 6. This paper is ended in Section 7 with the conclusion and future work.

2. DT-Assisted Simulation Framework of AV

According to the characteristics of DT technology and the experience of previous researches, and a DT-assisted simulation framework for AV is proposed. As shown in Figure 1, the DT-assisted simulation framework includes three parts: physical entity components, DT simulation components, and data processing and evaluation components. The AV of physical entity components acquires real data by physical sensors, and the DT simulation components is used for simulation verification by DT vehicle in DT scenario. The data processing and evaluation components are used for data storage in the DT-assisted simulation platform. Virtual perception data is sent to physical entity components from DT components. According to the acquired virtual information, the AV implements decision-making and control. Each is described in detail in the following sections.

2.1. Physical Entity Components

The physical data acquired by the real sensor devices in the physical entity components is used to implement the digitizing process from the physical space to the DT space. As for the autonomous driving simulation test, the AV equipped with the autonomous driving controller runs in the real test field and obtains real-time data (position, attitude, fault, etc.) through on-board IMU, GNSS, and other sensor devices. At the same time, the real-time acquired data is transmitted to the DT simulation components through network communication module, so as to realize the interconnection between the DT vehicle and AV under test.

On the other hand, the real-time data acquired by the physical entity components is also transmitted to the data processing and evaluation components for data storage, which is convenient for historical data reproduction and simulation evaluation. The autonomous driving controller will also receive virtual perception information. After receiving the virtual perception data, the AV make decision and control through the real test field and the real vehicle actuator, so as to gain more accurate simulation results.

2.2. DT Simulation Components

DT simulation components are digital description of certain historical moment or current real-time state in the physical entities’ components. The DT vehicle receives the real-time positioning data from real sensors in physical entity components and keeps the motion state of the DT vehicle consistent with AV at the appropriate frequency. The DT simulation components can provide repeatable and extreme simulation test scenarios, so the virtual sensors can sense the extreme scenarios information, and transmit the real-time dynamic simulation data to the data processing and evaluation components for data storage and data mining. The virtual perception information is fed back to the autonomous driving controller in physical entity components. At the same time, the DT simulation components is also used as visual module, which can observe the motion state, positioning information, and fault information of the physical vehicle through the DT vehicle.

2.3. Simulation Evaluation Components

The simulation evaluation components are the core part. In the process of DT-assisted simulation, the simulation evaluation components receive real-time data from the DT simulation components and the physical entity components for storage and mining. In addition, the data processing and evaluation components can perform simulation evaluation base on the real-time data fed back from the physical entity components and the simulation data fed back from the DT components. Besides, the simulation results are stored in simulation evaluation components, which can realize the rapid and convenient reproduction of dangerous accident scenarios and key simulation test process.

3. DT-Assisted Simulation Platform

At present, the mainstream simulation platforms of AV simulation are developed based on game engine. For instance, Carla was developed by Intel and Toyota based on Unreal4 [17], AirSim developed by Microsoft based on Unreal4 [18], SVL developed by LG based on Unity3D [19], and this AV simulation software based on game engine have good performance in the test process.

Unity3D has very efficient graphics rendering system, which can render realistic DT scenarios in real time. Besides, Unity3D can be developed on multiple platforms such as Windows, Linux, and Mac by using scripts such as C#. In addition, Unity3D can communicate with ROS through the TCP/IP protocol [20]. According to the characteristics of AV simulation test and DT-assisted simulation framework, this paper chooses Unity3D engine as the development platform. This part mainly introduces the implementation of DT-assisted simulation platform, including DT scenarios, DT vehicle, and network communication module.

3.1. DT Scenario

DT scenarios are particularly important for DT-assisted simulation test. As shown in Figure 2, the framework which is divided into two parts, the virtual environment and car-following scenarios. A virtual environment needs to reflect the characteristics of real environment from various aspects, so the virtual environment has high requirements for accuracy. Traditional 3D models are usually modeled by software such as 3DsMax, Blender, and Revit. However, those methods have the characteristics of low efficiency, long modeling cycle, and more manpower consumption and cannot meet the high-precision requirements of DT-assisted simulation test environment.

In this paper, ESRI City Engine, a three-dimensional city modeling software based on computer generated architecture rules, was used for integrated modeling. The ESRI City Engine can quickly create 3D scenarios from 2D data. At the same time, ESRI City Engine provides perfect support for ArcGIS, which enables the quick realization of high precision 3D modeling based on existing GIS data without conversion, shortens the construction cycle of DT environment, and ensures the high-precision requirements of the DT environment. The construction process of high-precision DT environment is presented: (i)Step1: acquisition of L19 satellite image and elevation data. Then, import the downloaded elevation data into ArcGIS and project it into 3857 coordinate system, and import the satellite image into ArcGIS to carry out the vectorization of buildings and roads(ii)Step2: the vectorized shp data and real materials in ArcGIS were imported into ERSI CityEngine, and integrated modeling was carried out based on the vectorized data. Then, the 3D model was fine-tuned according to the road parameters(iii)Step3: FBX models exported from ERSI CityEngine are imported to Pixyz for automatic model optimization and lightweight(iv)Step4: Unity3D generates a realistic and high-precision DT environment through high-definition render pipeline

For traffic scenarios, on account of that, the scenes of Unity3D is editable and extensible, and there are abundant resources in Asset Store; it is easy to simulate the traffic scenarios in the physical world. In addition, the traffic flow can also be controlled by the script to realize the traffic flow simulation of the AV. In the DT-assisted simulation platform, DT traffic scenarios can be designed according to simulation test requirements.

3.2. DT Vehicle (the Replica of AV)

The DT vehicle is replica of AV, playing an essential role in the simulation test of AV. The model of DT vehicle can be divided into four levels: physical model, connection model, data model, and virtual model [21]. As shown in Figure 2, the process from the physical model to the virtual model is the digital process. The physical model of the DT vehicle is defined to describe the motion characteristics of the AV to be tested. The connection model of the DT vehicle is the key model to realize the synchronization between the AV to be tested and the DT vehicle. The connection model includes coordinate transformation and data format transformation. The connection model interconnects the five hierarchical models of the DT vehicle. The data model drives the virtual model to keep in sync with the behavior of the physical AV to be tested, and the data model of the DT vehicle is also used to define the normal working threshold of each device and realizes the function of fault diagnosis. The virtual model of the DT vehicle is the visual model of the AV to be tested. In addition, modularized data storage can be carried out through the data model, which is convenient for subsequent AV simulation evaluation.

3.3. Network Communication Module

The network communication module is the bridge between physical space and DT space, and it is also the basis of DT-assisted simulation test. DT components need real-time operation data in physical space to be driven, and physical entities also need feedback data from DT components to be dynamically regulated. Unity3D supports common communication protocols. Considering the advantages and disadvantages of various communication protocols, the DT-assisted simulation platform in this paper uses WebSocket communication protocol for data transmission. The DT-assisted simulation test in this paper can be carried out by local area network (LAN) or 5G communication. As shown in Figure 2, if LAN is adopted, the data in the autonomous driving controller is sent to the on-board wireless router through controller area network (CAN), which realizes data transmission with the AP of the test field and finally is sent to the DT-assisted simulation platform through the switch. If 5G is adopted, the data in the autonomous driving controller is sent to 5G-CPE through CAN, subsequently sent to the wireless router through the 5G base station, and finally transmitted to the DT-assisted simulation platform. The DT-assisted simulation platform uses the same communication link to feed back the virtual perception information to the AV to be tested. The two communication modes can be switched automatically according to the strength of the signal to ensure low latency of DT-assisted simulation test.

4. The AV Simulation in Car-Following Scenario by DT-Assisted Method

In the process of car-following, it is defined as a car-following scenario with potential risks when the leading vehicle suddenly conduct emergency brake with large braking deceleration in this paper. Under the circumstances, if the following vehicle does not have effective emergency braking strategy, it will lead to serious accidents. Therefore, in the face of potential risks in the car-following scenario, the effective simulation verification of collision avoidance strategy is an indispensable part in the development process of AV. Collision avoidance strategies in car-following scenarios can be divided into four methods: time headway (THW), time to collision (TTC), time to stop (TTS), and time to react (TTR) [22]. In the car-following scenario, the most commonly used method to evaluate the potential risk and rear-end collision is TTC [23–25]. TTC represents the time it takes for the following vehicle to collide with the leading vehicle from the current state of motion. In this paper, the TTC model adopts to verify the effectiveness of the DT-assisted simulation method.

4.1. The Statement of AV Car-Following

This paper only considers the two-car following problem in single-lane. As shown in Figure 3, the vehicle in front is called the leading vehicle, which is represented by a MV that is an manned vehicle in traffic flow. The rear vehicle is represented by the DT vehicle for the following vehicle and is also the replica of AV in the physical world. The DT vehicle and AV maintain real-time information exchange and status synchronization. In the DT scenario of DT-assisted platform, it is assumed that the following vehicle (DT vehicle) and leading vehicle (MV) use the communication mode of V2V to realize information interaction. The DT vehicle sends the real-time virtual perception information obtained from the MV, to the AV in the physical world. In order to simulate the potential risks existing in the scenario of car-following, MV have four random motion states, that is stationary, accelerated motion, uniform motion, and decelerated motion. In order to simulate the potential risks in the scenario of car-following, the MV with random possibility of emergency braking is designed.

In the DT simulation components, the DT vehicle receives the speed, acceleration, and position information of MV through V2V and calculates the distance information between the two vehicles according to the position of the DT vehicle. The acceleration and speed information of the two vehicles, as well as the distance between the two vehicles and the safe distance, and the TTC are calculated by collision avoidance strategy. Different motion states of MV correspond to different TTC models. The same TTC model can be used when MV is in a state of uniform motion or at stationary, i.e. is equal to 0. When the MV is in the decelerating motion state, the potential risks need to be discussed separately. First, the two vehicles collided before the MV decelerated to stop. Second, the two vehicles collided after the MV stopped. If the relative velocities of between MV and DT vehicle are very small or zero, and the relative acceleration of both or the acceleration of DT vehicle is small, considering it to be a safe working condition for following vehicle, and TTC tends to infinity at this time. Since the larger TTC value has no real reference value, so the TTC value for this state is default to 30 s. If the TTC is less than 3 s, the output TTC is 3 s by default, indicating the risk of collision in the car-following scenario. Meanwhile, the MV and DT vehicles in the DT-assisted platform are assumed to be within the scope of V2V communication, and they interact and share information in real-time. At the same time, DT vehicle feedback the virtual perception information of MV to the AV in the physical world through the LAN in real time.

4.2. The Motion Control of AV in DT-Assisted Simulation

Longitudinal motion control of AV is closely related to the behavior of the car-following, and the longitudinal motion control module of the AV controls the vehicle to adjust its motion state (such as emergency braking) in time, so as to ensure the timely elimination of potential risks in the process of car-following. The control topology of AV can be divided into two parts: autonomous driving domain and vehicle chassis domain. As shown in Figure 4, the autonomous driving domain mainly consists of four parts: the autonomous driving controller, i-ECU, wireless router, and wireless receiver. The autonomous driving controller senses the surrounding environment and realizes the function of autonomous driving through V2X, Radar, Camera, LIDAR, GNSS, and other sensors. The chassis controller communicates with the (MCU) motor control unit, SBW (steer-by-wire), the braking system consisting of two parts, EPB (electronic parking braking system) and EHB (electronic-hydraulic braking system), and BMS (battery management system) through CAN.

Algorithm 1 is designed to conduct the collision avoidance warning and generate the longitudinal motion control command of the AV. According to the characteristics of AV and in the case of potential risks to ensure safety, the time threshold for collision avoidance warning is and the time threshold for braking is . The values of , , and TTC are taken as the input of Algorithm 1. If the TTC is less than the set threshold , the collision avoidance warning indicator () changed from 0 to 1, and the AV will send collision avoidance warning messages. If the TTC is less than the set threshold , the braking indicator () changed from 0 to 1, and the braking command will be generated and sent to the vehicle chassis domain controller through the i-ECU module and then sent to the EPB or EHB module through the CAN, and the AV will actuate braking.

Input: 1. TTC: collision time calculated by collision avoidance strategy; 2. : time threshold for collision avoidance warning; 3. : time threshold for braking;
Output: 1. Flag1: the collision avoidance warning indicator; 2. Flag2: the braking indicator;
1:
2: Flag1 0; Flag2 0;
3: if TTC then
4: Flag1 0;
5: Flag2 0;
6: end if
7: if TTC then
8: Flag1 1;
9: Flag2 0;
10:end if
11:if 0 TTC then
12: Flag1 1;
13: Flag2 1;
14:end if
15:return Flag1, Flag2

5. Real DT-Assisted Field Implementation

5.1. DT-Assisted Simulation Platform

According to the general framework of DT-assisted simulation test for AV proposed in this paper, the AV DT-assisted simulation platform has been developed. In this simulation platform, the high-precision model of the test field was established and realized the communication between the simulation platform and AV to be tested based on WebSocket protocol. In addition, the car-following scenario with potential risks is designed in the DT scenario. The relevant data and early warning information in the process of simulation test can be displayed in real time on the UI interface. As shown in Figure 5, the related data include leading vehicle motion state, collision avoidance warning information, AV operating state, and fault information and video. In the field implementation of AV simulation in the car-following scenario, the DT vehicle and MV play an important role in DT-assisted simulation platform. The DT vehicle model structure is consistent with the AV and it can interact with the AV in the physical world in real-time. Both of them have the same motion state and motion parameters. In the car-following scenario, MV is a virtual motor vehicle, which simulates the motor vehicle in the traffic flow.

(a)

(b)

5.2. Hardware Setup of AV Simulation in Car-Following

A 4-wheel steering AV is adopted in field implementation, of which length is 2.8 m, and width is 1.5 m. The AV has two sets of drive system, two sets of steering system, and a set of braking system, and it is also equipped with sensors such as lidars, millimeter-wave radar, and cameras. All the hardware equipment for field implementation included the following: DT-assisted simulation platform server, physical sensors, autonomous driving controller, and network communication devices.

5.2.1. DT-Assisted Simulation Platform Server

This module runs a DT-assisted simulation platform. As shown in Figure 6, the simulation platform server is equipped with Intel Core i9-10900K/i9-10900KF hybrid 3.7 GHz (10-core) CPU, 64G RAM, 1 TB SSD and 2 TB HDD, and GeForce RTX3090-24 G.

5.2.2. Physical Sensors

This module is equipped on the AV to be tested, and it is used to obtain real-time position, attitude, and speed data of AV. The above data is transmitted to the DT-assisted platform, and the DT vehicle can receive the real data and synchronize the motion state of the AV to be tested in real time. As shown in Figure 6, the Starneto Newton-M² inertial measurement unit (IMU) integrated navigation system is adopted in the implementation field, and the maximum data update frequency is 100 Hz. When the RTK mode is used and GNSS signal is good, positioning accuracy is 2 cm+1 ppm.

5.2.3. Network Communication Devices

As shown in Figures 6 and 5, APs are installed in the test field to ensure the full coverage of the LAN. The AV to be tested is connected with AP through on-board wireless router (as shown in Figure 6), when the vehicle passes the AP node, and it can automatically switch the AP node according to the signal strength. APs are connected with a DT-assisted simulation platform through a switch. The communication latency of the LAN is within a certain range, which meets the requirements of our field implementation.

5.2.4. Autonomous Driving Controller

As shown in Figure 6, this module is the control center for the AV, and the collision avoidance strategy and autonomous driving algorithm runs in the autonomous driving controller. In addition, it has the ability of multisensor fusion, path planning, decision, and control. It is equipped with GTX2060s, 256 GB SSD, and 16 GB RAM. The autonomous driving controller is connected to the vehicle chassis domain via the i-ECU unit, realizing the motion control of the AV.

5.3. Field Implementation Plan of DT-Assisted Simulation

The test field is located in East Campus of Sun Yat-sen University, Guangzhou, China, and the total length of the test field is 339 m. As GNSS+RTK positioning data obtained from the INS receiver is the latitude and longitude information of the WGS-84 protocol, it is not conducive to realize the position synchronization between the DT vehicle in the DT-assisted platform and AV. So, this paper takes the longitude and latitude of the starting point as the origin and convert the longitude, latitude, and height data into Cartesian coordinate data. In each time step, the position, attitude, speed, and acceleration information of the AV to be tested is provided by the INS system and communicate with the DT vehicle in real-time through the LAN. In the DT-assisted simulation platform, a certain distance is maintained between the DT vehicle and MV at the initial moment, and DT vehicle sends the received MV virtual perception information (including localization, speed, acceleration, and yaw) to the autonomous driving controller through the wireless router on the AV. The autonomous driving controller will input the virtual information received in real-time into the autonomous driving controller to calculate the TTC, then input the TTC into the longitudinal motion control instruction generation algorithm.

In order to simulate more special and dangerous collision avoidance scenarios, MV may break in an emergency during the movement. The effectiveness of the collision avoidance strategy can be judged by the braking time and braking effect of autonomous vehicle. In addition, it can also be judged by the collision between the DT vehicle and MV in the DT scenario. Field implementation is shown in Figure 5. In order to simplify the test, both the DT vehicle and MV were kept in the right lane, and only longitudinal collision avoidance was considered, without lateral collision avoidance. As shown in Table 1, the initial velocity of MV and AV is 0, and due to the short lane distance, the maximum speed of AV is limited to km/h, and the maximum speed of MV is limited to km/h. The forward collision warning time is 10 s, and the communication frequency of the MV and DT vehicle in the DT-assisted simulation platform is 15 Hz.

6. Result Evaluation

In this paper, the DT-assisted platform is verified by AV simulation in the car-following scenario. The field implementation by DT-assisted was carried out at 08:00 on June 27, 2021. The DT-assisted platform updates the status of the MV and the AV on 15 Hz. The effectiveness of the collision avoidance strategy can be evaluated according to the real-time velocity and TTC. The DT vehicle can reflect the state of AV in real time. On the other hand, in the process of DT-assisted simulation, DT vehicle needs to synchronize motion state of AV in real time and needs to feed back the virtual perceptual information to AV. In this work, if the communication latency is too large, which will lead to inaccurate simulation results, so the communication latency is particularly important in the simulation. The communication latency was also tested between AV and DT-assisted simulation platform.

6.1. Car-Following Simulation Result Evaluation

As is shown in Figure 7, for the first 12 seconds, MV stays stationary, and then, the leading vehicle speeds up. When there is no danger of collision between MV and AV, AV starts to speed up and keeps following the MV at 12.8 s. At 14.4 s, MV suddenly decelerates with a large acceleration. According to the collision avoidance strategy, there is a collision risk, and TTC reaches 3 s at 16.6 s. Therefore, AV adopts emergency braking to avoid collision with MV. At 19 s, MV accelerates with a large acceleration, and when the two vehicles enter the safe driving state, AV immediately accelerates and keeps car-following. At 2130 s, MV changes dynamically in three motion states of deceleration, acceleration, and constant speed. Due to the high speed of MV, when MV decelerates at 21.4 s, TTC is maintained at a large value. At 23 s, the TTC value is equal to 3 s according to calculate by collision avoidance strategy, and AV begins to stop in an emergency. In the last period, AV calculates TTC in real-time according to the motion state of MV and estimates the emergency degree of potential collision danger in the car-following scenario according to the TTC value, so as to adopt appropriate collision avoidance strategies to maintain the following state.

The effectiveness of collision avoidance strategy based on TTC in the car-following scenario with potential risks is demonstrated by the DT vehicle in the DT-assisted simulation platform. If the DT vehicle collides with the MV, this indicates that the strategy running in the AV is defective. If the DT vehicle follows the MV at safe distance, there is no collision between the DT vehicle and MV in the DT-assisted simulation platform, which indicates that the strategy in the AV is valid. In this process, the DT-assisted method using the real kinematics and dynamics model of AV, which greatly improved the reliability of the simulation results and also improved the efficiency of the simulation under the premise of ensuring safety. In particular, the DT-assisted method reduces the risk of simulation.

6.2. Communication Result Evaluation

To verify the communication latency in the DT-assisted simulation method is within an acceptable range, the communication latency experiment is performed. In this experiment, the communication latency is defined as the difference between the time when the AV sends a message and the time when the AV receives the feedback message from the DT-assisted simulation platform. First, AV sends real data to the DT-assisted simulation platform and records the time . After receiving the messages, the DT-assisted simulation platform will immediately send a message back to AV. When AV receives the feedback message, it records the time and the communication latency is . Figure 8 shows that communication latency in continuous 300 s and the histogram of communication latency distribution. As shown in Table 2, the maximum communication latency is 63.8 ms, the minimum communication latency is 32.3 ms, and the average communication latency is 52.3 ms. As shown in Figure 8(b), the results reveal that 85% of the communication latency is less than 56.5 ms, and 99% of the communication latency is less than 61 ms. In the field implementation, the communication frequency between the DT vehicle and AV is 15 Hz (66.6 ms), that is, the communication latency must be less than 66.6 ms to meet the requirements of DT-assisted simulation. According to the above communication latency test results, the maximum value is 63.8 ms less than 66.6 ms, which meets the requirement that DT vehicle can synchronize the motion state of AV smoothly to realize simulation in the car-following scenario.

(a)

(b)

7. Conclusion

The DT-assisted simulation method of AV is proposed in this paper, and the simulation platform is developed based on Unity3D. The AV simulation in the car-following scenario is implemented on the test field. The experiment results show that the DT-assisted method is effective within an acceptable communication latency. In the process of AV simulation, the real vehicle actuators and the vehicle directly interacts with the surface. Meanwhile, there is no need to consider vehicle kinematics and dynamics simulation, which can greatly improve the reliability of AV simulation. In particular, the DT-assisted simulation method can safely and efficiently implement simulation test that are difficult to complete in the physical world. There are many challenges in this paper. Firstly, since the physical world needs to be synchronized with the virtual world in real time, the accuracy of the DT scenario in the virtual world is critical. If the DT scenario is not accurate enough, the perception information outputted by the DT vehicle may be inaccurate. Secondly, if the DT-assisted simulation platform uses more sensors, huge amounts of data need to be transmitted between the virtual world and the real world, and it is difficult for general communication methods to meet the requirements. As a consequence, how to realize lightweight data transmission is an urgent problem to be solved.

Further, we need to continue to optimize the DT-assisted simulation platform and develop a method of lightweight data transmission for DT-assisted simulation. More virtual sensors need to be developed so as to realize AV simulation in various scenarios. On the other hand, the DT-assisted simulation method to reduce the communication latency as much as possible.

Data Availability

The experimental data used to support the findings of this study are available from the corresponding author upon request.

Conflicts of Interest

The authors declare that they have no conflicts of interest.

Acknowledgments

This work is supported by the Natural Science and Technology Special Projects under Grant 2019-1496, and Key-Area Research and Development Program of Guangdong Province (2020B0909050004).

References

Q. Xue, M. Xu, and C. Mullen, “Governance of emerging autonomous driving development in China,” Transportation Research Record, vol. 2674, no. 6, pp. 281–290, 2020.
View at: Publisher Site | Google Scholar
V. Singh, S. K. S. Hari, T. Tsai, and M. Pitale, “Simulation driven design and test for safety of ai based autonomous vehicles,” in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 122–128, Nashville, TN, USA, 2021.
View at: Google Scholar
L. J. Moukahal, M. Zulkernine, and M. Soukup, “Vulnerability-oriented fuzz testing for connected autonomous vehicle systems,” IEEE Transactions on Reliability, vol. 70, no. 4, pp. 1422–1437, 2021.
View at: Publisher Site | Google Scholar
Y. Shao, M. A. M. Zulkefli, Z. Sun, and P. Huang, “Evaluating connected and autonomous vehicles using a hardware-in-the-loop testbed and a living lab,” Transportation Research Part C: Emerging Technologies, vol. 102, pp. 121–135, 2019.
View at: Publisher Site | Google Scholar
Y. Chen, S. Chen, T. Xiao, S. Zhang, Q. Hou, and N. Zheng, “Mixed test environment-based vehicle-in-the-loop validation-a new testing approach for autonomous vehicles,” in IEEE Intelligent Vehicles Symposium (IV), pp. 1283–1289, Las Vegas, NV, USA, 2020.
View at: Google Scholar
M. Schluse, M. Priggemeyer, L. Atorf, and J. Rossmann, “Experimentable digital twins—streamlining simulation-based systems engineering for Industry 4.0,” IEEE Transactions on Industrial Informatics, vol. 14, no. 4, pp. 1722–1731, 2018.
View at: Publisher Site | Google Scholar
M. Grieves, “Digital twin: manufacturing excellence through virtual factory replication,” White paper, vol. 1, pp. 1–7, 2014.
View at: Google Scholar
A. Rassõlkin, T. Vaimann, A. Kallaste, and V. Kuts, “Digital twin for propulsion drive of autonomous electric vehicle,” in IEEE 60th International Scientific Conference on Power and Electrical Engineering of Riga Technical University (RTUCON), pp. 1–4, Riga, Latvia, 2019.
View at: Google Scholar
M. Szalai, B. Varga, T. Tettamanti, and V. Tihanyi, “Mixed reality test environment for autonomous cars using Unity 3D and SUMO,” in IEEE 18th World Symposium on Applied Machine Intelligence and Informatics (SAMI), pp. 73–78, Herlany, Slovakia, 2020.
View at: Google Scholar
Y. Ge, Y. Wang, R. Yu, Q. Han, and Y. Chen, “Research on test method of autonomous driving based on digital twin,” in IEEE Vehicular Networking Conference (VNC), pp. 1-2, Los Angeles, CA, USA, 2019.
View at: Google Scholar
A. C. F. da Silva, S. Wagner, E. Lazebnik, and E. Traitel, “Using a cyber digital twin for continuous automotive security requirements verification,” 2021, http://arxiv.abs/org/2102.00790.
View at: Google Scholar
S. Wang, B. Yu, and M. Wu, MVCM car-following model for connected vehicles and simulation-based traffic analysis in mixed traffic flow, IEEE Transactions on Intelligent Transportation Systems, 2021.
View at: Publisher Site
C. Osorio and V. Punzo, “Efficient calibration of microscopic car-following models for large-scale stochastic network simulators,” Transportation Research Part B: Methodological, vol. 119, pp. 156–173, 2019.
View at: Publisher Site | Google Scholar
F. Ma, J. Wang, S. Zhu et al., “Distributed control of cooperative vehicular platoon with nonideal communication condition,” IEEE Transactions on Vehicular Technology, vol. 69, no. 8, pp. 8207–8220, 2020.
View at: Publisher Site | Google Scholar
Y. Zhou, R. Wang, R. Ding, D. Shi, and Q. Ye, “Investigation on hier archical control for driving stability and safety of intelligent HEV during car-following and lane-change process,” SCIENCE CHINA Technological Sciences, vol. 65, no. 1, pp. 53–76, 2021.
View at: Google Scholar
T. Ard, L. Guo, R. A. Dollar et al., “Energy and flow effects of optimal automated driving in mixed traffic: vehicle-in-the-loop experimental results,” Transportation Research Part C: Emerging Technologies, vol. 130, p. 103168, 2021.
View at: Publisher Site | Google Scholar
A. Dosovitskiy, G. Ros, F. Codevilla, A. Lopez, and V. Koltun, Carla: an open urban driving simulator, JOURNAL OF MACHINE LEARNING RESEARCH, 2017.
S. Shah, D. Dey, C. Lovett, and A. Kapoor, AirSim: high-fidelity visual and physical simulation for autonomous vehicles, Journal of Field Robotics, 2018.
G. Rong, B. H. Shin, H. Tabatabaee et al., “LGSVL simulator: a high fidelity simulator for autonomous driving,” in IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), Rhodes, Greece, 2020.
View at: Google Scholar
Y. Yang, W. Meng, and S. Zhu, “A digital twin simulation platform for multi-rotor UAV,” in 7th International Conference on Information, Cybernetics, and Computational Social Systems (ICCSS), Guangzhou, China, 2020.
View at: Google Scholar
F. Tao, H. Zhang, A. Liu, and A. Y. Nee, “Digital twin in industry: state-of-the-art,” IEEE Transactions on Industrial Informatics, vol. 15, no. 4, pp. 2405–2415, 2018.
View at: Google Scholar
T. Li, D. Chen, H. Zhou, J. Laval, and Y. Xie, “Car-following behavior characteristics of adaptive cruise control vehicles based on empirical experiments,” Transportation Research Part B: Methodological, vol. 147, pp. 67–91, 2021.
View at: Publisher Site | Google Scholar
S. Glaser, B. Vanholme, S. Mammar, D. Gruyer, and L. Nouveliere, “Maneuver-based trajectory planning for highly autonomous vehicles on real road with traffic and driver interaction,” IEEE Transactions on Intelligent Transportation Systems, vol. 11, no. 3, pp. 589–606, 2010.
View at: Publisher Site | Google Scholar
J. Kim and D. Kum, “Collision risk assessment algorithm via lane-based probabilistic motion prediction of surrounding vehicles,” IEEE Transactions on Intelligent Transportation Systems, vol. 19, no. 9, pp. 2965–2976, 2018.
View at: Publisher Site | Google Scholar
Y. Li, L. Zhang, and Y. Song, “A vehicular collision warning algorithm based on the time-to-collision estimation under connected environment,” in 14th International Conference on Control, Automation, Robotics and Vision (ICARCV), pp. 1–4, Phuket, Thailand, 2016.
View at: Google Scholar

Copyright

Copyright © 2022 Huiyuan Xiong et al. This is an open access article distributed under the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.

PDF Download Citation

Download other formats

Order printed copies

Views

1522

Downloads

956

Citations