Abstract
Fifth-generation (5G) technology is one of the keys to the Industrial Revolution known as Industry 4.0 as it provides faster connectivity and allows a greater number of devices to be connected simultaneously. In the transport sector, newly produced vehicles are equipped with various sensors and applications to help drivers perform safe maneuvers. However, moving from semiautonomous to fully autonomous vehicles to cooperating systems remains a major challenge. Many researchers have focused on artificial intelligence (AI) techniques and the ability to share information to achieve this cooperative behavior. This information can be made up of different data, which can be obtained from different sensors such as laser imaging detection and ranging (LiDAR), radar, camera, global positioning system (GPS), or data related to the current speed, acceleration, or position. The combination of the different shared data is performed depending on the approach of each navigation algorithm. This data fusion will allow a better understanding of the environment but will overload the network, as the traffic generated will be massive. Therefore, this paper addresses the challenge of achieving this cooperation between vehicles from the point of view of network requirements and computational capacity. In addition, this study contributes to advancing theory into real-world practice by examining the performance of cooperative navigation algorithms in the midst of the migration of computational resources from onboard vehicle equipment to the cloud. In particular, it investigates the transition from a cooperative navigation algorithm based on a decentralized architecture to a semidecentralized one as computationally demanding processes previously performed onboard are performed in the cloud. Additionally, the paper discusses the indispensable role of 5G in fulfilling the escalating demands for high throughput and low latency in these services, particularly as the number of vehicles increases. The results of the tests show that the AI acting alone cannot achieve optimal performance, even using 100% of the computational capacity of the onboard equipment in the vehicle. However, a system that integrates 5G and AI-based joint decisions can achieve better performance, reduce the computational resources consumed in the vehicle, and increase the efficiency of collaborative choices by up to 83.3%.
1. Introduction
Every year, traffic accidents represent the leading cause of injury deaths worldwide, where 94% of the accidents are due to decisions on maneuvers made by drivers. One of the influencing factors is the complex interaction on the road, while other factors include atmospheric conditions [1]. Notably, in 2022, the European Union (EU) witnessed over twenty thousand fatalities resulting from road accidents, underscoring the severity of the issue [2]. A consequence of these traffic accidents is the physical effects, which can vary from an injury to the loss of life. In addition, there are psychological and socioeconomic consequences. This is because the people involved in these accidents may manifest symptoms of depression and anxiety. For instance, the study conducted on the tracking of a population of road traffic accidents in the Rhône (ESPARR) showed that if the accident causes a severe injury, there is a 32% probability of stopping work for at least one year, so the family economy is affected [3, 4].
Therefore, to have a safer road, the exchange of vehicle information and cooperative decision-making are two of the fundamental aspects to be taken into account. However, as a real-time service, the information must be sent with ultra-low latency, meaning that the time required to travel from one point to another must be extremely low.
The importance of latency is that it directly impacts the performance of the autonomous vehicle as its decisions are based on the data received. Moreover, these decisions must be made immediately after the vehicle receives the information. Therefore, if the data do not arrive in time, the vehicle will execute a maneuver with erroneous data, which can lead to an accident.
Consequently, successful cooperation will contribute to preventing and reducing the number of accidents, gas emissions, and traffic congestion, as well as saving fuel and mitigating environmental impact [5–8]. For instance, Liu et al. conducted a study showing that, with full market penetration of autonomous vehicles, average energy consumption is reduced by 62% [9]. However, Dikshit et al. conducted a study on AI-based traffic management systems, demonstrating that applying AI algorithms improves sustainable urban mobility. Within the study, specifically in case 2, they show the coordination of autonomous vehicles in Pittsburgh. With this, it is evident that their proposed algorithms allow for facilitating communication and coordination among vehicles [10]. In addition, Jafar et al. showed an in-depth study on how the application of multivehicle cooperation algorithms can significantly reduce travel times and, at the same time, reduce accidents leading to multiple collisions, as these types of accidents cause up to 50% of urban traffic congestion [11].
However, achieving multivehicle cooperation can be difficult due to the variety of possible situations in modern traffic, such as intersections, preferences, lane merging, freeway entry/exit, bottlenecks, and traffic lights. In addition, due to the constant increase in vehicles on the road, the amount of data traffic flow increases exponentially, thus requiring the use of multicast/broadcast messages.
Multicast messages will be used when a vehicle requires sending data to specific vehicles. In contrast, broadcast messages will be used when it requires sending data to all actors on the road. Therefore, to provide services such as vehicle maneuvering that rely on real-time data collected from onboard and road sensors, a robust, fast, and reliable network is needed. It is the time for 5G technology.
The potential benefit of vehicles knowing each other’s behavior on the road is evident in a platooning system. It allows increased efficiency and reduces fuel consumption through the use of shared information and joint actions that ensure a safe and efficient distance between vehicles. This system relies on vehicle-to-vehicle (V2V) communications to keep the follower at a prudential distance from the leader and vehicle-to-infrastructure (V2I) communications to allow the leader to decide about passing or not through an intersection [12].
The design of cooperative systems involves the coexistence of different onboard sensors, vehicle types, and algorithms. To tackle the problems of achieving coexistence, AI techniques have been introduced. AI has been applied to many fields, such as machine vision, mapping, route planning, and big data analysis. However, using machine learning (ML) algorithms implies a high computational cost. Therefore, onboard equipment will have to be high-tech, which will increase the monetary cost of deploying these systems.
In view of the high cost of deployment, this is a major concern at the time of design. To solve these concerns, the features referred to in the third-generation partnership project (3GPP) technical specification for 5G New Radio (NR) versions 16 to 18 are used. They allow a reduction in the computational capacity required in vehicles and enable the implementation of their functions in the cloud. This guarantees a high-performance system with a low-cost device in the vehicle.
Implementing services such as cooperative vehicles, autonomous vehicles, or Industry 4.0 applications such as holograms require many computational and communication network resources. For instance, Mercedes-Benz presented a prototype called the Mercedes-Benz S-Class S 500 Intelligent Drive, where they explained the technical conditions for the implementation of autonomous driving [13]. Similarly, Tesla has also shown a vehicle with full autonomous driving capability, where the systems are designed to allow the vehicle to drive both short and long distances with no intervention from the person behind the wheel [14]. It should be noted that for fully autonomous driving, in addition to AI algorithms, 5G technologies are needed to meet the communication network requirements [15].
Therefore, all the resources needed for these use cases lead to significant costs during their deployment. It is important to keep in mind that the theoretical performance offered is directly affected by the communication network and the hardware capacity of the equipment used by the vehicle. In view of the abovementioned fact, the contributions of this paper can be summarized as follows:(1)A study of the latest advances in integration between 5G technology and artificial intelligence (AI)(2)A study on how combining both technologies will impact cost reduction and improved decision-making efficiency of cooperative vehicle systems(3)A study of the throughput and latency required by a semidecentralized architecture of cooperative vehicles
The remainder of the document is organized as follows: Section 2 covers the first part of the technical background related to 5G technology. Section 3 covers the second part of the technical background related to artificial intelligence. Section 4 presents an explanation of the algorithms used. Section 5 evaluates and discusses the results obtained. Section 6 presents the main conclusions of the study. Section 7 highlights some of the main lines of future research.
2. Related Work: Part 1—The Role of 5G Technology
The preceding discussion highlighted the need for robust, reliable, and low-latency communications, where 5G plays a key role. 5G NR is the global standard for unified wireless communication with a higher data rate and support of three types of communication: enhanced mobile broadband (eMBB), massive machine-type communication (mMTC), and ultra-reliable and low-latency communication (URLLC).
Release 15 specification contains the initial description of the 5G. After that, the following specifications, from Release 16 to Release 18, address key elements for the deployment of these services, such as intelligent transportation system (ITS) that covers secure transportation services and solutions, vehicle-to-anything (V2X) technology that enables the exchange of information, and URLLC that provides highly reliable communication links [16, 17].
Release 16 focuses on very low latency and improved reliability by increasing the monitoring frequency of the control channel. These features are crucial for use cases that require communication between multiple services, such as factory automation, power distribution, and the transportation industry [18, 19]. Meanwhile, Release 17 is designed to support use cases with high mobile data traffic and NR customization demanded by, e.g., automotive, logistics, and media [18].
Release 18 will cover the area of AI/ML focused on deploying applications centered on data collection and improved mobility optimization, extended reality, and intelligent network solutions. Furthermore, it will include Reduced Capability (RedCap) solutions for user equipment (UE), which will allow the user to reduce device cost and power consumption by reducing the number of radio receivers and radio transmitters [20, 21]. Therefore, 5G technologies will enable the deployment of different V2X use cases, thanks to their higher bandwidth, ultra-low latency, and higher reliability.
As mentioned earlier, V2X use cases are expected to send a large volume of data over a network with low latency and high data rate, i.e., with a round-trip time of approximately 1 ms. To address these needs, new features such as flexible spectrum and flexible numerology were added to the 5G NR architectures.
These features will ensure efficient bandwidth utilization and enable lower latency communication. Therefore, 5G will support intelligent transportation use cases such as an autonomous car, as it is capable of providing a data rate of 100 Mb/s and 50 Mb/s for downlink (DL) and uplink (UL), respectively, with 99.99% reliability and user latency up to 1 ms [22]. However, the impact on data privacy of these new digital applications and services that have access to our information is a matter of concern for many users [23, 24].
2.1. Privacy Protection
The need to share information between different actors is one of the requirements in V2X use cases, as in the cooperative perception scenario. However, many concerns arise because of the different types of attacks that the network can receive, such as, attacks against availability, data integrity, authenticity, and confidentiality. In the case of the first one, an example is the jamming attack, which can degrade network performance by interfering with packet transmission. The second one can be mentioned as GPS signal spoofing, which involves falsifying GPS signal data such as position, velocity, and time (PVT) to fool a GPS unit or a specific receiver. In the case of the third one, we can find Sybil here; the attacking node spoofs many identities in the road network. Finally, in the last one, we can cite a location tracking attack; this consists of that the location of the vehicle through a certain period is considered as if it were a kind of personal data [26–29].
Several standards include deterministic security, and quality of service (QoS) guarantees to address the different security issues. For instance, in Release 16 of 3GPP, direct communication (PC5) and 5G network communication (5G-Uu) interfaces are operations to communications V2V and V2I, respectively [30–32].
2.2. V2X Use Cases
In use cases such as platooning or remote driving, it is necessary to transmit a large number of messages in real time with high reliability and low latency. For instance, the required latency for advanced vehicles is in the order of 3 to 100 ms. These messages may contain data provided by different sensors, such as a laser, velocimeter, accelerometer, and other information that can warn others about its future intentions. All this knowledge will allow the system to prioritize traffic optimization and have better energy management [33, 34]. There are many other possible use cases enabled by 5G features, but this paper focuses on those that are possible within the advanced driving concept, such as cooperative maneuvers and cooperative perception.
2.2.1. Cooperative Maneuvers
As the name implies, cooperative maneuvering allows coordinating moves between vehicles by exchanging messages (see Figure 1). For instance, vehicles can exchange messages about the traffic environment or the actions planned by the sender. They can also be cooperative messages to influence local planning among the participants or specific actions indicated by the sender [33, 35]. This precise information allows vehicles to have complete knowledge of their surroundings. For example, Cooperative Awareness Message (CAM) sends maneuver coordination information, while Maneuver Coordination Message (MCM) sends maneuvers between vehicles [36–38].
Another example is 5G for Connected and Automated Road Mobility in the European UnioN (5G-CARMEN) project, which showed a proprietary digital twin based on simulation for automated vehicles connected to 5G. One of the use cases presented focuses on cooperative maneuvers applied to a group of vehicles. These vehicles share messages in real time and privately about information collected by sensors such as LiDAR, radar, and onboard cameras. This approach demonstrated how vehicles can periodically decide to change lanes and even react safely when a vehicle within the system is manually driven [39]. In the meantime, Koopmann et al. proposed a decentralized approach called “game-theoretic.” This approach was designed to negotiate cooperative maneuvers at urban intersections. The authors assume that an intelligent traffic management system (TMS) is deployed with a complete view of the environment. Using this information, they generate movement recommendations for vehicles without directly controlling them [40].
Intervehicle cooperation can be realized using a centralized architecture where all data obtained by vehicle sensors are sent to the cloud, which processes them. Once processed, the cloud will be responsible for finding an optimal solution and notifying the vehicles of the most appropriate action to be executed. Another way of cooperation is through a decentralized architecture, which implies that there will be direct data exchanges between vehicles. Therefore, each vehicle will be in charge of finding its optimal action, considering nearby vehicles’ reactions. In other words, when a vehicle wants to initiate a cooperative process, it first sends a maneuver proposal to the other vehicles, and they can either accept it or send an alternative proposal. This approach requires a high computational capacity in each of the vehicles to achieve optimal coordination and thus avoid dangerous situations. For this reason, cooperative semidecentralized architecture is the subject of focus in this paper.
2.2.2. Cooperative Perception
In contrast, cooperative perception (see Figure 2) allows the vehicle to have a better awareness of its surroundings, also helps avoid road collisions, and improves the efficiency of the road [41]. For instance, if a vehicle has only a viewing angle of fewer than 120 degrees, this could cause blind spots for the vehicle and low detection accuracy. In other words, it cannot see objects behind corners, curves, or in invisible areas. In these cases, the vehicle can achieve a complete view of the area by sharing the raw data information from nearby vehicles. Moreover, in case of adverse weather conditions or poor lighting, redundant data can help mitigate the negative impact of these phenomena by improving data reliability [42, 43].
There are many techniques to achieve a cooperative perception; collaborative vision, selection of data to transmit, and federated learning (FL) are the most outstanding ones. Only important data are transmitted by selecting the data to be transmitted, so communications consume fewer resources because fewer data are transmitted. In contrast, FL enables vehicles to share the weights of the model with neighboring vehicles, thus reducing the need for massive data transfers [25, 44].
Finally, collaborative vision focuses on the fusion of high-level data from multiple sensors. Algorithms such as You Only Look Once (YOLO) use these data to implement three-dimensional (3D) object detection functionality. This enables traffic safety monitoring and driving assistance, all within 5G-based scenarios [45, 46].
A study of different projects of cooperative perception was presented by Caillot et al., where many machine learning algorithms were used to estimate positions, detect other vehicles and obstacles, and cover blind spots caused by other vehicles, especially at intersections and traffic circles. The different sensors used were lasers, cameras, radars, and GPS, and they were placed on bridges, and street lights on roads, especially at intersections [47]. Meanwhile, Zhou et al. suggested an enhanced cooperative information perception, in which an ultra-fast filtering system is proposed to optimize the received data. This means that, after filtering, only the information important for vehicle navigation is displayed, thereby eliminating visual noise and facilitating the understanding of the environment [48]. Additionally, an interesting approach was proposed by Cui et al. where a tilt model is obtained based on intervehicle perception; in other words, first, the information is encoded in a point-based representation to transmit compact messages between vehicles; secondly, the 3D LiDAR data from the different vehicles are aggregated and finally analyzed [49].
In short, vehicle perception of the environment is important for improved decision-making. However, equipping a vehicle with high technology increases the cost of deploying these services. However, if only low-tech sensors, such as a camera, are available, it is not certain that all relevant objects will be detected. On top of that, the sensor can fail due to different factors such as noise, environmental conditions, or even manufacturing defects. Considering these facts, the higher the level of technology onboard, the higher the accuracy of environmental detection and the cost of deployment. Therefore, the big question is as follows: How can onboard computation demand be reduced and still deploy successful cooperative systems? Here, AI plays an important role.
3. Related Work: Part 2—The Role of Artificial Intelligence
Besides stable communication, AI has become a promising field for deploying cooperative autonomous vehicles. AI is a branch of computer science that focuses on developing technology to give machines the ability to mimic human behavior. The purpose is to make their response to an event similar to human behavior. In other words, machines will be able to perform tasks that require skills such as learning, reasoning, and self-correction.
ML and deep learning (DL) are two subfields of AI. The main idea of both is to obtain data, analyze them, learn from them, and apply the acquired knowledge to solve or execute tasks. In the case of ML, the algorithm is trained to perform a specific task. For instance, entertainment platforms use data from user choices to create a user profile and predict what the user might like in order to recommend future content.
However, DL is a subset of ML that uses neural networks to make decisions, which are getting better and better as the neural network can self-correct to improve its prediction. For example, suppose we have a vehicle that wants to move from one point to another without a map, at the beginning of the journey. In that case, it will take the wrong direction, but as it collects data and the neural network is trained, it will be able to reach its destination successfully.
In addition, ML encompasses the multi-robot system (MRS) field, which focuses on studying scenarios shared by several robots, where the actions executed by each of them influence the behavior of the others. These robots can interact competitively in those scenarios in which they have different objectives, where each robot tries to obtain the highest reward and make the others obtain the lowest one. The reward represents the quality of the action or decision; i.e., if it is correct, it will have a positive incentive (reward), and if it is incorrect, it will have a negative one (punishment).
Multi-robots can focus on cooperative, competitive, or mixed tasks. When focusing on cooperative tasks, the algorithms can be designed for a static or dynamic environment depending on whether or not they consider the other vehicles as a part of the environment. During the interaction in cooperative mode, all robots work together to achieve a common goal, similar to the mixed behavior used in team video games. In these video games, robots work cooperatively when they belong to the same team but competitively when they belong to the opposing team. Regardless of the interaction, robots learn through the reward obtained, always maximizing the reward obtained. This can be maximized in a centralized or decentralized way. In the first case, all robots receive the same reward, whereas, in the second one, each robot gets its local reward [50].
This paper focuses on cooperative behavior, which can also be classified as aware or unaware. Being aware means that the robot knows the other robots in the environment. If a robot is aware, it can have different ways of coordinating with other robots (centralized or decentralized). Robots can also be classified according to their organization (centralized or distributed). If they have a distributed organization, all robots act independently. However, a centralized organization can be very strong as long as there is one leader telling the others what to do. If there are many leaders in the team, it is a weakly centralized system. In a distributed organization, robots can have dependent or independent communication.
Multiagent reinforcement learning (MARL) algorithms fall into the independent communication type, whose learning is based on trial and error. Figure 3 shows a summary of some MARL-based algorithms that interact in different ways to achieve satisfactory cooperation during the performance of a task. The advantage of MARL is that, although it focuses on robots, its concepts can be applied to vehicles. MARL in vehicles allows faster learning as it collects data from heterogeneous vehicles and fuses their capabilities.
The term heterogeneous vehicles refer to vehicles that have different sensors. This exchange of information is used to achieve better collaboration between them, but as the number of vehicles increases, so does the number of connections, which turns the information exchange into a process with a high computational cost. Furthermore, considering the data collected for training are extensive in volume, it involves using deep neural networks and 5G networks for processing and communication, respectively.
MARL has been used in many real-world scenarios. Among the use cases are multiplayer video games, intelligent transportation system control, and robot swarm coordination [51]. However, one of the biggest challenges that arises is the instability of the training. This is mainly because their environment will change from one that does not change over time (stationary) to one that is constantly changing (nonstationary). This instability makes it difficult to find a stable behavior policy. However, an additional problem is the very high data transmission speed and minimum latency required by the network to deploy these services successfully.
Therefore, to better understand MARL and how a machine can learn, it is necessary to go deeper into the topic of reinforcement learning (RL).
3.1. Reinforcement Learning
The idea behind RL is that the vehicle learns through interactions with its environment. This means the vehicle will represent the agent on the road, while the other vehicles and other actors, such as pedestrians, will represent the environment. In this kind of learning, each vehicle action will generate feedback in the form of rewards. These rewards can be positive (appropriate action) or negative (wrong action).
This feedback is the key that indicates whether the current behavior policy should be improved or not, where the policy is the control strategy that will determine the optimal action to be performed. This strategy uses different data for learning, which can include the position of the vehicle, data from its sensors, and the position of its target. In other words, a policy is mapping states (observations of the vehicle), actions, rewards, and following states (observations after a decision is performed). As a result, the future of the vehicle is only affected by the current state and not by past decisions.
One of the challenges faced by RL is the scalability problem. It is limited to low-dimensional problems as it uses tables to store possible combinations of actions and states. In addition, most of the proposed algorithms involve a single RL robot because cooperation between multiple RL robots represents a challenge in terms of fast learning, data security and privacy, and hardware and software limitations [52].
To address these problems, deep reinforcement learning (DRL) is used, as it combines the RL framework with neural networks, enabling learning with large amounts of data using neural networks instead of using long tables of combinations.
3.2. Deep Reinforcement Learning Algorithms
DRL algorithms are widely used in algorithms for autonomous driving, path planning, object detection, speech recognition, and cooperative perception. Table 1 summarizes the main advantages of using DRL in applications such as cooperative driving [53–57]. In the case of cooperative perception, each vehicle learns through the DRL which data should be sent to the other vehicles, taking into account their needs [58]. For instance, if , with a high-resolution camera, wants to send its data to , equipped with a low-resolution sensor, it will prefer to send high-resolution data as needs to have a better perception of its environment. However, if wants to send its data to equipped with a laser, it will send the data in low resolution just to reinforce the data already possessed by .
Due to congestion or latency in the network, the vehicle may experience loss of connectivity and, therefore, packets; Li et al. proposed a way to mitigate this problem by a method of merging data from nearby vehicles considering a loss of communication and using multiple deep learning layers [59]. In addition, these algorithms can speed up the process of finding an optimal policy by experience replay (ER), in which the collected data are buffered and then randomly sampled for the training process.
In this paper, the performance of two cooperative vehicle scenarios is shown. It is analyzed in terms of successful intervehicle cooperation when the computational processes executed in the vehicles are reduced. In other words, the computational capacity required in the vehicle is reduced by transferring the functions that require a high level of processing to the cloud. Transferring all the information to the cloud is feasible due to the low latency of 5G, as it is necessary to process the decision and send it back to the vehicle in a time low enough to justify offshoring the decision.
In the first scenario (see Figure 4(a)), different vehicles send information about their sensors to each other to train their own policies and estimate the action to be performed. This decentralized scenario uses 100% of the vehicle’s computational resources and sends high traffic between vehicles, making the vehicle’s response slow and susceptible to collisions. In the second scenario (see Figure 4(b)), being a semicentralized architecture, the vehicles send all the data to the cloud and get a common policy. Each vehicle will use this policy to choose the action; therefore, only 30% of the computational capacity of the vehicle’s onboard equipment is used.
(a)
(b)
For all of the abovementioned scenarios, the information sent is the information of each vehicle, such as position, velocity, acceleration, orientation, and the data read by its sensors, such as camera, LiDAR, and radar.
Finally, having studied AI and its importance in cooperative navigation, Table 2 contains a detailed summary of the different subfields of AI applied to cooperative navigation. In it, the distinguishing features of ML, RL, and DRL algorithms are highlighted. Each subfield is thoroughly examined in terms of its advantages, disadvantages, and various applications in cooperative navigation. This summary provides a clear view of the capabilities and limitations of each approach, allowing informed decisions to be made in the development and implementation of intelligent navigation systems [25, 60–70].
4. Previous Work
To test the performance of a navigation algorithm when the computational capacity required by each vehicle is moved to the cloud, we have used our previous work, adapting it to the desired scenario. For the first setting, we use an algorithm [57] designed to plan the trajectory from one point to another in an environment where only one vehicle is considered to move. In this algorithm, the architecture is decentralized as all computational processes are performed in the vehicle.
It is important to note that the algorithm focuses on navigation without the use of a preloaded map, i.e., of an unknown environment. The authors combine DL and RL with the HK algorithm. In addition, they use the benefits of ER, genetic algorithm (GA), and dynamic programming (DP) to find the policy optimally, allowing the vehicle to achieve all the set objectives without a map. In addition, it allows for optimizing the time needed to train the optimal behavioral policy and achieving a policy capable of tracing a short path, thus optimizing time and distance. Additionally, it uses a hybrid method using semiuniform distributed exploration (SUDE) to determine the probability that the chosen action is using directed knowledge, hybrid knowledge, or autonomous knowledge.
The approach uses a double deep Q-learning (DDQL) algorithm, where the architecture of a neural network corresponds to an input layer with six neurons, two hidden layers of 526 neurons, and an output layer of six neurons (forward, backward, and sideway actions with a given angle each). The summary of the characteristics of the architecture used in this approach, such as the number of layers and the optimizer, is detailed in Table 3.
As the objective is to test the performance obtained when more than one vehicle is used, two vehicles using the [57] algorithm were placed and moved in the same environment. Given that the original algorithm was designed for a single vehicle, a feature has now been added that enables it to receive and send its broadcast information to another vehicle.
The second scenario to be tested uses the algorithm proposed in [71], which was designed for cooperative navigation in a semidecentralized architecture. In contrast, this algorithm is based on semidecentralized learning. It employs the concepts of joint action learning (JAL) and independent learning (IL) to find an optimal approach. Additionally, it uses a self-advice module that estimates the correct action to be performed and thereby replaces the joint action defined in the JAL algorithms. With this, it is able to reduce the algorithmic complexity and resemble the benefits of IL as its complexity does not increase. At the same time, there are more vehicles in the system. Finally, it adds a collision controller module that is intended to mitigate the risk of collisions between different vehicles even more.
This approach uses two hidden layers, each with 526 neurons. It uses the deep deterministic policy gradient (DDPG) algorithm to train the policy, where the actor network has an input layer that has 28 neurons corresponding to the state (laser data, distance to the nearest object, heading between the agent and the target, current distance to the target, and initial heading to the target). However, the critical network has an input of 30 neurons, where 28 correspond to the state and the other two correspond to the different actions, one chosen by the agent’s behavior policy and the other chosen for the self-advice module. The network output has five neurons representing the five possible movements that the agent can perform: one forward, two to the right, and two to the left, with different angles each. Additionally, it is an episode-based approach where each episode ends when the agent performs 500 steps.
Table 3 summarizes the main parameters of the algorithms used. However, we encourage readers to learn more about how the algorithms work by reviewing the papers referenced in this table.
5. Evaluations and Discussion
To evaluate the performance of cooperative navigation, two algorithms were trained in the same environment for 19 hours, where the objectives to be achieved were the same but were randomly selected. From the results shown in Figure 5, it can be interpreted that the lower the percentage of processing performed in each vehicle and the higher the percentage of processing completed in the cloud, the better the collaborative process performs.
The first scenario (decentralized) shows a 4.2% successful cooperation rate, where it is observed that although the vehicle uses all its processing capacity, it does not guarantee a collaborative agreement as each vehicle has its behavioral policy and acts according to it.
In contrast, in the second scenario (semidecentralized), it is observed that moving the policy training function to the cloud increases cooperation by more than 62%. Therefore, using a common behavior policy and 70% less central processing unit (CPU), this approach can achieve six times more efficiency than the decentralized one.
Next, we will analyze the network requirements of the navigation algorithm to show how these real-time services need 5G technology for deployment.
Previously, we have examined semidecentralized and decentralized architectures and their performance in transferring their functions to the cloud. At this point, we will focus only on the semidecentralized architecture, given its interaction between the cloud and the vehicle. Regarding uplink data transmission, it poses minimal challenges to the network, as it just sends the required input data to the neural network, which sums to a total of twenty-eight data points. In contrast, downlink transmission presents significant challenges, as the cloud continuously sends the behavioral policy, which encompasses the weights of each neuron in the neural network, to the vehicle. Consequently, the deeper the network, the greater the volume of data to be transmitted.
It is worth noting that, for the analyses, the limited data rates are set at 250 Mbps for 5G and 50 Mbps for fourth generation (4G), referring to the report “Benchmarking the Global 5G Experience” published in June 2022 [72].
In Figure 6(a), a scenario is analyzed in which up to fifty vehicles are connected to the network and execute up to ten actions per second; i.e., the vehicles will send data to the cloud up to ten times per second. Therefore, the results show that only 5G technology provides the performance needed to handle this number of vehicles and actions in the system. In particular, 5G supports fifty vehicles executing ten actions, while 4G can accommodate fifty vehicles with eight actions or forty vehicles with ten actions.
(a)
(b)
(c)
In contrast, Figure 6(b) illustrates that for the same number of vehicles and actions, 4G downlink technology can only support 40 vehicles with one action or five vehicles with nine actions. In contrast, 5G technology meets the downlink throughput requirements for up to fifty vehicles with up to four actions per second or twenty vehicles with ten actions. This underscores the superior capability of 5G to deploy cooperative services.
Note that although 4G can handle up to forty vehicles with ten actions, only 5G meets the network requirements for uplink in a system involving up to fifty vehicles and up to ten actions, considering the relatively small size of the transmitted information. However, if we increase the number of vehicles on the uplink, we obtain similar results to those observed on the downlink (see Figure 6(c)), with the difference that now the number of vehicles can reach five hundred, that is, ten times the number of vehicles.
Now, we will analyze the latency experienced; for this, it is essential to take into account the bandwidth differences between 5G and 4G. With a bandwidth of 100 MHz versus 20 MHz for 4G, 5G technology offers five times more bandwidth. Figure 7 illustrates the latency behavior as the number of vehicles increases. It can be seen that as the number of vehicles increases, the latency remains nearly unchanged in 5G and conforms to the requirements of this type of application, typically set at 10 ms. In contrast, in 4G, the latency does not meet the 10 ms requirement after thirty-two vehicles. Beyond this threshold, the latency starts to scale exponentially, reaching values of up to 125 ms in a system with about fifty vehicles. Note that the simulations depicted in the figure earlier were performed using a discrete event network simulator called NS3, where the main parameters established are described in Table 4.
6. Conclusions
The implementation of effective cooperative systems is a fundamental pillar for improving safety and optimizing traffic management on roads. It is imperative that vehicles have a thorough understanding of road dynamics and can accurately predict their future positions, taking into account the presence of other vehicles. To foster this cooperation, the integration of advanced AI algorithms is indispensable in the development of global optimal behavioral policies. However, the search for an optimal policy requires access to large amounts of data.
This challenge is compounded by the significant computational processing demands supported by each vehicle as it trains its own network and formulates independent decisions. To cope with this computational load and ensure data privacy, the advent of 5G networks is critical. Thanks to their ability to transmit large volumes of data with low latency, 5G networks can transfer computationally intensive functions to the cloud. Consequently, the requirement for high levels of intelligence in individual vehicles is mitigated, as evidenced by the increased performance of intervehicle cooperation demonstrated in Figure 5. Additionally, the semidecentralized architecture reduced the computational burdens required by the vehicles while increasing the efficiency of cooperative decisions by up to 79%.
Furthermore, Figures 6 and 7 show that only 5G networks meet the throughput and latency characteristics needed to support cooperative navigation among multiple vehicles. Therefore, the seamless integration of 5G networks with AI is a key factor in the success of cooperative navigation.
7. Future Challenges
Considering this is an incipient area of research, several research challenges have to be addressed to improve the performance of cooperative maneuvering algorithms between vehicles.
One is to test real scenarios using models trained on simulated replicas of the scenarios but using data from actual human drivers in simple and complex traffic scenarios. Also, design algorithms can adapt to different scenarios and actors on the road. Finally, parallel computing techniques should be explored to reduce computational and memory costs and speed up the training process.
Data Availability
The data used to support the findings of this study are available from the corresponding author upon request.
Conflicts of Interest
The authors declare that they have no conflicts of interest.
Acknowledgments
Elizabeth Palacios’s research was funded by the Research and Development Grants Program (PAID-01-19) of the Polytechnic University of Valencia.