[Retracted] Deep and Reinforcement Learning Technologies on Internet of Vehicle (IoV) Applications: Current Issues and Future Trends

Elmoiz Alatabani, Lina; Sayed Ali, Elmustafa; Mokhtar, Rania A.; Saeed, Rashid A.; Alhumyani, Hesham; Kamrul Hasan, Mohammad

doi:https://doi.org/10.1155/2022/1947886

Journal of Advanced Transportation

On this page

Abstract Introduction Conclusion Data Availability Conflicts of Interest Acknowledgments References Copyright Related Articles

Review Article Retraction

!

This article has been Retracted. To view the article details, please click the ‘Retraction’ tab above.

Special Issue

Computational Intelligence Systems for Vehicular Ad Hoc Networks

View this Special Issue

Review Article | Open Access

Volume 2022 | Article ID 1947886 | https://doi.org/10.1155/2022/1947886

[Retracted] Deep and Reinforcement Learning Technologies on Internet of Vehicle (IoV) Applications: Current Issues and Future Trends

Lina Elmoiz Alatabani,¹Elmustafa Sayed Ali,²Rania A. Mokhtar,³Rashid A. Saeed,³Hesham Alhumyani,³and Mohammad Kamrul Hasan⁴

Academic Editor: Muhammad Arif

Received17 Jan 2022

Accepted14 Feb 2022

Published15 Apr 2022

Abstract

Recently, artificial intelligence (AI) technology has great attention in transportation systems, which led to the emergence of a new concept known as Internet of Vehicles (IoV). The IoV has been associated with the IoT revolution and has become an active field of research due to the great need, in addition to the increase in the various applications of vehicle communication. AI provides unique solutions to enhance the quality of services (QoS) and performance of IoV systems as well. In this paper, some concepts related to deep learning networks will be discussed as one of the uses of machine learning in IoV systems, in addition to studying the effect of neural networks (NNs) and their types, as well as deep learning mechanisms that help in processing large amounts of unclassified data. Moreover, this paper briefly discusses the classification and clustering approaches in predicative analysis and reviews their abilities to enhance the performance of IoV application systems.

1. Introduction

Deep learning (DL) is a technology scheme used to carry out the intelligent machine-learning concept, which processes the data and creates patterns that make decisions according to the input data [1]. The term “deep” represents the multihidden layers in the learning network, which consists of several connected processing nodes. In recent years, DL has great evolution in helping to enhance many IoT applications [2]. DL improves the IoV quality of services for different related vehicular applications such as autonomous driving and transportation traffic control and smart cities. AI applications are used in many communication fields and in IoV particularly to meet the needs and expectations of users. Attributing the importance of using AI, this paper provides details about deep learning techniques to improve services in IoV applications.

The paper is organized as follows: Section 2 provides a background to the DL approach and its evolution . The concept of using machine learning (ML) algorithms in IoV systems are briefly introduced in Section 3. In Section 4, some of the DL applications in IoV networks are discussed considering issues such as security and collision prediction. In Section 5, the performance of deep reinforcement learning (DRL) in IoV is presented showing its impact to enhance some aspects related to energy efficiency, resource management, and performance optimization. Moreover, the performance of DL-based IoV network is reviewed and different DL-driven IoV network levels related to network control, data analysis, and regression are discussed. Finally, in Section 7, the paper reviews the future directions and challenges related to the deployment of the ML and DL algorithms in IoV applications. And then, the paper is concluded in Section 8.

2. Deep Learning Evolution

The concept of the DL scheme is related to the human brain, where the DL network enables the creation of a computation model accordingly. In the sixties’ era, DL was developed and released as an algorithm to contribute to developing data processing and forecasting methods [3]. The seventies era was the beginning of the deep learning revolution which show up the ability of convolutional neural networks (CNNs), to create a computer model with learning capabilities to recognize visual patterns [4]. In the nineties’ era, the back propagation (BP) for the learning model was used in practice, and later, the deep learning approach has been developed significantly by taking the speed abilities of computers and processors. With the entry of the millennium era, what is known as artificial intelligence was launched, which is used to train neural networks [5].

Currently, there is a growing need to increase the speed of processing to enable rapid data analysis by deep learning, in addition to prediction purposes [6]. By using DL, many IoT applications can be improved and contributed to enhancing different related IoT aspects such as signal analysis, pattern recognition, and quality optimization [7]. The DL structure is consisting of several parts beginning from data input, passed through feature extraction, learning, and ending by predicted output data, as shown in Figure 1. The concept of a DL model is located within the learning process using the simulation of the human brain function to recognize patterns.

Current IoT applications depend on portable and embedded devices and sensors, in addition to services that are provided through these devices. The use of deep learning on IoT devices introduces a new type of application that has the ability of complex sensing and task recognition, in addition to linking the devices with humans interactively [8]. Different DL frameworks have been proposed. An example of such frameworks is a deep sense which consists of recurrent neural network (RNN) and convolution neural network (CNN) integrated into one model to provide a feasible solution for the problematics related to learning multisensor blending tasks. The deep sense enables to split the input neural network layer into time intervals for processing to provide time-series data estimation and classification [9].

The deployment of DL in IoT as deep IoT is facing challenges due to resource constraint, which will compress the deep neural network. In the deep learning IoT framework, the structure drops the hidden elements in the neural network according to the regularization scheme of deep learning (see Figure 2). The dropping process is known as dropout, which represents the probability of dropping hidden elements during the learning process [10]. The applications of DL in IoV enable to achieve an intelligent capability for vehicle systems and to provide smart services. Moreover, DL helps to improve the processes related to self-driving and automation for vehicles in the IoV network, in addition to optimizing the data streaming quality of services between IoV networks and existing systems [11].

3. Machine Learning-Based Systems

In the past decade, there has been an increasing need to use machine learning (ML), especially with the continuous development of various interactive communication devices. The rapid and increasing growth of data and the need to analyze it has helped in ML algorithm development for analysis and building decision-making systems. ML algorithms helped to enhance different applications such as driving automation and behavior prediction [12]. In some applications, the ML process may take a long time for learning calculations; hence, the use of distributed systems enables to enhance the parallelization processing and bandwidth to reduce the long runtime of training models [13]. However, sometimes, the centralized solution is not feasible because of the big data storing impact to the machine. To train large datasets using machine learning, algorithms have developed with the ability to perform parallel computations and distribute data, with the flexibility to handle failure [14].

There is a challenge facing the design of a comprehensive system that allows machine learning distribution effectively. The challenge is due to the unique communication pattern of the algorithms [15]. As shown in Figure 3, there are two phases related to the ML to represent the training and prediction processes. The training phase is responsible for training a large amount of data by using a suitable ML algorithm based on the specific set of parameters and application purposes. After doing the process of the training phase, the ML extracts the exact output trained model; then, in the prediction phase, the predicted results come out [16].

In distributed ML, the parallel approach takes two phases, known as the data and model, which are applied simultaneously [17]. The data-parallel phase allows dividing the data in equal for working nodes during the same algorithm operation on different datasets, while in the model parallel phase, the nodes process a copy of entire datasets [18, 19]. Distributed machine learning systems can be divided into three main categories; they are, database, general, and purpose.

3.1. Database Systems

In ML algorithms, the use of ordinary database management systems (DBMSs) based on SQL cannot be served. Recent studies are trying to develop a database system that a user can execute machine learning within the DBMS [20]. Such studies are related to BISMARCK and MADlib databases. In BISMARCK, the execution of ML is based on gradient descent. MADlib extends the extensions to SQL to allow users to perform machine-learning procedures built into databases [21].

3.2. General Systems

For distributed computing, the use of message passive-interface (MPI) framework enables to provide high performance of computations. MPI provides many primal operations, such as broadcasting and scattering, in addition to sending and receiving. Moreover, it enables to let users execute many applications for ML. However, it can be susceptible to error and labor exhaustive [22]. Another popular system in operation is Hadoop, which is an open-source system designed to execute large clusters of commodity machine workflows. Hadoop allows services such as automatic fault tolerance and simple programming, allowing users to analyze the large scale of data across many machines. However, it cannot support or provide iterative workflows, requiring the submission of a single job for every application [23].

3.3. Purpose-Built Systems

The purpose-built systems provide one of two options, machine learning domain-specific languages or algorithm optimization [4]. System ML enables a high-level language to provide R-like syntax programming used in data analysis, in addition to allowing built-in operators to perform matrix operations. Workflows are transformed into map-reduce jobs and rearranged to avoid multiple passes over input data. The Opti-ML provides a linear algebra-based language for scale embedded and domain specific [24]. It includes vector, matrix, and graph data types along with subdata types that allow additional optimization [25]. This is to summarise the available systems and is not intended to include all systems. Distributed machine learning is intended to allow users to conclude their desired results from massive datasets in a noticeably short time with the goal of resource optimization [26].

4. Deep Learning Applications on Internet of Vehicles

In recent years, IoV gained attention as a revolution in intelligent transport systems (ITSs) since it provides a great service in our lives [27]. The IoV deployment is very complicated and needs special considerations for special characteristics related to high mobility and dynamic change in topology. Due to this complication, standardization in communications is very important. Figure 4 shows the architecture standardization for VANET applications by the European Telecommunications Standards Institute (ETSI) [28].

In the ETSI architecture, the accommodation layers are accountable for handling the VANET-associated applications such as cooperative awareness messages (CAMs), decentralized notification messages (DENMs), and local dynamic maps (LDMs) as well as the communication process. Network and transport layers are combined in one layer; in addition, two additional layers were added to represent the management and security. Moreover, ITS dedicated stack integrating the geonetworking and addressing is updated [29, 30]. Nevertheless, such architecture does not clarify the interaction handle in case of handoff and other components that include outside gadget interaction. Figure 5 shows the four Cisco layer architecture [31].

IoV contains different communication application scenarios such as device to device (D2D), vehicle to vehicle (V2V), vehicles to the roadside unit (V2R), vehicle to infrastructure (V2I), roadside to roadside (R2R), vehicle to everything (V2X), vehicle to sensor (V2S), and vehicles to personal devices (V2P), in addition to manyother structures [32, 33]. Much novel research has been conducted using AI approaches and DL mechanisms in IoV-based services for different IoV aspects such as security, cloud-based IoV, and collision production, which were discussed in brief in the following sections.

4.1. Internet of Vehicle Security

Protecting IoV from attacks became noticeably important because of the considerable IoV growth. Different studies have been proposed for the use of ML in IoV security. Most recent studies propose the combined system containing AI approaches with IoV [34]. ML and DL are the most famous algorithms that provide a secure IoV. They enable the optimization of the IoV security system for security applications and services [34]. In IoV computing, the area security assurance within the versatile edge computing environment is very important and needs mechanisms to secure the scheduling collaborative resources [35, 36].

The use of deep learning methods enables the development of IoV networks decide the legitimacy of the watched information stream and to recognize potential security dangers [37]. For security purposes, blockchain in IoV enables progressing vehicular GPS situating precision, framework vigor, and security. Security thinking is concentrated on strength and security in terms of vehicular situating, data exchanging, and sharing [38]. Table 1 summarizes the most important studies related to AI approaches to use in IoV applications.

George et al. demonstrated an application to offload an IoV system through a persistent deep learning outage detection task. The creators utilize both a profound multilayer perceptron and repetitive neural arrange engineering to learn the worldly setting of distinctive assaults. The proposed study determines when the computation can be discharged with useful considering parameters related to network operation and processing requirements of the deep learning model.

Sharma et al. introduced the concept of V2X infrastructure to provide optimal and reliable communication services in smart cities. The paper touched on the possibility of using smart spectrum security systems (SSU) in IoV networks. The authors also presented a security system using deep learning that develops secure applications with high reliability. Deep learning works in the IoV system to monitor security threats. The proposed system has led to high performance in terms of monitoring accuracy and security. In a study proposed by Berger et al., the researchers presented several machine learning and deep learning methods and used them in the development of smart vehicle systems to improve the protection capabilities of the observed data flow and to identify the capabilities of potential security threats.

In a study by Pang et al., the researchers demonstrated a mechanism for collaborative scheduling of computing resources for Internet of vehicles so that privacy is adopted and protected in the edge computing environment during the movement of vehicles. The researchers adopted the use of a multiuser and multiregion MEC server system to enable vehicles to offload computing tasks to MEC servers in different regions. Researchers focused on developing a solution to the problem of revealing location privacy to the vehicle user in which the dual DQN algorithm was used to solve the optimal scheduling strategy to reduce the total cost of system consumption. Through experiments, the researchers concluded that the proposed system gives optimum performance by comparison with other scheduling algorithms.

In a study by Yanxing et al., the researchers presented several current collaborative positioning techniques that help improve the accuracy of vehicle positioning, where a new model of Internet of vehicles based on blockchain technology has been proposed to improve the accuracy of positioning in addition to increasing the degrees of safety. In this study, a deep neural network (DNN) algorithm was used to correct the positions of vehicles, and then, a blockchain system was used to protect the communications between smart vehicles, CoVs, and roadside units. The proposed model showed high accuracy in the identification and security of information transmission of the vehicle network.

Teodora and Nikolay showed the common diagrams of the ITS design and security issues. In expansion, the creators explore inventive approaches such as blockchain, sprout channel, haze computing, manufactured insights, diversion hypothesis, and ontologies. Those creators recently tried to figure out some intelligent ITS security procedures.

A study presented by Elmustafa et al. provides details about the security issues in the IoV edge computing offload model and the considerations related to the QoS requirements. The paper reviews most artificial intelligence approaches such as ML and DL algorithms and their impact on securing the IoV network. Praneeth et al. presented an intrusion prevention system (IPS) model based on the DL approach in the networks of the cognitive IoV by using binary classification that enables the identification of malicious packets. The model trains and tests packages in a cloud service for an open platform and validates the proposed prevention classifier model using a simulation dataset. Through the results, the researchers concluded that the proposed model gave an accuracy of 99.57% of the experiment compared to other smart models such as RNN and CNN.

Hasan et al. presented several concepts related to the use of smart cloud computing technology and its uses in decisions to improve road transport safety. The researchers also discussed what is known as cognitive Internet of vehicles (CIoV). The research discussed several axes related to the possibility of avoiding cybersecurity problems in IoV and cognitive design mechanisms to overcome potential security, privacy, and trust concerns. A study provided by Hbaieb et al. provided a comprehensive review of IoV management resources used to enhance the users’ QoS. The review gives technical concepts related to different IoV communication models and in addition to security issues.

4.2. Cloud-Based Mobile IoV Framework

Deep learning studies have been converted to be a very popular technology in image and information processing, in addition to other applications where smart data manipulation is needed [48]. Mobile cloud computing gives an improvement in results related to the training process and model repository in cloud platform transferring. It enhances the computational processes to the cloud making it faster. Moreover, it provides more security in data gathering processes that are transferred to mobile devices [49]. Deep learning topics have become a powerful technology for mobile cloud frameworks. Table 2 summarizes some of the studies related to the use of AI algorithms for cloud-based IoV frameworks.

In a study presented by Zhang et al., the authors presented a framework for scientific planning of allocating computing resources in edge computing for mobile IoV networks. The researchers used a deep reinforcement-learning (RL) network to improve the computing power of the service nodes and the speed of movement of the vehicle [50]. They also used Q Learning on a deep RL network to improve the stability of the neural network, where through the analysis, the researchers concluded that the proposed model gives an effective performance in managing the computing resources of IoV.

Saeed et al. provided a framework for the scientific planning of computing resource allocation in the edge computing of mobile IoV. The study presented a new model to enable smart resource allocation for mobile IoV networks based on smart computing. In this study, deep RL was used to improve computing efficiency, as the presented model gave a significant improvement in the resource management process.

According to what was presented by Grigorescu et al., the architecture of cloud computing systems based on deep learning is subject to cloud-level preparation and storage processes. ML empowers acknowledgment preparation and information gathering in versatile gadgets. The creators proposed such a system and communications to guarantee the victory of information transmission in unsteady arrange situations [51]. The structure identifies objects in recorded recordings amid driving, and the ML will give outflank discovery rate.

The proposed system by Claudio et al. offers context-aware genuine time and group administrations at the IoV edge [52]. The study creates a common ML-based system that leverages manufactured insights to estimate future activity requests and characterize activity highlights. This system can avoid the abuse of IoV edge computing activity requests and take strides in implementing basic controls in IoV arrangement. The proposed system can coordinate ML to move forward two distinctive organized instruments.

Ning et al. proposed a new offloading framework based on three layers to reduce the energy consumption in IoV. The study uses deep reinforcement learning to provide an optimal solution for offloading decisions. The framework is evaluated in real applications in China. The results show that the proposed idea is able to reduce the consumption of energy by 60% compared to other baseline algorithms. Razi et al. explored the information at the edge hubs to empower fog-based benefit. The creators emphasize the challenges included in advertising the context-aware administrations in an IoV environment [56]. The concept introduces an information analysis system for haze frameworks at the haze layer of conventional IoV engineering.

For self-driving cars and independent vehicles, Chen et al. displayed a novel system for creating AI deduction motors for independent driving applications based on learning modules. The proposed system empowers to prepare assignments that are sent flexibly over both cloud and edge assets, with the reason of lessening the desired organized transmission capacity, as well as moderating security issues [55]. The system demonstrated a compelling of AI deduction motors for independent vehicles, for environment discernment, and most likely way expectation.

4.3. Collision Prediction

Deep learning permits computational models to memorize representations of information with numerous levels of reflection. Table 3 shows the comparison between different DL applications for IoV collision prediction. In a study presented by Gupta et al., applications of deep learning for object detection and scene perception to avoid collisions in self-driving vehicles are shown in [57]. The authors discuss the gap between deep learning and self-driving cars and solutions to address image perception problems in real-time driving in autonomous driving applications.

In the study by Santos et al., several factors affecting the use of accident data in the Portugal region were identified as an example, where the researchers discussed how to address and develop smart models that classify the severity of traffic accidents [58]. The researchers also presented a proposal for a model that predicts future accidents based on antecedent data by using supervised ML and DL mechanisms.

In some IoV applications, irregular tunnel environments require an intelligent vehicle driving system. Fayyad et al. presented a real-time location strategy for autonomous vehicle driving on short range. The study reviews the use of the DL learning strategy to identify the vehicle locale within local vehicle environments.

Chang et al. presented a profound learning-based Web of Vehicle (IoV) framework known as deep crash [58]. The proposed framework enables the identification of collusion and the transmission of unfortunate location data to the cloud-based database server to enable autonomous collision vehicle fault acknowledgment. The evaluation of the deep crash system results in high traffic collision detection accuracy up to 96%, with a 7-second average response time.

In research by Chunjiao et al., the authors pretested a DL framework for the IoV system. The DL helps to detect traffic accident collisions in the IoV network and provides emergency announcements [61]. The study introduces the concept of implementing DL for driver assistance, which helps to avoid drowsiness driving accidents. The authors are benefited from the theory that shows the relationship between the feeling of sleepiness during long trips and the concentration of carbon dioxide in vehicles. The proposed approach gives up to 84.58% detection accuracy for traffic accident collisions.

5. Analysis of IoV Network Based on Deep Reinforcement Learning

In general, accidents happen by driver’s miscalculation in driving and mistakes. ML in IoVs provides solutions to raise the level of the driving experience as one of the most important requirements for smart city applications. The technology of IoV is continuously developing technologies and architectures to reduce traffic congestions and accidents transforming vehicles from regular driving tools to intelligent tools [60]. Reinforcement learning provides an architecture where a network learns from previous experience to increase some reward signal. In learning to control, an agent directly receives high-dimensional input such as voice and video in an extremely monotonous task known as the curse of dimensions. Many suggested solutions have been proposed for this dimension’s issue such as the use of linear function approximation, hierarchical representation, and state aggregation [62].

For decision making of multistage process in a probabilistic environment, the control process must be accurate. The use of the Markov decision process (MDP) enables the provision of descriptive formal standard methodology as a discrete-time stochastic control process. Accordingly, the process is in a state x at each time step so that it allows a decision maker to move the process to the new state to give a corresponding decision to reward . The possibility of transition of the process to the new state is determined by the given act according to the state transition function to fulfil the Markov property.

During the MDP process, a problem of finding decision maker policy may appear because of the total independence of past states concerning the current state x and next state [31]. By defining the function for a specific action to the decision maker for choosing when the state is x, we can derive the purpose of MDP that has to find a policy for to maximize the cumulative function by the following equation [63]:where n represents the time step. Denotes discount factor between 0 and 1. By setting the factor, the process of the decision can be specified. In the case of = 0, it will make the agent short sighted, and if the value is 1, this makes it look to the future high reward.

The operation of classic MDP enables to enhance the decision through two approaches, the value iteration or policy iteration. However, this enhancement takes place according to the assumptions that the decision maker has an accurate value for the transition function and reward for all states in the application environment [64]. While in the actual operation, the decision maker cannot recognize the transition function. Accordingly, the use of Q learning helps to overcome such problems.

Q-learning (QL) is a form of RL that enables to teach the decision maker how to behave in an MDP environment when the transition function and/or reward are unknown [65]. In this approach, each state is assigned as an initial Q value, which can be estimated by an online incremental update stochastic QL algorithm, as shown in the following equation:where represents the single-step reward. denotes step-size learning rate between 0 and 1. In the case of , no Q values are updated, and no learning process is occurred [66].

The previous approach is known as the value-iteration algorithm which fulfils the optimal action-value function as . Dimensional curse in the classical QL algorithm is occurred due to a large amount of time to converge. To overcome this issue in MDPs, functional approximation techniques can be used.

The use of neural networks helps to find good features for high-dimensional input data. It can be represented within the value of action function by considering the current system state and action in input to obtain suitable Q-value output. This approach is representing the concept of deep RL. The QL with weight is trained to learn the parameter of the action function by reducing the sequence of the loss function . The is stated in the following equation [67]:where is the neural network parameter. The part is the target for iteration i.

The goal is to save the cost expression as small as possible by using a gradient descent algorithm, which repeats the computation of the gradient expressed in the following equation [68]:

The gradient descent algorithm becomes slow when processing huge datasets; however, the use of stochastic gradient descent (SGD) will help to reduce the problem of slow processing [69]. The following sections show the impact of the RL algorithms in IoV networks related to energy efficiency and overall performance optimization.

5.1. Scheduling Algorithms for Energy Efficiency

The ITS applications based on vehicular communications and IoV may consist of different vehicular networking architectures such as V2V or V2I which are possibly taking information exchange by the roadside units (RSUs). The use of the V2I communication structure enables the provision of a reliable and efficient driving experience and safety for vehicular communications [70]. The data exchanged in such a structure are scheduled between vehicles and RSUs with a policy to enable V2I communication management to preserve the batteries in the RSUs and prolong the lifetime of the network. Accordingly, the V2I communication will ensure better quality-of-service (QoS) levels [71]. The deployment of RSUs depends on the battery recharge in time intervals using energy harvesting via solar or by physically recharging.

Deep reinforcement learning (DRL) based-scheduling scheme enables to manage and organize the communications in RSUs, and to enhance the lifetime of IoV as well. Scheduling DRL approach is similar to action value function (AVF) and Q-learning algorithms in terms of simplicity and robustness. The use of a scheduling policy framework helps to optimize the energy efficiency in IoV, and accordingly, the task scheduling system, as shown in Figure 6, enables to provide an energy-efficient overall system that increases the lifetime of the network [72].

In Figure 6, the task scheduling is defined according to the time until the IoT gateway is cutoff considering vehicle speed. As observed from the figure, a deep Q network (DQN) enables the balance of the power consumption in IoV networks [73]. Compared with other algorithms, DQN gives better performance. The performance of the random vehicle selection algorithm and prioritizing departing vehicles (PDVs) is constant for all considered vehicle speed. The two other algorithms greedy power conservation (GPC) algorithm and greedy prioritize departing vehicle (GPDV) algorithm give an average network lifetime and keep a convergent performance for all ρ vehicle speed. In general, the task decorrelation is the stage responsible for the scheduling policy to organize tasks and reduce the power consumption [47].

5.2. RL Algorithms for Resource Management

The provisions quality of service (QoS) and quality of experience (QoE) are the most urgent metrics for vehicle users, which are required to maintain at acceptable levels. Thus, many resource management algorithms have anticipated for IoV communications to improve the network performance. Software-defined network-based resource management algorithms were introduced to ensure vehicle users’ QoS [74]. There is a need to provide highly efficient and reliable communications such as device-to-device communication using V2V with the possibility of improvements by reducing transmission delay and power consumption while providing local packet distribution, improving spectrum efficiency, and reducing gain characteristics [75].

An efficient transfer AC (ETAC) learning method enables the achievement of an intelligent resources’ allocation in the IoV network. Intelligent resource management is important to empower the V2X communication with intelligent decision making and satisfy multiple QoS requirements such as latency and reliability requirements for V2V communication, in addition to ensuring suitable data rate requirements for V2I channels. RL, as a solution that enables the adoption of the Markov decision process and with the actor-critic (AC) framework, will provide an intelligent resource allocation mechanism. This mechanism enables a new Markov reward function to allocate smart resources in the IoV network mentioned in the following equations [44]:where represents the sum data rate, and pare instant cost function. The coefficients are the weights of , , and and are used to balance the utility and cost. represents the D2D-V2V pair sets. In the IoV networks, each agent chooses a policy to capitalize the reward representing that the state-action function is a summed discount return for starting the network s with an assumed policy stated in the following equation [46]:

Intelligent resource management aims to find the policy best describe the maximization of network objective reward, stated in the following equation:where represents the state spreading function, and represents the stochastic policy with the state s with completed the current action a. It reveals the conditional probability density of the action (a) at the state (s). Given the above situation, the policy improved numerically by applying the value iteration methods with the RL algorithms [76].

The probability of satisfied V2I and V2V be increased by the contribution of ETAC and obtains absolut speed of the vehicle and considering the variation of device vehicle users within D2D-V2V communications. ETAC outperforms other RL algorithms, as shown in Figure 7.

The AC agent might use to apply intelligent resource management to satisfy the user of the vehicle QoS. The critic process: the aim of the critic process of the AC learning agent is to assess the policy quality that the learning system is supposed to search for [45]. The player process: the stochastic-policy-gradient (PG) technique usually employs the player part to parameterize policies to improve the policy thoroughly to enhance the goal function in the following equations:where is the future vector. Next, the policy parameter vector is updated based on the gradient of the objective reward given in the following equation [43]:where is actor-learning rate.

ETAC enables the vehicle users to import learned strategies from experience VUE and gives an independent learning process. The employment of the parameter vector ν is rationalized in the critic part. The policy is rationalized by the actor [77].

The problems with the standard AC approach can be addressed by pursuing qualifications to improve the efficiency of the learning setting. The feature of coordinating the work with the inaccurate parameter reduces the fluctuation with angle. The activity methodology helps to enhance the overall quality of learning by increasing the speed of convergence [56, 78]. Performance evaluation of the ETAC is shown in Figure 8. By observing the varying number of D-VUEs, it is clear that ETAC gives better results compared to other reinforcement learning approaches with an improved sum rate.

5.3. Performance Optimization Algorithms

In the IoV network structure, a huge amount of data is exchanged among different IoV vehicles, RSUs, and infrastructure. The data should be stored and interchanged in a secured environment to enhance traffic safety and efficiency. Two types of operations are considered in IoV network data storage and sharing. A blockchain system with reinforcement learning will help to optimize the performance of the IoV network [79].

The blockchain framework consists of two main parts. The first part, block producers, is related to the number of node assumptions, block producers, candidates block, a set of nodes, and computational resources. The second part, consensus models, applies the practical byzantine fault tolerant (PBFT) algorithm for consensus considered as a very robust protocol [61, 77, 80]. The client issues a block with the transaction number stored or shared and validated by broadcasting it to other validators to have consensus. The process involves exchanging and verifying messages, as shown in Figure 9. The time-varying transmission links are modeled through the finite-state Markov chain (FSMC) [80].

Among different schemes applying blockchain systems, the average latency/time to finality (TTF) decreases with the increase of average computational resources because an agreement can become more rapidly among more computationally competent validators [82].

6. Deep Learning-Based IoV Network Performance

IoV networks that provide high mobility broadband access have gained much attention from industry and scientists. Due to the increasing number of traffic accidents, there is an urgent need for faster and smarter analytical systems; thus, many scientists provide much research in this field, employing deep learning techniques for its proven results and high performance in areas of collision detection, analysis, and notification [83]. Like the OSI model, the IoV architecture consists of multiple layers such as network and application layers, which depend on the intelligent transportation system (ITS).

6.1. Deep Learning-Based IoV Network-Level Data Analysis

DL approach enables to the development of a multilayer architecture that includes intelligent computing and big data analytics with IoV dimensions. IoV dimension is divided into layers to support the operations related to data analysis [84]. These layers are stated as follows:(i)Perception layer consists of most IoV devices such as sensors, actuators, vehicles, and smart mobiles(ii)Infrastructure network layer contains the main IoV infrastructure components such as RSUs and base stations, which are considered as the backbone of the IoV network(iii)Artificial intelligence layer enables the procedure related to computational intelligent algorithms and architectures(iv)Communication layer provides the required communication technologies such as 5G and 4 G/LTE [85].

Various DL techniques are used for data analysis image classification. Deep learning proposing inception v3 network architecture is adopted for binary image classification to achieve the best image classification results. [35]. Inception v3 employs two deep learning network architectures, densely connected convolutional networks (DensNet) which increase the depth of high-dimensional neural networks. In addition, squeeze-and-excitation networks (SENet) are responsible for filtering the last output feature to cancel or remove features that are not needed for the current task [85].

6.2. Deep Learning-Driven App-Level IoV Data Analysis

The function of the application layer is to provide data and control the APIs to allow applications to use the distributed data store and to determine the configuration of the deployment. This layer is responsible for supplying the IoV network with applications through the common communication architectures in IoV [87]. Real-time data gathering and analysis are required in IoV networks to ensure immediate and appropriate actions; for example, an ambulance needs real-time data on traffic information to avoid traffic jams and save lives.

Maintaining the QoS requirements is crucial in the IoV applications to improve the operation of IoV services. At the application layer, the web services in the service layer provide applications by subscribing data deployment strategies [88]. Such strategies are published/subscribe strategy access, which lets applications register their interest to notify about the service updates. In general, the applications can be categorized into three main sets, real time (RT), batch information required-based application, and near real time (NRT).

The strategy of push and publish is considered the best choice for RT and batch applications. While for the near real time (NRT), pull or subscribe is the best fit. The mentioned strategies used with the DL approach enable to enhance the adaptive-traffic message-scheduling application. Historic data is required by the batch data requirement; thus, it has a low QoS requirement in terms of latency [89]. Process offloading and safety applications as real-time QoS requirement are needed for real-time applications; these applications help in complex data processing such as video processing on the IoV network.

6.3. Deep Learning-Driven IoV Network Control

In 5G-based IoV communication, the networks consist of both wired and wireless networks that needs an intelligent method to deal with the huge growth in the data traffic of the network for their varying nature of network sceneries such as cloud computing, mobility, and big data manipulation [90]. To make IoV networks smarter, DL enables to learn how to route network traffic between routers and optimize the network performance. DL has shown promising results in network control, by dividing the framework into phases. Phase 1 is the initial phase, where the related data are obtained for training the deep system. Phase 2 is the training phase that depends on collected data by the previous phase, applying supervised learning algorithms to train the DL-based architecture. Phase 3 represents the running phase where we put the learned algorithm into action where traffic patterns are used as an input that outputs a new path or route after learning the network behaviour based on deep learning algorithms [91].

The virtualization approach is a new advancement to the IoV network to have more control over the network resources and to elevate the network performance. The sole source input/output virtual (SS-IOV) environment is proposed to allow I/O devices utilized by VMs with no decrement in the run time throughput of the overall network. SR-IOV is capable of creating virtual functions that allow guests to have direct access [35]. It is an efficient network device that provides the benefit of input/output performance and decreases central processing unit (CPU) employment though significant scalability increment and the sharing abilities of the device [92].

6.4. Predictive Analytics “Regression” Problem in Deep Learning

ML uses supervised learning in having an input variable x and output variable y using a learning algorithm in between to study the input/output function of mapping . The ML process simply having inputs and the output is predicted after the learning process is completed.. Approaches to supervised machine learning include linear and logistic regression, classification, decision tree, and support vector machine [93]. Regression states the operation of modelling the association among one or more autonomous variables and a dependent variable; its main objective is prediction. Problems usually occur when regression is supposed to predict numerical values such as prices, the number of vehicles, predicting demand forecasting, and the length of waiting at a queue.

6.4.1. Simplified Dependent and Independent Variable Regressions

These models give conventions about the average functions expressed by and the variance function, with the assumption that for some transformation, can be written as

Stating the relation between y and u, simple linear regression is operating with only one predictor.

6.4.2. Multiple Regression

Problems of regression that occurred have triggered the need to consider operating with more than one predictor, having the dependent variable y that depends simultaneously on predictors x₁, x₂, …, x_n. We begin with n predictors x₁, …, x_n building a set of k terms based on the predictors u_i-1, u_i, …, u_k-i to have the multiple linear regression mean function [93].

DL architectures enable to solve the regression problem, proven for its demonstrated execution in the vision of computer errands such as picture categorization, question discovery, and picture examination [95]. The common engineering comprises several convolutional layers, taken after a few fully connected layers and a classification delicate max layer [96, 97]. Deep regression algorithms have given results in traditional vision relapse issues such as human posture estimation and a facial point of interest discovery.

7. Machine Learning-Based IoV Future Trends and Deployment Challenges

By 2030, it is expected that the services related to autonomous vehicles will become fully automated, covering most of the applications related to intelligent driving and transportation safety [98]. This expectation will soon become a reality by using machine and deep learning mechanisms, which play an important role in developing most applications related to automated driving. These mechanisms have the ability to build algorithms that carry out intelligent learning and predictions related to network security, control, and resource management.

With the increase of technologies in the future, vehicle networks and the links between communications, computing, and resource management become more complex, and most recent studies have tended to use deep learning techniques to improve the quality-of-service requirements [42]. When considering how to apply deep learning mechanisms to IoV applications, researchers face a few challenges that represent a rich environment for future research. These challenges are related to complications related to big data computations, control processing, and resource management, which require extensive improvements to obtain a coherent system that meets service quality and user quality requirements as well.

8. Conclusion

The fast development of DL approaches, models, and frameworks with the thought of quick development of IoV applications and administrations activated more and broad inquires about to progress the way of life. Profound learning scheme for IoV is discussed in this paper counting convolutional, deep reinforcement learning (DRL), conventional neural network (CNN), and recurrent neural networks (RNNs). These schemes have demonstrated a tall execution when connected to IoV applications and administrations, i.e., in independent driving, activity checking, mishap avoidance, activity direction frameworks, secure route, electronic fee collection, and secure route. In the future, the ML guarantees to optimize the arrangements to the Qo/QoS for perceptive future IoV systems. The enhancements will upgrade information-gushing quality for amusements and activity administrations. [99] In any case, the sending of ML on IoV will confront diverse challenges due to the huge sum of traded information and distinctive asset availability.

Data Availability

The datasets and codes generated and/or analyzed during the current study are available from the corresponding author on reasonable request.

Conflicts of Interest

The authors declare that they have no conflicts of interest.

Acknowledgments

This research was supported by Taif University Researchers with Supporting Project number (TURSP-2020/216), Taif University, Taif, Saudi Arabia.

References

D. Shetty, C. A. Harshavardhan, M. Jayanth Varma, S. Navi, and M. R. Ahmed, “Diving deep into deep learning: history, evolution, types and applications,” International Journal of Innovative Technology and Exploring Engineering (IJITEE) ISSN, vol. 9, no. 3, pp. 2278–3075, 2020.
View at: Google Scholar
S. Yao, Y. Zhao, A. Zhang et al., “Deep learning for the internet of things,” Computer, vol. 51, no. 5, 2018.
View at: Publisher Site | Google Scholar
R. A. S. Al-Sakib Khan Pathan, M. A. Feki, and N. H. Tran, “Integration of IoT with future internet,” Journal of Internet Technology, vol. 15, no. 2, 2014.
View at: Google Scholar
J. Verbraeken, M. Wolting, J. Katzy, J. Kloppenburg, T. Verbelen, and S. Rellermeyer, “A survey on distributed machine learning,” 2019, https://arxiv.org/abs/1912.09789.
View at: Google Scholar
S. Ibrahim, R. A. Saeed, and A. Mukherjee, “Resource management in vehicular cloud computing,” The Book with Title Vehicular Cloud Computing for Traffic Management and Systems, IGI Global, Hershey, PA, USA, 2018.
View at: Publisher Site | Google Scholar
D. Amodei, “Deep speech 2: end-to-end speech recognition in English and Mandarin,” Learning, JMLR: W&CP, vol. 48, 2016.
View at: Google Scholar
M. Bojarski, D. Del Testa, D. Dworakowski et al., “End to end learning for self-driving cars,” 2016, https://arxiv.org/abs/1604.07316.
View at: Google Scholar
A. E. Khandani, A. J. Kim, and A. W. Lo, “Consumer credit-risk models via machine-learning algorithms,” Journal of Banking & Finance, vol. 34, no. 11, pp. 2767–2787, 2010.
View at: Publisher Site | Google Scholar
D. Peteiro-Barral Bertha Guijarro-Berdiñas, “A survey of methods for distributed machine learning,” Progress in Artificial Intelligence volume, vol. 2, pp. 1–11, 2013.
View at: Google Scholar
J. Qiu, Q. Wu, G. Ding, Y. Xu, and S. Feng, “A survey of machine learning for big data processing,” EURASIP Journal on Applied Signal Processing, vol. 67, 2016.
View at: Publisher Site | Google Scholar
M. Abdelgadir, R. A. Saeed, and A. Babiker, “Mobility routing model for vehicular ad-hoc networks (VANETs), smart city scenarios,” Vehicular Communications, vol. 9, pp. 154–161, July 2017.
View at: Publisher Site | Google Scholar
L. Liu and M. Tamer, Özsu. Encyclopedia of Database Systems, Springer Nature, Berlin, Germany, 2018.
N. M. Elfatih, M. K. Hasan, Z. Kamal et al., “Internet of vehicle’s resource management in 5g networks using ai technologies: current status and trends,” Wiley, Hoboken, NJ, USA, 2021.
View at: Publisher Site | Google Scholar
S. Ruder, “An overview of gradient descent optimization algorithms,” 2017, https://arxiv.org/abs/1609.04747.
View at: Google Scholar
J. Duchi Elad Hazan Yoram Singer, “Adaptive subgradient methods for online learning and stochastic optimization,” Journal of Machine Learning Research, vol. 12, 2011.
View at: Google Scholar
M. D. Z. Adadelta, “An adaptive learning rate method,” 2012, https://arxiv.org/abs/1212.5701.
View at: Google Scholar
D. P. Kingma and J. L. Adam, “A method for stochastic optimization,” 2017, https://arxiv.org/abs/1412.6980.
View at: Google Scholar
R. C.-H. Hsu Shangguang Wang, “Internet of vehicles – technologies and services,” in Proceedings of the 1st International Conference, Beijing, China, September 2014.
View at: Google Scholar
Z. Ahmed, R. A. Saeed, and A. Mukherjee, “Vehicular cloud computing models, architectures, applications, challenges and opportunities,” The Book with Title Vehicular Cloud Computing for Traffic Management and Systems, IGI Global, Hershey, PA, USA, 2018.
View at: Google Scholar
S. Sharma, K. K. Ghanshala, and S. Mohan, “A security system using deep learning approach for internet of vehicles (IoV),” in Proceedings of the 9th IEEE Annual Ubiquitous Computing, Electronics & Mobile Communication Conference (UEMCON), New York, NY, USA, November 2018.
View at: Google Scholar
M. Abdelgadirand and R. A. Saeed, “Evaluation of performance enhancement of OFDM based on cross layer design (CLD) IEEE 802.11p standard for vehicular ad-hoc networks (VANETs), city scenario,” International Journal of Signal Processing Systems, vol. 8, no. 1, 2020.
View at: Publisher Site | Google Scholar
Y. Piyush and D. Sarkar, “Traffic prediction framework for open street mapusing deep learning based complex event processing and open traffic camera,” 2020, https://arxiv.org/abs/2008.00928.
View at: Google Scholar
C. Chen, H. Xiang, T. Qiu, C. Wang, Y. Zhou, and V. Chang, “A rear-end collision prediction scheme based on deep learning in the Internet of Vehicles,” Journal of Parallel and Distributed Computing, vol. 21, 2017.
View at: Google Scholar
K. O’Shea and R. Nash, “An introduction to convolutional neural networks,” 2015, https://arxiv.org/abs/1511.08458.
View at: Google Scholar
S. Albawi, T. A. Mohammed, and S. Al-Zawi, “Understanding of a convolutional neural network,” in Proceedings of the 2017 International Conference on Engineering and Technology (ICET), Antalya, Turkey, August 2017.
View at: Publisher Site | Google Scholar
J. C. Principe, N. R. Euliano, and W. Curt Lefebvre, “Neural and adaptive systems: fundamentals through simulation,” Wiley, Hoboken, NJ, USA, 1997.
View at: Google Scholar
S. Yoon and D. Kum, “The multilayer perceptron approach to lateral motion prediction of surrounding vehicles for autonomous vehicles,” in Proceedings of the 2016 IEEE Intelligent Vehicles Symposium (IV), Gothenburg, Sweden, June 2016.
View at: Google Scholar
M. D. Ferreira, D. Cristina Corröea, L. G. Nonato, and R. Fernandes de Mello, “Designing architectures of convolutional neural networks to solve practical problems, to appear in: expert systems with applications,” Expert Systems with Applications, vol. 94, 2017.
View at: Google Scholar
A. Islam, T. Hossan, and Y. M. Jang, “Convolutional neural network scheme–based optical camera communication system for intelligent Internet of vehicles,” International Journal of Distributed Sensor Networks, vol. 14, no. 4, 2018.
View at: Publisher Site | Google Scholar
R. DiPietro and G. D. Hager, Deep Learning: RNNs and LSTM, Handbook of Medical Image Computing and Computer Assisted Intervention, Elsevier, Hoboken, NJ, USA, 2020.
K.-L. Du and M. N. s. Swamy, Neural Networks and Statistical Learning, Springer link, Berlin, Germany, 2019.
S. Virmani and S. Gite, “Performance of convolutional neural network and recurrent neural network for anticipation of driver’s conduct,” in Proceedings of the 8th International Conference on Computing, Communication and Networking Technologies (ICCCNT), Delhi, India, July 2017.
View at: Google Scholar
A. A. Eltahir and R. A. Saeed, “V2V communication protocols in cloud assisted vehicular networks,” The Book with Title Vehicular Cloud Computing for Traffic Management and Systems, IGI Global, Hershey, PA, USA, June 2018.
View at: Publisher Site | Google Scholar
E. Sayed Ali Ahmed, Z. T. Mohammed, M. Bakri Hassan, and R. A. Saeed, “Algorithms optimization for intelligent IoV applications,” Handbook of Research on Innovations and Applications of AI, IoT, and Cognitive Technologies, IGI Global, Hershey, PA, USA, pp. 1–25, 2021.
View at: Publisher Site | Google Scholar
Z. Ning, P. D. Senior, X. Wang et al., “Deep reinforcement learning for intelligent internet of vehicles: an energy-efficient computational offloading scheme,” IEEE Transactions on Cognitive Communications and Networking, vol. 5, no. 4, 2019.
View at: Publisher Site | Google Scholar
R. F. Atallah, C. M. Assi, and M. J. Khabbaz, “Scheduling the operation of a connected vehicular network using deep reinforcement learning,” IEEE Transactions on Intelligent Transportation Systems, vol. 20, no. 5, 2018.
View at: Google Scholar
R. O. Dirar, A. Saeed, M. K. Hasan, and M. Mahmud, “Persistent overload control for backlogged machine to machine communications in long term evolution advanced networks,” Journal of Telecommunication, Electronic and Computer Engineering, vol. 9, no. 3, 2017.
View at: Google Scholar
R. Atallah, C. Assi, and M. Khabbaz, “Deep reinforcement learning-based scheduling for roadside communication networks,” in Proceedings of the 2017 15th International Symposium on Modeling and Optimization in Mobile, Ad Hoc, and Wireless Networks (WiOpt), Paris, France, May 2017.
View at: Google Scholar
A. Hbaieb, S. Ayed, and L. Chaari, “A survey of trust management in the Internet of Vehicles,” Computer Networks, vol. 203, Article ID 108558, 2022.
View at: Publisher Site | Google Scholar
K. F. Hasan, O. Anthony, K. Ansari, G. S. Ramachandran, and J. Raja, “Security, privacy and trust: cognitive internet of vehicles,” 2021, https://arxiv.org/abs/2104.12878.
View at: Google Scholar
V. Praneeth, K. R. Kumar, and N. Karyemsetty, “Security: intrusion prevention system using deep learning on the internet of vehicles,” International Journal of Safety and Security Engineering, vol. 11, no. 3, pp. 231–237, 2021.
View at: Publisher Site | Google Scholar
E. Sayed Ali, M. K. Hasan, R. Hassan et al., “machine learning technologies for secure vehicular communication on internet of vehicles: recent advances and applications,” Security and Communication Networks, vol. 2021, Article ID 8868355, 23 pages, 2021.
View at: Google Scholar
S. Yanxing, Y. Fu, F. R. Yu, and L. Zhou, “Blockchain-enabled internet of vehicles with cooperative positioning: a deep neural network approach,” IEEE Internet of Things Journal, vol. 7, no. 4, pp. 3485–3498, 2020.
View at: Publisher Site | Google Scholar
M. Pang, L. Wang, and N. Fang, “A collaborative scheduling strategy for IoV computing resources considering location privacy protection in mobile edge computing environment,” Journal of Cloud Computing, vol. 9, no. 1, p. 52, 2020.
View at: Publisher Site | Google Scholar
T. Mecheva and N. Kakanakov, “Cybersecurity in intelligent transportation systems,” Computers, vol. 9, no. 4, p. 83, 2020.
View at: Publisher Site | Google Scholar
I. Berger, R. Rieke, M. Kolomeets, A. Chechulin, and I. Kotenko, “Comparative study of machine learning methods for in-vehicle intrusion detection,” Computer Security. SECPRE 2018, CyberICPS 2018, Springer, Berlin, Germany, vol. 11387, 2019.
View at: Publisher Site | Google Scholar
E. S. Ali, M. B. Hassan, and R. A. Saeed, “Machine learning technologies in internet of vehicles,” Intelligent Technologies for Internet of Vehicles. Internet of Things (Technology, Communications and Computing), Springer, Berlin, Germany, 2021.
View at: Publisher Site | Google Scholar
M. Cheng, J. Li, and S. Nazarian, “DRL-cloud: deep reinforcement learning-based resource provisioning and task scheduling for cloud service providers,” in Proceedings of the 23rd Asia and South Pacific Design Automation Conference (ASP-DAC), Austin, TX, USA, June 2018.
View at: Google Scholar
H. Yang, X. Xie, B. Rong, and M. Kadoch, “Intelligent resource management based on reinforcement learning for ultra-reliable and low-latency iov communication networks,” IEEE Transactions on Vehicular Technology, vol. 68, 2019.
View at: Google Scholar
Y. Zhang, M. Zhang, and C. Fan, “Computing resource allocation scheme of IOV using deep reinforcement learning in edge computing environment,” EURASIP Journal on Applied Signal Processing, vol. 33, 2021.
View at: Publisher Site | Google Scholar
S. Grigorescu, T. Cocias, B. Trasnea, A. Margheri, F. Lombardi, and L. Aniello, “Cloud2Edge elastic AI framework for prototyping and deployment of AI inference engines in autonomous vehicles,” Sensors, vol. 20, no. 19, p. 5450, 2020.
View at: Google Scholar
F. Claudio, C. Zhang, P. Patras, A. Banchs, and J. Widmer, “A machine-learning-based framework for optimizing the operation of future networks,” IEEE Communications Magazine, vol. 58, no. 6, pp. 20–25, 2020.
View at: Publisher Site | Google Scholar
Z. Ning, P. Dong, X. Wang et al., “Deep reinforcement learning for intelligent internet of vehicles: an energy-efficient computational offloading scheme,” IEEE Transactions on Cognitive Communications and Networking, vol. 5, no. 4, pp. 1060–1072, 2019.
View at: Publisher Site | Google Scholar
R. Iqbal, T. Ashraf Butt, T. A. Butt, M. W. A. Talib, and T. Umar, “Context-aware data-driven intelligent framework for fog infrastructures in internet of vehicles,” IEEE Access, vol. 6, pp. 58182–58194, 2018.
View at: Publisher Site | Google Scholar
C. Chen, C. Lee, and W. Lu, “A mobile cloud framework for deep learning and its application to smart car camera,” Internet of Vehicles – Technologies and Services. IOV, Springer, Berlin, Germany, vol. 10036, 2016.
View at: Publisher Site | Google Scholar
C. Fiandrino, C. Zhang, P. Patras, A. Banchs, and J. Widmer, “A machine-learning-based framework for optimizing the operation of future networks,” IEEE Communications Magazine, vol. 58, no. 6, pp. 20–25, 2020.
View at: Publisher Site | Google Scholar
A. Gupta, “Deep learning for object detection and scene perception in self-driving cars: survey, challenges, and open issues,” Array, vol. 10, Article ID 100057, 2021.
View at: Publisher Site | Google Scholar
D. Santos, J. Saias, P. Quaresma, and V. B. Nogueira, “Machine learning approaches to traffic accident analysis and hotspot prediction,” Computers, vol. 10, no. 12, p. 157, 2021.
View at: Publisher Site | Google Scholar
F. Aadil, W. Ahsan, Z. U. Rehman, P. A. Shah, S. Rho, and I. Mehmood, “Clustering algorithm for internet of vehicles (IoV) based on dragonfly optimizer (CAVDO),” The Journal of Supercomputing, vol. 74, no. 9, pp. 4542–4567, 2018.
View at: Publisher Site | Google Scholar
W. Chang, L. Chen, and K. Su, “DeepCrash: a deep learning-based internet of vehicles system for head-on and single-vehicle accident detection With emergency notification,” IEEE Access, vol. 7, pp. 148163–148175, 2019.
View at: Publisher Site | Google Scholar
C. Dong, C. Shao, J. Li, and Z. Xiong, “An improved deep learning model for traffic crash prediction,” Journal of Advanced Transportation, vol. 2018, Article ID 3869106, 13 pages, 2018.
View at: Publisher Site | Google Scholar
J. Brownlee, “Machine learning algorithms. logistic regression for machine learning,” 2019, https://machinelearningmastery.com/logistic-regression-for-machine-learning/.
View at: Google Scholar
A. Zhang, Z. C. Lipton, L. Mu, J. Alexander, and J. Smola, “Dive into deep learning,” 2020, https://arxiv.org/abs/2106.11342.
View at: Google Scholar
L. E. Alatabani, E. S. Ali, and R. A. Saeed, “Deep learning approaches for IoV applications and services,” Intelligent Technologies for Internet of Vehicles. Internet of Things (Technology, Communications and Computing, Springer, Berlin, Germany, 2021.
View at: Google Scholar
M. B. Hassan, E. Sayed Ali, R. A. Mokhtar, R. A. Saeed, and S. Bharat, “Nb-IoT: concepts, applications, and deployment challenges,” LPWAN Technologies for IoT and M2MApplications, Elsevier, Hoboken, NJ, USA, 2020.
View at: Google Scholar
Z. E. Ahmed, R. A. Saeed, S. N. Ghopade, and A. Mukherjee, “Energy optimization in LPWANs by using heuristic techniques,” LPWAN Technologies for IoT and M2M Applications, Elsevier, Hoboken, NJ, USA, 2020.
View at: Google Scholar
S. Raza, S. Wang, M. Ahmed, and M. R. Anwar, “A survey on vehicular edge computing: architecture, applications, technical issues, and future directions,” Wireless Communications and Mobile Computing, vol. 2019, Article ID 3159762, 3 pages, 2019.
View at: Publisher Site | Google Scholar
M. B. Hassan, E. Sayed Ali, N. Nurelmadina, and R. A. Saeed, “Artificial intelligence in IoT and its applications, book: intelligent wireless communications,” Deep Machine Learning—A New Frontier in Artificial Intelligence Research, IET book publisher, 2020.
View at: Google Scholar
R. Miotto, F. Wang, S. Wang, X. Jiang, T. Joel, and Dudley, Deep Learning for Healthcare: Review, Opportunities and Challenges, Oxford University Press, Oxford, UK, 2017.
Y. Dong, Z. Yu, and G. Rose, “SR-IOV networking in xen: aarchitecture, design and implementation,” in Proceedings of the First Workshop on I/O Virtualization, San Diego, CA, USA, December 2008.
View at: Google Scholar
J. Eom, H. Kim, S. H. Lee, and S. Kim, “DNN-assisted cooperative localization in vehicular networks,” Energies, vol. 12, no. 14, 2019.
View at: Publisher Site | Google Scholar
L. Lei, Y. Tan, K. Zheng, S. Liu, K. Zhang, and X. Shen, “Deep reinforcement learning for autonomous internet of things: model, applications and challenges,” IEEE Communications Surveys & Tutorials, vol. 22, no. 3, 2020.
View at: Publisher Site | Google Scholar
S. Grilli and G. Panza, “AI-based traffic queue detection for IoV safety services in 5G networks,” Intelligent Technologies for Internet of Vehicles. Internet of Things (Technology, Communications and Computing), Springer, Berlin, Germany, 2021.
View at: Publisher Site | Google Scholar
S. Ruder, “An overview of gradient descent optimization algorithms,” 2016, https://arxiv.org/abs/1609.04747.
View at: Google Scholar
S. Sharma, K. K. Ghanshala, and S. Mohan, “A security system using deep learning approach for internet of vehicles (IoV),” in Proceedings of the 2018 9th IEEE Annual Ubiquitous Computing, Electronics & Mobile Communication Conference (UEMCON), pp. 1–5, New York, NY, USA, November 2018.
View at: Publisher Site | Google Scholar
G. Loukas, T. Vuong, R. Heartfield, G. Sakellari, Y. Yoon, and D. Gan, “Cloud-based cyber-physical intrusion detection for vehicles using deep learning,” IEEE Access, vol. 6, pp. 3491–3508, 2018.
View at: Publisher Site | Google Scholar
L.-B. Chen, K.-Y. Su, Y.-C. Mo et al., “An implementation of deep learning based IoV system for traffic accident collisions detection with an emergency alert mechanism,” in Proceedings of the 2018 IEEE 8th International Conference on Consumer Electronics - Berlin (ICCE-Berlin), pp. 1-2, Berlin, Germany, September 2018.
View at: Publisher Site | Google Scholar
M. S. H. Sassi and L. C. Fourati, “Investigation on deep learning methods for privacy and security challenges of cognitive IoV,” in Proceedings of the 2020 International Wireless Communications and Mobile Computing (IWCMC), pp. 714–720, Harbin, China, July 2020.
View at: Publisher Site | Google Scholar
F. A. Alqurashi, Machine Learning Techniques in Internet of UAVs for Smart Cities Applications, IOS Press, Amsterdam, Netherlands, 2021.
M. Y. Abdelsalam, R. A. Saeed, and R. A. Alsaqour, “The effect of route routability procedure to the route optimization distribution binding update,” International Journal of Engineering and Technology, vol. 5, no. 2, pp. 669–679, 2013.
View at: Google Scholar
J. Kim, “Vehicle detection using deep learning technique intunnel road environments,” Symmetry,, vol. 12, 2020.
View at: Google Scholar
J. Joon and H.-J. Kim, “An automobile environment detection system based on deep neural network and its implementation using iot-enabled in-vehicle air quality sensors,” Sustainability, vol. 12, 2020.
View at: Google Scholar
J. Dong, W. Wu, Y. Gao, X. Wang, and P. Si, “Deep reinforcement learning based worker selection for distributed machine learning enhanced edge intelligence in internet of vehicles” Intelligent and Converged Networks,” vol. 1, no. 3, 2020.
View at: Publisher Site | Google Scholar
D. Kwon, J. Kim, A. David, and L. Wonjun, “Self-adaptive power control with deep reinforcement learning for millimeter-wave Internet-of-vehicles video caching”,” Journal of Communications and Networks, vol. 22, no. 4, 2020.
View at: Publisher Site | Google Scholar
Q. Qi, J. Wang, Z. Ma et al., “Knowledge-driven service offloading decision for vehicular edge computing: a deep reinforcement learning approach,” IEEE Transactions on Vehicular Technology, vol. 68, no. 5, 2019.
View at: Publisher Site | Google Scholar
X. He, H. Lu, M. Du, Y. Mao, and K. Wang, “QoE-based task offloading with deep reinforcement learning in edge-enabled internet of vehicles,” IEEE Transactions on Intelligent Transportation Systems, Early Access Article, vol. 22, 2020.
View at: Google Scholar
F. Fu, Y. Kang, Z. Zhang, F. Richard Yu, and T. Wu, “Soft actor–critic DRL for live transcoding and streaming in vehicular fog-computing-enabled IoV,” IEEE Internet of Things Journal, vol. 8, no. 3, 2018.
View at: Google Scholar
C. Chen, Z. Wang, Q. Pei, C. He, and Z. Dou, “Distributed computation offloading using deep reinforcement learning in internet of vehicles,” in Proceedings of the 2020 IEEE/CIC International Conference on Communications in China (ICCC), Chongqing, China, August 2020.
View at: Google Scholar
F. Naeem, S. Seifollahi, Z. Zhou, and M. Tariq, “A generative adversarial network enabled deep distributional reinforcement learning for transmission scheduling in internet of vehicles,” IEEE Transactions on Intelligent Transportation Systems, Early Access Article, 2020.
View at: Google Scholar
P. Zhou, X. Chen, Z. Liu, T. Braud, H. Pan, and J. Kangasharju, “DRLE: decentralized reinforcement learning at the edge for traffic light control in the IoV,” IEEE Transactions on Intelligent Transportation Systems, vol. 22, 2020.
View at: Google Scholar
A. Kang, X. Yan, T. Liang, and W. Lu, “Mobility prediction based vehicular edge caching: a deep reinforcement learning based approach,” in Proceedings of the 2019 IEEE 19th International Conference on Communication Technology (ICCT), Xi’an, China, October 2019.
View at: Google Scholar
T. Yuan, R. Wilson, C. E. Rothenberg, K. Obraczka, C. Barakat, and T. Turletti, “Dynamic controller assignment in software defined internet of vehicles through multi-agent deep reinforcement learning,” IEEE Transactions on Network and Service Management, vol. 18, no. 1, 2021.
View at: Publisher Site | Google Scholar
Y. Guo, Z. Ning, and R. Kwok, “Deep reinforcement learning based traffic offloading scheme for vehicular networks,” in Proceedings of the 2019 IEEE 5th International Conference on Computer and Communications (ICCC), Chengdu, China, December 2019.
View at: Google Scholar
S.-S. Lee and S. Lee, “Resource allocation for vehicular fog computing using reinforcement learning combined with heuristic information,” IEEE Internet of Things Journal, vol. 7, no. 10, 2020.
View at: Publisher Site | Google Scholar
Y. Dai, D. Xu, Y. Lu, S. Maharjan, and Y. Zhang, “Deep reinforcement learning for edge caching and content delivery in internet of vehicles,” in Proceedings of the 2019 IEEE/CIC International Conference on Communications in China, ICCC), Changchun, China, August 2019.
View at: Google Scholar
X. He, H. Lu, Y. Mao, and K. Wang, “QoE-driven task offloading with deep reinforcement learning in edge intelligent IoV,” in Proceedings of the GLOBECOM 2020 - 2020 IEEE Global Communications Conference, Taiwan, China, December 2020.
View at: Google Scholar
Q. Qi, L. Zhang, J. Wang et al., “Scalable parallel task scheduling for autonomous driving using multi-task deep reinforcement learning,” IEEE Transactions on Vehicular Technology, vol. 69, no. 11, 2020.
View at: Publisher Site | Google Scholar
Ge Wang and F. Xu, “Regional intelligent resource allocation in mobile edge computing based vehicular network,” IEEE Access, vol. 8, 2020.
View at: Publisher Site | Google Scholar
H. Lin, S. Garg, H. Jia, K. Georges, M. Min Peng, and H. Shamim, “Blockchain and deep reinforcement learning empowered spatial crowdsourcing in software-defined internet of vehicles,” IEEE Transactions on Intelligent Transportation Systems, vol. 22, 2020.
View at: Google Scholar

Copyright

Copyright © 2022 Lina Elmoiz Alatabani et al. This is an open access article distributed under the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.

PDF Download Citation

Download other formats

Order printed copies

Views

1591

Downloads

945

Citations