Abstract
In view of the problem of long migration time and low availability of VM (virtual machine) migration in MD (mobile devices) in the MEC (mobile edge computing) environment, this paper designs a MD2N (Mobile Device to Network) model. This model no longer uploads requests to the cloud center, but centers on the edge server, scheduling and selecting migration services through the communication between the servers. And, a virtual machine migration strategy based on Markov decision and greedy algorithm is proposed. Model the edge server based on its geographic location, imitating the traditional cellular network method, and then the improved greedy algorithm is used to dynamically select the best edge device to migrate VM. Simulation and comparison experiments show that the proposed scheme has small migration fluctuations, stable migration status, short migration time, low average time delay, and high virtual machine migration availability after migration.
1. Introduction
With the explosive growth of the number of mobile devices (MD) and the resulting large amount of data, cloud computing has not been able to meet user requirements in terms of time delay and energy efficiency. Therefore, edge computing [1–4] has achieved tremendous development as a supplementary computing paradigm to overcome the above challenges. Among them, mobile edge computing (MEC), as a product of the integration of the Internet and Communication Technology (ICT), has become a key technology for the next generation of mobile networks [5, 6] enabling mobile devices to distribute their computing tasks to servers located on the “edge” of the radio access network [7], and it creates more practical value while reducing core network congestion, such as lower latency, location awareness, and network context information. [8, 9].
Due to the limited coverage of edge servers and the random mobility of users, sinking the computing power of cloud servers to the edge will bring new challenges. For example, when users are receiving continuous services, they may roam in wireless areas served by different edge servers. In order to ensure user satisfaction with service performance, dynamic service migration needs to be considered.
If the MEC allows the MD to migrate its computing tasks to the MEC server through the wireless access network, the computing overhead and the time delay caused by mutual communication must be considered at the same time. In order to solve the above problems, this paper proposes a dynamic programming algorithm based on Markov decision and greedy algorithm. It selects computing nodes (local equipment, edge server, and remote cloud) to run its services by predicting the behavior trajectory and migration needs of mobile users. Use Markov model to deduce the location of MD, prepare for its migration in advance, and then use greedy algorithm to select the optimal migration strategy.
2. Related Work
Mobile edge computing has received more and more attention in recent years [10], especially some previous studies that focused on service placement issues [11–13]. The issue of user mobility management is a major challenge for service placement in MEC. Existing mobility management work can be roughly divided into three categories. (1) Always migrate, as long as the mobile device enters the new MEC server service range, immediately migrate the computing task from the source MEC server to the current MEC server. (2) Never migrate, no matter how the mobile device moves, the calculation task is always executed on the original MEC server, and communicates with the source server through the transmission of information. (3) Partial migration, by weighing the communication delay and migration time delay, and making the optimal migration decision according to the actual application situation. Obviously, the first method reduces communication delays, but it will bring additional migration costs, such as insufficient bandwidth usage, potential service interruption, or even handover failure, and the second method will cause longer communication delays as the network distance increases. Although the third method has the highest benefit, it often consumes a lot of computing resources when making a decision. Therefore, how to pay the minimum cost when making a decision becomes the main research goal.
The challenge mentioned above can be described as dynamically keeping all services following the user when the user moves, which will lead to the migration of VM. Raad et al. [14] considered large-scale real-time VM migration, and evaluated the locator/identifier separation protocol (LISP) based on flat message definition to improve migration efficiency. Machen et al. [15] proposed a three-layer service migration protocol using Linux container (LXC) and kernel-based virtual machine (KVM) technologies. Cerroni and Callegati [16] designed sequential and parallel multiple VMs and compared the performance of the two methods to achieve parallel migration with lower latency but higher resource consumption. Although the above methods can solve the problems of time delay and dynamic migration, the designed solution occupies more resources and cannot be realized in the edge environment with limited resources. In this paper, to adapt to the shortage of limited resources in the edge environment, a sequential migration method is used to migrate services.
The service migration method is a key component of the right management in MEC [17–19]. When there is user migration, it is difficult to decide what is the best choice to migrate the running service. Although VM migration is closer to the user’s location, it will cause an increase in cost. Nadembega et al. [20] use a mobile-based prediction scheme to balance execution overhead and transmission delay. Wang et al. [21] place services by predicting the cost of future data transmission, processing and service migration. W. Zhang et al. [22] observe the correlation between bandwidth and geographic location based on historical data, and use this to predict user location to optimize wireless transmission time.
But in actual operation, it is more difficult to accurately predict the future information of MD based on historical data, and it will cause a large computational overhead, and may also cause a larger time delay in an edge-computing environment. Taleb and Ksentini [23] used Markov chains to analyze the possibility of user mobility. Ksentini et al. [24] and Wang et al. [25] both tried to model the entire service migration process and design the optimal migration decision through the Markov decision process. The research of T. Ouyang et al. [26] does not require any prior knowledge about the future mobility of users, and uses Lyapunov optimization to solve the service placement problem.
But, these works are mainly concentrated on the management and optimization of service placement within the system, and the scheduler understands complete system information. In actual use, the scheduler may not necessarily have the future location information of the mobile device, and it also faces the uncertainty of edge server selection. In order to meet these challenges, Sun et al. [27] proposed a learning-based service placement framework, using the theory of MAB (multiarmed bandit) to enable vehicles to provide their services by learning the best neighboring vehicle selection.
Although the above method solves the problem of service placement, it does not consider the time dependence of sequential decisions when tasks are migrated from one server to another. Sun et al. [28], user-centric EMM (energy-aware mobility management) solution, continuously optimize the time delay caused by wireless access and computing, and select BS and MEC servers for users and when to handover. Alfakih et al. [29] proposed a RL-SARSA (state-action-reward-state-action algorithm based on reinforcement learning) to solve the problem of resource management in edge servers, and through the best foaming decision to minimize the system cost, including energy consumption and calculation time delay. This method is called SARSA based on offloading decision (OD-SARSA).
Pavlos et al. [30] regarded the MEC server option as a public resource pool with uncertain user returns, and local computing as a security option for each user, following the attributes of prospect theory, and using local computing in the case of probabilistic uncertainty and uninstall overhead options to develop the user’s prospect theory utility. The solution MIGRATE of Santa et al. [31] proposed the concept of virtual mobile devices (vMDs). When the devices are moved to different management domains to access the network, virtual mobile nodes in the form of digital twins must be able to “migrate” to the new edge virtualization field. Liang et al. [32] proposed a multiuser task offloading scheme based on edge-cloud joint computing, designed a greedy algorithm based on submodule theory and fully utilized the computing and communication resources of the cloud and edge. This solution still has the participation of the cloud. Although the transmission speed is slightly better than the traditional migration method, the communication delay from the edge to the cloud still exists. The solution proposed in this article does not depend on the cloud center, and the main work is carried out in the edge network. There is no remote communication delay, which can change the traditional migration method and improve the transmission efficiency.
3. Scheme Design
As shown in Figure 1, when a user is roaming or has a migration request, in order to obtain a lower perceived delay, users tend to place their services on the most appropriate MEC server through 4G/5G/WiFi access networks. In order to ensure the continuity of services, it is necessary to take the migration time into consideration and use the predictive migration method [33, 34] to migrate virtual machines. In this paper, we design the Mobile Device to Network (MD2N) model based on the mobile device’s movement and service conditions in real life. The specific model is shown in Figure 1. Suppose there are N servers arranged by index , each of which is directly associated with one or more base stations (BS), and all MEC servers can communicate with each other through wired channels. Any MEC server in the edge network environment can provide multiple MD2N services for mobile devices. The mobile device selects the best base station according to the signal strength and communicates with the MEC server associated with the base station. In addition, the MEC server can be used as a router, so that mobile devices can communicate with the MEC server remotely. Under this premise, multiple MEC servers can provide services for the same mobile device.
Divide the actual physical time into continuous time gaps, represented by , , a time gap is assumed to be seconds; the state of the mobile edge network is represented by an matrix, the (-, -) th element (G)i,j∈[0, ηmax) (ηmax is the maximum channel capacity) in the matrix indicates whether there is a direct connection between the -th and -th MEC server, if , means no connection. Assuming that is a constant matrix that does not change with time, that is, within a specified time, the state of the mobile edge network will not change, so a time parameter is defined to represent the channel resources available in the network. represents the minimum channel resources required to ensure communication. In fact, for the service in the network, in order to ensure its stable transmission in the case of channel congestion and flow fluctuations, excessive channel resources should be provided. In addition, the more congested the channel, the fewer available resources, so can also indicate the number of mobile devices in the channel. (bits/s) represents the available channel capacity between MEC server and , which can be calculated using according to the RIP (Routing Information Protocol).
We assume that according to the MEC server indexed by , connected mobile devices can use up to M MD2N services; we can define a matrix represented by to record the mobile device’s future. The migration strategy in the next time slot, where is an integer use to represent the target MEC server hosting service in the time slot ( represents the start time), correspondingly in the matrix, if the MEC server will not host the service in the future, it will be represented by . Based on the definition of the above matrix, we can use the vector to describe the service position in the time slot , therefore, we can use the time slot in the vector to rewrite the migration strategy. Similarly, for any time slot , you can use to rewrite the migration strategy.
Using a model similar to Markovian to derive the location of MD, divide the network into regular hexagonal units (as shown in Figure 2), where one hexagonal unit represents the service range of an MEC server. Consider a random walking movement model, in which MD visits any of the six adjacent cells with probability . Figure 2 shows a ring with a service area of . When the MD’s position is hops away from the initial position, the service migration and relocation for the MD starts. Let denote the distance from the MD to the initial server at time (calculated in hop count). The system forms a CTMC (Continuous-Time Markov Chain), the state space is .
In Figure 2, it can be seen that the MD in the 1st ring moves to any adjacent unit with probability . The MD in the second ring returns to the 1st ring with probability , remains in the second ring with probability , and moves to the third ring with probability , increasing the distance, so all the rings in the first ring can be aggregated into one state. For the third ring, there are two types of rings: (1) similar to ring 1, there are three 3-rings, two 2-rings, and one 1-ring adjacent to it; (2) on the three rings, each has two rings adjacent to it. Depending on where the MD is in the second ring, there may be a or probability to move to the third ring. Therefore, two aggregation states are obtained: and . In the same way, any ring has aggregation states and , where .
Figure 3 shows a Markov chain in each aggregation state with maximum of 5. According to the research of Langar et al. [35], the ring number and superscript are used to indicate the aggregation state of each ring , and the equilibrium state of its migration strategy is shown in formulas (1)–(9):
Among them, the values of and are
Among them, the value of
Among them, the value of
Among them, represents the current position, formula (1) represents the possible position and probability of the next step when it is located in the corresponding position in Figure 3, and formula (2) represents the corresponding position in Figure 3. The possible position and probability of the second step, (3) is the prediction of the possible position and probability of the next step according to its law. equations (4)–(9) are the reasoning and simplification of (3).
Let denote the average distance from MD to the MEC server. From (1)–(9), we can see that the size of depends on the value of , so the average distance from MD to the initial position is
4. Migration Decision
4.1. Transmission and Calculation Consumption
The widely used three-parameter model is used to describe each MD2N service ; input the data that needs to be processed with the size of (bits), the calculation intensity (CPU cycles/bit) represents the requirement of each service requiring CPU resources to calculate one bit of input data, and the maximum allowable delay (seconds); in addition, define to represent the frequency of message sending.
Data is transmitted from one MEC server to another MEC server through a wired channel. Assuming that the wireless transmission consumption between the mobile device and the serving base station BS is a constant, the transmission consumption of MD2N service is
Among them, is a constant, representing the consumption of wireless transmission, represents the channel capacity from to .
Each MEC server can provide computing services for multiple tasks of connected mobile devices at the same time. Use to describe the CPU frequency that MEC server can allocate to MD2N service . Because MEC server has powerful computing capabilities, will not change. If the MEC server is selected to provide service , the calculated delay is
When the migration strategy vector is , we consider the communication delay of the sum of data transmission and processing in the specified time slot
4.2. Migration Cost
Add a new parameter (in bits) to indicate the size of the service sample data, where is the maximum possible size of the sample data, for each MD2N service , execute in the time slot . The time delay of the migration strategy is
Among them, is the channel capacity from to .
The goal of this paper is to minimize the average time delay of service migration. Therefore, once the MD’s future movement trajectory is obtained through probability calculation, a migration strategy is proposed to reduce the time delay as much as possible based on the consideration of communication and migration costs. For service , the total service delay in time slot is
The delay defined in equation (15) represents the response time delay of a single message between the MEC server and the MD. When the migration of the VM also occurs in the time slot , it is obvious that this will lead to an increase in the response time. The response time is the sum of communication costs and migration costs. If in the time slot , service chooses not to migrate, the migration cost at this time is zero, the response time represents the communication time delay, and the formula (15) is expressed as
Generally speaking, MD2N services can always be completed within 1 s [36], which is much smaller than the time gap . The use of VM migration technology makes the migration time of each service not lasting too long. In view of this, the following assumptions can be made; even under the minimum available channel resource , the migration of each MD2N service can be completed within a time gap, namely
For each service , the importance of MD equipment is different, so use to represent the weight of each service. Although different MD2N services have great differences in actual delay and end time, according to the importance of the service, the time delay of equation (15) is standardized using . Therefore, the standardized average time delay of multiple MD2N services can be expressed as
MD2N associates a series of possible future state transition behaviors to according to the user’s current state . For a given behavior , when switching from one state to another state , an instantaneous reward , the reward is based on the size of migration overhead, . Therefore, the Markov Decision Process (MDP) can be expressed as
Among them, represents the conversion rate when switching from state to another state under behavior . Define an attenuation factor , then the migration strategy can be expressed as
According to the research of Taleb et al. [37], the solution of the optimal equation of (20) is equivalent to the optimal decision , which determines which MEC server MD chooses in the future and whether to perform the migration decision. The algorithm is described as follows
| ||||||||||||||||
Algorithm 1 uses the dynamic programming algorithm to obtain the optimal migration strategy matrix of MD migration. The user performs migration according to the matrix. Use to represent the MD queue that is about to face the migration decision, and define to limit the migration overhead to a time slot . The algorithm of the migration service is described as follows:
Algorithm 2 is a migration service algorithm based on Algorithm 1. Considering the communication delay and migration cost, we will serve it according to the migration queue obtained by Algorithm 1 under the premise of minimizing the delay and cost as much as possible.
5. Simulation Results and Performance Analysis
5.1. Experimental Configuration
This section evaluates the system utility of the proposed dynamic programming algorithm based on Markov decision and greedy algorithm under the edge-computing environment through simulation experiments. In order to be consistent with the environment of the comparison experiment, this article uses KubeEdge on the Dell PowerEdge R720 server to simulate an area with 26 edge servers and 1 cloud server. Each edge server is configured with dual-core CPUs. Since all edge servers are cloned, here, we assume that the computing power of all edge servers is the same, and consistent with the comparison experiment, it is 10 GHz, and the computing tasks of edge devices are randomly and evenly distributed between (500 and 1000) cycles/bit. The user’s mobility data is generated by the currently widely used, the one simulator [38].
In the experiment, the migration time delay is counted, and the time of the migration failure case cannot be counted. Therefore, only the time consumption and time delay of the successfully migrated VM are recorded in this experiment. In terms of standardized average time delay and actual average time delay, this experiment compares the PDOA [39] and MDP schemes and compares them with the optimal solution; in terms of actual running time and VM availability after migration, this experiment compares the PDOA, MDP, and AUSP [38] schemes and compares them with the completely offline scheme and the optimal solution.
5.2. Evaluation Index
This experiment mainly evaluates from the aspects of standardized time delay, channel available resources, number of servers, and the availability of virtual machines after migration.
5.2.1. Standardized Time Delay
Comparing the actual standardized average time delay and the average standardized time delay of each program in different time slots, through the fluctuation of each program, it can be judged whether the program is stable.
5.2.2. Channel Available Resources
Set different channel available resource rates, run each plan under different channel available resource rates, count the running time, and compare the applicability of the plans.
5.2.3. Number of Servers
Change the number of servers from small to large, count and compare the actual running time of each program.
5.2.4. The Availability Rate of the Virtual Machine after Migration
Count the ratio of the number of available virtual machines to the total number of virtual machines after the migration of each program is completed, so as to evaluate the pros and cons of each program in practical applications.
Finally, comprehensively compare all the indicators and draw a conclusion.
5.3. Experimental Results
Figure 4 compares the standardized average time delay of each time slot , namely formula (18), the dotted line in the figure represents the calculated value, which does not change with time, for comparison, and the solid line represents the actual measured value. Here we assume that MD2N services are used, and the available channel resource . When , the migration time of each scheme is close (Figure 5), which is convenient for comparison. At the same time, starting from , the average time extension tends to be horizontal (Figure 6), and less resources are more representative. It can be seen from the experimental results that the BMS scheme and PDOA scheme proposed in this paper are close to the optimal solution, and the mean value of the standardized delay is much smaller than the MDP scheme. Although the average value of the BMS scheme proposed in this paper is slightly higher than that of the PDOA scheme, the difference is very small and can be ignored, and the fluctuation of the BMS scheme is smaller and the most stable among the four schemes.
Since the PDOA solution is mainly for devices in the Internet of Vehicles, the application range is limited. In order to highlight the practicability of this scheme, an experiment was first compared with the PDOA scheme on a certain street, as shown in Figure 5. According to the change of the average time delay under the conditions of different channel available resource rates, it can be seen that as the channel available resource rate increases, the time delay gradually decreases. When increases to a certain degree (0.65), the decreasing trend of the average time delay is no longer obvious. The small graph in Figure 5 shows how the average time delay changes when is between 0.5 and 0.8. It can be seen that the overall performance of the PDOA solution is slightly better than this solution. This is because the PDOA scheme adopts a predictive algorithm, and the behavior of cars driving on the highway is regular, so the time consumption in calculation is less than this scheme. But when the user’s behavior is not regular, the prediction algorithm of the PDOA scheme no longer has the advantage. The specific results are reflected in the following experiment.
Figure 6 shows the effect of channel available resource on the long-term normalized average time delay. In the experiment, was increased from 0.1 to 0.8. It can be seen from the experimental result graph that as increases, the average time delay gradually decreases. This is because as the channel’s available resources increase, the available resources increase, the queues waiting for migration will decrease, and the time required will decrease accordingly. It can be seen that when the channel available resources are less (from 0.1 to 0.4), the time delay of MDP is the longest, and the BMS and PDOA schemes are very close to the optimal solution. This is because the MDP scheme treats all MD2N services equally, while the BMS, PDOA, and optimal solutions are dynamically migrating MD2N services. When channel resources are sufficient (after 0.65), it can be seen from the small graph in Figure 6 that the average time delay of the BMS and PDOA solutions is slightly higher than that of the MDP solution. This is because with the increase of channel resources, the BMS solution proposed in this paper still maintains the previous migration strategy. Since the calculation scale has not changed, it is difficult to improve the time. Therefore, the average delay is slightly higher than the MDP solution, but on the whole, this time difference is very small, which can be tolerated in practical applications. In addition, although the average time delay of the BMS scheme is slightly higher than that of the PDOA scheme, it is on the same level as a whole. Moreover, as can be seen from Figure 4, the stability of the BMS scheme is better than that of the PDOA scheme.
Figure 7 records the running time of each scheme under different numbers of servers in a certain time slot . Figure 8 shows the running time chart of the number of servers in Figure 5 between 12 and 26. As can be seen from the figure, the time used by the BMS, AUSP, and PDOA solutions is significantly shorter than the offline algorithm time, and the three solutions consume basically the same time, especially with the increase in the number of servers, the time consumed by offline algorithms has increased dramatically, while the time consumed by other algorithms tends to stabilize. Although the time-consuming of the MDP scheme has been stabilizing, its average delay is longer and its stability is poor. Although the running time of the BMS solution proposed in this article is slightly higher than the other two, the difference is not large and can be ignored. It can be seen from Figure 4 that the BMS solution has the best stability.
Figure 9, respectively, counts the availability of VMs on the target device under the conditions of channel available resources of 0.2, 0.4, 0.6, and 0.8 after different migration schemes are executed. The specific situation is shown in Figure 9. It can be seen that the offline and MDP solutions always maintain a high service availability rate, but the running time of the offline solution varies greatly. When the number of servers is large, it takes a long time, which is not conducive to practical applications. The average time delay of the MDP scheme has large fluctuations and poor stability. With the increase of channel available resources, the service availability rates of BMS, AUSP, and PDOA tend to increase, but from an overall point of view, the service availability rate of the BMS solution proposed in this paper is higher than the other two solutions.
6. Conclusion and Future Work
Aiming at the problem of virtual machine migration in mobile devices in the edge environment, this paper proposes a dynamic programming scheme BMS based on Markov decision and greedy algorithm. Firstly, formulate the time delay and other costs incurred in the VM migration process, then use a method similar to cellular networks to model the geographic location, and use Markov decision and greedy algorithms to select the server to receive the VM to be migrated. Unlike traditional prediction algorithms, this is a probability-based choice, which improves accuracy to a certain extent and ensures the availability of the migrated VM on the target device. The innovation of the scheme proposed in this paper is mainly reflected in (i)Abandon the traditional way of relying on cloud servers to migrate VMs in mobile devices. Instead, it relies on edge servers to migrate. The cloud data center only functions to manage edge servers and does not directly participate in the migration of VMs. Edge servers are closer to users. With the popularization of 5G technology, transmission speeds increase, response times are shorter, and time delays are shortened accordingly(ii)Compared with the traditional relying only on MDP algorithm or prediction algorithm, this scheme adopts a combination of MDP algorithm and greedy algorithm. The impact of prediction accuracy on the migration result is removed, and at the same time it is ensured that the selected service is optimal or close to the optimal solution
The experimental results show that the stability of the BMS scheme is the best among the several schemes compared, and the running time and time delay are also of the best. Although using MEC servers close to users to serve users can reduce the transmission time delay, the computing power of MEC servers is not as good as that of cloud servers. Therefore, it will take some time to make the optimal migration decision, so it is slightly higher than AUSP and PDOA, but has high stability. And in terms of the availability of VMs after migration, the BMS solution is significantly better than the other two. To sum up, considering various factors, the BMS solution is the best choice.
Using the edge server closer to the user as the migrating dispatch center can reduce transmission delay, but due to the limited computing power of the edge server itself, the processing task speed is far lower than that of the cloud server, which will cause a large time overhead. Although the total execution time and time delay have been improved, they have not improved much. These problems will be solved through continuous improvement of the algorithm in subsequent research. But overall, the solution proposed in this paper has short running time, small average delay, best stability, and the highest availability on the target device, which is the best solution.
Data Availability
The data used to support the findings of this study are available from the corresponding author upon request.
Conflicts of Interest
The authors declare that they have no conflicts of interest.
Acknowledgments
This work was supported by the National Natural Science Foundation of China (No. 61572170) and the Natural Science Foundation of Hebei Province, China (F2019201290).