Abstract
Computation offloading is an effective way to alleviate the resource limited problem of mobile devices. However, the offloading is not an always advantageous strategy for under some circumstances the overhead in time and energy may turn out to be greater than the offloading savings. Therefore, an offloading decision scheme is in demand for mobile devices to decide whether to offload a computation task to the server or to execute it in a local processor. In this paper, the offloading decision problem is translated into a dynamic execution mode selection problem, the objective of which is to minimize the task execution delay and reduce the energy consumption of mobile devices. A novel execution mode adjustment mechanism is introduced to make the execution process more flexible for real-time environment variation. Numerical results indicate that the proposed scheme can significantly reduce the task execution delay in an energy-efficient way.
1. Introduction
Nowadays, advancements in computing technology have made mobile devices more and more smart and powerful, for example, smart phones, handheld computers, ambient sensors, and autonomous robots. The increasing processing and storage capacity have made it possible for them to run complex applications. However, mobile devices are inherently resource limited and energy constrained [1], for the more complex and energy-intensive programs are emerging in an explosive speed [2].
Advanced computing architectures, for example, wireless/mobile computing [3] and cloud computing [4], can provide attractive solutions to alleviate the resource limitation problem. These solutions all introduce a new strategy, that is, the task offloading, also known as task migration, where the mobile terminals (MTs) can make use of the ample computing resources of the wireline domain, sending their computation tasks to remote computation servers and receiving the computation results afterwards.
Is the offloading really beneficial? From the perspective of saving time, the relatively faster server processor can accelerate the execution speed, however, overheads are introduced by wireless communication between MTs and the server. On the other hand, it is hard to conclude whether the offloading can reduce the MT’s energy consumption or not. Most energy is consumed by the MT’s CPU when the task is executed locally. If offloaded, this part of energy can be saved but the MT’s transceiver will consume additional energy to accomplish the uplink and downlink data transmission.
Therefore, the offloading decision problem needs to be considered from many aspects, for example, task properties, wireless bandwidth, and MT’s CPU capacity. As [5] concludes, offloading is beneficial when large amounts of computation are needed with relatively small amounts of communication. Except typical applications which are suitable for offloading (chess games, face recognition applications, etc.), there are many computation tasks that are hard to be classified as “tasks suitable for offloading” or “tasks suitable for local processing,” for example, the online file editing or the relatively simple image processing, the local computation volume, and offloading communication volume which are comparative and the wireless bandwidth plays an important role in the offloading decision process. A weak wireless link can slow down the communication and raise power consumption at the transceiver; the overhead in time and energy may turn out to be greater than the offloading savings.
In this paper, we consider the computing architecture which has been introduced in [6]. The MT can execute the computation task in one of the following three modes: (1) locally: executing the task at the MT’s local processor, (2) remotely: offloading the task to a remote server by wireless transmission, and (3) combinedly: executing the task in the former two ways simultaneously. Before task execution, the MT has to decide which mode to adopt. Therefore, the offloading decision problem is equivalent to an execution mode selection problem.
Based on the previous description, an adaptive execution mode selection scheme (AEMSS) is proposed to help MTs make offloading decisions. Main contributions of this paper are described as follows.(i)For the existing offloading schemes, decisions are made in advance and cannot be changed once the execution has begun. The proposed AEMSS allows the task to change execution mode during processing, which can make the task execution more flexible for the changing wireless condition.(ii)The execution mode selection problem is formulated as a finite-horizon Markov decision process (MDP) and the user’s tolerance for execution delay is subtly formulated as the last decision epoch of the model. A heavy penalty factor for timeout can guarantee the delay performance of the proposed scheme.(iii)The MDP is solved by the backward induction algorithm (BIA) and numerical results indicate that the proposed AEMSS can achieve a good performance on execution delay and energy efficiency.
The remainder of this paper is organized as follows. The related work is summarized in Section 2. System model and the problem formulation of AEMSS are presented in Section 3. The detailed finite-horizon MDP model and the solution are described in Section 4. In Section 5, some implementation issues are discussed. Simulation assumptions and the numerical results are presented in Section 6. Finally the conclusions are drawn in Section 7.
2. Related Work
In this section, we provide a discussion on the existing algorithms for offloading decision, which can be partitioned into two categories, that is, the static decisions and the dynamic decisions [7].
Static decision is that the program is partitioned during development—which part to execute locally and which part to offload. Algorithms to make static offloading decisions mostly appear in the work of earlier years [8–10]. In [8], an algorithm is proposed to divide the program into server tasks and client tasks such that the energy consumed at the client is minimized. Reference [9] presents a task partition and allocation scheme to divide the distributed multimedia processing between the server and a handheld device. Authors in [10] introduce a strategy that executes the program initially on the mobile system with a timeout. If the computation is not completed after the timeout, it is offloaded. These static offloading decisions have the advantage of low overhead during execution. However, this kind of approach is valid only when the parameters, for example, wireless transmission capacity, can be accurately predicted in advance.
In contrast, dynamic decisions can adapt to various run-time conditions, for example, fluctuating network bandwidths and varying server load. Prediction mechanisms are usually used in dynamic approaches for decision making. For example, the bandwidth is monitored and predicted using a Bayesian scheme in [11]. Offloading decision algorithms in recent work are most dynamic ones [12–16]. In [12], a theoretical framework of energy-optimal mobile cloud computing under stochastic wireless channel is provided. In [13], a study on the feasibility of applying machine learning techniques is presented to address the adaptive scheduling problem in mobile offloading framework. Reference [14] presents a collaborative WiFi-based mobile data offloading architecture targeted at improving the energy efficiency for smartphones. Reference [15] proposes a fine grained application model and a fast optimal offloading decision algorithm where multiple offloading decisions are made per module based on the execution paths leading to the module. Authors in [16] also formulate the offloading decision problem based on MDP and solve it by using a linear programming approach. They address the problem of extending the lifetime of a battery powered mobile host in a client-server wireless network by using task migration and remote processing.
The work in this paper is inspired by [6], where a task migration jointly with the terminal power management mechanism is formulated in the framework of dynamic programming. The solution is a policy specifying when the terminal should initiate task migration versus executing the task locally, in conjunction with the power management. Although there have been various offloading decision algorithms proposed in the literature, our work has some original and advanced characters. We put the focus on the offloading decision making phase, with the optimization objective of improving the task execution efficiency and reducing the energy consumption of mobile devices by determining the “best fit” execution mode at each decision epoch.
3. Problem Formulation
3.1. Computing Architecture
In this paper, we consider a computing architecture, where the MT can execute its computation task in one of the following three modes:(1)locally: executing the task at the MT’s processor;(2)remotely: offloading the task to a remote computation server via the wireless network;(3)combinedly: executing the task with the former two options simultaneously.Modes 2 and 3 involve the practice of offloading, which is modeled into a three-step process as follows.(i)Data Uploading. MT sends the task specification and input data to the remote computation server.(ii)Server Computing. The remote computation server performs the task.(iii)Results Downloading. The MT downloads the computation results from the remote computation server.
3.2. Execution Completion Time
Let denote the local computation volume of the task. is the speed of local processor, which is a constant value determined by the MT’s CPU capacity. The task execution completion time under the local execution mode can be expressed as
In the remote execution mode, let and denote the data volume for uplink and downlink transmission and the corresponding average transmission capacities of the wireless system are and , respectively. and are determined by multiple parameters, for example, the wireless bandwidth, transmission power, and average channel fading coefficient. For simplicity, the remote server is assumed to have a perfect process capacity so that the computation delay can be ignored. Thus the execution completion time under the remote execution mode can be expressed as the sum of the uplink transmission delay and the downlink transmission delay ; that is,
When the task is executed combinedly, the local processor and the remote server work simultaneously. The execution process ends when either of them finishes the task. So the execution completion time can be expressed as
3.3. Energy Consumption of MT
Energy is primary constraint for mobile devices; therefore, we only consider the energy consumption of the MT. In the local execution mode, is the power of the local processor; the energy consumption of the MT is
In the remote execution mode, the power of transmitting antenna and receiving antenna is and , respectively. The energy consumption of the MT is where and are power consumption of the data uploading and results downloading phases, respectively. When executed combinedly, the energy consumption of the MT can be expressed as
3.4. Adaptive Execution Mode Selection
Under the previous assumptions, the offloading decision problem can be translated into an execution mode selection problem. Based on the optimization object, that is, minimizing task execution delay as well as reducing MTs’ energy consumption, an adaptive execution mode selection scheme (AEMSS) is proposed. Functions of the offloading decision maker include(i)determining the most appropriate execution mode for a specific computing task, that is, local execution, remote execution, or combined execution;(ii)determining if the execution mode needs to be adjusted during the execution process when the unstable factors, for example, the wireless environment condition, have dramatically changed.
The AEMSS makes decisions based on a series of parameters, including the task properties (), MT’s CPU capacity, and the wireless transmission capacity (). The problem is formulated into a finite-horizon Markov decision process (MDP) model and solved by the backward induction algorithm (BIA). Details are elaborated in the next section.
4. The Finite-Horizon MDP-Based AEMSS
In general, an MDP model consists of five elements, that is, (1) decision epochs; (2) states; (3) actions; (4) transition probabilities; and (5) rewards. In this section we will describe how the offloading decision problem can be formulated into a finite-horizon MDP model from these five aspects.
4.1. Decision Epochs and State Space
For simplicity, the time is slotted into discrete decision epochs and indexed by . As Figure 1 shows, time point denotes the observation end time in the MDP model, not the moment the task is just being completed. Time has another sense, that is, the longest task completion time that the user can sustain. The task is considered failed if it is still uncompleted upon time .
At decision epoch , the system state reflects the current task completion progress and the real-time wireless condition, which can be expressed as a tuple the elements in which are explained below:(i): the execution mode adopted in the last time slot ; that is, (ii): the remaining computation volume for local processing by time ;(iii): the remaining data volume for uplink transmission by decision epoch ; if or the uplink transmission has already finished, ;(iv): the remaining data volume for downlink transmission by decision epoch ; if , ;(v): the transmission capacity the wireless network can provide at decision epoch , which is assumed to be static within a time slot and i.i.d. between slots.
In addition, there are two kinds of special states in the state space, that is, the initial states and a terminal state . The initial states are specific at decision epoch and indicate that the task is untreated, while the terminal state indicates that the task execution process has already been completed. Therefore, the state space can be expressed as where , , , , and are subspaces of , the definitions of which are listed in Table 1.
4.2. Action Space and Policy
In this model, there are four actions in the action space ; that is, At decision epoch , AEMSS chooses an action based on the current state . Different actions represent the different execution modes the task will adopt in the following time slot; that is, where indicates that the task has already been completed and nothing needs to be done in the following time slot.
At decision epoch , the decision maker chooses an action from the feasible action set according to the decision rule . In MDP, a policy specifies the decision rule to be used at all decision epochs. It provides the decision maker with a prescription for action selection under any possible future state [17]. In this model, the decision rules at all decision epochs are different; for example, when the task has already been completed, is the only available action. Therefore, the policy obtained is a “nonstationary policy.” In the following parts we will show how the actions can transform the system states.
At time , the system state is . After action is taken the system will transform to the state by time . The feasible cases of states transformation are listed in Table 2. The cases can be split into two categories, that is, (cases 1–3, the current execution mode will continue to be adopted) and (cases 4–9, the current execution mode will be adjusted).
In Table 2, cases 4–9 indicate that the execution mode is changed within two successive time slots as follows.(i)In cases 4 and 5, the task is being executed locally/remotely at time when the decision maker decides to change the execution mode to the remote/local one. In these cases the execution progress before time will be cleared and the task will be forced to be executed from scratch with a different execution mode.(ii)In cases 6 and 7, when the task is being executed locally or remotely, the decision maker wants it to be executed with both modes simultaneously in the next time slot. In these cases the current execution progress will be preserved and a new execution process with another execution mode will begin in the next slot.(iii)In cases 8 and 9, the task is being executed with local mode and remote mode simultaneously, but the decision maker judges that one of them is unnecessary. In the next time slot the execution progress of this mode will be cleared and another one will continue.
4.3. State Transition Probabilities
From Table 2 we can conclude that when a specific action is selected, the system state of the next decision epoch can be determined except the element of wireless transmission capacity . Therefore the state transition probability between two successive decision epochs can be written as The long-term probability distribution of the wireless transmission capacities is denoted as ; thus, the steady state probability distributions of each task state at each decision epoch are denotes the steady state probability of state at decision epoch . denotes the set containing all feasible states at time epoch , that is, states with a steady state probability of .
4.4. Rewards and Value Function
In this model, the system reward function within time interval , , is defined as is the delay reward function and is the energy consumption function. , are weight factors satisfying . is the penalty factor for exceeding the execution delay limit, which is defined as It can be seen that the task will be regarded as a failure if it is still uncompleted at the final decision epoch . The delay reward function is given by where , , , and is the task completion factor at decision epoch , the definition of which is It can be seen that is the percentage completion of the task. is the extra reward the system gains each time slot after the task is completed. The penalty factor and extra reward have the same function, that is, promoting the task to be completed as early as possible.
At decision epoch , the energy consumption function is given by It can be seen that, under the combined execution mode, the task can be accomplished fastest with the price of the highest energy consumption. During the whole time domain from , the expected total reward, that is, the value function, can be expressed as where denotes the expectation value of under policy with the initial state . The optimization objective is to find an optimal policy which satisfies for all initial states and all . is the set containing all feasible policies.
4.5. Solution for Finite-Horizon MDP
For an infinite-horizon MDP, there are various mature algorithms available for reference, for example, value iteration, policy iteration, and action elimination algorithms [17]. In this part, the solution for the proposed finite-horizon MDP model is discussed.
On time domain , sequence is called a “history” of the system and . Let for denote the total expected reward obtained by using policy at decision epochs with a history of ; that is, When , . From [17], the optimal equations are given by for and , .
From above we can see that corresponds to the maximum total expected reward . The solutions satisfying the optimal equations are the actions , , and which can achieve the maximum total expected reward; that is,
In this paper, the BIA [17] is employed to solve the optimization problem given by (23) as follows.
The Backward Induction Algorithm (BIA)(1)Set and for all .(2)Substitute for and compute for each by Set (3)If , stop. Otherwise return to step 2.
5. Model Implementation Issues
In this section, discussions are made on some issues occurring when the proposed MDP-based AEMSS is implemented to a real system.
5.1. The Offloading Decision Process
Figure 2 illustrates the workflow of the AEMSS, which can be partitioned into two parts, that is, the offline policy making and the online action mapping.(i)Offline Policy Making. The execution mode selection policy is calculated offline via BIA, the input data of which includes task properties, wireless network characteristics, and the MT’s capacities. Before execution, the computation task needs to be profiled first to determine the description parameters, that is, , , , and the time threshold . Then the parameters will be submitted to the server for policy determination. The server will also evaluate the wireless channel condition to determine the transition probability matrix of the wireless capacities. In this model, the optimal policy achieved, that is, the output of BIA, is a nonstationary policy. Thus for each decision epoch , there will be a matrix reflecting the correspondence between states and actions and all the matrixes can form a -dimension state-to-action mapping table.(ii)Online Action Mapping. By employing BIA, the optimal execution mode selection policy can be obtained and stored in advance. At decision epoch after the execution begins, the MT evaluates the channel state information by measuring the fading level of the reference signal and reports it to the parameter collection module, which monitors the task execution progress and collects necessary parameter values such as . All the real-time parameter values are mapped into a certain system state . Then by looking up the state-to-action mapping table, the optimal action can be chosen and the task execution mode in the following time slot is determined.
5.2. Measurement of the State Transition Probabilities
The changing process of a wireless network is complicated and difficult to predict. There are many factors that can influence the wireless transmission capacity, for example, bandwidth, user number, and resource allocation scheme. In this paper, a system simulation for a typical 3GPP network is conducted to estimate the state transition probabilities, where users arrive and depart from the network according to a Poisson process. Main parameters in simulation are listed in Table 3. The proposed scheme is adaptive and applicable to a wide range of conditions. Different wireless networks have different state transform situations, which can be obtained by changing the parameter set in the simulation.
6. Numerical Results and Analysis
In this section, the performance of the proposed MDP-based AEMSS is evaluated with four other offloading schemes, including three static schemes and a dynamic one. The policy achieved by solving the finite-horizon MDP is denoted as ; other policies for comparison are referred to as , , , and , the descriptions of which are listed in Table 4.
Firstly, a qualitative analysis on the optimal policy is provided to reflect the relationship between the chosen execution modes and the task characteristics. A series of computation tasks with different characteristics (, , and ) are analyzed. The probability of choosing a certain action is defined as where is the action decision rule at decision epoch . The definition of operator is
Figure 3 shows the probability of adopting the three execution modes versus the ratio . It can be seen that when is relatively small to , the probability of adopting the local execution mode is high. With the rising of , the probability of adopting the remote execution mode goes to 1. The conclusion is obvious: offloading is beneficial when large amounts of local computation volume are needed with relatively small amounts of data transmission volume, and vice versa.
Next we consider a computation task with a comparative local computation volume and data transmission volume so that making offloading decisions is relying more on the real-time environmental information. The task description parameters adopted in the simulation are other parameters are listed in Table 5.
The performance metric adopted is the expected total reward defined in Section 4 with different weight factors and different initial states. Figure 4 shows the performance of the MDP-based AEMSS and other four schemes, that is, always local, always offloading, and always combined schemes and a dynamic scheme that makes offloading decisions based on the wireless transmission capacity at the beginning of the execution (called DY scheme afterwards).
(a)
(b)
(c)
(d)
From Figures 4(a)–4(d) it can be concluded that (a) with higher wireless transmission capacity, the MDP-based policy gains a better performance, while the performance of always local scheme stays at a consistent level for the wireless transmission condition has no effect on the local execution process. (b) The always offloading scheme gains a pretty good performance, almost equivalent with the proposed AEMSS when the wireless transmission capacity is high whereas when the wireless transmission capacity decreases, the performance gap between them gets wider. (c) When the weight factor of energy consumption function is high, the performance of always combined policy is poor, because executing a task in local and remote modes simultaneously is an energy-intensive practice. However, when the weight factor of delay reward function increases, its performance improves and is equal to the AEMSS when the weight factor of delay reward function is 1. Under these circumstances, the combined execution mode is the optimal mode for its task completion time is shortest. (d) The performance of the DY mechanism is superior to the other three static policies, for it can react to the real-time environment condition. The performance gap between it and the AEMSS is caused mainly by the execution mode adjustment mechanism of AEMSS.
We integrate the expected total reward with different initial states by and the integrated performance of different policies is shown in Figure 5. Figures 4 and 5 reflect a phenomenon that the expected total reward increases linearly with the weight factor . This is driven by the design of the reward function, not indicating that a higher weight factor of the delay reward function is better. As defined in (16), the system will gain an extra reward at each time slot after the task execution is completed. A higher will push the task execution to be finished earlier; therefore, the system can gain more extra reward until time . When employing the AEMSS, the weight factors are determined by the designer’s preference, that is, delay oriented or energy oriented.
indicates that the task execution process has been completed at time epoch . Therefore, the completion probability of the task can be estimated by the steady state probability of the terminal state at decision epoch ; that is, Figure 6(a) depicts the task completion probabilities at each decision epoch with different policies when . We can see that the always combined scheme can complete the task fastest. The local computation volume is set as in the simulation; therefore by time the always local and always combined schemes can achieve a task completion probability of 1. The always offloading policy can complete the task with the highest probability when , but this may also leave the task uncompleted when with the highest probability. The delay performance of proposed MDP-based AEMSS is at an intermediate level because it also takes the energy consumption into consideration. Figure 6(b) illustrates the task completion probabilities with different weight factors. We can see that, with a higher weight factor of the delay reward function, the task execution will be finished faster. When , the combined execution mode will be adopted with probability of 1; therefore, the task will be finished with probability 1 at time ().
(a) Task completion probabilities under different policies
(b) Task completion probability with different weight factors
Figure 7 illustrates the tradeoff between the time saving and energy consumption of the AEMSS when the weight factors are varying. At , the delay performance and the cumulative energy consumption under the optimal policy are plotted. It can be concluded that, with a higher , the task will be completed faster and the energy consumption will increase accordingly. This is because the combined execution mode is more likely to be adopted when the delay requirement is strict and executing the task both locally and remotely is energy intensive.
As described in Section 4, the AEMSS can adjust the execution mode during the task execution process when the wireless condition has dramatically changed. That is the main reason behind the performance improvement in our proposed scheme compared to the general dynamic offloading schemes. An observation is taken on the execution mode adjustment frequency at all the decision epochs. At decision epoch , an “execution mode adjustment” event, which is denoted as , occurs when and the occurrence probability of event at decision epoch is defined as Figure 8 shows the execution mode adjustment probability at all decision epochs. Along with the timeline, the execution mode adjustment probabilities reduce to zero gradually. The reason is that, with the growth of the execution progress, adjusting the execution mode will cost a heavier price.
7. Conclusion
In this paper, MTs can execute their computation tasks either (1) locally; (2) remotely; or (3) combinedly. To determine the most appropriate execution mode, a dynamic offloading decision scheme, that is, the AEMSS, is proposed. The problem is formulated into a finite-horizon MDP with the objectives of minimizing the execution delay and reducing the energy consumption of MTs. Offloading decisions are made by taking the task characteristic and the current wireless transmission condition into an overall consideration. In the design of reward function, an execution threshold time is introduced to make sure that the task execution can be completed with an acceptable delay. In addition, a novel execution mode adjustment mechanism is introduced to make the task execution process more flexible for the real-time environment variation. By solving the optimization problem with the BIA, a nonsteady policy describing the correspondence of states and actions is obtained. The policy is equivalent to a state-to-action mapping table which can be stored for looking up during the decision making phase. The performance of the proposed scheme is evaluated with other several offloading schemes and the numerical results indicate that the proposed scheme can outperform other algorithms in an energy-efficient way.
Conflict of Interests
The authors declare that there is no conflict of interests regarding the publication of this paper.
Acknowledgments
This work was supported in part by the Fundamental Research Funds for the Central Universities (no. 2014ZD03-02), National Key Scientific Instrument and Equipment Development Project (2013YQ20060706), and National Key Technology R&D Program of China (2013ZX03003005).