Abstract
Scalable video coding allows an efficient provision of video services at different quality levels with different energy demands. According to the specific type of service and network scenario, end users and/or operators may decide to choose among different energy versus quality combinations. In order to deal with the resulting trade-off, in this paper we analyze the number of video layers that are worth to be received taking into account the energy constraints. A single-objective optimization is proposed based on dynamically selecting the number of layers, which is able to minimize the energy consumption with the constraint of a minimal quality threshold to be reached. However, this approach cannot reflect the fact that the same increment of energy consumption may result in different increments of visual quality. Thus, a multiobjective optimization is proposed and a utility function is defined in order to weight the energy consumption and the visual quality criteria. Finally, since the optimization solving mechanism is computationally expensive to be implemented in mobile devices, a heuristic algorithm is proposed. This way, significant energy consumption reduction will be achieved while keeping reasonable quality levels.
1. Introduction
The evolution of multimedia encoding techniques allows efficiently provisioning video services at different quality levels. However, resulting streams lead also to different energy consumptions making it difficult to simultaneously satisfy both energy consumption and quality requirements. Therefore, an energy versus quality compromise solution is commonly required. In commercial cellular networks, users are used to dealing with these trade-offs either manually or automatically (i.e., using small widgets to reduce display brightness, disable radio interfaces, etc.) and normally maintaining the same play-out quality. However, reduced energy consumption becomes a truly severe constraint in specific communication scenarios such as mobile emergency networks or distributed sensors. Additionally, any solution will also depend on the characteristics of the video players although higher resolution video could improve visual quality for high-end mobile devices, for others no visible quality improvement is achieved due to available screen resolution, codecs, or CPU power. So, additional energy consumption, higher data bandwidth, and spectrum use would have no real impact on users satisfaction. Energy- and visual quality-aware video dynamic transmission schemes would allow network operators and users to avoid such waste of resources.
In order to cope with the heterogeneity of mobile devices and user requirements for efficient mobile video delivery, a multilayer scheme is broadly considered as the best solution. In this paradigm, each video is encoded into a single stream with multiple layers, where each layer is only transmitted once. Scalable Video Coding (SVC) standard, an extension for H.264/AVC standard, makes this multilayering possible, becoming the most promising encoding technology for solving the problem of multiuser video streaming in most mobile environments (see [1, 2]). This mechanism for content delivery provides quality differentiation, so that the same content is sent simultaneously in different qualities without replicating the original information. This way, users that demand lower energy consumption can maintain the reproduction but accepting lower quality level.
For example, [3–5] have proposed broadcasting schemes that would allow mobile devices to receive and decode the most suitable number of layers, maintaining the perceived video quality proportional to the consumed energy in a DVB-H scenario. Under typical system parameters of mobile TV networks, the proposed schemes allow mobile devices to achieve energy savings between 60% and 95% depending on how many layers they receive. In [5], not only does the proposed scheme enable each device to achieve energy saving proportional to perceived quality but also low channel switching delays are guaranteed, which is also important to user experience.
However, in these works the expected visual quality level as perceived by the user is not quantified, which is fundamental to find a relation between energy consumption and user Quality of Experience (QoE).
In [6], authors follow a similar approach but in a 802.11e environment by using sleep cycles of wireless adapter for energy saving. In this case, a simple QoE estimation algorithm is used to trigger specific power save protocol operations. However, they only modify the behaviour of the receiver with a single version of the content. Furthermore, no analysis of the optimality of their approach is included.
Such kind of optimization in 802.11 is carried out in [7], including both linear programming techniques and heuristics. Unfortunately, only energy is considered as optimization criteria and no effect into quality is analyzed.
Finally, [8] proposes a method for statically selecting the best number of layers for the whole duration of a video with energy constraints. However, no compromise solution is provided and DVB-H scenarios only are considered.
In order to cover these lacks, in this paper we analyze the optimal strategy to be applied if we can modify dynamically the number of layers during the video reproduction considering energy and quality constraints. Therefore, the main contributions of the paper are as follows. (1) We analyze the energy versus QoE trade-offs considering either energy or quality as constraints. (2) We formulate the aforementioned scenarios in terms of single-objective linear programming (SOLP) problems and analyze the optimal sets in both the input and the objective space. (3) We study the multiobjective problem therefore allowing deployers to fine tune the relative weights of green consciousness and quality criteria. (4) We propose lightweight heuristics for energy- and QoE-aware optimization that ensure a feasible implementation on the end user while providing near-optimal solutions.
Therefore, the rest of this paper is structured as follows. In Section 2, we analyze the dual energy minimizing and QoE maximizing problem while alternating the number of reproduced layers during a video session. Section 3 considers users that have both energy and QoE related criteria, and Section 4 summarizes achieved results.
2. Energy/QoE Single Optimization
In this section, we will express the Energy versus QoE trade-off in terms of a typical optimization problem with different objectives and constraints, analyze the optimal strategy, and compare it with the traditional ones.
Traditional QoE-aware energy-constrained video reproduction strategies select the maximum number of layers that the available battery [8] and/or CPU load [9] would allow for the full video playout in an static way, so that the decision is taken just once. We will generalize and refer to this kind of strategies as basic strategy.
Considering (1) that each additional layer provides different QoE level (see, e.g., [10, 11]) and energy consumption and (2) that SVC players [12] support switching from a layer to another, we will instead propose a dynamic method for triggering layer switching considering energy constraints.
We will therefore focus on selecting the best set of different time periods so that during layers will be reproduced.
For simplicity purposes, we will consider 4 layers to illustrate the method. For a video of duration it is clear then that . In our optimization problem will be the input variables.
In order to define our optimization problem completely, we will consider that the average of the satisfaction over the whole video reproduction is a good estimator of video quality. Then, we have an optimization problem consisting of the following. (A2.1) Minimizing energy consumption for a given video and certain minimum acceptable visual quality. (A2.2) Maximizing user QoE for a given video and certain maximum energy constraint.
Any single-objective optimization problem (SOP) [13] like (A2.1) and (A2.2) aims at choosing the “best” possible combination of input parameters in order to optimize the one considered criterion.
Let be a vector of input variables of the optimization problem.
The SOP can be stated as follows: where is the set of feasible points in the input space delimited by inequalities and equalities such as Thus, the SOP could be summarized as “finding the set of input variables belonging to region so that is ,” where is constrained by inequalities and equalities .
If both is linear and is defined by linear conditions (therefore both and are linear functions as well), then the SOP is called single objective linear programming problem (SOLP) [14].
The basic metric that we will use for estimating the whole user satisfaction regarding video quality will be the average value of the Mean Opinion Score (MOS) according to the number of layers reproduced along each time period. Therefore, where is the evolution of the QoE measured in the MOS scale along the th period.
If the visual quality is considered roughly constant for a certain bitrate or number of layers and we denote as the MOS estimated for a certain number of layers according to subjective tests, then for the whole period. Therefore, are the normalized quality coefficients, and (3) can be expressed as follows:
Similarly, according to [8, 15], the battery consumption in mobile video players shows some kind of dependence on the number of layers received. Therefore, the total energy consumption will depend on the number of layers that will be received so that total consumption along the whole reproduction time can be expressed as follows: where is the vector of normalized coefficients of battery consumption.
This way, problem (1) can be expressed as a SOLP problem for both (A2.1) and (A2.2).
2.1. Minimizing Energy Consumption Assuring a QoE Threshold (A2.1)
In this case, the objective function to be minimized is the energy consumption so that . Minimum energy consumption will be constrained by the QoE threshold, the user will stand () in terms of . This constraint will lead to the associated inequality in (7). Similarly, the sum of the time period for all the layer numbers must be the total duration of the video leading to an equality constraint expressed with the identity vector . Consider the following: where the natural condition for all is included in the matrix notation in (7) using the identity matrix .
Then, the SOLP problem in (7) can be solved by typical well-known optimization problem resolution mechanisms such as the simplex method [16]. Therefore, we can easily calculate the solution of the SOLP for a wide range of constraints and parameters. In fact, considering the proposed transmission scheme in [8], we have carried out the optimization for the parameters collected in Table 1 and obtained Figures 1 and 2. Note that the normalized available battery is measured in seconds to avoid specific details about battery characteristics (like voltage and capacity in mAh).
For mobile videos with different motion levels or Content Types (CTs) (referred to as Low Motion (LM), Medium Motion (MM), and High Motion (HM)) and Spatial Resolutions (SRs) we have calculated the QoE-optimal layer selection strategy with the energy constraint. Note that, although we have considered a specific scenario in our study, the method can be applied to any multilayer broadcasting technology just by considering related and coefficients.
In order to calculate realistic QoE constraints, we must consider and : the minimum and maximum achievable considering and , respectively (i.e., receiving only 1 layer—worst quality—or 4 layers—best quality—during the whole video length). According to the metric used and . Finally, since the additional constraint of the total energy consumption must be less than the available battery (if a handheld device) or the energy budget, not all points will be feasible.
In Figure 1, the objective space (namely, minimum battery consumption versus QoE threshold) and the finite decision points associated to the traditional selection of a fixed number of layers for the whole video are shown for a certain CT/SR combination. Considered remaining battery constraint results in a Feasible Point (FP) that represents the maximum achievable QoE level. Both this FP and discrete no. L points are shown (the points related to reproducing a certain number of layers during the whole play-out time, ).
We can see how, by changing the number of layers reproduced along the video duration, we can optimize the energy consumption with finer grain QoE constraints. This way, for traditional static strategies would result in selecting 2 layers (2L) for the whole duration of the video and consuming associated energy. Our dynamic approach would instead allow the selection of different (energy, QoE) points. On the other hand, if we compare in detail both approaches for this concrete CT and SR (LM and QVGA), obtained optimum strategy does not match the simplest one in 2L point. This clearly reflects the energy saving achieved as for the same MOS less energy is consumed.
Additional results are depicted in Figure 2 for every combination of video types and encoding resolutions considered. Examining in detail the figure we can see how optimization strategy does not match the traditional static approaches (fixed no. L points). Furthermore, vertical lines show the result of ensuring certain QoE levels for different CTs. For example, for selected thresholds, the QCIF versions are better in terms of energy consumption for HM and MM videos, while QVGA is better for the threshold considered in the LM case.
2.2. Maximizing User QoE Constrained by Maximum Energy Consumption (A2.2)
For (A2.2), in an analogous way to Section 2.1, the objective function to be maximized is the MOS so that . Maximum achievable QoE will be constrained by the maximum battery consumption . This constraint will lead to the associated inequality in the following:
Figure 3 depicts the evolution in the objective space of the proposed optimal strategy in comparison with the basic one considering #L points only. Once more, we can see how the proposed scheme allows us to set a continuous range of energy constraints resulting in different values of . In this case, an available energy budget in the range would collapse into the same 1L point, therefore allowing only minimum .
When we compare the obtained results for different CT/SR combinations (in Figure 4), we can see, again, how the QoE maximizing quality selection mechanism would choose QCIF (for the particular energy saving constraints marked with a vertical line in the figure) for LM and MM videos, while QVGA for the HM one.
2.3. Comparison between SOLP Approaches
In Sections 2.1 and 2.2, we have used the simplex optimization method for SOLPs in order to optimize either energy for a given QoE threshold or QoE for a given maximum battery consumption. Due to the nature of the linear optimization problem, if we compare both optimization approaches, we can see in Figure 6 that both lead to the same shape in the objective space. This conclusion is consistent with the nature of the dual problem itself, since energy and QoE are opposite objectives, and the constraint in one problem becomes the objective function in the other.
Since carrying out the simplex method for the optimization of every video reproduction would be inviable for handheld devices due to the high CPU power needed, we focus on developing heuristics capable of providing near-optimal solutions. This way, both video operators and end users themselves would be able to reduce their energy consumption while maintaining QoE levels. In order to do so, we have analyzed more deeply the shape of the optimal set (namely, Pareto front in the objective space) for three different situations and compared it with the #L points. We can see (Figure 5) how the optimal strategy for energy versus QoE, depending on and evolution: (1)follows exactly the line between 1L and 4L (including 2L and 3L), constrained by the FP;(2)follows the polygon (the 2D polytope in the objective space) 1L-2L-3L-4L constrained by the FP;(3)follows the 1L and 4L but without going through 2L and 3L points. We conclude that, regardless of the specific parameters of the SOLP problem to be solved, the optimal strategy always includes 1L and 4L points (or associated FP point if 4L is not feasible due to the battery constraint). This result is quite evident since reproducing just 1 or all the 4 layers gives the minimum and maximum QoE and battery. However, both 2L () and 3L () points are not always optimal points (i.e., they do not belong to the Pareto front). As a result, the simple strategy of following the 1L-2L-3L-4L path must be carefully reviewed.
(a)
(b)
Therefore, in order to propose an optimization strategy, we must evaluate the evolution of the Pareto front in both the objective and the input space for every possible situation (1, 2, and 3).
In Figures 6 and 7, we can see such evolution for the objective and input space in the aforementioned Situations 1 and 2. Figure 6 confirms the superposition of both (A2.1) and (A2.2) Pareto fronts in the objective space for both situations. In Figure 7, we depict the 4D input space with the points that belong to the Pareto front in both situations. In order to do so, space coordinates correspond to , , and , respectively. The fourth dimension is represented by the area of the sphere in the point. Finally, the result of the objective function is represented by the color of the sphere according to the colormap shown. , , , and points are also depicted with spheres.
(a)
(b)
When comparing Situations 1 and 2 in the input space, we can see how, for the former, the optimal path does not follow any simple strategy. However, the latter follows a linear path in a set of consecutive planes, which leads to easy-to-implement heuristics.
After comparing the figures for for every considered video CTs and SRs and the evolution of the energy consumption versus the number of layers, we conclude that all of them follow the Situation 2. In fact, if we analyze the sufficient conditions leading to Situation 2 it is clear that, if the function in the objective space is convex, the resulting shape would belong to this group. Most QoE studies aiming at mapping satisfaction versus network performance parameters use logarithmic expressions (see [17]), so that they are convex. In our case, due to this convexity of the MOS expression and the proportional dependency of the energy with for all , an increment achieved by the simplex algorithm in terms of would be caused by a displacement vector leading to a point above the 1L–4L line in the objective space. An equivalent conclusion will be obtained if we calculate the gradient of both MOS and energy, or if we consider that many optimization methods indeed provide the convex part of the Pareto front.
Therefore, in this case the associated optimization heuristic is straightforward, regardless either optimization approach 1 (minimize energy for a certain QoE) or 2 (maximize QoE for a certain energy budget) is applied. Algorithm 1 shows the complete optimization heuristic procedure. The algorithm is based on the fact that the optimal polytope will follow the 1L-2L-3L-4L path (constrained by the FP). Additionally, we take into account that, due to the linear constraints, the polytope will be formed by the intersection of planes leading to lines in the different 2D planes formed by the successive input parameters. So, since is the 4D input space, the optimization heuristic will consider the polytope delimited by 1L, 2L, 3L, and 4L/FP points and planes , and . Note here that along the different figures in this paper, we have set a maximum energy constraint leading to an FP between 3L and 4L for illustration purposes. In a real scenario, the FP could be anywhere in the 1L–4L Pareto Front.
| ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
Regardless our aim is minimizing energy for a certain QoE or maximizing QoE with a certain energy budget, we will calculate the 2D plane where the target point in the input space will be located at. Since both energy and QoE are monotonically growing with for all either the QoE or the battery constraint will allow us to select the plane by evaluating the or intervals where . Later, the linear dependence will allow us to express as a function of provided constraints (see lines 18–21 and 28–30).
In Figure 8, the results obtained with the heuristics for both Situations 1 and 2 in the objective space are depicted. Note that, even for Situation 1 the heuristic provides values very close to the Pareto set. The reason is that 2L, 3L and the points in the 1L–4L path belong to the Pareto front. At the same time, when approaching these points from and , there exist different combinations of that result in the equivalent point in the objective space. Therefore, depending on the path followed by the simplex algorithm and the stopping thresholds, the simulation would provide different points in the input space, but the heuristic is still capable of providing equivalent optimal points in the objective space.
(a)
(b)
Figure 9 shows the effect of applying developed algorithm for Situation 3. As already mentioned, although each case should be evaluated in terms of and in order to estimate associated situation, the convex shape of the MOS function makes this situation highly improbable. Anyway, the equivalent heuristic is again quite simple, since the Pareto front follows the 1L–4L path.
(a)
(b)
3. Hybrid Approach: Optimizing Energy Consumption and QoE
In the previous section, we have focused on analysing the problem of optimizing one single objective (i.e., either energy or QoE) in our mobile video scenario. Therefore, we have expressed both optimization problems in terms of SOP. The considered metric allowed us to reduce such problems to SOLP ones and describe them with the matrix notation in (7) and (8).
However, real users do not usually consider a single criterion while evaluating a product or a service [18]. Generally speaking, most users will not care about energy consumption or quality in an isolated way but will take into account both criteria. On one hand, commercial users would probably prefer assuring higher quality for playing movies. In other scenarios, such as aforementioned emergency networks, they would instead put the emphasis on preserving battery. However, minimum image quality levels should be also provided to keep quality of information so that, for example, first responders would be still capable of evaluating the risks of an emergency. Therefore, the two original SOPs merge into a more complex Multiple Objective Optimization Problem (MOOP) including two objectives: minimizing energy consumption (i.e., ) and maximizing QoE (i.e., ).
Then, the MOOP is an extension of the SOP, which can be defined as follows: where is the th criterion function.
The simplest solution for the MOOP problem consists of finding the input vector so that In most of the cases, there will not exist such which maximizes all the criteria simultaneously. So, we will have to redefine the nature of the problem by introducing the concept of utility function, . Then, the real formulation of the MOOP can be expressed mathematically as follows:
Then, any MOOP requires the definition of a utility function that collects users’ preferences regarding different considered criteria as in (11). Many authors have considered the linear composition of preferences with different weights to express the articulation of preferences in communications systems with methods such as the Analytic Hierarchy Process (AHP; see, e.g., [19]) to infer the weight of each criterion out of user surveys. If we consider that our utility function follows this linear approach: where, if we normalize and between the same value ranges (i.e., as in the MOS scale), associated weights will follow so that we could express the utility function in a linear way as in the following:
Since both and (normalized ) are related to QoE and battery criteria, respectively, if we consider similar expressions as those in (7) and (8) for the objective functions, we could expand expression (13) as follows: where is the vector of normalized battery parameters so that . Note that, contrary to , coefficients will decrease with for all since users satisfaction regarding battery will decrease when the number of layers and therefore the battery consumption grows. If we manipulate this expression for the utility function, we can express the complex MOOP into a simplified SOLP similar to (7) and (8) as follows: where . In order to calculate , we carry out a mapping between the actual battery consumption and associated satisfaction, where will be associated with the maximum satisfaction and with the lowest one (maximum consumption).
Once is calculated and maximum battery and minimum QoE are set, we can use simplex again in order to optimize users’ satisfaction considering both battery and QoE. Figure 10 depicts the evolution of the optimal strategy in the objective space for different values of , the relative weight of both criteria. Both hybrid optimization (with proposed utility function) together with maximizing QoE only policy, static assignment and proposed heuristics are shown. The different values for represent how important is the “Green characteristic” of the device for the user and/or video provider when compared with video quality (with completely important and not important at all).
We have computed only the range , since the same figures were obtained for lower values of . We can see how, for all , the optimal shape is restricted to a single point (corresponding to 1L). The reason is that, considering the low importance of QoE, the optimization algorithm considers that it is always more convenient to restrict the energy consumption rather than receive more layers and obtain better video quality. The higher the value, the closer the Pareto fronts gets to our original single-objective optimization.
In Figure 11, we carry out the equivalent analysis in the input space. Once again, the Pareto front for considered and leads to a simplified optimization strategy for . In this case, however, the evolution is from plane to plane .
In order to get the exact point for every and constraint pair, we carry out the same equations as in lines 18–21 and 28–30 of Algorithm 1.
Therefore, in order to solve the MOLP problem, we could use any LP optimization technique (i.e., simplex or Interior Points Method) or analyze the problem in a case per case basis in order to infer whether a simple per-plane heuristic could be applied. In any case, the complexity of the solution is low. For example, although simplex’s complexity analysis is rather a problem dependent leading even to worst-case exponential, due to the small size of the input matrix, results are obtained in less than 20 iterations, and empirical tests have led to less than 30?ms of CPU consumption.
4. Conclusions
In this paper, we analyze the trade-off between energy consumption and visual quality for mobile video systems.
Different optimization approaches have been evaluated. The simplest static strategy comprises receiving the highest number of video layers while coping with the video duration requirements. Thus, taking as inputs the video length and the amount of remaining battery, we always select the best possible visual quality. The main drawback of this approach is that all the battery is available to be used in the video playout.
In Section 2, we introduce a single-objective optimization problem as a way to provide an automated decision making process to the mobile device. Two approaches have been defined and solved by linear programming: energy consumption minimization constrained to a minimal QoE threshold, and QoE maximization constrained to a maximum level of battery consumption. Contrary to the previous case, the decision maker may provide a noninteger number of layers, providing a finer grain resolution for quality optimization. Yet although we are able to introduce additional energy constraints to the automated decision making, the desired remaining battery level must be a priori computed without further information of the achievable QoE level.
Additionally, once the energy or quality constraints are assured, the single-objective optimization will lead to the feasible point of maximum quality or minimum energy consumption. Therefore, this approach does not explore the intermediate points as possible optimum solutions, where we can make use of the different relations between increased energy consumption and enhanced visual quality.
In order to overcome this drawback, we analyze in Section 3 the problem from a multiobjective optimization standpoint. Both energy and quality are considered as objective functions by means of a weighted utility function, which allows us to solve the problem as a single-objective linear programming problem. Different weights have been evaluated, which entail different priority to energy saving or required quality. These weights could be used to define different user profiles, different device energy saving modes, or dynamically adapted based on the status of the device battery.
Since the implementation of the optimization algorithm in a mobile handset may result on a resource-consuming process, we propose the use of a heuristic algorithm. From the analysis of the evolution of both the objective function and input variable spaces, different alternatives are found for the shape of the Pareto front. However, considering the logarithmic shape of the evolution of most MOS-related utility functions, a simple heuristic has been proposed. As a result, the proposed algorithm can be run on a mobile device as a decision making process to trigger the switching between layers.
Acknowledgments
The research leading to these results has received funding from the European Union Seventh Framework Programme (FP7/2007–2013) under Grant agreement 284863 (FP7 SEC GERYON) and from the Basque Country under BASAJAUN Project.