Abstract
Network Functions Virtualization (NFV) is a network architecture concept where network functionality is virtualized and separated into multiple building blocks that may connect or be chained together to implement the required services. The main advantages consist of an increase in network flexibility and scalability. Indeed, each part of the service chain can be allocated and reallocated at runtime depending on demand. In this paper, we present and evaluate an energy-aware Game-Theory-based solution for resource allocation of Virtualized Network Functions (VNFs) within NFV environments. We consider each VNF as a player of the problem that competes for the physical network node capacity pool, seeking the minimization of individual cost functions. The physical network nodes dynamically adjust their processing capacity according to the incoming workload, by means of an Adaptive Rate (AR) strategy that aims at minimizing the product of energy consumption and processing delay. On the basis of the result of the nodes’ AR strategy, the VNFs’ resource sharing costs assume a polynomial form in the workflows, which admits a unique Nash Equilibrium (NE). We examine the effect of different (unconstrained and constrained) forms of the nodes’ optimization problem on the equilibrium and compare the power consumption and delay achieved with energy-aware and non-energy-aware strategy profiles.
1. Introduction
In the last few years, power consumption has shown a growing and alarming trend in all industrial sectors, particularly in Information and Communication Technology (ICT). Public organizations, Internet Service Providers (ISPs), and telecom operators started reporting alarming statistics of network energy requirements and of the related carbon footprint since the first decade of the 2000s [1]. The Global e-Sustainability Initiative (GeSI) estimated a growth of ICT greenhouse gas emissions (in GtCO2e, Gt of CO2 equivalent gases) to 2.3% of global emissions (from 1.3% in 2002) in 2020, if no Green Network Technologies (GNTs) would be adopted [2]. On the other hand, the abatement potential of ICT in other industrial sectors is seven times the size of the ICT sector’s own carbon footprint.
Only recently, due to the rise in energy price, the continuous growth of customer population, the increase in broadband access demand, and the expanding number of services being offered by telecoms and providers, has energy efficiency become a high-priority objective also for wired networks and service infrastructures (after having started to be addressed for datacenters and wireless networks).
The increasing network energy consumption essentially depends on new services offered, which follow Moore’s law, by doubling every two years, and on the need to sustain an ever-growing population of users and user devices. In order to support new generation network infrastructures and related services, telecoms and ISPs need a larger equipment base, with sophisticated architecture able to perform more and more complex operations in a scalable way. Notwithstanding these efforts, it is well known that most networks and networking equipment are currently still provisioned for busy or rush hour load, which typically exceeds their average utilization by a wide margin. While this margin is generally reached in rare and short time periods, the overall power consumption in today’s networks remains more or less constant with respect to different traffic utilization levels.
The growing trend toward implementing networking functionalities by means of software [3] on general-purpose machines and of making more aggressive use of virtualization—as represented by the paradigms of Software Defined Networking (SDN) [4] and Network Functions Virtualization (NFV) [5]—would also not be sufficient in itself to reduce power consumption, unless accompanied by “green” optimization and consolidation strategies acting as energy-aware traffic engineering policies at the network level [6]. At the same time, processing devices inside the network need to be capable of adapting their performance to the changing traffic conditions, by trading off power and Quality of Service (QoS) requirements. Among the various techniques that can be adopted to this purpose to implement Control Policies (CPs) in network processing devices, Dynamic Adaptation ones consist of adapting the processing rate (AR) or of exploiting low power consumption states in idle periods (LPI) [7].
In this paper, we introduce a Game-Theory-based solution for energy-aware allocation of Virtualized Network Functions (VNFs) within NFV environments. In more detail, in NFV networks, a collection of service chains must be allocated on physical network nodes. A service chain is a set of one or more VNFs grouped together to provide specific service functionality and can be represented by an oriented graph, where each node corresponds to a particular VNF and each edge describes the operational flow exchanged between a pair of VNFs.
A service request can be allocated on dedicated hardware or by using resources deployed by the Service Provider (SP) that processes the request through virtualized instances. Because of this, two types of service deployments are possible in an NFV network: (i) on physical nodes and (ii) on virtualized instances.
In this paper, we focus on the second type of service deployment. We refer to this paradigm as pure NFV. As already outlined above, the SP processes the service request by means of VNFs. In particular, we developed an energy-aware solution to the problem of VNFs’ allocation on physical network nodes. This solution is based on the concept of Game Theory (GT). GT is used to model interactions among self-interested players and predict their choice of strategies to optimize cost or utility functions, until a Nash Equilibrium (NE) is reached, where no player can further increase its corresponding utility through individual action (see, e.g., [8] for specific applications in networking).
More specifically, we consider a bank of physical network nodes (in this paper, we also use the terms node and resource to refer to the physical network node) performing tasks on requests submitted by players’ population. Hence, in this game, the role of the players is represented by VNFs that compete for the processing capacity pool, each by seeking the minimization of an individual cost function. The nodes can dynamically adjust their processing capacity according to the incoming workload (the processing power required by incoming VNF requests) by means of an AR strategy that aims at minimizing the product of energy consumption and processing delay. On the basis of the result of the nodes’ AR strategy, the VNFs’ resource sharing costs assume a polynomial form in the workloads, which admits a unique NE. We examine the effect of different (unconstrained and constrained) forms of the nodes’ optimization problem on the equilibrium and compare the power consumption and delay achieved with energy-aware and non-energy-aware strategy profiles.
The rest of the paper is organized as follows. Section 2 discusses the related work. Section 3 briefly summarizes the power model and AR optimization that was introduced in [9]. Although the scenario in [9] is different, it is reasonable to assume that similar considerations are valid here, too. On the basis of this result, we derive a number of competitive strategies for the VFNs’ allocation in Section 4. Section 5 presents various numerical evaluations, and Section 6 contains the conclusions.
2. Related Work
We now discuss the most relevant works that deal with the resource allocation of VNFs within NFV environments. We decided not only to describe solutions based on GT, but also to provide an overview of the current state of the art in this area. As introduced in the previous section, the key enabling paradigms that will considerably affect the dynamics of ICT networks are SDN and NFV, which are discussed in recent surveys [10–12]. Indeed, SPs and Network Operators (NOs) are facing increasing problems to design and implement novel network functionalities, following rapid changes that characterize the current ISPs and telecom operators (TOs) [13].
Virtualization represents an efficient and cost-effective strategy to exploit and share physical network resources. In this context, the Network Embedding Problem (NEP) has been considered in several recent works [14–19]. In particular, the Virtual Network Embedding Problem (VNEP) consists of finding the mapping between a set of requests for virtual network resources and the available underlying physical infrastructure (the so-called substrate), ensuring that some given performance requirements (on nodes and links) are guaranteed. Typical node requirements are computational resources (i.e., CPU) or storage space, whereas links have a limited bandwidth and introduce a delay. It has been shown that this problem is NP-hard (it includes as subproblem the multiway separator problem). For this reason, heuristic approaches have been devised [20].
The consolidation of virtual resources is considered in [18], by taking into account energy efficiency. The problem is formulated as a mixed integer linear programming (MILP) model, to understand the potential benefits that can be achieved by packing many different virtual tasks on the same physical infrastructure. The observed energy saving is up to 30% for nodes and up to 25% for link energy consumption. Reference [21] presents a solution for the resilient deployment of network functions, using OpenStack for the design and implementation of the proposed service orchestrator mechanism.
An allocation mechanism, based on auction theory, is proposed in [22]. In particular, the scheme selects the most remunerative virtual network requests, while satisfying QoS requirements and physical network constraints. The system is split into two network substrates modeling physical and virtual resources, with the final goal of finding the best mapping of virtual nodes and links onto physical ones according to the QoS requirements (i.e., bandwidth, delay, and CPU bounds).
A novel network architecture is proposed in [23], to provide efficient coordinated control of both internal network function state and network forwarding state, in order to help operators achieve the following goals: (i) offering and satisfying tight service level agreements (SLAs); (ii) accurately monitoring and manipulating network traffic; and (iii) minimizing operating expenses.
Various engineering problems, where the action of one component has some impacts on the other components, have been modeled by GT. Indeed, GT, which has been applied at the beginning in economics and related domains, is gaining much interest today as a powerful tool to analyze and design communication networks [24]. Therefore, the problems can be formulated in the GT framework, and a stable solution for the components is obtained using the concept of equilibrium [25]. In this regard, GT has been used extensively to develop understanding of stable operating points for autonomous networks. The nodes are considered as the players. Payoff functions are often defined according to achieved connection bandwidth or similar technical metrics.
To construct algorithms with provable convergence to equilibrium points, many approaches consider network models that can be mapped to specially constructed games. Among this type of games, potential games use a real-valued function that represents the entire player set to optimize some performance metric [8, 26]. We mention Altman et al. [27], who provide an extensive survey on networking games. The models and papers discussed in this reference mostly deal with noncooperative GT, the only exception being a short section focused on bargaining games. Finally, Seddiki et al. [25] presented an approach based on two-stage noncooperative games for bandwidth allocation, which aims at reducing the complexity of network management and avoiding bandwidth performance problems in a virtualized network environment. The first stage of the game is the bandwidth negotiation, where the SP requests bandwidth from multiple Infrastructure Providers (InPs). Each InP decides whether to accept or deny the request when the SP would cause link congestion. The second stage is the bandwidth provisioning game, where different SPs compete for the bandwidth capacity of a physical link managed by a single InP.
3. Modeling Power Management Techniques
Dynamic Adaptation techniques in physical network nodes reduce the energy usage, by exploiting the fact that systems do not need to run at peak performance all the time.
Rate adaptation is obtained by tuning the clock frequency and/or voltage of processors (DVFS, Dynamic Voltage and Frequency Scaling) or by throttling the CPU clock (i.e., the clock signal is disabled for some number of cycles at regular intervals). Decreasing the operating frequency and the voltage of the node, or throttling its clock, obviously allows the reduction of power consumption and of heat dissipation, at the price of slower performance.
Advantage can be taken of these adaptation capabilities to devise control techniques based on feedback on the incoming load. One of the first proposals is that examined in a seminal paper by Nedevschi et al. [28]. Among other possibilities, we consider here the simple optimization strategy developed in [9] aimed at minimizing the product of power consumption and processing delay with respect to the processing load. The application of this control technique gives rise to quadratic dependence of the power-delay product on the load, which will be exploited in our subsequent game theoretic resource allocation problem. Unlike the cited works, our solution considers as load the requests rate of services that the SP must process. However, apart from this difference, the general considerations described are valid here, too.
Specifically, the dynamic frequency-dependent part of the processor power consumption is proportional to the clock frequency and to the square of the supply voltage [29]. However, if DVFS is used, there is proportionality between the frequency and the voltage raised to some power [30]. Moreover, the power scaling effect induced by alternating active-idle periods in the queueing system served by the physical network node can be accounted for by multiplying the dynamic power consumption by an increasing concave function of the resource utilization of the form , where is a parameter and and are the workload (processing requests per unit time) and the task processing speed, respectively. By taking (which implies a cubic relationship in the operating frequency), the overall dependence of the power consumption on the processing speed (which is proportional to the operating frequency) can be expressed as [9]where is a proportionality constant. This expression will be exploited in the optimization problems to be defined and studied in the next section.
4. System Model of Coordinated Network Management Optimization and Players’ Game
We consider the situation shown in Figure 1. There are VNFs that are represented by the incoming workload rates , competing for physical network nodes. The workload to the th node is given bywhere is VNF ’s contribution. The total workload vector of player is and, obviously,
The computational resources have processing rates , and we associate a cost function with each one; namely,
Cost functions of the form shown in (4) have been introduced in [9] and represent the product of power and processing delay, under assumption on each node’s queue. They actually correspond to the average energy consumption that can be attributed to functions’ handling (considering that the node will be active whenever there are queued functions). Despite the possible inaccuracy in the representation of the queueing behavior, the denominator of (4) reflects the penalty paid for approaching the resource capacity, which is a major aspect in a model to be used for control purposes.
We now consider two specific control problems, whose resulting resource allocations are interconnected. Specifically, we assume the presence of Control Policies (CPs) acting in the network node, whose aim is to dynamically adjust the processing rate, in order to minimize cost functions of the form of (4). On the other hand, the players representing the VNFs, knowing the policy adopted by the CPs and the resulting form of their optimal costs, implement their own strategies to distribute their requests among the different resources.
The simple control structure that we envisage actually represents a situation that may be of interest in a number of different operational settings. We only mention two different cases of current high relevance: (i) a multitenant environment, where a number of ISPs are offered services by a datacenter operator (or by a telecom operator in the access network) on a number of virtual machines, and (ii) a collection of virtual environments created by a SP on behalf of its customers, where services are activated to represent customers’ functionalities on their own private virtual LANs. It is worth noting that in both cases the nodes’ customers may be unwilling to disclose their loads to one another, which justifies the decentralized game optimization.
4.1. Nodes’ Control Policies
We consider two possible variants.
4.1.1. CP1 (Unconstrained)
The direct minimization of (4) with respect to yields immediately:
We must note that we are considering a continuous solution to the node capacity adjustment. In practice, the physical resources allow a discrete set of working frequencies, with corresponding processing capacities. This would also ensure that the processing speed does not decrease below a lower threshold, avoiding excessive delay in the case of low load. The unconstrained problem is an idealized variant that would make sense only when the load on the node is not too small.
4.1.2. CP2 (Individual Constraints)
Each node has a maximum and a minimum processing capacity, which are taken explicitly into account Then,Therefore,
In this case, we assume explicitly that .
4.2. Noncooperative Players’ Optimization
Given the optimal CPs, which can be found in functional form, we can then state the following.
4.2.1. Players’ Problem
Each VNF wants to minimize, with respect to its workload vector , a weighted (over its workload distribution) sum of the resources’ costs, given the flow distribution of the others:
In this formulation, the players’ problem is a noncooperative game, of which we can seek a NE [31].
We examine the case of the application of CP1 first. Then, the cost of VNF is given bywhere is the total flow from the other VNFs to node .
The cost in (9) is convex in and belongs to a category of cost functions previously investigated in the networking literature, without specific reference to energy efficiency [32, 33]. In particular, it is in the family of functions considered in [33], for which the NE Point (NEP) existence and uniqueness conditions of Rosen [34] have been shown to hold. Therefore, our players’ problem admits a unique NEP. The latter can be found by considering the Karush-Kuhn-Tucker stationarity conditions with Lagrange multiplier :from which, for ,Excluding the negative solution, the nonzero components are those with the smallest and equal partial derivatives that, in a subset , yield ; that is,from which can be determined.
As regards form (7) of the optimal node cost, we can note that if we are restricted to (a coupled convex constraint set), the conditions of Rosen still hold. If we allow the larger constraint set , the nodes’ optimal cost functions are no longer of polynomial type. However, the composite function is still continuously differentiable (see the Appendix). Then, it would (i) satisfy the conditions for a convex game (the overall function composed of the three parts is convex), which guarantee the existence of a NEP [34, Th. 1], and (ii) possess equivalent properties to functions of Type A as defined in [32], which lead to uniqueness of the NEP.
5. Numerical Results
In order to evaluate our criterion, we present numerical results deriving from the application of the simplest Control Policy (CP1), which implies the solution of a completely quadratic problem (corresponding to costs as in (9) for the NEP). We compare the resulting allocations, power consumption, and average delays with those stemming from the application (on non-energy-aware nodes) of the algorithm for the minimization of the pure delay function that was developed in [35, Proposition ]. This algorithm is considered a valid reference point in order to provide good validation of our criterion. The corresponding cost of VNF iswhereThe total demand is assumed to be less than the total operating capacity, as we have done with our CP2 in (7).
To make the comparison fair, we have to make sure that the maximum operating capacities (that are constant in (14)), are compatible with the quadratic behavior stemming from the application of CP1. To this aim, let denote the total input workload to node corresponding to the NEP obtained by our problem; we then choose
We indicate with , the total node input workloads and those corresponding to VNF , respectively, produced by the NEP deriving from the minimization of costs in (13). The average delays for the flow of player under the two strategy profiles are obtained asand the power consumption values are given by
Our data for the comparison are organized as follows. We consider normalized input rates, so that We generate randomly values corresponding to the VNFs’ input rates from independent uniform distributions; then,(1)we find the NEP of our quadratic game, by choosing the parameters , also from independent uniform distributions (with , resp.);(2)by using the workload profiles obtained, we compute the values , from (15) and find the NEP corresponding to costs in (13) and (14), by using the same input rates;(3)we compute the corresponding average delays and power consumption values for the two cases, by using (16) and (17), respectively.
Steps (1)–(3) are repeated with a new configuration of random values of the input rates for a certain number of times; then, for both games, we average the values of the delays and power consumption per VNF, and of the total delay and power consumption averaged over the VNFs, over all trials. We compare the results obtained in this way for four different settings of VNFs and nodes; namely, The rationale of repeating the experiments with random configurations is to explore a number of different possible settings and collect the results in a synthetic form.
For each VNFs-nodes setting, 30 random input configurations are generated to produce the final average values. In Figures 2–10, the labels EE and NEE refer to the energy-efficient case (quadratic cost) and to the non-energy-efficient one (see (13) and (14)), respectively. Instead, the label “Saving (%)” refers to the saving (in percentage) of the EE case compared to the NEE one. When it is negative, the EE case presents higher values than the NEE one. Finally, the label “User” refers to the VNF.
The results are reported in Figures 2–10. The model implementation and the relative results have been developed with MATLAB® [36].
In general, the energy-efficient optimization tends to save energy at the expense of a relatively small increase in delay. Indeed, as shown in the figures, the saving is positive for the energy consumption and negative for delay. The energy saving is always higher than 18% in every case, while the delay increase is always lower than 4%.
However, it can be noted (by comparing, e.g., Figures 5 and 7) that configurations with lower load are penalized in the delay. This effect is caused by the behavior of the simple linear adaptation strategy, which maintains constant utilization, and highlights the relevance of setting not only maximum, but also minimum values for the processing capacity (or for the node input workloads).
Another consideration arising from the results concerns the variation of the energy consumption, by changing the number of players and/or the nodes. For example, Figure 6 (case ) shows total power consumption significantly higher than the one shown in Figure 8 (case ). The reasons for this difference are due to the lower workload managed by the nodes of the case (greater number of nodes with an equal number of users) making it possible to exploit the AR mechanisms with significant energy saving.
Finally, Figure 10 shows the product of the power consumption and the delay for each studied case. As described in the previous sections, our criterion aims at minimizing this product. As can be easily seen, the saving resulting from the application of our policy is always above 10%.
6. Conclusions
We have examined the possible effect of introducing simple energy optimization strategies in physical network nodes performing services for a set of VNFs that request their computational power within an NFV environment. The VNFs’ strategy profiles for resource sharing have been obtained from the solution of a noncooperative game, where the players are aware of the adaptive behavior of the resources, and aim at minimizing costs that reflect this behavior. We have compared the energy consumption and processing delay with those stemming from a game played with non-energy-aware nodes and VNFs. The results show that the effect on delay is almost negligible, whereas significant power saving can be obtained. In particular, the energy saving is always higher than 18% in every case, with a delay increase always lower than 4%.
From another point of view, it would also be of interest to investigate socially optimal solutions stemming from a Nash Bargaining Problem (NBP), along the lines of [37], by defining suitable utility functions for the players. Another interesting evolution of this work could be the application of additional constraints to the proposed solution such as, for example, the maximum delay that a flow can support.
Appendix
We check the maintenance of continuous differentiability at the point . By noting that the derivative of the cost function of player with respect to has the formit is immediate to note that we need to compareWe haveThen, let us check when
Conflict of Interests
The authors declare that there is no conflict of interests regarding the publication of this paper.
Acknowledgments
This work was partially supported by the H2020 European Projects ARCADIA (http://www.arcadia-framework.eu/) and INPUT (http://www.input-project.eu/), funded by the European Commission.