Abstract
Smart grid applications experience an extremely wide range of communication delay. Data flows of those applications are normally aggregated at industrial network routers in substations, form infinite (long) queues termed bufferbloat issue, and might damage the operation of transmission control protocol. The default queue management scheme, DropTail, in such routers just drops packets if queue is full while the others in literature are mostly based on one-loop feedback control where an optimal point of performance between queue length and drop rate is limited. In this paper, we study the problem of managing a long queue of industrial router at substation under heterogeneous smart grid networks. Specifically, we propose an enqueue-dequeue dropping cascade control using a two-loop design method to control both window size and queue length. Moreover, our proposal can be easily implemented into router firmware with provided discrete expressions. Finally, our simulation results are presented to validate the possible benefits that can be gained from cascade control and compare the existing queue management methods as well.
1. Introduction
1.1. Overview
Smart grid is going to be a geographically widespread system soon, which integrated many diverse real-time applications (e.g., sensor) together with energy management applications. Smart grid, which is a digitally-enhanced version of the traditional electric grid, provides infrastructure to support diverse services such as finance, information, and electrical delivery among consumers, assets, and those users who have authorized access. A conceptual model for smart grid communication is presented in [1]. At the customer house, energy consumption information is monitored and then sent to service provider for further analysis and better support. At the operator side, data is collected to make decision according to demand response or energy saving.
With an emerging smart grid, various applications can be enabled to improve quality of service and consumer satisfaction. Some popular examples are building automation, automated meter reading, outage and restoration management, and electric vehicles. Most of the smart grid applications have a strict latency requirement in the range of 100 milliseconds to 5 seconds [2]. In [3], the delay ranges of different smart grid applications were approximately reported. For example, teleprotection applications allow traffic delay from 8 to 10 ms while they allow traffic delay for more than one second for interval measurement from smart meters. To respect it, we also need a fast communication infrastructure that can handle a huge amount of exchanging data and is able to provide a near real-time response. Latency is defined as the time interval between when the state occurred and when it was acted upon by a smart grid application [2]. Various applications own different latency requirements that depend on the kind of system response it is dealing with. In such scenario, network equipment including switches, Internet wired/wireless routers, and operating systems (O/S) are the fundamental components to provide a fast and reliable smart grid communication. Table 1 lists latency and bandwidth requirements for some specific applications in smart grid networks [4].
In such a heterogeneous environment with multiple types of applications, special equipment like industrial routers at substations are necessary to handle. Commercially, we have seen an example product from the company, Virtual Access, that is, GW2027, which is an industrial router with applications like machine-to-machine (M2M), telemetry, supervisory control and data acquisition (SCADA), roadside, and wireless. Figure 1 illustrates such smart grid network situation in detail, where industrial router has to process data from both wired and wireless links. Among several great software features like remote configuration and fault reporting and so forth we, however, realize that a scheme to manage the input queue of the industrial router is not specified in the data sheet [5] and is usually Drop Tail, a simple scheme which drops packets in case of full buffer and allows packets otherwise.
1.2. Motivation
In general, each part of every network equipment or substation industrial router is usually preinstalled or configured with some amount of buffering, whether it is enough or not, to handle bursts of arriving packets and departing packets to the next link. It is important to ensure good utilization of the network link, especially in cases where arriving rate is greater than departing rate causing bottleneck point to be built up. The buffer then absorbs high-rate traffic packets which wait and are later served on the slower outgoing links [7]. Not enough buffering results in high dropping rate of most of the packets and low egress link utilization. For example, if a user transfers files using transmission control protocol (TCP), the user satisfaction is measured by how quickly the file transfer can complete, which is directly related to how effectively the protocol can utilize network links, that is, more buffering is better. Moreover, as the cost of buffers/memory keeps reducing, it is foreseen that large buffering would be put into every piece of network equipment and even preconfigured in operating systems. However, large buffering results in long latency experience for users as well. Besides, most of TCP schemes (Reno, New Reno, CUBIC, etc.) use loss-based congestion control. It means that a loss-based TCP sender continues increasing its congestion window size (i.e., sending rate) until a loss signal is detected by time-out or incorrect acknowledgment packets. In [8], it is suggested that “the widely used rule of thumb leads to buffers which are much larger than they need to be” and could be reduced by scarifying a small amount of bandwidth utilization. Using too much buffering, it obviously takes longer time for loss detection and therefore TCP may function incorrectly. The excessive long latency and the damage of TCP due to too much buffering are two major consequences of a problem recently termed as bufferbloat.
The up-to-date solutions to the bufferbloat issue, or debloat, consist of traffic shaping, TCP window size modification, and active queue management (AQM). In the scope of this paper, we concentrate on AQM algorithms because they can manage queuing delay efficiently, by either a direct or an indirect way. Most of Linux kernels and some open source routers (CeroWRT) have been integrated with some modern AQM algorithms [9, 10]. For example, depending on where the bottlenecks are, a Linux user can change the computer to use a different AQM qdisc (queuing discipline) via a sysctl or the tc (traffic control) command. Literature has witnessed several coexisting AQM candidate algorithms including Controlled Delay (CoDel) [11], Stochastic Flow Queuing CoDel (sfqCoDel) [12], and Proportional Integral Enhanced (PIE) [13] that have been proposed using varied theory tools (e.g., optimization, queuing, and control theory). Among them, the control theory approach owns some advantages to be implemented in smart grid network equipments, for example, parameterization controllers and well reference tracking. Theoretically, control-based AQM algorithms exploit the additive increase multiplicative decrease (AIMD) model of TCP and the continuous fluid-flow queue approximation [14, 15]. Then the specific controller’s parameters are designed according to a closed-loop transfer function of the whole system. We, however, realize that these models almost design one-loop control for queue length only which has some problems of large overshoots and unacceptable lags (delay) in mixed-traffic scenarios. More loops operating in different network layers should improve the whole system performance, especially the input delay transient (overshoot) behaviour of the aforementioned debloat schemes.
1.3. Our Contributions
In this paper, we develop a novel active queue management using cascade control framework in control theory that is able to reduce large overshoot and therefore improve delay transient behaviour. To the best of our knowledge, our work is the first attempt to adapt cascade control method to bufferbloat research field. Using the well-known AIMD and fluid queue model [14], we decompose transfer functions into an inner and an outer loop. The inner one adjusts window size based on changing of traffic and feedback window size and the outer one mission adjusts queue length based on feedback queue length value at time instant. Each loop’s parameters are tuned using optimal gain and phase margin method [16]. This scheme operates at both the network transport layer (adjusting TCP congestion window size) and the network link layer (adjusting queue length). A difficulty when considering this cascade design is the interaction in time scale between two loops. The other side effect is complexity due to two additional controllers. Nevertheless, we show that better delay performance results can be achieved with EDDC via both numerical and simulation results (Section 5.2). Our primary contributions are summarized as follows:(i)We develop a cascade control active queue management (EDDC) scheme by decomposing the TCP/AQM fluid-flow model into two cascaded loops for two independent controllers.(ii)By applying the reliable optimal gain and phase margin tuning method, we derive a set of parameters for these controllers based on a given example from single-loop AQM design.(iii)Our simulations are conducted in network simulator ns-2 using different smart grid application models, including file data transmission (FTP), voice over IP (VoIP), and video conference, which are close to the realistic smart grid applications. In most of cases, the related-delay results using EDDC are well controlled around the target delay (10 ms) while the large overshoot phenomenon disappears.
2. Related Works
In this section, we discuss the features of the famous existing AQM schemes: one mature (RED) and two modern AQM algorithms (CoDel, sfqCoDel) which are designed specifically for bufferbloat mitigation.
Random Early Detection (RED) is one of the first viable AQM algorithms [17]. However, it is proved to be so difficult to configure properly that hardly anybody uses it, even though many carrier grade routers implement it. With an appropriate set of parameters, RED is an effective algorithm. However, dynamically predicting this set of parameters was found to be difficult. As a result, RED has not been enabled by default, and its present use in the Internet is limited [18]. Other AQM algorithms have been developed since RFC2309 was published, some of which are self-tuning within a range of applicability. Hence, while this memo continues to recommend to deploy AQM, it no longer recommends that RED or any other specific algorithm is used as a default; instead it provides recommendation on how to select appropriate algorithms and that a recommended algorithm is able to automate any required tuning for common deployment scenarios.
“Controlled Delay” (CoDel) AQM algorithm [11] tries to address the problems that RED could not. First, the input signal into the algorithm (sojourn time versus average queue length) is of a different quality; second, CoDel (in its plain form) does drop/mark on dequeue (departure queue), while RED drops/marks on enqueue (arrival queue). Therefore TCP congestion control loop using CoDel responds much quicker than using RED; thus the reaction by the sender will probably be timely and relevant for that congestion epoch. With RED, the congestion signal (lost packet) has to traverse the filled-up buffer first; thus, the control loop time is much larger (includes the instantaneous queue length of the buffer) and is further delayed by the averaging going on. By design, CoDel does not need to be tuned specifically for one particular drain rate (bandwidth) of the queue unlike RED; so it adjusts much better to variable bandwidth MACs (e.g., Wi-Fi and DOCSIS link).
CoDel itself, however, drops packet of a group during each interval without considering the packet’s priority level or user side. Recently, Stochastic Flow Queue CoDel (sfqCoDel) [12] is a promising AQM algorithm which demonstrates a satisfied performance to mitigate bufferbloat for users, for example, increase x speed of a network under load [9], and it has been implemented in open source routers (i.e., OpenWRT [10]). The sfqCoDel is a hybrid of deficit round-robin scheduling (DRR) and CoDel. It is renamed from “Fair Queuing” to “Flow Queuing” because flows that build a queue are treated differently than flows that do not. It stochastically classifies incoming packets into different queues; each queue is managed by the CoDel AQM algorithm. Packet ordering within a queue is preserved since queues have FIFO ordering [19].
3. System Model
3.1. TCP State-Space Model
We exploit the TCP fluid-flow model described by nonlinear differential equations that has been extensively studied in network routers interacting with TCP sources (e.g., [14, 20, 21]). This model captures the additive increase multiplicative decrease (AIMD) feature from TCP [22], without slow start and time-out mechanisms. However, this lack only affects initial start-up of the TCP system. Once the system reaches the stable point, the differential equations solver is able to track changes in the network well [23].where is average TCP window size (packets); is queue length at cable modems (packets); is packet dropping probability function (); is transmission capacity of link (packets/sec); is round-trip time (sec); with being propagation delay; and is number of TCP sessions.
The first equation (1) describes the TCP-Reno-sender behaviour based on AIMD while the second one (2) models a fluid queue, which allows traffic arrivals to be continuous rather than discrete, as in a classic queuing model like M/M/1 queue. The operating point of model (1) can be derived at with and as follows
By doing linearization around operating points [14], with ; ; and , a linearized small signal model is obtained as:
By defining system state vectors as ; control input vector as ; external disturbance as ; and system output vector as , we then rewrite the TCP system as a continuous time-invariant type using state-space modeling approach.where matrices , , , , and represent state (system), input, output, and disturbance matrix, respectively, as follows:
The final system model equations which map the input (traffic flow) to the output (queue length) are
Using the above state-space TCP/IP system model, there are several approaches to design an efficient controller with the following requirements:(i)Queue length is as short as possible.(ii)Sensitivity to network parameters is low.(iii)Link utilization is high, which means we do not waste the cost of link.(iv)Packet drop rate is low.
We, however, admit a trade-off to satisfy all of those aforementioned requirements. For example, a shorter queue length comes along with a higher packet drop rate. Our algorithm in the next section seeks for an optimal point.
3.2. Optimization Formulation of Infinite Buffer (Bufferbloat)
The classic M/M/1 queuing model well represents the infinite buffer or bufferbloat phenomenon of smart grid substation routers. Optimizing the performance of even simple queue like M/M/1 is a difficult problem because of nonlinearity of objective functions and constraints and running time scales exponentially with the problem size. In [24], a convex optimization problem was proposed for M/M/1 queues and with purpose of minimizing the state probability only. However, minimizing queue length is the main purpose when dealing with the bufferbloat issue. Therefore, we formulate our optimization problem to directly optimizing queue length of infinite buffer, with constraints following the aforementioned TCP state-space model.
The optimization variables are and , and the constant parameters are , , and .
4. Proposed Algorithm
In this section, we propose an algorithm, named an Enqueue-Dequeue Dropping Cascade control scheme (EDDC). We present two main design steps of our proposed algorithm from the above TCP state-space system model. The first step is to design in the continuous time domain. The second step is transform dropping probability formula into discrete domain so that we are able to implement it into the smart grid industrial router firmware (Figure 2).
4.1. EDDC in Continuous Time Domain
We design two controllers for each loop. The inner controls drop probability based on traffic information and the outer controls drop probability based on difference between measured average queue length and reference queue length.
(i) Inner Loop. An important design requirement is that the inner loop controller should behave quickly. From (7), the inner control objective is a linear first-order type: , where and . Hence, we design an integral I controller for inner loop:
The close-loop transfer function of the inner loop is
The transfer function is converted into frequency domain, with as frequency:
One of quality requirements of closed-loop control system, which is represented by , is that the output is same to the input signal or the controller should bring , , which can be called optimal gain and phase margin tuning method [16]. However, due to several reasons of real system, that requirement is rarely satisfied for all frequencies . An acceptable design is that , in a wide band of low frequencies . Hence, we propose to choose such that or . This close-form expression of is used to make decision for controller .
(ii) Outer Loop. The close-loop transfer function of the outer loop is
The outer control objective into zero-pole form iswhere ; ; ; and .
For the outer loop, the objective function is linear third-order type, due to inclusion of . Hence, we choose proportional-integral-derivative (PID) controller by using the same method at the inner loop design, or .with , , and .
4.2. EDDC in Discrete Time Domain
(i) Discrete Outer Loop Controllerwhere is the output of the outer loop controller and is the difference between measured queue length and desired queue length at the outer loop. We apply backward difference method to get the z-transform of the outer loop controller :
Transforming again to time domain, we obtainor a compact form of the outer loop controller
(ii) Discrete Inner Loop Controllerwhere is the output of the inner loop controller and is the difference between and congestion window size . Doing similarly as discrete outer loop steps above, we obtain a compact form of the inner loop controller in discrete domain as follows, with :
5. Performance Evaluation
5.1. Simulation Setup
NS2 is a discrete event network simulator, which simulates packet-level events and facilitates the development of communication network scenarios considering the new protocols involved, either wired or wireless technologies. We use and modify the NS2 TCP evaluation tool originally developed by Wang et al. [25] for our AQM algorithm evaluation purpose. We also develop our own EDDC source code files based on the basic proportional integral (PI) algorithm code from NS2 source code tree. The simulation network topology as multiple cross bottlenecks is our choice because of its realistic smart grid IP networks (Figure 3). The simulation configuration is presented in Table 2.
5.2. Simulation Results
We evaluate our proposed algorithm performance according to three main characteristics of the bottleneck link: bandwidth (Mbps), propagation round-trip time (ms), and packet error rate ratio. Each experiment is compared between AQM algorithms (in Table 2) with respect to mean queue length, link utilization, and packet drop rate. We expect that mean queue length is as low as possible, link utilization is about , and packet error rate should be low. These requirements of three criteria, however, cannot be achieved at the same time due to trade-off. Therefore, our algorithm searches for a balance point among three of them.
5.2.1. Experiment 1: Varied Link Bandwidth
In the first experiment, we vary the bottleneck link bandwidth from Mbps to Mbps. This variation represents different types of links in heterogeneous smart grid networks. For example, a wired link such as Ethernet has a bandwidth of Mbps while a wireless link from wireless adapter has varied bandwidth because of wireless signal attenuation and fading.
Firstly, Figure 4 presents our comparison results when we change the value of link bandwidth from Mbps to Mbps. Our algorithm achieves very low and stable mean queue length (≈10%); even bandwidth is varied in a wide range. Meanwhile, the link utilization of EDDC is about , which is higher than the default DropTail or RED and nearly approaches the modern sfqCoDel algorithm (). The packet drop rate performance of EDDC, however, is almost the same as the others. It makes sense that when we use a low-bandwidth link, the packet drop rate is high due to full buffer phenomenon. We conclude that our EDDC algorithm promisingly maintain the queue length of substation router, while link utilization or the busy level of link is acceptable under different kinds of links.
(a) Queue length
(b) Link utilization
(c) Packet drop rate
5.2.2. Experiment 2: Varied Link Round-Trip Time
Figure 5 shows our comparison of AQM algorithms performance in case of varied link propagation round-trip time (RTT). This criterion represents the latency of packets when using a loss-based TCP congestion control scheme, for example, TCP Reno. As mentioned before, one consequence of a large buffer (bufferbloat) is that TCP operation might be damaged because of untimely congestion information feedback of acknowledgment (ACK) packets. The higher the propagation RTT is, the higher chance of incorrect ACK feedback to the sources is. We observe the same results of EDDC performance for the mean queue length and link utilization. In Figure 5(c), the packet drop rate results of EDDC, however, are higher than the other AQM algorithms in the RTT range from ms to ms.
(a) Queue length
(b) Link utilization
(c) Packet drop rate
5.2.3. Experiment 3: Varied Link Packet Error Rate
Lastly, we change the packet error rate (PER) of the congested link. A wired link, for example, Ethernet cable, usually has a PER value of zero. A wireless link, however, has the wide range of PER values because of wireless signal characteristics such as attenuation or fast fading. In order to simulate them, we vary PER from to ; for instance, a link with means that if we transmit packets, only packet is successfully delivered to destination.
In Figure 6(a), we see that EDDC keeps the router queue length at which is lower and better than DropTail, RED, FB-OCQ, and CoDel algorithms. The modern sfqCoDel using the stochastic fair method seems to manage queue length to be too short (nearly ) which might increase the number of packet drop. It is, in fact, confirmed from Figure 6(c) that sfqCoDel has the highest packet drop rate (≥2%), while our algorithm EDDC only drops packets smartly at ratio.
(a) Queue length
(b) Link utilization
(c) Packet drop rate
The above experiments support us to conclude some important advantages of our queue management algorithm EDDC. By smartly dropping packets with EDDC scheme, the mean queue length is kept at a stable and low level (2–5%) comapred with the other popular algorithms. Moreover, link utilization value is reasonable (80–95%) while the packet drop rate of EDDC is lower than the newest and modern algorithm (sfqCoDel).
6. Conclusions
An algorithm for better queue management of heterogeneous smart grid traffic was proposed. The algorithm, based on cascade control theory, smartly dropped packets waiting in the queue based on information of inflow smart grid traffic and current state of input queue. The algorithm was shown to effectively manage the queue length or the number of packets that are waiting in queue at a stable and low level (2–5%) while the link utilization does not reduce too much.
Competing Interests
The authors declare that there is no conflict of interests regarding the publication of this paper.
Acknowledgments
This research was supported by the MSIP (Ministry of Science, ICT and Future Planning), Korea, under the Grand Information Technology Research Center Support Program (IITP-2016-R71181610050001002) supervised by the IITP (Institute for Information & Communications Technology Promotion). This work was also supported by a 2-year research grant of Pusan National University.