Modeling Slotted Aloha as a Stochastic Game with Random Discrete Power Selection Algorithms

Abstract

We consider the uplink case of a cellular system where 𝑚 bufferless mobiles transmit over a common channel to a base station, using the slotted aloha medium access protocol. We study the performance of this system under several power differentiation schemes. Indeed, we consider a random set of selectable transmission powers and further study the impact of priorities given either to new arrival packets or to the backlogged ones. Later, we address a general capture model where a mobile transmits successfully a packet if its instantaneous SINR (signal to interferences plus noise ratio) is lager than some fixed threshold. Under this capture model, we analyze both the cooperative team in which a common goal is jointly optimized as well as the noncooperative game problem where mobiles reach to optimize their own objectives. Furthermore, we derive the throughput and the expected delay and use them as the objectives to optimize and provide a stability analysis as alternative study. Exhaustive performance evaluations were carried out, we show that schemes with power differentiation improve significantly the individual as well as global performances, and could eliminate in some cases the bi-stable nature of slotted aloha.

1. Introduction

Aloha [1] and slotted aloha [2] have long been used as randomly distributed medium access protocols for radio channels. They are used in satellite networks and cellular telephone networks for the sporadic transfer of data packets. In these protocols, packets are transmitted sporadically by various users. If packets are sent simultaneously by more than one user, then they collide. After a packet is transmitted, the transmitter receives the information on whether there has been a collision (and retransmission is needed) or whether it has been well received. All colliding packets are assumed to be corrupted which get backlogged and are retransmitted after some random time. We focus on the slotted aloha [3], in which time is divided into units. At each time unit a packet may be transmitted, and at the end of the time interval, the sources get the feedback on whether there was zero, one, or more transmissions (collision) during the time slot. A packet that arrives at a source is immediately transmitted. Packets that are involved in a collision are backlogged and are scheduled for retransmission after a random time.

Interest has been growing in recent years in studying competition of networking in general and access to a common medium in particular, within the frame of noncooperative game theory; see, for example, the survey paper [4]. Various game formulations of the standard slotted aloha (with a single power) have been derived and studied in [5–10] for the noncooperative choice of transmission probabilities. Several papers study slotted aloha with power diversity but without differentiating between transmitted and backlogged packets and without the game formulation. In [11] it is shown that the system capacity could be increased from 0.37 to 0.53 if one class of terminals always uses high power and the other always uses low power level. In [12], power diversity is studied with the capture model that we use as well as with another capture model based on signal-to-noise ratio. The workin [13] studies power diversity under three types of power distribution between the power levels and provides also a detailed stability analysis. The workin [14] proposes a model and evaluates the throughput that can be achieved in a system of 𝑁 mobiles using generalized aloha like protocols where the mobiles transmit data using a two-state decision system. For cooperative systems, it gives the throughput bounds and explores the trade-off between throughput and short-term fairness. But our proposal is different here; we address the effect of randomization in power levels for both cooperative and noncooperative setups.

When multiple users share a common channel and contend for access, a typical conflict problem arises. Recently, the selfish behavior of users in MAC protocols has been widely analyzed using game theory with all its powerful solution concepts. It was shown in [5, 7, 15, 16] that the users selfish behavior likely leads to a network collapse, where a typical prisoners dilemma situation occurs. This illustrates, in fact, that Nash equilibrium (NE) is not efficient in some situations. (A Nash equilibrium is a strategies profile where no player can improve its reward by deviating unilaterally from its strategy.) This way, full system utilization requires coordination among users using explicit message exchanges or presence of an arbitration mechanism [17], which may be impractical given the distributed nature and arbitrary topology change (due to mobility, ended calls, new calls, environment, etc.) of wireless networks. To achieve a better performance without coordination schemes, users need to sustain cooperation or priority. It is beneficial to design a set of users whose mission is to provide incentives for other users to behave cooperatively as well as respect the defined priority; this mechanism may limit the aggressiveness level (access to the channel) and resolve the contention/random problem. Another way to reduce the access contention is to introduce a transmission cost. Indeed, it was shown in [18] that costs have a stabilizing effect; being rational, users will defer packet transmissions when congestion develops and the cost for successful transmission becomes high. This way, users will drop packets when the total transmission costs are high which can cause a huge delay and then the analysis seems to be not applicable to delay-sensitive traffic. Authors in [19] showed that introducing users hierarchy improves the performances of the whole system and the enhancement depends on the offered load. At low load, this improvement is due to the fact that users (both of leaders and followers) retransmit with high probability at low load; hence the backoff time is reduced. At average and high loads, leaders turn to be generally less aggressive than Nash equilibrium case. This way, the collision probability is reduced and then the performance of the hierarchical scheme is enhanced. A pretty phenomena, when the number of followers becomes larger than the number of leaders, is that those latter users become more friendly and reduce their retransmission probability, whereas the followers become very aggressive and transmit at probability close to 1. Here, we extend the model first proposed in [15] and introduce new schemes in which multiple power levels are used for transmission. When several packets are sent simultaneously, one of them can often be successfully received due to the power capture effect. In this paper, we consider a general capture model where a mobile transmits successfully a packet if his instantaneous SINR (signal-to-interferences-plus-noise ratio) is larger than the fixed threshold. In particular, we introduce the differentiation between new packets and backlogged packets allowing prioritization of one or the other in terms of transmit power. We study and compare the following schemes.

(1)The first scheme considers the power diversity but defines no prioritization in transmission or retransmission. This scheme will mainly show the effect of power diversity on system performances; (2)In the second scheme, each new packet is immediately transmitted with the lowest power, whereas backlogged packets are sporadically retransmitted at a random power selected among 𝑁−1 larger distinct levels.(3)Here, new packets are transmitted with the highest power, and backlogged packets are retransmitted at a random power level picked from 𝑁−1 lower distinct levels. (4)This scheme gives more priority to new arrivals. Indeed, backlogged packets are retransmitted with the lowest power level and a new packet is transmitted at a random power selected among 𝑁−1 larger distinct levels. (5)All previous schemes are compared with standard slotted aloha taken as reference; this allows to compare and analyze performances of each scheme.

The capture model used in [15] is not realistic; authors assume therein that when a unique mobile chooses the highest power, compared to other mobiles, its transmission is succeeded independently of the other mobiles and their respective choices. This assumption could not be always true. Indeed, the aggregate signal of other mobiles may jam the signal of the tagged mobile, that is, whose power level is the highest; therefore no successful transmission exists. In this paper, a terminal succeeds its transmission if it chooses the most elevated power level compared to other mobiles and its instantaneous SINR is greater than a given threshold 𝛾_th. Under more general capture model, we study the team problem in which we optimize transmission probabilities for the various schemes so as to achieve the maximum throughput or to minimize the expected delay. We discover however that in heavy load, the optimality is obtained at the expense of huge expected delay of backlogged packets (EDBPs). We therefore consider the alternative objective of minimizing the EDBP. We study both the throughput as well as the delay performance of the global optimal policy. We also solve the multicriteria problem in which the objective is a convex combination of the throughput and the EDBP. This allows, in particular, to compute the transmission probabilities that maximize the throughput under a constraint on EDBP, which could be quite useful for delay-sensitive applications (e.g., Video streaming, Voice, VoD, etc.). We show that schemes with priority do not only improve the average performances considerably but they are also able in some cases to eliminate the bistable nature of the slotted aloha. Furthermore, we study the game problem in which each mobile chooses its transmission probability selfishly in order to optimize its own objective. This gives rise to a game theoretic model in which we study the equilibrium properties (Nash equilibrium). We show that the power diversity and the prioritization profit to mobiles also in this competitive scenario even if the advantage is less notorious than in the team's behavior.

The rest of this paper is organized as follows. Section 2 describes the problem and introduces the Markovian model. We discuss in Section 3 the team formulation of the problem. Next, we formulate the game setting in Section 4. We finally discuss the performance of different schemes using some numerical examples in Section 5 and finish by some concluding remarks in Section 6.

2. Problem Formulation

We consider a collision channel used by one central receiver and 𝑚 mobiles without buffer; that is, mobiles do not generate new packet till the current one is successfully transmitted. This assumption can have realistic application in the context of signaling. Indeed, it is natural to assume that a source does not start generating a new signaling packet (e.g., a new reservation) as long as the current signaling packet is not transmitted successfully. Here, the process of attempts to retransmit a new packet from a source after the previous packet has been successfully transmitted corresponds well with our bufferless model.

At the beginning of any slot, each mobile can transmit a packet using a power level among 𝑁 available levels 𝒫={𝑝₁,𝑝₂,…,𝑝_𝑁}. We consider a general capture model where a packet transmitted by a mobile 𝑖 is received successfully when and only when its instantaneous SINR is larger than some given threshold 𝛾_th. Let 𝑝_𝑖 be the transmit power chosen by mobile 𝑖 in the current slot, and let 𝜎² be the spectral density of the background noise which is assumed to be AWGN and time-independent. Let us denote by 𝑔_𝑖 the gain experienced by mobile 𝑖 over the channel. It depends on the distance between the mobile and the AP, but it is also impacted by the reflective paths. The instantaneous SINR of mobile 𝑖 received by the central receiver is given by

𝛾_𝑖=𝑔^𝑖𝑝_𝑖∑^𝑘𝑗=1𝑔_𝑗𝑝_𝑗𝐼(𝑗)+𝜎²,(1) where 𝐼(𝑗) is indicator of the event that at the current slot, user 𝑗 is transmitting. We address here a random power selection fashion where transmit powers are selected according to a probability distribution 𝑋=[𝑥₁,𝑥₂,…,𝑥_𝑁]; that is, power level 𝑝_𝑗 is chosen with probability 𝑥_𝑗. The impact of selection probability distribution will be discussed in Section 5.1 In particular, we propose to discuss the case of (1) uniform distribution; (2) high power levels are prioritized; that (3) the case where low power levels are prioritized.

Here, we extend the Markovian model first described by Altman et al. [5, 6, 15] in order to incorporate the instantaneous capture effect. The arrival probability of the packets to the source 𝑖 follows a Bernoulli process with parameter 𝑞_𝑎 (i.e., at each time slot, there is a probability 𝑞_𝑎 of a new arrival at a source, and all arrivals are independent). As long as there is a packet at a source (i.e., as long as it is not successfully transmitted) new packets to that source are blocked and lost (because we consider sources without buffer). The arrival processes of different sources are assumed to be independent. Similarly, we consider that a backlogged packet at source 𝑖 is retransmitted with probability 𝑞^𝑖𝑟. We should restrict in our control and game problems to simple policies in which 𝑞^𝑖𝑟 does not change in time. We also will be interested in the case of symmetric sources; we should then find a symmetric optimal solution, that is, a retransmission probability 𝑞_𝑟 which does not depend on 𝑖. Next, we consider as a state of the system the stochastic process representing the number of backlogged packets in the beginning of a slot; we denote it by 𝑛. For any choice of values 𝑞^𝑖𝑟∈(0,1], the state process is a Markov chain that contains a single ergodic chain (and possibly transient states as well). Define 𝐪_𝑟 to be the vector of retransmission probabilities for all users (whose 𝑗th entry is 𝑞^𝑗𝑟 ). We note the transition matrix of the Markov chain by 𝑃(𝐪_𝑟). Let 𝝅(𝐪_𝑟) be the corresponding vector of the steady-state probabilities where its 𝑛th entry 𝜋_𝑛(𝐪_𝑟) denotes the probability of 𝑛 backlogged mobiles. When all entries of 𝐪_𝑟 are the same, say 𝑞, we shall write (with some abuse of notation) 𝝅(𝑞) instead of 𝝅(𝐪_𝑟).

We introduce further notation; assume that there are 𝑛 backlogged packets, and all use the same value 𝑞_𝑟 for retransmission rate. We denote by 𝑄_𝑟(𝑖,𝑛) the probability that 𝑖 mobiles out of the 𝑛 backlogged packets retransmit at the current slot. Then

𝑄_𝑟(𝑖,𝑛)=𝑛𝑖(1−𝑞_𝑟)^𝑛−𝑖𝑞_𝑟^𝑖.(2)

Let 𝑄_𝑎(𝑖,𝑛) be the probability that 𝑖 unbacklogged mobiles transmit packets in a given slot (i.e., that 𝑖 arrivals occurred at mobiles without backlogged packets). We have

𝑄_𝑎(𝑖,𝑛)=𝑚−𝑖𝑛1−𝑞_𝑎^{𝑚−𝑛−𝑖}𝑞_𝑎^𝑖.(3)

Let 𝑄_𝑟(𝑖,0)=0 and 𝑄_𝑎(𝑖,𝑚)=0.

For ease of reading, we summarize the assumptions of our contention model as follows.

(i)A finite set of 𝑚 bufferless mobiles interacts over a single collision channel. (ii)Time is divided into multiple equal and synchronized slots. Transmission feedbacks (success or collision) are received in the end of the current slot. (iii)Each user 𝑖 is assumed to be nonsaturated; packets arrive from higher layers according to a Bernoulli process with parameter 𝑞_𝑎. (iv)Each user 𝑖 retransmits, in every slot, its packets with probability 𝑞^𝑖𝑟. (v)The average throughput or minus expected delay of backlogged packets is the objective functions to maximize.

3. The Team Problem

In this section we study slotted aloha in a team problem point of view; that is, all mobiles optimize the same objective function (maximize the system throughput or minimize the average delay). Here, we analyze the case when there exists a central entity (base station, dedicated device, etc.) that calculates the optimal strategy profile and broadcasts it to operating mobiles. This entity should get the information of the number of mobiles as well as the individual new arrivals intensity 𝑞_𝑎. Next, we allow prioritization by incorporating a selective transmit power allocation mechanism. On the beginning of each slot, each mobile picks a power level from the 𝑁 available power levels and decides to transmit/retransmit its packet. Based on power allocation fashion, we analyze the following four schemes. The corresponding Markov chain is depicted in Figure 1.

Figure 1 

Markov chain for the team problem. The state of the system is the backlog, it can decrease by at most one per transition but can increase by an arbitrary amount less than or equal 𝑚−𝑛.

3.1. Scheme 1: Random Power Levels without Priority

In this approach, there is no preference between new packets or backlogged ones. A mobile chooses to transmit using any power level among 𝑁 different levels. In case all mobiles use the same transmit rate 𝑞, the transition probability of the system is given by

𝑃_{𝑛,𝑛+𝑖}=⎧⎪⎪⎪⎪⎪⎪⎪⎪⎪⎪⎪⎪⎪⎪⎨⎪⎪⎪⎪⎪⎪⎪⎪⎪⎪⎪⎪⎪⎪⎩𝑄^𝑎(𝑚−𝑛,𝑛)^𝑛_𝑗=0𝑄𝑄𝑖_𝑟=(𝑗,𝑚𝑛)−𝑛1,−𝑖≥𝐴2,_{𝑗+𝑚−𝑛𝑎},(𝑖,𝑛)^𝑛_𝑗=0+𝑄_𝑎𝑄_𝑟(𝑗,𝑛)1−𝐴_𝑗+𝑖(𝑖+1,𝑛)^𝑛_𝑗=0𝑄2≤𝑄𝑖_𝑟<(𝑗𝑚,𝑛−)𝐴𝑛,_{𝑗+𝑖+1𝑎},(1,𝑛)^𝑛_𝑗=1+𝑄_𝑎𝑄_𝑟(𝑗,𝑛)1−𝐴_𝑗+1(2,𝑛)^𝑛_𝑗=0𝑄𝑄_𝑟(𝑗,𝑛)𝐴_𝑗+2,𝑖=1,_𝑎(0,𝑛)𝑄_𝑟(0,𝑛)+^𝑛_𝑗=2+𝑄_𝑎𝑄_𝑟(𝑗,𝑛)1−𝐴_𝑗(1,𝑛)^𝑛_𝑗=0𝑄𝑄_𝑟(𝑗,𝑛)𝐴_𝑗+1,𝑖=0,_𝑎(0,𝑛)^𝑛_𝑗=1𝑄_𝑟(𝑗,𝑛)𝐴_𝑗,𝑖=−1,(4) where 𝐴_𝑘 is the probability of a successful transmission among 𝑘≥2. Denote by 𝑎^𝑘𝑖 the event that transmission of some tagged mobile 𝑖 is successful when having 𝑘−1 other simultaneous transmissions. It can be derived by the following decomposition 𝑎^𝑘𝑖=∑^𝑁𝑡=2𝑃 (mobile 𝑖 transmits with power level 𝑝_𝑡; other mobiles transmit with powers less than 𝑝_𝑡; SINR of mobile 𝑖 is greater than the threshold 𝛾_th). Since mobiles are assumed to be symmetric, then 𝐴_𝑘=∑^𝑘𝑖=1𝑎^𝑘𝑖=𝑘⋅𝑎^𝑘𝑖, and it follows that

𝐴_𝑘=𝑘^𝑁−2_{𝑙=0𝑘−1}_{𝑘1=0𝑘−1𝑘2=0}⋯^𝑘−1_𝑘𝑁_{−𝑙−1=0}𝑥^𝑘1₁⋅𝑥^𝑘2₂⋯𝑥^{𝑘𝑁−𝑙−1}_{𝑁−𝑙−1}⋅𝑥^1𝑁−𝑙⋅𝛿𝑘−1−^{𝑁−𝑙−1}_𝑠=1𝑘_𝑠⋅𝑢∑^{𝑁−𝑙−1𝑠=1}𝑝𝑝_𝑠𝑘^𝑁−𝑙_𝑠+𝜎²/𝑔−𝛾^th(5) with 𝐴₀=0 and 𝐴₁=1. 𝑥_𝑠 denotes the probability that a user (with new arrival or backlogged packet) (re)transmits using power level 𝑝_𝑠.𝑝_𝑁−𝑙 is the power level chosen by the terminal whose transmission maybe potentially succeed, that is, its corresponding to the highest power selected in the current slot, whereas 𝑘_𝑠 denotes the number of terminals that choose the power level 𝑝_𝑠. 𝛿(𝑡) (Dirac distribution) and 𝑢(𝑡) (unite echelon) are defined as follows:

𝛿(𝑡)=10,,iefls𝑡e=,0,𝑢(𝑡)=10,,iefls𝑡e≥.0,(6)

Calculating the success probability is a very hard issue. The difficulty in formula (5) is to consider one single transmitting mobile at the highest power level and list all the cases where the 𝑘−1 remaining mobiles transmit at lower power. This corresponds exactly to the set of partitions of the positive integer 𝑘−1 considering all possible permutations. (A partition of a positive integer 𝑛 is a way of writing 𝑛 as a sum of positive integers.) Generating all the partitions of an integer is widely studied in the literature and several algorithms were proposed; for example, see [20, 21]. The computational complexity of such algorithms is very high and may take long time to list the set of all partitions as well as their permutations. However, in our model the success probability depends on none of the following: the instantaneous backlog of the system 𝑛, the arrival probability 𝑞_𝑎, and, the retransmission probability 𝑞_𝑟. Henceforth, success probability matrix 𝐀=(𝐴_𝑘), 𝑘=1⋯𝑚 can be calculated once and reused to derive the transition matrix.

3.2. Scheme 2: Retransmission with More Power

Now, backlogged packets have more priority; mobile having a backlogged packet retransmits using a random power level among the 𝑁 available, while a mobile with a new arrived packet uses always the lowest level (𝑝₁). Successful capture is occurred when one of the backlogged packets transmits with a power level which is larger than the one chosen by all other transmitters and its corresponding SINR is greater than the threshold 𝛾_th when arriving at the 𝐴𝑃, or a single new arrival occurs and there is no retransmission attempt of any backlogged packet. The transition probabilities are given by

𝑃_{𝑛,𝑛+𝑖}=⎧⎪⎪⎪⎪⎪⎪⎪⎪⎪⎪⎪⎪⎪⎪⎨⎪⎪⎪⎪⎪⎪⎪⎪⎪⎪⎪⎪⎪⎪⎩𝑄^𝑎(𝑚−𝑛,𝑛)^𝑛_𝑗=1𝑄𝑖𝑄=_𝑟𝑚(𝑗,−𝑛)𝑛,1𝑖≥−𝐵2,_{𝑗,𝑚−𝑛𝑎},(𝑖,𝑛)^𝑛_𝑗=0+𝑄_𝑎𝑄_𝑟(𝑗,𝑛)1−𝐵_𝑗,𝑖(𝑖+1,𝑛)^𝑛_𝑗=1𝑄2≤𝑖𝑄<_𝑟(𝑚𝑗,−𝑛)𝑛𝐵,_{𝑗,𝑖+1𝑎},(1,𝑛)^𝑛_𝑗=1+𝑄_𝑎𝑄_𝑟(𝑗,𝑛)1−𝐵_𝑗,1(2,𝑛)^𝑛_𝑗=1𝑄𝑄_𝑟(𝑗,𝑛)𝐵_𝑗,2,𝑖=1,_𝑎(0,𝑛)𝑄_𝑟(0,𝑛)+^𝑛_𝑗=2+𝑄_𝑎𝑄_𝑟(𝑗,𝑛)1−𝐵_𝑗,0(1,𝑛)^𝑛_𝑗=0𝑄𝑄_𝑟(𝑗,𝑛)𝐵_𝑗,1,𝑖=0,_𝑎(0,𝑛)^𝑛_𝑗=1𝑄_𝑟(𝑗,𝑛)𝐵_𝑗,0,𝑖=−1,(7) where the probability of a successful transmission among 𝑘 retransmissions and 𝑘^ new arrival packets when 𝑘+𝑘^≥2 is given by

𝐵_𝑘,𝑘^′=𝑘^𝑁−2_{𝑙=0𝑘−1𝑘1=0}⋯^𝑘−1_{𝑘𝑁−𝑙−1=0𝑁−𝑙−1𝑖=1}𝑥^𝑘𝑖_𝑖⋅𝑥^1𝑁−𝑙⋅𝛿𝑘−1−^{𝑁−𝑙−1}_𝑠=1𝑘_𝑠⋅𝑢𝑝_𝑁−𝑙∑^{𝑁−𝑙−1𝑠=1}𝑝_𝑠𝑘_𝑠+𝑘^𝑝₁+𝜎²/𝑔−𝛾^th(8) with 𝐵_0,0=0, 𝐵_0,1=1, and 𝐵_1,0=1.

3.3. Scheme 3: Retransmission with Less Power

New arrivals are transmitted with the highest power 𝑝_𝑁, whereas mobiles having backlogged packets attempt new retransmission using a random power picked from 𝑁−1 distinct lower power levels. The transition matrix is summarized by

𝑃_{𝑛,𝑛+𝑖}=⎧⎪⎪⎪⎪⎪⎪⎨⎪⎪⎪⎪⎪⎪⎩𝑄^𝑎𝑄(𝑖,𝑛),2≤𝑖,_𝑎(1,𝑛)^𝑛_𝑗=1𝑄𝑄_𝑟(𝑗,𝑛)1−𝐶_𝑗,1,𝑖=1,_𝑎(0,𝑛)𝑄_𝑟(0,𝑛)+^𝑛_𝑗=2+𝑄_𝑎𝑄_𝑟(𝑗,𝑛)1−𝐶_𝑗,0(1,𝑛)^𝑛_𝑗=0𝑄𝑄_𝑟(𝑗,𝑛)𝐶_𝑗,1,𝑖=0,_𝑎(0,𝑛)^𝑛_𝑗=1𝑄_𝑟(𝑗,𝑛)𝐶_𝑗,0,𝑖=−1,(9) where the probability of a successful transmission when 𝑘≥1 mobiles attempt retransmissions is calculated using the following formula:

𝐶_𝑘,1=𝑘^𝑘−1_{𝑘1=0𝑘−1𝑘2=0}⋯^𝑘−1_{𝑘𝑁−1=0𝑁−1𝑖=1}𝑥^𝑘𝑖_𝑖⋅𝑥^1𝑁−𝑙⋅𝛿𝑘−^𝑁−1_𝑠=1𝑘_𝑠⋅𝑢𝑝_𝑁∑^{𝑁−1𝑠=1}𝑝_𝑠𝑘_𝑠+𝜎²/𝑔−𝛾^th.(10)

The probability of a successful retransmission among 𝑘≥2 simultaneous attempts is given by

𝐶_𝑘,0=𝑘^𝑁−2_{𝑙=1𝑘−1}_{𝑘1=0𝑘−1𝑘2=0}⋯^𝑘−1_{𝑘𝑁−𝑙−1=0𝑁−𝑙−1𝑖=1}𝑥^𝑘𝑖_𝑖⋅𝑥^1𝑁−𝑙⋅𝛿𝑘−1−^{𝑁−𝑙−1}_𝑠=1𝑘_𝑠⋅𝑢𝑝_𝑁−𝑙∑^{𝑁−𝑙−1𝑠=1}𝑝_𝑠𝑘_𝑠+𝜎²/𝑔−𝛾^th,(11) whereas 𝐶_𝑘,𝑘^′=0 if 𝐾^≥2,𝐶_0,1=1, and 𝐶_1,0=1.

3.4. Scheme 4: Retransmission with the Lowest Power

In this last proposal, a new transmitted packet uses a power among 𝑁−1 higher available power levels. Mobiles having backlogged packet retransmit with the lowest power level (𝑝₁). The transition matrix of the Markov chain is given by

𝑃_{𝑛,𝑛+𝑖}=⎧⎪⎪⎪⎪⎪⎪⎪⎪⎪⎪⎪⎪⎪⎨⎪⎪⎪⎪⎪⎪⎪⎪⎪⎪⎪⎪⎪⎩𝑄^𝑎(𝑚−𝑛,𝑛)^𝑛_𝑗=0𝑄𝑄𝑖_𝑟=(𝑗,𝑚𝑛)−𝑛1,−𝑖≥𝐷2,_{𝑗,𝑚−𝑛𝑎},(𝑖,𝑛)^𝑛_𝑗=0+𝑄_𝑎𝑄_𝑟(𝑗,𝑛)1−𝐷_𝑗,𝑖(𝑖+1,𝑛)^𝑛_𝑗=0𝑄2≤𝑖𝑄<_𝑟(𝑚𝑗,−𝑛)𝑛𝐷,_{𝑗,𝑖+1𝑎},(1,𝑛)^𝑛_𝑗=1+𝑄_𝑎𝑄_𝑟(𝑗,𝑛)1−𝐷_𝑗,1(2,𝑛)^𝑛_𝑗=0𝑄𝑄_𝑟(𝑗,𝑛)𝐷_𝑗,2,𝑖=1,_𝑎(0,𝑛)𝑄_𝑟(0,𝑛)+^𝑛_𝑗=2+𝑄_𝑎𝑄_𝑟(𝑗,𝑛)1−𝐷_𝑗,0(1,𝑛)^𝑛_𝑗=0𝑄𝑄_𝑟(𝑗,𝑛)𝐷_𝑗,1,𝑖=0,_𝑎(0,𝑛)𝑄_𝑟(1,𝑛),𝑖=−1,(12) where 𝐷_𝑘,𝑘^′ represents the probability of a successful new transmission among 𝑘 backlogged packets and 𝑘^ new packets such that 𝑘^+𝑘≥2. The value 𝐷_𝑘,𝑘^′is given by

𝐷_𝑘,𝑘^′=𝑘^𝑁−2_{𝑙=0𝑘−1𝑘′1=0}⋯^𝑘′−1_{𝑘′𝑁−𝑙−1=0𝑁−𝑙−1𝑖=1}𝑥^𝑘′𝑖_𝑖⋅𝑥^1𝑁−𝑙⋅𝛿⎛⎜⎜⎝𝑘^−1−^{𝑁−𝑙−1}_𝑝_𝑙=1𝑘^𝑝_𝑙⋅𝑢𝑝_𝑁−𝑙⎞⎟⎟⎠∑^{𝑝𝑁−𝑙−1}_𝑙=1𝑝_𝑙𝑘^𝑝_𝑙+𝑘𝑃₁+𝜎²/𝑔−𝛾^th,(13) where 𝐷_0,0=0, 𝐷_0,1=1, and 𝐷_1,0=1.

3.5. Performance Metrics

We now turn to present the performance measures (average throughput and expected delay) of interest for optimization as a function of the steady-state probabilities of the Markov chain. Let us denote by 𝜋_𝑛(𝑞) the equilibrium probability that the network is in state 𝑛 (number of backlogged packets at the beginning of a slot). Hence the equilibrium state equations are

𝝅(𝑞)=𝝅(𝑞)⋅𝑃(𝑞),_𝑚_𝑛=0𝜋𝜋_𝑛(𝑞)=1,_𝑛(𝑞)≥0,𝑛=0,1,…,𝑚.(14)

System (14) yields the equilibrium probabilities vector. The average number of backlogged packets can then simply be calculated by

𝑆(𝑞)=^𝑚_𝑛=0𝜋_𝑛(𝑞)⋅𝑛.(15)

Similarly, the system throughput (defined as the sample average of the number of packets that are successfully transmitted) is given almost surely by the constant

^𝑚_{𝑛=0𝑚−𝑛𝑛𝑖=0}_𝑗=0𝜋_𝑛(𝑞)𝑄_𝑎(𝑖,𝑛)𝑄_𝑟(𝑗,𝑛)𝐴_𝑗+𝑖,Scheme1,_𝑚_𝑛=0𝜋_𝑛(𝑞)^𝑚−𝑛_𝑛𝑖=0_𝑗=1𝑄_𝑎(𝑖,𝑛)𝑄_𝑟(𝑗,𝑛)𝐵Sc_𝑗,𝑖h+em𝑄e_𝑎2(1,_𝑚,𝑛)𝑄_𝑟(0,𝑛),_𝑛=0𝜋_𝑛(𝑞)𝑄_𝑎(0,𝑛)^𝑛_𝑗=1⋅𝑄_𝑟(𝑗,𝑛)𝐶_𝑗,0+𝑄_𝑎(1,𝑛),_𝑛_𝑗=0𝑄_𝑟(𝑗,𝑛)𝐶_𝑗,1Scheme3,_𝑚_𝑛=0𝜋_𝑛(𝑞)^𝑚−𝑛_𝑛𝑖=1_𝑗=0𝑄_𝑎(𝑖,𝑛)𝑄_𝑟(𝑗,𝑛)𝐷Sc_𝑗,𝑖h+em𝑄e_𝑎4(,_𝑚0,𝑛)𝑄_𝑟(1,𝑛),_𝑛=0𝜋_𝑛(𝑞)𝑄_𝑎(0,𝑛)𝑄_𝑟(1,𝑛)S+am𝑄e_𝑎(p1o,w𝑛e)r𝑄._𝑟(0,𝑛),(16)

Using the rate balance equation (i.e., input=output) at the steady state, the throughput satisfies (and thus can be computed more easily through)

th𝑝(𝑞)=𝑞_𝑎𝑚_𝑛=0𝜋_𝑛(𝑞)(𝑚−𝑛)=𝑞_𝑎(𝑚−𝑆(𝑞)).(17)

Indeed, the throughput is the expected number of arrivals at a time slot (which actually enter the system), and this is expressed in the equation for th𝑝(𝑞) by conditioning on 𝑛. The throughput should be equal to the expected number of departures (and thus the throughput) at stationary regime, which is expressed in (17). The expected delay of transmitted packets 𝐷 is defined as the average time, in slots, that a packet takes from its source to the receiver. Then, Little's result yields

𝐷(𝑞)=1+𝑆(𝑞)th𝑝(𝑞)=1+𝑞_𝑎(𝑚𝑆−(𝑞𝑆)(𝑞)).(18)

Analyzing (17) and (18) it is easy to show that maximizing the global throughput is equivalent to minimizing the average delay of transmitted packets. We shall therefore restrict in our numerical investigation to maximization of the throughput. However, we shall consider the delay of backlogged packets as yet another objective to minimize.

3.6. Performance Measures for Backlogged Packets

If we denote by Δ the throughput of new arrivals, that is, the amount of arrivals whose first transmission attempt is successful, then the throughput of backlogged packets for each scheme is given by th𝑝^𝑐(𝑞)=th𝑝(𝑞)−Δ, where Δ is calculated from Markov chain and given by

^𝑚_{𝑛=0𝑚−𝑛𝑛𝑖=1}_𝑗=0𝑖𝑖+𝑗𝜋^𝑛(𝑞)𝑄_𝑎(𝑖,𝑛)𝑄_𝑟(𝑗,𝑛)𝐴_𝑖+𝑗,Scheme1,_𝑚_𝑛=0𝑄𝜋_𝑛(𝑞)𝑄_𝑎(1,𝑛)𝑄_𝑟(0,𝑛),Scheme2,_𝑎(1,𝑛)^𝑛_𝑗=0𝑄_𝑟(𝑗,𝑛)𝐶_𝑗,1,Scheme3,_𝑚_{𝑛=0𝑚−𝑛𝑛𝑖=1}_𝑗=0𝜋_𝑛(𝑞)𝑄_𝑎(𝑖,𝑛)𝑄_𝑟(𝑗,𝑛)𝐷_𝑗,𝑖,Scheme4,_𝑚_𝑛=0𝜋_𝑛(𝑞)𝑄_𝑎(1,𝑛)𝑄_𝑟(0,𝑛),Samepower.(19)

The expected delay of backlogged packets 𝐷^𝑐, which is defined as the average time, in slots, that a backlogged packet takes to go from the source to the receiver, can also be calculated by applying Little's result. Hence

𝐷^𝑐(𝑞)=1+𝑆(𝑞)th𝑝^𝑐(𝑞).(20)

3.6.1. Team Problem Resolution

The optimal solution of the team problem is obtained by resolving the following optimization problem:

max_𝑞objective(𝑞)s.t.⎧⎪⎪⎨⎪⎪⎩𝝅(𝑞)=𝝅(𝑞)⋅𝑃(𝑞),_𝑚_𝑛=0𝜋𝜋_𝑛(𝑞)=1,_𝑛(𝑞)≥0,𝑛=0,1,…,𝑚,(21) where objective(𝑞) is replaced by the average throughput or minus expected delay. We note that the solution can be obtained by computing recursively the steady-state probabilities, as Problem 4.1 in [3].

3.6.2. Singularity at q = 0

The only point where the Markov chain 𝑃 does not have a single stationary distribution is at 𝑞=0, where it has two absorbing states: 𝑛=𝑚 and 𝑛=𝑚−1. All remaining states are transient (for any 𝑞_𝑎>0), and the probability to end at one of the absorbing states depends on the initial distribution of the Markov chain. We note that if the state 𝑚−1 is reached, then the throughput is 𝑞_𝑎 w.p. 1. However, if the state 𝑚 is reached, then the throughput equals 0, which means that it is a deadlock state. For 𝑞_𝑎>0 and 𝑞_𝑟=0, the deadlock state is reached with positive probability from any initial state other than the absorbing state 𝑚−1. we shall therefore exclude the case of 𝑞_𝑟=0 and optimize only on the range 𝜖<𝑞_𝑟≤1.

3.6.3. Existence of a Solution

The steady-state probabilities 𝝅(𝑞) are continuous over 0<𝑞≤1. This is not a closed interval, therefore a solution needs not exist. However, as we restrict to the closed interval [𝜖,1], where 𝜖>0, an optimal solution indeed exists. Therefore for any 𝛿>0, there exists some 𝑞^∗>0 which is 𝛿-optimal. 𝑞>0 is said to be 𝛿-optimal for the throughput maximization if it satisfies th𝑝(𝑞^∗)≥th𝑝(𝑞)−𝛿 for all 𝑞∈[𝜖,1]. A similar definition holds for any objective function (e.g., delay minimization).

3.7. Stability

Another qualitative way to compare schemes is the stability characteristics of the protocol. Slotted aloha is known to have a bi-stable behavior, we hence shall check whether this is also the case in our four schemes, if answer is yes, under which conditions?

Let us denote 𝑝^succ_𝑛 the expected number of successful transmissions in the current slot having 𝑛 backlogged packets. Based on the derived Markov chains, the probability of a successful transmission is given by

𝑝=^succ_𝑛⎧⎪⎪⎪⎪⎪⎪⎪⎪⎪⎨⎪⎪⎪⎪⎪⎪⎪⎪⎪⎩(𝑞)_𝑚−𝑛_𝑛𝑖=0_𝑗=0𝑄_𝑎(𝑖,𝑛)𝑄_𝑟(𝑗,𝑛)𝐴_𝑗+𝑖,Scheme1_𝑚−𝑛_𝑛𝑖=0_𝑗=1𝑄𝑄_𝑎(𝑖,𝑛)𝑄_𝑟(𝑗,𝑛)𝐵_𝑗,𝑖+𝑄Sc_𝑎h(e1m,𝑛e)2_𝑎𝑄_𝑟(0,𝑛),(0,𝑛)^𝑛_𝑗=1𝑄_𝑟(𝑗,𝑛)𝐶_𝑗,0+𝑄_𝑎(1,𝑛)^𝑛_𝑗=0Scheme𝑄3_{𝑟𝑚−𝑛}(𝑗,𝑛)𝐶_𝑗,1,_𝑛𝑖=1_𝑗=0𝑄𝑄_𝑎(𝑖,𝑛)𝑄_𝑟(𝑗,𝑛)𝐷_𝑗,𝑖+Sc𝑄he_𝑎m(0e,4_𝑎𝑛)𝑄_𝑟(1,𝑛),(0,𝑛)𝑄_𝑟(1,𝑛)+𝑄_𝑎(1,𝑛)𝑄_𝑟(0,𝑛).Slottedaloha(22)

Define now the drift 𝐷_𝑛 in state 𝑛, as the expected change in backlog from one slot to the next slot, which is the expected number of arrivals, that is, 𝑞_𝑎(𝑚−𝑛), less the expected number of successful departures 𝑝^succ_𝑛, that is

𝐷_𝑛=𝑞_𝑎(𝑚−𝑛)−𝑝^succ_𝑛.(23)

For standard slotted aloha it has been shown that three equilibria may exist. System equilibrium points occur where the curves 𝑝^succ_𝑛 and the straight line 𝑞_𝑎(𝑚−𝑛) intersect. When the drift, which represents the difference between the straight line and the curve, is positive the system state tends to increase, because the system input rate becomes greater than its output rate. Whereas it decreases when the drift is negative. This explains why the middle equilibrium point is definitively unstable and the other two are stable. A bi-stable situation as in the standard aloha is hence undesirable since it means, in practice, that the system spends long time in each of the stable equilibria, including the one with large backlog 𝑛 corresponding to a congestion situation (low throughput and large delay).

Let us examine the stability behavior of slotted aloha and our new schemes for 𝑚=40 mobiles, threshold SINR 𝛾_th=10 dB, arrival probability 𝑞_𝑎=0.01, and 𝑁=5 selectable powers; see Figure 2. We note that no scheme suffers from the bistability problem under low aggressiveness 𝑞_𝑟=0.1, where the departure rate of our schemes is, all the time, greater than the arrival rate. It follows that slotted aloha under our algorithms is stable and the average number of backlogged packets is very low (which decreases significantly the expected delay). Under 𝑞_𝑟=0.25 slotted aloha and scheme 4 become instable whereas other schemes with random power keep stability whatever the average number of backlogged packetis. In contrast to standard slotted aloha and through simple computation of equilibria, the expected number of backlogged packets for schemes 1–4 can be approximated by the desired stable equilibrium which provides a very interesting feature. That means that in the bistability case for schemes 1–4, the system spends most of the time at that desired equilibrium. Next, we note that success probability 𝑃^succ_𝑛 decreases with 𝑛 and vanishes for all schemes for 𝑞_𝑟>0.5, where all schemes acquire a bi-stable behavior. It follows that the stability region is tightening with transmission rate 𝑞_𝑟. Indeed slotted aloha and all other schemes become instable when mobiles retransmit aggressively. Then the collision probability is close to 1 and the departure rate becomes less than the arrival rate which causes absorption of the system by the undesired equilibrium point. Under this situation the average number of backlogged packets can be approximated by the nondesired equilibrium. Here, the system spends most of the time at that equilibrium.

(a) backlogged packets 𝑛

(b) backlogged packets 𝑛

(c) backlogged packets 𝑛

Figure 2 

Stability of slotted aloha with random power selection algorithms: Probability of success transmission versus backlogged packets 𝑛 for all the schemes. We consider 𝛾th=10 dB, transmit powers 𝒫=[1,5,25,125,625]mW, arrival probability 𝑞𝑎=0.01 and different retransmission probabilities (aggressiveness levels) 𝑞𝑟=0.1 (sub-figure a), 𝑞𝑟=0.25 (sub-figure b) and 𝑞𝑟=0.5 (sub-figure c).

The observed bipolar behavior of slotted aloha as well as in our schemes can be avoided by decreasing the probability of retransmission. Yet, decreasing 𝑞_𝑟 expands the departure rate curve which removes all intersections with arrival straight line but the desired stability point. The drawback is that expected delay increases since a packet will wait longer at a backlogged node before a successful transmission and the maximum throughput decreases slightly.

4. Game Problem

Implementation of a centralized system is a real issue since it needs a high technology, advanced software performances, extra signaling protocol, and full information about mobiles and their instantaneous QoS (Quality of Service) requirements. This way, a high amount of bandwidth should be reserved for signaling. Moreover, mobile users are not forced to cooperate and may act in a selfish way. We derive in this section an alternative framework which is decentralized. In this context, the team model does not hold any more, we shall then formulate a distributed model by the help of game theory tools. Yet, the decentralized model is more powerful and appropriate to slotted aloha and has high interest for analyzing access games in general. Therefore, Nash equilibrium concept will replace the optimality concept used in the team problem. It possesses a robustness property: at equilibrium and assuming rationality of mobiles, no mobile has incentive to deviate. The elements of our contention game are listed as follows.

(i)A finite set of 𝑚+1 bufferless users interacts over a single collision channel.(ii)Each user 𝑖 retransmits, in every slot, its packets with probability 𝑞^𝑖𝑟. The open interval 𝑞^𝑖𝑟∈(0,1] corresponds to the set of all possible actions of user 𝑖. (iii)There is no cost of (re)transmitting packets. (iv)The individual throughput or minus expected delay of backlogged packets is the utility functions to maximize.

For any instance of the game, we denote the policy vector of retransmission probabilities for all users by 𝐪_𝑟 whose 𝑗th entry is 𝑞^𝑗𝑟. Define (𝑞^𝑟−𝑖,𝑞^𝑖𝑟) to be a retransmission policy at a slot, where user 𝑖 retransmits with probability 𝑞^𝑖𝑟 and any other user 𝑗 retransmits with probability 𝑞^𝑗𝑟 for all 𝑗≠𝑖. Each user 𝑖 seeks to maximize his own objective_𝑖(𝐪_𝑟); each mobile either maximizes its own throughput or maximizes minus expected delay. The problem, we are interested in, is then to find a symmetric equilibrium policy 𝐪^∗𝑟=(𝑞_𝑟,𝑞_𝑟,…,𝑞_𝑟) such that for any user 𝑖 and any retransmission probability 𝑞^𝑖𝑟 for that user,

objective_𝑖𝐪^∗𝑟≥objective_𝑖𝐪^∗𝑟^−𝑖,𝑞^𝑖𝑟.(24)

Without any loss of generality, we restrict to symmetric policy 𝐪^∗𝑟 where all mobiles are balanced-payoff. We shall also identify it (with some abuse of notation) with the actual transmission probability which is the same for all users. We first note that due to symmetry, to see whether 𝐪^∗𝑟 is an equilibrium, it suffices to check (24) for a single player. We shall thus assume that 𝑚 users retransmit with a given probability [𝐪_𝑟]^−(𝑚+1)=(𝑞^𝑜𝑟,𝑞^𝑜𝑟,…,𝑞^𝑜𝑟) and the user 𝑚+1 retransmits with probability 𝑞^𝑟𝑚+1. Define the set 𝒬^𝑚+1(𝐪^𝑜𝑟) as the set of best response strategies of user 𝑚+1; it can be written as

𝒬^𝑚+1𝐪^𝑜𝑟=argmax_{𝑞𝑟𝑚+1∈[𝜖,1]}objective^𝑚+1𝐪^𝑜𝑟^−(𝑚+1),𝑞^𝑟𝑚+1,(25) where 𝐪^𝑜𝑟 denotes the policy where all users retransmit with probability 𝑞^0𝑟 and the maximization is taken with respect to 𝑞^𝑟𝑚+1. Then 𝐪^∗𝑟 is a symmetric equilibrium if

𝐪^∗𝑟∈𝒬^𝑚+1𝐪^∗𝑟.(26)

To compute the performance measures of interest, we introduce again a Markov chain with a two dimensional state; see Figure 3. The first state component corresponds to the number of backlogged packets among users 1,…,𝑚, and the second component is the number of backlogged packets (either 1 or 0) of user 𝑚+1.

Figure 3 

Bi-dimensional Markov chain for the game setting. The state of the system is the backlog vector; the first component corresponds to the number of backlogged packets for 𝑚 first mobiles whereas the second component indicates the number of backlogged packets of the tagged mobile 𝑚+1 (either 0 or 1).

4.1. Scheme 1: Random Power without Priority

We consider the game problem in which packets are transmitted/retransmitted with a random power (according to the distribution probability 𝑋=[𝑥₁,𝑥₂,…,𝑥_𝑁]) selected from 𝑁 levels. In this scheme, there is no priority for both classes of packets. The transition probabilities when a given mobile uses 𝑞^𝑟𝑚+1 and the 𝑚 other mobiles use 𝑞^0𝑟 are given in Appendix A.

4.2. Scheme 2: Retransmission with More Power

We consider here the proposal in which backlogged packets are retransmitted with random power picked from 𝑁−1 high levels. A new arrival packet is always transmitted with the lowest power level, that is, 𝑝₁. The transition probabilities when 𝑚 mobiles use 𝑞^0𝑟 and the 𝑚+1 mobile transmits at a rate 𝑞^𝑟𝑚+1 are clarified in Appendix B.

4.3. Scheme 3: Retransmission with Less Power

In this scheme, we assume that new arrivals always transmit over the channel using the highest power level 𝑝_𝑁, whereas backlogged packets attempt retransmission with some power among remaining 𝑁−1 lower levels. The corresponding transition matrix whose elements are 𝑝_{(𝑛,𝑎),(𝑛+𝑖,𝑏)} is summarized in Appendix C.

4.4. Scheme 4: Retransmission with the Lowest Power

We finally consider the noncooperative scenario where backlogged packets are retransmitted with the lowest power level 𝑝₁. Here, new arrival packets are transmitted with a power selected among 𝑁−1 distinct higher levels. The transition matrix of this scheme is given in Appendix D.

4.5. Performance Metrics

Let 𝜋_𝑛,𝑎 be the steady state of the Markov chain where 𝑛 is the number of backlogged packets of the 𝑚 first mobiles and 𝑎 is the binary-valued number of backlogged packets of user 𝑚+1. The average number of backlogged packets of source 𝑚+1 is written as

𝑆_𝑚+1𝐪^𝑜𝑟^−𝑚+1,𝑞^𝑟𝑚+1=^𝑚_𝑛=0𝜋_𝑛,1𝐪^𝑜𝑟^−(𝑚+1),𝑞^𝑟𝑚+1,(27) and the average throughput of user 𝑚+1 is almost surely given by

th𝑝_𝑚+1𝐪^𝑜𝑟^−(𝑚+1),𝑞^𝑟𝑚+1=𝑞^𝑎𝑚_𝑛=0𝜋_𝑛,0𝐪^𝑜𝑟^−(𝑚+1),𝑞^𝑟𝑚+1.(28) Thus, the expected delay of transmitted packets at source 𝑚+1 for all schemes verifies (Little's result)

𝐷_𝑚+1𝐪^𝑜𝑟^−(𝑚+1),𝑞^𝑟𝑚+1=1+𝑆^𝑚+1𝐪^𝑜𝑟^−(𝑚+1),𝑞^𝑟𝑚+1th𝑝_𝑚+1𝐪^𝑜𝑟^−(𝑚+1),𝑞^𝑟𝑚+1.(29)

4.5.1. Performance Measures for Backlogged Packets

Let us denote the throughput of backlogged packets (i.e., packets that arrive and become backlogged) at source 𝑚+1 by

th𝑝^𝑐𝑚+1𝐪^𝑚+1=^𝑚_𝑚𝑛=0_𝑛′=0𝑝_{(𝑛,0),(𝑛′,1)}𝐪^𝑚+1𝜋_𝑛,0𝐪^𝑚+1.(30)

Thus, the expected delay of backlogged packets at source 𝑚+1 is

𝐷_𝑚+1𝐪^𝑚+1=1+𝑆^𝑚+1𝐪^𝑚+1th𝑝^𝑐𝑚+1𝐪^𝑚+1.(31)

5. Numerical Investigation

We conduct here a numerical investigation of the discussed cooperative and noncooperative frameworks of slotted aloha under power diversity, packets priority, and capture effect. We fix throughout this section 𝜖=10⁻⁴; that is, the solution 𝑞^∗ will be searched in the closed interval [𝜖,1] instead of ]0,1]; see Section 3.6.

5.1. Impact of System Parameters

In the following we investigate the impact of each parameter on the protocol performance. For illustrative purpose, we use scheme 1 and yield similar results for other schemes. We also focus on the throughput as a measurement metric; other metrics such as delay provide similar trends. We plot the throughput as a function of arrival probability when changing the number of available power levels; Figure 4(a). We note that thethroughput is improved when increasing the number of power levels. This is quite intuitive since mobiles will have larger choices and then high chance to decode correctly the received signal by the central receiver.

(a)

(b)

Figure 4 

Impact of the number of available power levels (a) and threshold SINR 𝛾th (b) on the global throughput under scheme 1.

Next we depict the average throughput for different values of threshold SINR 𝛾_th; see Figure 4(b). The system performances are deteriorated while increasing 𝛾_th. When the access point requires a high signal quality (i.e., high value of 𝛾_th) to decode correctly the received signals, the loss probability becomes very high and therefore the throughput decreases. Furthermore, we note that the best performances are obtained when the available power levels follow a geometric progression (see Figure 5(a)) whereas same power scenario (the case of standard slotted aloha) provides the lowest performance. This can be explained by the distance between power levels and its strong impact on the instantaneous SINR value. For a tagged mobile 𝑖, with geometric or more rapid progression, only successive power levels may interfere significantly with the chosen power 𝑝^𝑖𝑙, whereas all other power levels have high interfering capability on transmitted packet when using arithmetic progression. It is also interesting to show that performances are improved when increasing the geometric step. Another parameter which influences on the whole performances is the probability distribution 𝑋=[𝑥₁,𝑥₂,…,𝑥_𝑁] to select transmit powers. Figure 5(b) shows that uniform distribution (i.e., 𝑥_𝑖=1/𝑁) is the best, whereas prioritizing high power levels provides the lowest performances. Yet, prioritizing some power levels leads the system to behave as an equivalent system with less number of available power levels, which explains the observed decrease of performance. When available power levels are uniformly distributed, they all have the same chance to be used and henceforth that fashion outperforms other distributions.

(a)

(b)

Figure 5 

This figure shows the impact of power levels (a) and selection probabilities distribution (b) on the global throughput under scheme 1.

5.2. Team Problem

5.2.1. Aggregate Throughput Maximization

In the following, we are interested in the symmetric solution that maximizes the global throughput. We depict in Figures 6–8 the throughput, expected delay of backlogged packets (EDBPs), and optimal retransmission probability for all addressed schemes. We fix the threshold SINR to 𝛾_th=10 dB and consider five selectable power levels 𝒫=[1,5,25,125,625]mW for all schemes. Note that slotted aloha can be obtained from scheme 1 using same power policy and infinite value of 𝛾_th.

(a)

(b)

Figure 6 

Aggregate throughput and expected delay of backlogged packets for 4 mobiles under the team problem and throughput maximization.

(a)

(b)

Figure 7 

Aggregate throughput and expected delay of backlogged packets for 10 mobiles under the team problem and throughput maximization.

(a)

(b)

Figure 8 

Optimal retransmission probability for schemes 1–4 and slotted aloha versus arrival rate 𝑞𝑎 for both 4 mobiles and 10 mobiles when throughput is maximized.

First, we evaluate the system performance in terms of aggregate throughput for 4 mobiles. In Figure 6, we plot the global throughput as a function of arrival probability 𝑞_𝑎. At very low load (𝑞_𝑎<0.1), all schemes have likely same performance which can be approached by a linear function in 𝑞_𝑎. For low load (0.1<𝑞_𝑎<0.24), scheme 2 seems to perform slightly better than other schemes. When arrival rate is average (between 0.24 and 0.6), scheme 2 performs better and provides higher throughput. This is due to the fact that scheme 2 prioritizes the retransmission of backlogged packets exploiting the fact that there is few new arrivals. But at high load the throughput of scheme 4 becomes the highest because it prioritizes new arrivals. Clearly, new arrival packets have an extended choice of power levels and therefore benefit from prioritization and power diversity.

We remark that scheme 3 (retransmission with less power) which is the same as the one first proposed in [15] presents the least performance compared to other schemes. This is due to the negative effect of large choice of power levels reserved to backlogged packets and in particular to the penalizing capture effect. This way, the instantaneous SINR of new arrival becomes strongly noised by backlogged packets. However, we note that all schemes with random power selections and capture effect outperform standard slotted aloha.

In terms of expected delay of backlogged packets, at low and average load and for all schemes (see Figure 6(b)), we obtain a slightly increasing function of the arrival probability, and it can be approached by a semiconstant value. For instance, the average backoff duration of schemes 1–4 and slotted aloha can be approximated by 𝑑₁≃7, 𝑑₂≃4, 𝑑₃≃10, 𝑑₄≃4, and 𝑑₀≃11 slots, respectively. It is clear that schemes 3 and 4 are the best in terms of throughput and delay. At heavy and very heavy loads (0.75<𝑞_𝑎), delay of schemes 1–3 and slotted aloha increases exponentially, whereas scheme 4 holds a semiconstant value for expected delay. Under scheme 4, backlogged packets have extended choices (all power levels indexed as 𝑝₁,𝑝₂,…,𝑝_𝑁−1), and this gives advantage to retransmissions when there is no new arrival. Hence, latter fashion provides a very good amount of successful transmission under a low delay tradeoff. The dramatic huge of delay of other schemes decreases significantly the system reliability by causing very large backoff stage. Then schemes 1–3 and slotted aloha are not recommended for the delay sensitive applications such as real time services (e.g., voice, streaming etc.) elsewhere the system reliability and QoS guaranteeing become a hard issue, whereas scheme 4 seems to be perfectly adapted to support these classes of services since its respective backoff stage is strongly reduced.

Next we plot the of optimal retransmission probability versus offered load. We remark that for 𝑚=4, see Figure 8(a), all schemes optimal retransmission probabilities 𝑞_𝑟 are decreasing with 𝑞_𝑎 until to be semiannulled (𝑞_𝑟≃10⁻⁴=𝜖 because we only consider solutions in [𝜖,1]) for schemes 1–3 and slotted aloha. This explains the huge EDBP seen for these schemes: each backlogged packet stays a long time in the system. whereas retransmission rate keeps a constant value (around 0.3) for scheme 4 (for 𝑞_𝑎 over 0.5) because it prioritizes new packets and then it hurts not from backlogged packets. This later scheme seems to be the most fair since both new arrivals as well as backlogged packets take advantage from a high throughput and low delay and then may meet a good channel utilization.

For 𝑚=10, see Figures 7(a), 7(b) and 8(b), we observe similar trends in terms of throughput and delay for all schemes. In fact even if the number of mobiles becomes large, a good whole performance is handled by decreasing retransmission rates so as to avoid/reduce potential collisions. We remark that at heavy load the base station asks mobiles to decrease their retransmission probabilities to avoid collisions; therefore the system keeps a very good amount of successful departure. Henceforth an optimal value of throughput is achieved and is much better compared to slotted aloha.

5.2.2. Delay Minimization

When maximizing the global throughput, we observed a huge EDBP for all schemes 1–3 and slotted aloha, in particular at heavy load, whereas scheme 4 keeps a constant low delay. This may be very harmful for many applications which are delay-sensitive (real-time applications). Now we shall investigate the problem of minimizing EDBP and the impact of this optimization (Figures 9(a) and 9(b)) on the throughput performance. We note in particular that throughput performance in the four schemes improves considerably with respect to slotted aloha. Scheme 1 is slightly better in terms of throughput only at light load, scheme 2 is almost better at medium load whereas scheme 4 outperforms remarkably all other schemes at high (0.55<𝑞_𝑎) and very high loads. The case of 10 mobiles provides similar trends (Figure 10).

(a)

(b)

Figure 9 

Aggregate throughput and expected delay of backlogged packets for 4 mobiles under the team framework. The objective is to minimize the expected delay of backlogged packets.

(a)

(b)

Figure 10 

Aggregate throughput and expected delay of backlogged packets for 10 mobiles under the team framework. The objective is to minimize the expected delay of backlogged packets.

When EDBP is minimized, for 𝑚=4 and 𝑁=5, retransmission probability decreases with 𝑞_𝑎, so standard aloha and scheme 4 have optimal retransmission probability of around 0.3 at heavy load whereas algorithms 1–3 have much higher retransmission probabilities (Figure 11(a)). For larger mobile population 𝑚=10 (Figure 11(b)), we observe that optimal retransmission rate falls down rapidly for all schemes. Numerically, when traffic is high, slotted aloha mobiles retransmit with probability around 0.13, around 0.1 for scheme 4, and around 0.19 for schemes 1–3 at average and heavy load. In the team problem we note that the optimal retransmission probability 𝑞_𝑟 decreases when increasing arrival probability 𝑞_𝑎 and vice versa. This means that mobiles have to cooperate (adapt their retransmission probabilities according to the load) to reach the best performance either when maximizing global throughput or where minimizing EDBP.

(a)

(b)

Figure 11 

The optimal retransmission probability for schemes 1–4 and slotted aloha as function of arrival rate 𝑞𝑎 for both 4 mobiles and 10 mobiles when throughput is maximized.

5.2.3. Discussion and Remarks

In previews simulations we considered the extreme cases of maximizing independently the throughput and minimizing the EDBP. Radunovic and Le Boudec [22] suggest that considering the total throughput as a performance objective may not be a good objective, and this yields also for delay. In practice it may be more interesting to have a multicriteria optimization in which a convex combination of both the throughput and EDBP is optimized. We consider the following objective 𝛼th𝑝(𝑞)+(1−𝛼)/𝐷^𝑐(𝑞),0≤𝛼≤1. This allows in particular handling QoS constraints: by varying 𝛼 one can find appropriate tradeoff between the throughput and the expected delay, so that the throughput is maximized while keeping the EDBP bounded by some constant. This improves considerably the system reliability and makes the system able to support several kinds of services with different QoS requirements.

At low load (𝑞_𝑎=0.3) and under schemes 2 and 4, see Figures 12(a) and 12(b), the optimal throughput and EDBP are slightly constant and so insensitive to the values of the weight 𝛼 under different loads, because optimal retransmission probabilities under separated objectives (maximizing throughput or minimizing delay) are so close. This means that for schemes 2 and 4, when throughput is maximized, the EDBP is also optimized which corroborates previous result in Figures 9 and 10. At high load, when we give more weight to the throughput (by increasing 𝛼), throughput and EDBP for scheme 1 (without prioritization) and scheme 3 (retransmission with less power) increase slightly. In Figures 13(a) and 13(b) we plot the performance when 𝑁=5,𝑚=4 and 𝛾_th=10dB at high load (𝑞_𝑎=0.9); we note that, for schemes 1 and 3, the throughput improves when increasing 𝛼 whereas it decreases a little for scheme 4. Despite this hybrid optimization, a huge delay is seen for schemes 1 and 3 while schemes 2 and 4 keep constant value because of prioritization and large power randomization given to backlogged packets.

(a)

(b)

Figure 12 

Joint throughput and delay convex optimization. This figures shows the impact of multicriteria factor 𝛼 on aggregate throughput and expected delay of backlogged packets at low load 𝑞𝑎=0.3.

(a)

(b)

Figure 13 

Impact of multicriteria factor 𝛼 on aggregate throughput and expected delay of backlogged packets at heavy load 𝑞𝑎=0.9.

We note, through simulations, that our algorithms perform much better than the standard slotted aloha and remark that schemes behavior is changing with the offered load. Yet, scheme 2 is generally more efficient at average load whereas scheme 4 seems to be the most performing at high and very high loads. These remarks motivate us to propose a load adaptive algorithm described as follows:

𝑞^∗=max_scheme=1⋯4⎛⎜⎜⎜⎜⎝max_𝑞⋅s.t.⎧⎪⎪⎨⎪⎪⎩objective(𝑞)𝝅(𝑞)=𝝅(𝑞)⋅𝑃(𝑞),_m_𝑛=0𝜋𝜋_𝑛(𝑞)=1,_𝑛(𝑞)≥0,𝑛=0,1,…,𝑚.⎞⎟⎟⎟⎟⎠.(32) Our mixed algorithm can be implemented in practice as the SP (Service Provider) is continuously monitoring the state of its radio system and then can estimate the instantaneous load as well as the number of communicating mobiles and the average number of backlogged packets. Here, one can use history statistics or any adapted algorithm such the Pseudo-Bayesian Algorithm described in Problem 4.2 in [3]. Having this information, the SP will be capable to judiciously switch its system to the best scheme and therefore take advantage from collected data. However, the base station should integrate a powerful software coupled with a high-technology measurement devices.

5.3. Game Problem

5.3.1. Throughput Maximization

Figures 14(a) and 14(b) show the variation of Nash equilibrium throughput and expected delay of backlogged packets versus the offered load for 4 mobiles. The throughput of slotted aloha increases with the arrival rate till achieving a maximum throughput of th𝑝^max=0.34 at 𝑞_𝑎≃0.14, and then it decreases till getting annulled for 𝑞_𝑎≥0.32. This throughput collapse causes a huge delay for slotted aloha. Schemes 1 and 3 have similar behavior but keep a constant value of throughput although the arrival rate continues to increase. We have, respectively, th𝑝^1max=0.38 at 𝑞_𝑎≃0.15 and th𝑝^3max=0.53 at 𝑞_𝑎≃0.23.

(a)

(b)

Figure 14 

Nash equilibrium throughput and EDBP for 4 mobiles when the payoff function is the individual throughput.

Under these parameters values, an interesting feature occurs. In contrast to [15] where scheme 3 seems to achieve a performance close to the team problem, the throughput obtained for scheme 2 in the game setting matches closely the throughput of the team setting. Here, scheme 3 turns to perform less better than our schemes 1, 2, and 4. This shows that prioritizing backlogged packets rather than new arrival can be a good policy and constitutes a tradeoff to support services with different QoS requirements.

5.3.2. Behavior under Large Mobile Population

When the population size becomes large, we note that the throughput is only improved at low load for schemes 1–3 compared to slotted aloha, (Figures 15(a) and 17). An interesting result is that scheme 4 outperforms all other schemes and does not suffer from throughput collapse in average and high loads, and the throughput falls exponentially only at very high load. This behavior can be explained by the fact that new arrivals are prioritized and can succeed their transmissions even if backlogged mobiles become very aggressive (retransmission probability close to 1). In term of EDBP, at low load scheme 4 seems to outperform all other schemes. Whereas it performs less better than schemes 1–3 when the load becomes average or high (Figure 15(b) and 17(b)), indeed schemes 1–3 keep a constant value of EDBP (scheme 2 that prioritizes backlogged packets is the best). Plotting the retransmission probability in Figure 16 shows why the throughput vanishes and the expected delay becomes huge. Contrary to the small population size case where mobiles become aggressive only at heavy load, here, users are very aggressive and transmit at probability close to 1 at average and high loads. One can note that schemes 1 and 2 are similar and provide same average throughput; however a slight difference is seen in terms of EDBP where scheme 2 performs a bit better. Scheme 3 seems to perform bad at large population but still slightly better than slotted aloha, in particular by providing a smaller expected delay.

(a)

(b)

Figure 15 

Nash equilibrium throughput and EDBP for 10 mobiles when the payoff function is the individual throughput.

(a)

(b)

Figure 16 

Retransmission rate (Nash equilibrium strategy) when maximizing individual throughput for 4 and 10 mobiles.

(a)

(b)

Figure 17 

Nash equilibrium throughput and EDBP for 40 mobiles when maximizing individual throughput.

5.3.3. Delay Minimization

When the expected delay of backlogged packets is minimized, see Figures 18 and 19, we obtain similar behavior as when maximizing the individual throughput we note that in some cases the throughput obtained when minimizing the delay is better than the corresponding one when throughput is maximized. This situation is similar to the prisoners dilemma; it shows in fact that Nash equilibrium is not efficient in some situations (very known result in literature).

(a)

(b)

Figure 18 

Throughput and delay when minimizing EDBP for 4 mobiles under the game problem.

(a)

(b)

Figure 19 

Retransmission probability when minimizing EDBP for 4 and 10 mobiles under the game problem.

6. Concluding Remarks

We define the throughput gain by 𝐺_𝑖=20log((th𝑝_𝑖)/(th𝑝_𝑠)), 𝑖=1⋯4, where th𝑝_𝑖 is the throughput of scheme 𝑖 and th𝑝_𝑠 is the corresponding throughput of slotted aloha. Figures 20(a) and 20(b) show that priority and power control improve considerably the system performance both in terms of throughput and delay. This gain becomes more important, in particular, at high load where congestion situation may be efficiently avoided or attenuated using our new schemes. The gain function looks like an S-shaped function and can be divided into two regions. The first region corresponds to low load where throughput is slightly improved and the average and high loads where throughput is considerably improved (throughput vanishes for slotted aloha under game problem) as well as expected delay.

(a)

(b)

Figure 20 

The throughput gain when the payoff function is the individual throughput for 4 mobiles, under both optimization (a) and game settings (b).

Our schemes improve the channel utilization and allows the system to support several services requiring different QoS. The team formulation examines the case when coordination between users and control of their retransmission probabilities are possible. This needs some cognition capabilities and therefore requires to upgrade the already existent access points to support power control and coordination mechanisms. This solution consumes valuable bandwidth and requires advanced devices with complex computing capability; this is why we studied the game problem. Latter setting requires no coordination between the central access point and mobile users and represents well the nature of aloha system; each mobile optimizes its own objective function according to the other users strategies and its collected information such as estimated value of the instantaneous backlog of the system. We found that performance indicators obtained at Nash equilibrium are, in general, less than those of the team setting. The experienced SINR is increasing as the number of available power levels is large as well as the distance between two consecutive power levels is larger. We also showed that introducing power differentiation and priority may improve the stability of the protocol.

However unfortunately, for large mobiles population and such slotted aloha, our schemes may also suffer from bistability and experience a throughput break down causing thereby a huge delay as the offered load becomes heavy. This is visible in particular under game setting. This is due to the fact that contention becomes very important and then mobiles are more aggressive. Controlling the retransmission probability 𝑞_𝑟 as in team setting seems to be an efficient way to stabilize the slotted aloha system and reach high performance. The idea behind this is to reduce the collision probability by decreasing the aggressiveness of communicating mobile users. Here, each mobile has to estimate the instantaneous backlog state 𝑛 based on its feedback. there are many learning solutions, for example, see [23, 24], based on the following statement: increase 𝑞_𝑟 when experiencing an idle slot and decrease 𝑞_𝑟 when a collision occurs. One can also reduce the expected delay by limiting the number of retransmissions per packet; note that each packet is then definitively dropped after 𝐾 retransmissions. The problem of throughput collapse in overload condition can also be resolved using a dynamic retransmission control which fine-tunes the retransmission limit according to the instantaneous backlog state. By means of game theory, one can also shrink mobiles aggressiveness by adding a (re)transmission cost to reduce access contention and then improve the stability of the system. Finally, it should be noted that aloha's characteristics are still not much different from those experienced today by CSMA/Wi-Fi and similar contention-based systems that have no carrier sense capability. Indeed, there is a certain amount of inherent inefficiency in this family of systems. For instance, 802.11b sees about a 2–4 Mbit/s effective throughput with a few stations talking versus its theoretical maximum of 11 Mbit/s. These remarks show that our presented result can be extended to other contention systems.

Appendices

A. Transition Probabilities for the Game Problem under Scheme 1

The transition probabilities 𝑃_{(𝑛,𝑎),(𝑛+𝑖,𝑏)} are given by the following expression:

𝑞^𝑟𝑚+1𝑄_𝑎(𝑖,𝑛)∑^𝑛_𝑗=0𝑄_𝑟(𝑗,𝑛)𝐴^𝑗+1,𝑖𝑄𝑗+𝑖+1,𝑎=1,𝑏=0_𝑎𝑄(𝑖,𝑛)𝑞∑_{𝑎𝑛𝑗=0}𝑄_𝑟(𝑗,𝑛)1−𝐴_𝑗,𝑖+1,𝑎=0,𝑏=1_𝑎(𝑖,𝑛)1−𝑞_𝑎∑^𝑛_𝑗=0𝑄_𝑟(𝑗,𝑛)1−𝐴_𝑗,𝑖+𝑞∑_{𝑎𝑛𝑗=0}𝑄_𝑟(𝑗,𝑛)𝐴^𝑗,𝑖+1𝑄𝑗+𝑖+1,𝑎=0,𝑏=0_𝑎(𝑖,𝑛)1−𝑞^𝑟𝑚+1∑^𝑛_𝑗=0𝑄_𝑟𝑞(𝑗,𝑛)1−𝐴_𝑗,𝑖+𝑞^{𝑟𝑛𝑚+1}∑_𝑗=0𝑄_𝑟(𝑗,𝑛)1−𝐴_𝑗+1,𝑖,𝑎=1,𝑏=1⎫⎪⎪⎪⎪⎪⎬⎪⎪⎪⎪⎪⎭𝑖=(𝑚−𝑛)≥2,_𝑟𝑚+1𝑄_𝑎(𝑖,𝑛)∑^𝑛_𝑗=0𝑄_𝑟(𝑗,𝑛)𝐴^𝑗+1,𝑖𝑞𝑗+𝑖+1,𝑎=1,𝑏=0_𝑎𝑄^𝑎(𝑖,𝑛)∑^𝑛_𝑗=0𝑄_𝑟(𝑗,𝑛)1−𝐴_𝑗,𝑖+1+𝑄_𝑎(𝑖+1,𝑛)∑^𝑛_𝑗=0𝑗+𝑖+1𝑗+𝑖+2𝑄^𝑟(𝑗,𝑛)𝐴_𝑗,𝑖+21−𝑞_𝑎,𝑎=0,𝑏=1𝑄_𝑎(𝑖,𝑛)∑^𝑛_𝑗=0𝑄_𝑟(𝑗,𝑛)1−𝐴_𝑗,𝑖+𝑄_𝑎(𝑖+1,𝑛)∑^𝑛_𝑗=0𝑄_𝑟(𝑗,𝑛)𝐴_𝑗,𝑖+1+𝑞_𝑎𝑄_𝑎(𝑖,𝑛)∑^𝑛_𝑗=0𝑄_𝑟(𝑗,𝑛)𝐴^𝑗,𝑖+11−𝑞_𝑟𝑚+1𝑗+𝑖+1,𝑎=0,𝑏=0𝑄_𝑎(𝑖,𝑛)∑^𝑛_𝑗=0𝑄_𝑟(𝑗,𝑛)1−𝐴_𝑗,𝑖+𝑄_𝑎(𝑖+1,𝑛)∑^𝑛_𝑗=0𝑄_𝑟(𝑗,𝑛)𝐴_𝑗,𝑖+1+𝑞_𝑟𝑚+1𝑄^𝑎(𝑖,𝑛)∑^𝑛_𝑗=0𝑄_𝑟(𝑗,𝑛)1−𝐴_𝑗+1,𝑖+𝑄_𝑎(𝑖+1,𝑛)∑^𝑛_𝑗=0𝑗+𝑖+1𝑗+𝑖+2𝑄^𝑟(𝑗,𝑛)𝐴_{𝑗+1,𝑖+1}𝑞,𝑎=1,𝑏=1⎫⎪⎪⎪⎪⎪⎪⎪⎪⎬⎪⎪⎪⎪⎪⎪⎪⎪⎭2≤𝑖<𝑚−𝑛,_𝑟𝑚+1𝑄_𝑎(1,𝑛)∑^𝑛_𝑗=0𝑄_𝑟(𝑗,𝑛)𝐴^𝑗+1,1𝑞𝑗+2,𝑎=1,𝑏=0_𝑎𝑄^𝑎(1,𝑛)∑^𝑛_𝑗=0𝑄_𝑟(𝑗,𝑛)1−𝐴_𝑗,2+𝑄_𝑎(2,𝑛)∑^𝑛_𝑗=0𝑗+2𝑗+3𝑄^𝑟(𝑗,𝑛)𝐴_𝑗,31−𝑞_𝑎,𝑎=0,𝑏=1𝑄_𝑎(1,𝑛)∑^𝑛_𝑗=0𝑄_𝑟(𝑗,𝑛)1−𝐴_𝑗,1+𝑄_𝑎(2,𝑛)∑^𝑛_𝑗=0𝑄_𝑟(𝑗,𝑛)𝐴_𝑗,2+𝑞_𝑎𝑄_𝑎(1,𝑛)∑^𝑛_𝑗=0𝑄_𝑟(𝑗,𝑛)𝐴^𝑗,21−𝑞_𝑟𝑚+1𝑗+2,𝑎=0,𝑏=0𝑄_𝑎(1,𝑛)∑^𝑛_𝑗=1𝑄_𝑟(𝑗,𝑛)1−𝐴_𝑗,1+𝑄_𝑎(2,𝑛)∑ⁿ_𝑗=0𝑄_𝑟(𝑗,𝑛)𝐴_𝑗,2+𝑞_𝑟𝑚+1𝑄^𝑎(1,𝑛)∑^𝑛_𝑗=0𝑄_𝑟(𝑗,𝑛)1−𝐴_𝑗+1,1+𝑄_𝑎(2,𝑛)∑^𝑛_𝑗=0𝑗+2𝑗+3𝑄^𝑟(𝑗,𝑛)𝐴_𝑗+1,2𝑞,𝑎=1,𝑏=1⎫⎪⎪⎪⎪⎪⎪⎪⎪⎬⎪⎪⎪⎪⎪⎪⎪⎪⎭𝑖=1,_𝑟𝑚+1𝑄_𝑎(0,𝑛)∑^𝑛_𝑗=0𝑄_𝑟(𝑗,𝑛)𝐴^𝑗+1,0𝑞𝑗+1,𝑎=1,𝑏=0_𝑎𝑄^𝑎(0,𝑛)∑^𝑛_j=0𝑄_𝑟(𝑗,𝑛)1−𝐴_𝑗,1+𝑄_𝑎(1,𝑛)∑^𝑛_𝑗=0𝑗+1𝑗+2𝑄^𝑟(𝑗,𝑛)𝐴_𝑗,21−𝑞_𝑎,𝑎=0,𝑏=1𝑄_𝑎(0,𝑛)∑^𝑛_{𝑗=0,𝑗≠1}𝑄_𝑟(𝑗,𝑛)1−𝐴_𝑗,0+𝑄_𝑎(1,𝑛)∑^𝑛_𝑗=0𝑄_𝑟(𝑗,𝑛)𝐴_𝑗,1+𝑞_𝑎𝑄_𝑎(0,𝑛)∑^𝑛_𝑗=0𝑄_𝑟(𝑗,𝑛)𝐴^𝑗,11−𝑞_𝑟𝑚+1𝑗+1,𝑎=0,𝑏=0𝑄_𝑎(0,𝑛)∑^𝑛_{𝑗=0,𝑗≠1}𝑄_𝑟(𝑗,𝑛)1−𝐴_𝑗,0+𝑄_𝑎(1,𝑛)∑^𝑛_𝑗=0𝑄_𝑟(𝑗,𝑛)𝐴_𝑗,1+𝑞_𝑟𝑚+1𝑄^𝑎(0,𝑛)∑^𝑛_𝑗=1𝑄_𝑟(𝑗,𝑛)1−𝐴_𝑗+1,0+𝑄_𝑎(1,𝑛)∑^𝑛_𝑗=0𝑗+1𝑗+2𝑄^𝑟(𝑗,𝑛)𝐴_𝑗+1,1𝑞,𝑎=1,𝑏=1⎫⎪⎪⎪⎪⎪⎪⎪⎪⎬⎪⎪⎪⎪⎪⎪⎪⎪⎭𝑖=0,_𝑎𝑄_𝑎(0,𝑛)∑^𝑛_𝑗=1𝑗𝑗+1𝑄^𝑟𝑄(𝑗,𝑛)𝐴_𝑗+1,𝑎=0,𝑏=1_𝑎(0,𝑛)1−𝑞^𝑟𝑚+1∑^𝑛_𝑗=1𝑄_𝑟(𝑗,𝑛)𝐴_𝑗,0+𝑞^{𝑟𝑛𝑚+1}∑_𝑗=1𝑗𝑗+1𝑄^𝑟(𝑗,𝑛)𝐴_𝑗+11−𝑞_𝑎,𝑎=1,𝑏=1𝑄_𝑎(0,𝑛0),∑^𝑛_𝑗=1𝑄_𝑟(𝑗,𝑛)𝐴_𝑗,0,𝑎𝑎==01,,𝑏𝑏==00⎫⎪⎪⎪⎪⎬⎪⎪⎪⎪⎭𝑖=−1.(A.1)

B. Transition Probabilities for the Game Problem under Scheme 2

The transition probabilities 𝑃_{(𝑛,𝑎),(𝑛+𝑖,𝑏)} are given by the following expression:

𝑄𝑞^𝑟𝑚+1𝑄_𝑎(𝑖,𝑛)∑^𝑛_𝑗=0𝑄_𝑟(𝑗,𝑛)𝐵_𝑗+1,𝑖,𝑎=1,𝑏=0_𝑎𝑄(𝑖,𝑛)𝑞∑_{𝑎𝑛𝑗=0}𝑄_𝑟(𝑗,𝑛)1−𝐵_𝑗,𝑖+1,𝑎=0,𝑏=1_𝑎𝑄(𝑖,𝑛)1−𝑞_𝑎∑^𝑛_𝑗=0𝑄_𝑟(𝑗,𝑛)1−𝐵_𝑗,𝑖,𝑎=0,𝑏=0_𝑎𝑞(𝑖,𝑛)1−𝑞^𝑟𝑚+1∑^𝑛_𝑗=0𝑄_𝑟(𝑗,𝑛)1−𝐵_𝑗,𝑖+𝑞^{𝑟𝑛𝑚+1}∑_𝑗=0𝑄_𝑟(𝑗,𝑛)1−𝐵_𝑗+1,𝑖,𝑎=1,𝑏=1⎫⎪⎪⎪⎪⎪⎬⎪⎪⎪⎪⎪⎭𝑖=(𝑚−𝑛)≥2,_𝑟𝑚+1𝑞𝑄_𝑎(𝑖,𝑛)∑^𝑛_𝑗=0𝑄_𝑟(𝑗,𝑛)𝐵_𝑗+1,𝑖𝑎=1,𝑏=0_𝑎𝑄^𝑎(𝑖,𝑛)∑^𝑛_𝑗=0𝑄_𝑟(𝑗,𝑛)1−𝐵_𝑗,𝑖+1+𝑄_𝑎(𝑖+1,𝑛)∑^𝑛_𝑗=1𝑄_𝑟(𝑗,𝑛)𝐵_𝑗,𝑖+21−𝑞_𝑎,𝑎=0,𝑏=1𝑄_𝑎(𝑖,𝑛)∑^𝑛_𝑗=0𝑄_𝑟(𝑗,𝑛)1−𝐵_𝑗,𝑖+𝑄_𝑎(𝑖+1,𝑛)∑^𝑛_𝑗=0𝑄_𝑟(𝑗,𝑛)𝐵_𝑗,𝑖+11−𝑞_𝑟𝑚+1,𝑎=0,𝑏=0𝑄_𝑎(𝑖,𝑛)∑^𝑛_𝑗=0𝑄_𝑟(𝑗,𝑛)1−𝐵_𝑗,𝑖+𝑄_𝑎(𝑖+1,𝑛)∑^𝑛_𝑗=1𝑄_𝑟(𝑗,𝑛)𝐵_𝑗,𝑖+1+𝑞_𝑟𝑚+1𝑞𝑄_𝑎(𝑖,𝑛)∑^𝑛_𝑗=0𝑄_𝑟(𝑗,𝑛)1−𝐵_𝑗+1,𝑖,𝑎=1,𝑏=1⎫⎪⎪⎪⎪⎪⎪⎬⎪⎪⎪⎪⎪⎪⎭2≤𝑖<𝑚−𝑛,_𝑟𝑚+1𝑞𝑄_𝑎(1,𝑛)∑^𝑛_𝑗=0𝑄_𝑟(𝑗,𝑛)𝐵_𝑗+1,1,𝑎=1,𝑏=0_𝑎𝑄^𝑎(1,𝑛)∑^𝑛_𝑗=0𝑄_𝑟(𝑗,𝑛)1−𝐵_𝑗,2+𝑄_𝑎(2,𝑛)∑^𝑛_𝑗=0𝑄_𝑟(𝑗,𝑛)𝐵_𝑗,31−𝑞_𝑎,𝑎=0,𝑏=1𝑄_𝑎(1,𝑛)∑^𝑛_𝑗=1𝑄_𝑟(𝑗,𝑛)1−𝐵_𝑗,1+𝑄_𝑎(2,𝑛)∑^𝑛_j=1𝑄_𝑟(𝑗,𝑛)𝐵_𝑗,21−𝑞_𝑟𝑚+1,𝑎=0,𝑏=0𝑄_𝑎(1,𝑛)∑^𝑛_𝑗=1𝑄_𝑟(𝑗,𝑛)1−𝐵_𝑗,1+𝑄_𝑎(2,𝑛)∑^𝑛_𝑗=1𝑄_𝑟(𝑗,𝑛)𝐵_𝑗,2+𝑞_𝑟𝑚+1𝑞𝑄_𝑎(1,𝑛)∑^𝑛_𝑗=0𝑄_𝑟(𝑗,𝑛)1−𝐵_𝑗+1,1,𝑎=1,𝑏=1⎫⎪⎪⎪⎪⎪⎪⎬⎪⎪⎪⎪⎪⎪⎭𝑖=1,_𝑟𝑚+1𝑞𝑄_𝑎(0,𝑛)∑^𝑛_𝑗=0𝑄_𝑟(𝑗,𝑛)𝐵_𝑗+1,0,𝑎=1,𝑏=0_𝑎𝑄^𝑎(0,𝑛)∑^𝑛_𝑗=1𝑄_𝑟(𝑗,𝑛)1−𝐵_𝑗,1+𝑄_𝑎(1,𝑛)∑^𝑛_𝑗=1𝑄_𝑟(𝑗,𝑛)𝐵_𝑗,21−𝑞_𝑎,𝑎=0,𝑏=1𝑄_𝑎(0,𝑛)∑^𝑛_{𝑗=0,𝑗≠1}𝑄_𝑟(𝑗,𝑛)1−𝐵_𝑗,0+𝑄_𝑎(1,𝑛)∑^𝑛_𝑗=0𝑄_𝑟(𝑗,𝑛)𝐵_𝑗,1+𝑞_𝑎1−𝑞𝑄_{𝑎𝑟𝑚+1}(0,𝑛)𝐵_0,1,𝑎=0,𝑏=0𝑄_𝑎(0,𝑛)∑^𝑛_{𝑗=0,𝑗≠1}𝑄_𝑟(𝑗,𝑛)1−𝐵_𝑗,0+𝑄_𝑎(1,𝑛)∑^𝑛_𝑗=0𝑄_𝑟(𝑗,𝑛)𝐵_𝑗,1+𝑞_𝑟𝑚+1𝑞𝑄_𝑎(0,𝑛)∑^𝑛_𝑗=1𝑄_𝑟(𝑗,𝑛)1−𝐵_𝑗+1,0,𝑎=1,𝑏=1⎫⎪⎪⎪⎪⎪⎪⎪⎬⎪⎪⎪⎪⎪⎪⎪⎭𝑖=0,_𝑎𝑄𝑄_𝑎(0,𝑛)∑^𝑛_𝑗=1𝑄_𝑟(𝑗,𝑛)𝐵_𝑗,1,𝑎=0,𝑏=1_𝑎1−(0𝑞,_𝑎𝑛)1−𝑞^𝑟𝑚+1∑^𝑛_𝑗=1𝑄_𝑟(𝑗,𝑛)𝐵_𝑗,0,𝑎=1,𝑏=10,𝑄_𝑎(0,𝑛)∑^𝑛_𝑗=1𝑄_𝑟(𝑗,𝑛)𝐵_𝑗,0,𝑎𝑎==01,,𝑏𝑏==00⎫⎪⎪⎪⎪⎬⎪⎪⎪⎪⎭𝑖=−1.(B.1)

C. Transition Probabilities for the Game Problem under Scheme  3

𝑄𝑄_𝑎(𝑖,𝑛)1−𝑞_𝑎,𝑎=0,𝑏=0_𝑎𝑄(𝑖,𝑛),𝑎=1,𝑏=1_𝑎𝑄(𝑖,𝑛)𝑞_𝑎,𝑎=0,𝑏=1⎫⎪⎬⎪⎭2≤𝑖,_𝑎𝑄(1,𝑛)1−𝑞_𝑎∑^𝑛_𝑗=1𝑄_𝑟(𝑗,𝑛)1−𝐶_𝑗,1,𝑎=0,𝑏=0_𝑎(1,𝑛)+𝑞1−_{𝑟𝑛𝑚+1}𝑞^𝑟𝑚+1∑^𝑛_𝑗=1𝑄_𝑟(𝑗,𝑛)1−𝐶_𝑗,1𝑄∑_𝑗=0𝑄_𝑟(𝑗,𝑛)1−𝐶_𝑗+1,1,𝑎=1,𝑏=1_𝑎𝑄(𝑖,𝑛)q_𝑎,𝑎=0,𝑏=1⎫⎪⎪⎪⎪⎬⎪⎪⎪⎪⎭𝑖=1,_𝑎(0,𝑛)1−𝑞_𝑎∑^𝑛_{𝑗=0,𝑗≠1}𝑄_𝑟(𝑗,𝑛)1−𝐶_𝑗,0+𝑞∑_{𝑎𝑛𝑗=0}𝑄_𝑟(𝑗,𝑛)𝐶_𝑗,1+𝑄_𝑎𝑄(1,𝑛)1−𝑞_𝑎∑^𝑛_𝑗=0𝑄_𝑟(𝑗,𝑛)𝐶_𝑗,1,𝑎=0,𝑏=0_𝑎(0,𝑛)1−𝑞^𝑟𝑚+1∑^𝑛_{𝑗=0,𝑗≠1}𝑄_𝑟(𝑗,𝑛)1−𝐶_𝑗,0+𝑞^{𝑟𝑛𝑚+1}∑_𝑗=1𝑄_𝑟(𝑗,𝑛)𝐶_𝑗+1,0+𝑄_𝑎(1,𝑛)1−𝑞^𝑟𝑚+1∑^𝑛_{𝑗=0,𝑗≠1}𝑄_𝑟(𝑗,𝑛)𝐶_𝑗,1+𝑞^{𝑟𝑛𝑚+1}∑_𝑗=0𝑄_𝑟(𝑗,𝑛)𝐶_𝑗+1,1𝑄,𝑎=1,𝑏=1_𝑎𝑄(0,𝑛)𝑞∑_{𝑎𝑛𝑗=1}𝑄_𝑟(𝑗,𝑛)1−𝐶_𝑗,1,𝑎=0,𝑏=1_𝑎(0,𝑛)𝑞^{𝑟𝑛𝑚+1}∑_𝑗=01𝑗+1𝑄^𝑟𝑄(𝑗,𝑛),𝑎=1,𝑏=0⎫⎪⎪⎪⎪⎪⎪⎪⎪⎬⎪⎪⎪⎪⎪⎪⎪⎪⎭𝑖=0,_𝑎𝑄(0,𝑛)1−𝑞_𝑎∑^𝑛_𝑗=1𝑄_𝑟(𝑗,𝑛)𝐶_𝑗,0,𝑎=0,𝑏=0_𝑎(0,𝑛)1−𝑞^𝑟𝑚+1∑^𝑛_𝑗=1𝑄_𝑟(𝑗,𝑛)𝐶_𝑗,0+𝑞^{𝑟𝑛𝑚+1}∑_𝑗=1𝑗𝑗+1𝑄^𝑟(𝑗,𝑛)𝐶_𝑗+1,00,otherwise.,𝑎=1,𝑏=1⎫⎪⎪⎬⎪⎪⎭𝑖=−1,(C.1)

D. Transition Probabilities for the Game Problem under Scheme  4

The transition probabilities 𝑃_{(𝑛,𝑎),(𝑛+𝑖,𝑏)} are given by the following expression:

𝑄𝑄_𝑎(𝑖,𝑛)𝑞∑_{𝑎𝑛𝑗=0}𝑄_𝑟(𝑗,𝑛)1−𝐶_𝑗,𝑖+1,𝑎=0,𝑏=1_𝑎(𝑖,𝑛)1−𝑞_𝑎∑^𝑛_𝑗=0𝑄_𝑟(𝑗,𝑛)1−𝐶_𝑗,𝑖+𝑞∑_{𝑎𝑛𝑗=0}𝑄_𝑟(𝑗,𝑛)𝐶^𝑗,𝑖+1𝑄𝑖+1,𝑎=0,𝑏=0_𝑎𝑞(𝑖,𝑛)1−𝑞^𝑟𝑚+1∑^𝑛_𝑗=0𝑄_𝑟(𝑗,𝑛)1−𝐶_𝑗,𝑖+𝑞^{𝑟𝑛𝑚+1}∑_𝑗=0𝑄_𝑟(𝑗,𝑛)1−𝐶_𝑗+1,𝑖,𝑎=1,𝑏=1⎫⎪⎪⎪⎬⎪⎪⎪⎭𝑖=(𝑚−𝑛)≥2,_𝑎𝑄^𝑎(𝑖,n)∑^𝑛_𝑗=0𝑄_𝑟(𝑗,𝑛)1−𝐶_𝑗,𝑖+1+𝑄_𝑎(𝑖+1,𝑛)∑^𝑛_𝑗=0𝑖+1𝑖+2𝑄^𝑟(𝑗,𝑛)𝐶_𝑗,𝑖+2𝑄,𝑎=0,𝑏=1_𝑎(𝑖,𝑛)1−𝑞_𝑎∑^𝑛_𝑗=0𝑄_𝑟(𝑗,𝑛)1−𝐶_𝑗,𝑖+𝑞∑_{𝑎𝑛𝑗=0}𝑄_𝑟(𝑗,𝑛)𝐶^𝑗,𝑖+1+1−𝑞_𝑎𝑖+1𝑄𝑄_𝑎(𝑖+1,𝑛)∑^𝑛_𝑗=0Q_𝑟(𝑗,𝑛)𝐶_𝑗,𝑖+1,𝑎=0,𝑏=0_𝑎+(𝑄𝑖,_𝑎𝑛)1−𝑞^𝑟𝑚+1∑^𝑛_𝑗=0𝑄_𝑟(𝑗,𝑛)1−𝐶_𝑗,𝑖+𝑞^{𝑟𝑛𝑚+1}∑_𝑗=0𝑄_𝑟(𝑗,𝑛)1−𝐶_𝑗+1,𝑖(𝑖+1,𝑛)1−𝑞^𝑟𝑚+1∑^𝑛_𝑗=0𝑄_𝑟(𝑗,𝑛)𝐶_𝑗,𝑖+1+𝑞^{𝑟𝑛𝑚+1}∑_𝑗=0𝑄_𝑟(𝑗,𝑛)𝐶_{𝑗+1,𝑖+1}𝑞,𝑎=1,𝑏=1⎫⎪⎪⎪⎪⎪⎪⎪⎬⎪⎪⎪⎪⎪⎪⎪⎭2≤𝑖<𝑚−𝑛,_𝑎𝑄^𝑎(1,𝑛)∑^𝑛_𝑗=0𝑄_𝑟(𝑗,𝑛)1−𝐶_𝑗,2+𝑄_𝑎(2,𝑛)∑^𝑛_𝑗=023𝑄^𝑟(𝑗,𝑛)𝐶_𝑗,3𝑄,𝑎=0,𝑏=1_𝑎(1,𝑛)1−𝑞_𝑎∑^𝑛_𝑗=1𝑄_𝑟(𝑗,𝑛)1−𝐶_𝑗,1+𝑞∑_{𝑎𝑛𝑗=0}𝑄_𝑟(𝑗,𝑛)𝐶^𝑗,2+1−𝑞_𝑎2𝑄𝑄_𝑎(2,𝑛)∑^𝑛_𝑗=0𝑄_𝑟(𝑗,𝑛)𝐶_𝑗,2,𝑎=0,𝑏=0_𝑎+(𝑄1,_𝑎𝑛)1−𝑞^𝑟𝑚+1∑^𝑛_𝑗=1𝑄_𝑟(𝑗,𝑛)1−𝐶_𝑗,1+𝑞^{𝑟𝑛𝑚+1}∑_𝑗=0𝑄_𝑟(𝑗,𝑛)1−𝐶_𝑗+1,1(2,𝑛)1−𝑞^𝑟𝑚+1∑^𝑛_𝑗=0𝑄_𝑟(𝑗,𝑛)𝐶_𝑗,2+𝑞^{𝑟𝑛𝑚+1}∑_𝑗=0𝑄_𝑟(𝑗,𝑛)𝐶_𝑗+1,2𝑞,𝑎=1,𝑏=1⎫⎪⎪⎪⎪⎪⎪⎪⎬⎪⎪⎪⎪⎪⎪⎪⎭𝑖=1,_𝑎𝑄^𝑎(0,𝑛)∑^𝑛_𝑗=1𝑄_𝑟(𝑗,𝑛)1−𝐶_𝑗,1+𝑄_𝑎(1,𝑛)∑^𝑛_𝑗=012𝑄^𝑟(𝑗,𝑛)𝐶_𝑗,2𝑄,𝑎=0,𝑏=1_𝑎𝑄(0,𝑛)𝑞^𝑟𝑚+1𝑄_𝑟(0,𝑛),𝑎=1,𝑏=0_𝑎(0,𝑛)1−𝑞_𝑎∑^𝑛_{𝑗=0,𝑗≠1}𝑄_𝑟(𝑗,𝑛)1−𝐶_𝑗,0+𝑞∑_{𝑎𝑛𝑗=0}𝑄_𝑟(𝑗,𝑛)𝐶_𝑗,1+𝑄_𝑎𝑄(1,𝑛)1−𝑞_𝑎∑^𝑛_𝑗=0𝑄_𝑟(𝑗,𝑛)𝐶_𝑗,1,𝑎=0,𝑏=0_𝑎+(𝑄0,_𝑎𝑛)1−𝑞^𝑟𝑚+1∑^𝑛_{𝑗=0,𝑗≠1}𝑄_𝑟(𝑗,𝑛)1−𝐶_𝑗,0+𝑞^{𝑟𝑛𝑚+1}∑_𝑗=1𝑄_𝑟(𝑗,𝑛)1−𝐶_𝑗+1,0(1,𝑛)1−𝑞^𝑟𝑚+1∑^𝑛_𝑗=0𝑄_𝑟(𝑗,𝑛)𝐶_𝑗,1+𝑞^{𝑟𝑛𝑚+1}∑_𝑗=0𝑄_𝑟(𝑗,𝑛)𝐶_𝑗+1,11−𝑞_𝑎,𝑎=1,𝑏=1⎫⎪⎪⎪⎪⎪⎪⎪⎬⎪⎪⎪⎪⎪⎪⎪⎭𝑖=0,1−𝑞_𝑟𝑚+1𝑄_𝑎(0,𝑛)𝑄_𝑟(1,𝑛),𝑎=0,𝑏=00,otherwis𝑄e._𝑎(0,𝑛)𝑄_𝑟(1,𝑛),𝑎=1,𝑏=1𝑖=−1,(D.1)