#### Abstract

We use antagonistic stochastic games and fluctuation analysis to examine a single-server queue with bulk input and secondary work during server's multiple vacations. When the buffer contents become exhausted the server leaves the system to perform some diagnostic service of a minimum of jobs clustered in packets of random sizes (event A). The server is not supposed to stay longer than units of time (event B). The server returns to the system when A or B occurs, whichever comes first. On the other hand, he may not break service of a packet in a middle even if A or B occurs. Furthermore, the server waits for batches of customers to arrive if upon his return the queue is still empty. We obtain a compact and explicit form functional for the queueing process in equilibrium.

#### 1. Introduction

In this paper we use a game-theoretic analysis for a single-server queue with maintenance. When the queue becomes empty, the server leaves the system. It is usually associated with vacations. However, in our case, the server is vacant from the system not just to perform some unspecified maintenance (as it is the case in most systems with server vacations), but to render a random quantity of jobs. This maintenance can be associated with a semiroutine diagnostic, as computer servers render from time to time. When the server has a break, it is a right time to perform a background work.

In our case, the server will be facing random amounts of various diagnostic procedures and he will have to do some minimum quantity of them. In addition, his vacation from the main system should also be limited by some reasonable time limit. Consequently, the server is scheduled to be vacant until one of the two events will take place first: the total amount of jobs performed is or the time he spends in the maintenance facility is . The server's absence policy is however not rigid, due to one more factor. Jobs are included in packets of random quantities (as it is the case in computer networks and telecommunications), so that by no means can the server break a packet with unfinished jobs and return to the system even if the total amount of jobs crosses or his time spent with maintenance expires beyond . The server alone cannot break a single job in a middle if his time in the maintenance facility is expired. The server will ultimately return to the primary system after he is done with a packet. This policy is more reasonable than a rigid one, but the system modeling is obviously more complex.

Consequently, the return time, the amount of done jobs, and most importantly, the number of customers so far accumulated in the buffer become a nontrivial problem that is investigated by means of fluctuation analysis. More specifically, it seems plausible to introduce a multivariate random walk process, with two antagonistic components (the number of jobs and the cumulative time), say players A and B. In light of stochastic game theory, the game will be over when a player wins. However, the information on player's victory is not available in the real time, but at one of the epochs of time designated for (random) observations. It means that the end of the game does not occur at the moment when the defeat takes place, but with a random delay. Thus, the information collected about the game can be crude or fine dependent on the intensity of the observation process. Now, due to delays, and because the game will actually continue even after one of the players is defeated, a delayed information can cause the winner to endure damages exceeding his threshold. For example, if server's time in the maintenance facility is expired, but he is not done with a packet (an analog of a delayed observation), he remains in the facility until he completes the packet. Meanwhile, the total quantity of jobs he processed by then also exceeds . Then, he returns to the system with both, the amount of jobs done greater than and the time spent at the facility greater than . (Here the game is pronounced to be over, at the opportune observation epoch, with both players winning.)

We need to notice that there is one more player C involved in the game, though passively. This player receives strikes from both players A and B, but never responds and his defeat is imminent. It occurs either during the game between players A and B, or thereafter when either one of the players or their coalition attacks C until C is defeated. In our case, player C is represented by the flow of customers arriving to the main system during server's work at the maintenance facility and possibly during server's waiting time in the system.

In a nutshell, the game aspect of this analysis lies in two active players exerting random damages to each other at random times. The first phase of the game continues until the casualties to one of the players exceed a certain threshold ( for player A and for player B) upon one of the legitimate “observation” times (successive completion of servicing packets). When this takes place, the first phase of the game is over. There is a third player (C) who suffers casualties (from both players) but does not actively participate in neither phase of the game. At the beginning of phase two, the total amount of casualties to player C is observed and the entire game is over if there is at least one. Otherwise, the game enters its second phase and continues until the player C suffers one or more hits. (Each strike carries out multiples hits.) Player C is represented by the input flow of customers to the system and the “total casualties” to player C is the amount of customers accumulated in the system during the two phases of the game.

In the past analyses of antagonistic games [1–13], the focus was on warfare [2–6, 8, 12], economics [1, 5, 7, 9, 11], and ecology [10]. For the first time (to the best of our knowledge), stochastic games are directly applied to queueing in this form.

*Description of the System*

The input to the system is a marked Poisson process with position independent marking and the FIFO discipline of lined up units. The server takes only one unit at a time and have it departed once being processed. (More on this below.)

When the queue becomes exhausted, the server leaves the system (for special vacations) beginning with his diagnostic checkup. Suppose that the server first observes a batch of jobs that he needs to complete. Each job needs a random amount of time beginning with . So, the jobs the server work on sequentially willbe done in units of time.

After this is finished, the server turns to another batch of jobs, and so on. We assume that , a.s. The total amount of jobs to be rendered is limited but some minimum of them, say , must be rendered before the server returns to the system. In addition, the total amount of time needed to complete these jobs is also limited by a fixed positive number . We need to clarify this constraints. In general the total quantity of all jobs is limited by . However, the server must not break a batch to comply with this rule. In other words, if for some , he will be serving batches with

while

Analogously, the time the server spends on diagnostic is limited by , but never at expense of breaking a job or even a packet (i.e., a batch), so that

but

Therefore the server returns to the system when one of the two events take place (whichever comes first): the cumulative amount of jobs exceeds or the total time spent with the jobs exceeds after completion of a last batch of jobs (any of these taking place for the first time).

If is as in (1.3), we obviously have

as the time spent for jobs.

We need to find how many jobs were actually done upon server's return to the system, as well as the return time, and most importantly, how many customers were so far accumulated in the buffer. We first define the random indices

With so defined, we see that will be the *return time* (also referred to as the *first passage time* or the *exit time*), while and are the total quantity of jobs done during the maintenance time and the total number of customers fill in the buffer, respectively, upon server's return to the system. Notice that if the server, upon his return, finds no customers in the system, he will rest and wait for a first batch to come. We will discuss this new phase in due course.

Consequently, we will be interested in the joint functional

in which both and are “active components,” while is a “passive component”. The reason for these names is that and both actively involved in server's return and their values are related to and , respectively, while is not and its value is only assumed on server's return.

is the amount of jobs rendered by the server upon his return to the system, is the number of customers by the end of the maintenance period, and is the first passage time, that is, the time when the server returns to the system and thus exits the maintenance mode.

*The Game*

In one of the setting for stochastic antagonistic games [4, 14–17], there is a bivariate marked point process
representing incremental casualties and to players A and B observed at time The players actions in reality take place at different times, but they are observed jointly upon times . The first passage time takes place at one of the observed epochs from when the casualties to one of the players become “unbearable.”

In our case, however, the two players will be represented by incremental marks and instead, while marks will be continually observed and associated with the casualties to the third player C whose status will be raised upon moving on to phase two of the game. (One of the authors of this paper has been involved in multiphase games [15, 18–20] directly related to this model.)

Without loss of generality, we will be using the same notation for the random measure , however meaning 's and 's as active components representing players A and B, while is meant to be a passive component representing player C.

In the second phase the game turns to

with and are the times when the winner of phase one exerts hits to player C in increments of until the total casualties to C at cross level . The total game is over.

*The Queue on Phase II*

When the server returns to the system at , he finds customers in the buffer, and unless , the server does not resume his service and waits for more units to arrive at times Once, , a new busy period begins. In this paper, we assume for simplicity, and leave a more general case of for an upcoming work (in its association with the -policy). Nevertheless, the use of fluctuation analysis on the second phase with is almost mandatory, because we are not able to discern analytically whether or not .

*The Layout of the Paper*

The introductory section is followed by the formalism of our model and the use of fluctuation analysis. The named functional is obtained in a closed form with the focus on the marginal functional (in Section 3) giving the number of units accumulated in the buffer upon exit from the secondary work (phase I). The tractability of is then demonstrated on a special case of the marked point process in phase I (with no restriction on the queueing system), allowing us in Section 4 to arrive at fully explicit formulas. Section 5 deals with phase II of the game (waiting time after server's return) and the output functional where is the number of customers in the system at time (the exit from phase II and the beginning of a busy period). Section 6 offers a Kendall-like compact formula for the queue in the steady state in the form of the probability generating function. The paper concludes with a summary of the results obtained and the discussion of the upcoming work on continuous time parameter queueing process and an extended global control under the revival of rendered jobs during the maintenance period.

*A Relationship to the Existing Literature*

The present model generalizes the very old classical queue [21, page 301], but it goes beyond it. It also modifies - and *T*-policy (cf. Tian and Zhang [22, pages 27-28 and 32], resp.), multiple vacations systems (cf. Jain et al. [23, page 357]) all in one, let alone the inclusion of the secondary work, and not just vacations.

We use the method of fluctuation analysis [4–6, 14–19, 24, 25], some of which previously applied to antagonistic games [4, 5, 15, 18, 19] and described the system (at least in part) as a game. To the best of our knowledge such an approach is novel and only fragments of this model are related to the past work. A lion portion of this paper is related to sequential games [15, 18–20]. The latter includes two or more phases of a game, mostly a conflict between two players. For example article [15] introduced a game of two players A and B, in which phase I involved a series of one-sided attacks on player A (such as economic sanctions, small ambushes, or threats). They continued until damage to player A became serious enough to respond. The latter turned the game to phase II with mutual attacks, in which attacks to player A became more vigorous than during phase I. In another paper modeling a sequential game [19], phase I runs a mutual conflict with weak attacks on each other that were “tolerated” until cumulative damages to one of the players were crossing some thresholds. Once this took place, the second phase began with more aggressive and dangerous attacks leading to the destruction of one of the players. The paper dealt with the exit time from the game as well as the cumulative damages to either player at the exit. In various variants of such or a similar game, the authors treated discrete- or continues-valued components (cf. [4]). In some cases, like in the present variant of the conflict, the components are mixed. The variety of the nature of the component suggests different types of inverse operators involved to solve the problem. Article [25] by the second author offers an introduction to such different cases. Other work on sequential games can be found in [26–30].

The notion of *T*-policy as stated in Tian and Zhang [22] and earlier in Heyman [31] is to call the server back as soon as his vacation time hits . A more plausible scenario of *T*-policy is to call the server back after his vacation time hits *T*, but not at expense of breaking his vacation in the middle, especially if it is a multiple vacation case. Furthermore, if the server (as it is in our case) performs explicit and not virtual jobs, then it makes sense to restrict the exit from his vacations by not only having him to end a job, but also a packet of jobs. So, formally we do not generalize [31], but modify it to a seemingly more realistic scenario. As far as *N*-policy alone (i.e., if ), we also modify it due to the server's obligation not to break a random batch of jobs until it is finished. A classic and most often cited to paper on *N*-policy and multiple vacations is by Lee et al. [32], which is somewhat related to our present model.

While a majority of work on *N*-policy with (or without) multiple vacations use the “supplementary variable” technique by finding the probability generating function (pgf) of the continuous time parameter queueing process (cf. Lee et al. [32]), we work on the embedded process upon departures. However, the results we obtain here will be used later on to have the pgf of the continuous time parameter queue using techniques of time-sensitive functionals, which offers benefits of no mandatory assumption on the service time distribution to be absolutely continuous (used in the supplementary variables method).

As far as the techniques utilized in this paper, we relate to fluctuations of stochastic processes [24, 33–35] widely used in queueing [36, 37], economics [14, 17, 38–40], physics [41–44], astronomy [42], and biology [42]. Pertinent cases of fluctuation phenomena in single- and multivariate marked point processes are treated by the second author and his collaborators in [14, 16, 17, 24, 25]. The reader is also advised to see more general work in [33–35].

Finally our present paper is also related to Abolnikov et al. [36] about “hybrid queues” which comes closest of all to our present work. The authors there considered a queue also with maintenance jobs similar to ours, but without a time restriction, and most importantly, without explicit treatment of the process during the maintenance period.

#### 2. Formalism of the Model and the Main Past Results

To formulate a result previously obtained in Dshalalow and Liew [16] we utilize a generic bivariate marked point process. Let

(is the Dirac mass) be a delayed marked renewal process on a probability space . We assume that is with position dependent marking such that

Given some fixed thresholds and , we define

the random index such that the cumulative marks and will exceed or , respectively. Notice that is the only one of the three random components that assumes a value at , making a passive component. Since and are directly related to the control levels and , they are referred to as *active components* of . The following functional:
will describe the status of the process uponits exit from the rectangle at .

We will make use of the Laplace-Carson transform

whose inverse obviously is where denotes the inverse of the Laplace transform.

Next, we introduce another (discrete) transformation

where is an integrable function defined on set . The inverse operator below can restore , if we apply it for all :

where the inverse is

if applied to a function analytic at zero in the first variable.

Theorem 2.1 by Dshalalow and Liew [16] given below establishes an explicit formula for .

Theorem 2.1. *The functional of (2.4) satisfies the following formula:
*

In the sequel, we will drop the delay in by setting

thereby making

The latter will reduce (2.10) to

In particular, for we obtain the marginal functional

We will now focus on . With the conventional formula for the double expectation we have

Now, we can regard as the sum of customers that enter the system during the times , that is,

with

Since are conditionally (given -algebra ) independent and jointly identically distributed random vectors, we have

where

is the common Laplace-Stieltjes transform (LST) of the r.v.'s Using (2.18)-(2.19) we have

where

is the probability generating function (pgf) of r.v. .

Finally, substituting (2.20) in (2.14), we get

giving the joint marginal transform of r.v.'s (the number of units upon server's return to the system) and (the time of server's return). We note that the closest functional to (2.22) was the one considered in Abolnikov et al. [36], but without , as no restriction in was there imposed. Consequently, the vacation model there was simpler from the fluctuations stand. In the rest of the comprehensive literature on vacations (cf. the monograph [22]), there is none to deal with secondary jobs and fluctuation analysis that will bear fruits as we will see it from the upcoming sections.

#### 3. Adopted to the Queueing System

In this section we return to the queueing system and adopt the generic bivariate marked point process of the previous section, now interpreting the component as the maintenance routine taking place during server's vacationing from the main system as observed over . The component describes the accumulation process of the incoming units in the system during the server's absence. Considering that the input to the system is marked Poisson (also referred to as *compound Poisson*), with position independent marking and intensity ,

where the marks 's are iid r.v.'s with the common pgf and the mean

Assuming that the times needed to process successive jobs during the maintenance period are iid r.v.'s with the common LST

we easily conclude that the joint functional of the number of customers entering the main facility during processing any job (left behind in (2.19)) will then be

equation (3.4) is due to the usual double expectation routine occurring in queueing.

Consequently, with (3.4), formula (2.22) for the joint transform of the customers accumulated during the entire maintenance process and the server's return time will be

Specifically, for , degenerates to

#### 4. A Special Case of the Maintenance Process

First, we will avoid a relative simplicity with , as in this case the operator will degenerate. So, we assume that . Next, make two assumptions on the maintenance process and go with the special case.

(i)There is the LST of the processing time of a job (ii)The distribution of the batch sizes is geometric with parameter and pgfGiven the above assumptions, we calculate of (2.20) as needed for (3.5). After straightforward algebra we have

From (3.5), using (4.4) for and (4.17)–(4.21), after simple algebra, we finally have

In particular,

where

From (3.6) and (4.3), for we get

In particular,

In a nutshell we have the following.

Theorem 4.1. *Given assumptions (4.1) and (4.2), the number of customers that enter the system during the server's maintenance work (i.e., between his departure from the system and his return at ) along with the time and the number of jobs rendered by in the form satisfies formulas (4.18)–(4.22) and (4.26). The corresponding marginal functional satisfies formulas (4.19)-(4.20), (4.23)–(4.25), (4.27).*

#### 5. Phase 2: The Waiting Period

Now, when the server returns to the system, he finds customers in the buffer, which does not have to be positive. In case if the server does not leave the system as he did in the conclusion of a busy period, and so he waits for any batch of customers. Consequently, his service finally resumes when a group of units enters the system. However, it sounds like most likely customers will wait for the server rather than other way around. The problem is not trivial as it is for a regular queue, with the server waiting for the first batch of customers unconditionally. We turn again to fluctuation analysis considering a generic delayed marked renewal process

with generally position dependent marking. The process runs its course until its cumulative mark

crosses some integer at . In other words, with

at the exit time . We describe the process by

We are interested in the joint functional

The latter is known [16] to obey the formula

with

when bearing in mind a period when the server returns to the system, with the history of

Note that the server would then wait until customers or more will accumulate upon one of the arrivals. (Recall that the input is bulk.) In the event, , will contain factor (with the rest being analytic in ) and thus make the second term of (5.6) vanishes, as

In this case, would equal . Thus, formula (5.6) will take care of the case when by itself.

Obviously,

in our case of the marked Poisson process with position independent marking. Consequently, we have the marginal functional

Furthermore, we assume that the threshold (although in some upcoming versions we would like to retain the generality of ) thereby further reducing (5.11) to

where

To find explicitly we turn to formulas (4.18)–(4.21):

The above can be summarized as Theorem 5.1. For convenience, we bring all formulas together in one place.

Theorem 5.1. *The number of units accumulated in the buffer upon the beginning of a busy period , jointly with for , is given by the transform
**
where
**
The marginal functional can be obtained from (5.15)-(5.16) by letting **
where, in particular,
**
For *

For the sequel, we will need and for thus, we need . After straightforward computations, we arrive at the following.

Proposition 5.2. *The mean number of units at the beginning of a busy period satisfies the following formula:
**
where
**
with satisfying (5.18) and being defined in (3.2).*

#### 6. The System on a Busy Period and in Equilibrium

In Section 5 (Theorem 5.1) we calculated the pgf of the quantity of the units accumulated in the buffer at the beginning of a busy period. The time during which the system gained in customers consisted of two phases. The first phase was when the server performed a maintenance completing amount of jobs. Phase 2 involved a waiting period in the event the buffer was empty.

The rest of the analysis (as regards the embedded process) is almost identical to that of the usual bulk input queue. We therefore restrict ourselves to the main formalism and present the final formulas.

(i)The input to the system is bulk Poisson with position independent marking and , , as per (3.1)-(3.2), and intensity of its point process (ii)Service is general and independent including independence of the input that is, service times are iid r.v. with a common LST (iii) is the right continuou squeueing process (the number of customers at time ). (iv) are successive departures of individual units.(v) is the embedded Markov chain upon departures.(vi)The transition probability matrix of is a delta-2 matrix (cf. [36]) similar to that of queue, with only zero row different, with no impact on ergodicity condition (necessary and sufficient). The zero row contains the transition probabilities over the two-phase maintenance with waiting and service period (zero-service cycle), so that the queue length upon the end of such cycle is and its pgf, after a straightforward calculation, is . So, the pgf of the zero-row in the system is where (of Theorem 5.1) will replace in the system [21].(vii)Consequently, if Kendall's formula (most often referred to as*Pollaczek-Khinchin formula*) in the system is (cf. Medhi [21]) all we need to do is to replace with to get Also, in Kendall's formula, we have So, we also replace with of Proposition 5.2. Here is the pgf of the embedded queueing process in equilibrium.

We can summarize the above as the follows.

Theorem 6.1. *The embedded queueing process in the type queue with a two-phase maintenance is ergodic given . Under this condition the pgf of the invariant probability measure satisfies the Kendall-like formula
**
with and of Proposition 5.2 and Theorem 5.1, respectively. *

*Summary*In conclusion, we studied an exhaustive type queue with a single server performing secondary maintenance jobs during his vacationing periods. Unlike most of queues with vacations, in which the vacationing time is “anonymous,” our server takes on real jobs. In addition, we introduce a so-called “modified *T*-Policy” that nonrigidly restricts the server to a time to be vacant from the system. However, if the server is in a middle of a batch of jobs, he must finish them first before coming back. Thus, the return will take place at the first opportune time. Furthermore, the server must also complete a minimum of jobs before returning to the system. Therefore, his discharge from maintenance will take place at some epoch of time (first passage time) when the server first completes a minimum of jobs (without interrupting an individual batch) or being vacant during units of time (with the same restriction), whichever of these two occurs first. To find the number of units present in the system (if any) upon server's return, we use game-theoretic modeling (of two players A and B whoever wins first) and fluctuation analysis for multivariate random walk processes.

Now, coming back, if no unit is waiting in the buffer, then the server waits himself for new units to come. We use again fluctuation analysis to find the total quantity of units upon server's return to work and complete the paper with Kendall-like formula for the stationary distribution of units present in the system upon departure epochs.

In the upcoming work with this model, we plan to use time-sensitive functionals to interpolate the probabilities on departures to those for any time (time dependent process). This requires a different approach and would expand the paper. We would also like to consider a multivalued process which would also revive the number of jobs done during the maintenance periods in order to more comprehensively optimize the values of and . Again all these would run out of the scope of this paper that aims at introducing a game-theoretic approach and fluctuations to find the pgf of the equilibrium queue distribution. We notice that a supplementary variables approach would be very difficult if not impossible for our model. In addition, it also restricts the service time distributions to those absolutely continuous.

#### Acknowledgments

The authors are grateful to anonymous referees and the associate editor for their valuable suggestions that led to a considerable improvement of the paper. This research is supported by the U.S Army Grant no. W911NF-07-1-0121.