Abstract
The class of counting processes constitutes a significant part of applied probability. The classic counting processes include Poisson processes, nonhomogeneous Poisson processes, and renewal processes. More sophisticated counting processes, including Markov renewal processes, Markov modulated Poisson processes, age-dependent counting processes, and the like, have been developed for accommodating a wider range of applications. These counting processes seem to be quite different on the surface, forcing one to understand each of them separately. The purpose of this paper is to develop a unified multivariate counting process, enabling one to express all of the above examples using its components, and to introduce new counting processes. The dynamic behavior of the unified multivariate counting process is analyzed, and its asymptotic behavior as is established. As an application, a manufacturing system with certain maintenance policies is considered, where the optimal maintenance policy for minimizing the total cost is obtained numerically.
1. Introduction
A stochastic process is called a counting process when is nonnegative, right continuous and monotone nondecreasing with . The classic counting processes of importance include a Poisson process, a nonhomogeneous Poisson process (NHPP) and a renewal process. More sophisticated counting processes have been developed in order to accommodate a wider range of applications. A Markov renewal process, for example, extends an ordinary renewal process in that the interarrival time between two successive arrivals has a probability distribution depending on the state transition of the underlying Markov chain, see, for example, Pyke [1, 2]. In Masuda and Sumita [3], the number of entries of a semi-Markov process into a subset of the state space is analyzed, while a Markov-modulated Poisson process (MMPP) is introduced by Neuts [4] where jumps of the MMPP occur according to a Poisson process with intensity whenever the underlying Markov-chain is in state .
The MMPP is generalized subsequently into several different directions. Lucantoni et al. [5] develop a Markovian arrival process (MAP), where a Markov-chain defined on with , and is replaced to a state as soon as it enters a state with probability and the counting process describes the number of such replacements occurred in . If we set and , and an absorbing state can be reached only from its counter part with instantaneous replacement back to itself, the resulting MAP becomes an MMPP. Another direction for generalizing the MMPP is to replace the underlying Markov-chain by a semi-Markov process, which we call a semi-Markov-modulated Poisson process (SMMPP). To the best knowledge of the authors, the SMMPP is first addressed by Dshalalow [6], where a systematic approach for dealing with modulated random measures is proposed in a more abstract way. Such modulated random measures include the MMPP and the SMMPP as special cases. An application of the SMMPP to queueing theory is discussed in Dshalalow and Russell [7]. More recently, in a series of papers by Agarwal et al. [8] and Dshalalow [9, 10], the original approach above has been further extended and refined. The SMMPP is studied in detail independently by Özekici and Soyer [11].
In the SMMPP, the counting process under consideration is modulated according to state transitions of the underlying semi-Markov process. A further generalization may be possible by considering a counting process whose arrival intensity depends on not only the current state of the semi-Markov process but also the current age of the process. This line of research is originated by Sumita and Shanthikumar [12] where an age-dependent counting process generated from a renewal process is studied. Here, items arrive according to an NHPP which is interrupted and reset at random epochs governed by a renewal process.
All of these counting processes discussed above seem to be quite different on the surface, forcing one to understand each of them separately. The purpose of this paper is to develop a unified multivariate counting process which would contain all of the above examples as special cases. In this regard, we consider a system where items arrive according to an NHPP. This arrival stream is interrupted from time to time where the interruptions are governed by a finite semi-Markov process on . Whenever a state transition of the semi-Markov process occurs from to , the intensity function of the NHPP is switched from to with an initial value reset to . In other words, the arrivals of items are generated by the NHPP with when the semi-Markov process is in state with denoting the time since the last entry into state . Of particular interest in analysis of such systems are the multivariate counting processes and where counts the cumulative number of items that have arrived in while the semi-Markov process is in state and represents the cumulative number of the state transitions of the semi-Markov process from to in . The joint multivariate counting process enables one to unify many existing counting processes in that they can be derived in terms of the components of . Because of this reason, hereafter, we call the unified multivariate counting process. The dynamic behavior of is captured through analysis of the underlying Laplace transform generating functions, yielding asymptotic expansions of the means and the variances of the partial sums of its components as .
Applications of the unified multivariate counting process can be found, for example, in modern communication networks. One may consider a high-speed communication link for transmitting video signals between two locations. Video sequences are transmitted as streams of binary data that vary over time in traffic intensity according to the level of movement, the frequency of scene changes, and the level of transmission quality. Consequently, efficient transmission of video traffic can be achieved through variable bit rate coding. In this coding scheme, data packets are not generated at a constant rate from the original sequence, but rather at varying rates. By doing so, one achieves less fluctuation in transmission quality level and, at the same time, transmission capacity can be freed up whenever possible. As in Maglaris et al. [13], such a mechanism may be implemented by using multimode encoders where each mode reflects a certain level of data compression, and the change between modes is governed by the underlying video sequence according to buffer occupancy levels. A system of this sort can be described in the above framework with representing the number of packet arrivals at the origination site and describing the number of the encoder changes in . The state of the underlying semi-Markov process at time then corresponds to the current mode of the encoder. Other types of applications include system reliability models where the semi-Markov process describes the status of the system under consideration while the interruptions correspond to system failures and replacements. A cost function associated with such a system may then be constructed from the unified multivariate counting processes . In this paper, a manufacturing system with certain maintenance policies is considered, where the unified multivariate counting process enables one to determine numerically the optimal maintenance policy for minimizing the total cost.
The structure of this paper is as follows. In Section 2, key transform results of various existing counting processes are summarized. Detailed description of the unified multivariate counting process is provided in Section 3 and its dynamic behavior is analyzed in Section 4 by examining the probabilistic flow of the underlying stochastic processes and deriving transform results involving Laplace transform generating functions. Section 5 is devoted to derivation of the existing counting processes of Section 2 in terms of the components of the unified multivariate counting process. Asymptotic analysis is provided in Section 6, yielding asymptotic expansions of the means and the variances of the partial sums of its components. An application is discussed in Section 7, where a manufacturing system with certain maintenance policies is considered and the optimal maintenance policy for minimizing the total cost is obtained numerically. Some concluding remarks are given in Section 8.
Throughout the paper, matrices and vectors are indicated by double underlines (, etc.) and underlines (, etc.) respectively. The vector with all components equal to is denoted by and the identity matrix is written as . A submatrix of is defined by .
2. Various Counting Processes of Interest
In this section, we summarize key transform results of various counting processes of interest, which can be expressed in terms of the components of the unified multivariate counting process proposed in this paper as we will see. We begin the discussion with one of the most classical arrival processes, the Poisson process.
2.1. Poisson Process
Poisson process of intensity is characterized by a sequence of independently and identically distributed (i.i.d.) exponential random variables with common probability density function (p.d.f.) . Let . Then, the associated Poisson process is defined as a counting process satisfying If a system has an exponential lifetime of mean and is renewed instantaneously upon failure, represents the lifetime of the th renewal cycle. The Poisson process then counts the number of failures that have occurred by time .
Let and define the probability generating function (p.g.f.) by It can be seen, see, for example, Karlin and Taylor [14], that where for . Multiplying on both sides of (2.3) and summing from to , one then finds that Since where if statement is true and otherwise, one has . Equation (2.4) can then be solved as
2.2. Nonhomogeneous Poisson Process (NHPP)
An NHPP differs from a Poisson process in that the failure intensity of the system is given as a function of time . Accordingly, (2.3) should be rewritten as By taking the generating function of (2.6), one finds that With , this equation can be solved as The reader is referred to Ross [15] for further discussions of NHPPs.
2.3. Markov-Modulated Poisson Process (MMPP)
Let be a Markov-chain in continuous time on governed by a transition rate matrix . Let and define the associated diagonal matrix . An MMPP characterized by is a pure jump process where jumps of occur according to a Poisson process with intensity whenever the Markov-chain is in state .
Let and define . The infinitesimal generator associated with the Markov-chain is then given by For , let and define the associated matrix generating function by It can be seen that In matrix notation, this can be rewritten as By taking the generating function of (2.13) together with (2.9), one sees that Since , one has , where is the identity matrix, so that the above differential equation can be solved as where for any square matrix . It should be noted that , which is the transition probability matrix of as it should be. By taking the Laplace transform of both sides of (2.15), is given by
In general, the interarrival times generated by an MMPP are not independent nor identically distributed. In multimedia computer and communication networks, data packets are mingled together with voice and image packets generated from analogue sources. Since arrival patterns of such packets differ from each other, MMPPs have provided useful means to model arrival processes of packets in multimedia computer and communication networks, see, for example, Heffes and Lucantoni [16] and Sriram and Whitt [17]. Burman and Smith [18], and Knessl et al. [19] studied a single server queuing system with an MMPP arrival process and general i.i.d. service times. Neuts et al. [20] established characterization theorems for an MMPP to be a renewal process in terms of lumpability of the underlying Markov-chain . The reader is referred to Neuts [4] for further discussions of MMPP.
An MMPP can be extended by replacing the underlying Markov-chain in continuous time by a semi-Markov process as discussed in Section 1. This process is denoted by SMMPP. To the best knowledge of the authors, the SMMPP is first addressed by Dshalalow [6], where a systematic approach for dealing with modulated random measures is proposed in a more abstract way. Such modulated random measures include the MMPP and the SMMPP as special cases. An application of the SMMPP to queueing theory is addressed in Dshalalow and Russell [7]. More recently, the original approach above has been further extended and refined in a series of papers by Agarwal et al. [8] and Dshalalow [9, 10]. The SMMPP is studied in detail independently by Özekici and Soyer [11] including transient characterizations and ergodic analysis. Both MMPP and SMMPP will be proven to be expressible in terms of the components of the unified multivariate counting process proposed in this paper.
2.4. Renewal Process
Renewal processes can be considered as a generalization of Poisson processes in that a sequence of i.i.d. exponential random variables are replaced by that of any i.i.d. nonnegative random variables with common distribution function . The resulting counting process is still characterized by (2.1). Let as before. One then sees that where denotes the -fold convolution of with itself, that is, and which is the step function defined as for and else.
Let and . By taking the Laplace transform of both sides of (2.17), it follows that By taking the generating function of the above equation with , one has The reader is referred to Cox [21], or Karlin and Taylor [14] for further discussions of renewal processes.
2.5. Markov Renewal Process (MRP)
An MRP is an extension of an ordinary renewal process in that, in the interval , the former describes the recurrence statistics for intermingling classes of epochs of an underlying semi-Markov process, whereas the latter counts the number of recurrences for a single recurrent class of epochs. More specifically, let be a semi-Markov process on governed by a matrix p.d.f. where and is a stochastic matrix which is assumed to be ergodic. Let be a recurrent class consisting of the entries of the semi-Markov process to state for , and define to be a counting process describing the number of recurrences for given that there was an epoch of at time . Then is called an MRP.
The study of MRPs can be traced back to early 1960s represented by the two original papers by Pyke [1, 2], followed by Keilson [22, 23], Keilson and Wishart [24, 25], Çinlar [26, 27] and McLean and Neuts [28]. Since then, the area attracted many researchers and a survey paper by Çinlar [29] in 1975 already included more than 70 leading references. The study has been largely focused on the matrix renewal function with , the associated matrix renewal density, and the limit theorems. For example, one has the following result concerning the Laplace transform of by Keilson [23]: where is the Laplace transform of . The unified multivariate counting process of this paper contains an MRP as a special case and provides more information based on dynamic analysis of the underlying probabilistic flows.
2.6. Number of Entries of a Semi-Markov Process into a Subset of the State Space (NESMPS)
Another type of counting processes associated with a semi-Markov process on governed by a matrix p.d.f. is studied in Masuda and Sumita [3], where the state space is decomposed into a set of good states and a set of bad states satisfying and . The counting process is then defined to describe the number of entries of into by time .
While is a special case of MRPs, the detailed analysis is provided in [3], yielding much more information. More specifically, let be the age process associated with , that is, where , and define where One then has The associated matrix Laplace transform generating function can then be defined as It has been shown in Masuda and Sumita [3] that Here, with and for , the following notation is employed:
As we will see, the unified multivariate counting process proposed in this paper enables one to deal with multidimensional generalization of NESMPSs as a special case.
2.7. Markovian Arrival Process (MAP)
As for Poisson processes, a renewal process requires interarrival times to form a sequence of i.i.d. nonnegative random variables. As we have seen, a class of MMPPs enables one to avoid this requirement by introducing different Poisson arrival rates depending on the state of the underlying Markov chain. An alternative way to avoid this i.i.d. requirement is to adapt a class of MAPs, originally introduced by Lucantoni et al. [5]. We discuss here a slightly generalized version of MAPs in that a set of absorbing states is not necessarily a singleton set.
Let be an absorbing Markov-chain on with and , where all states in are absorbing. Without loss of generality, we assume that , and . For notational convenience, the following transition rate matrices are introduced.
The entire transition rate matrix governing is then given by
A replacement Markov-chain on is now generated from . Starting from a state in , the process coincides with within . As soon as reaches state , it is instantaneously replaced at state with probability and the process continues. Let where . Then the transition rate matrix and the infinitesimal generator of are given as where with Let be the transition probability matrix of with its Laplace transform defined by . From the Kolmogorov forward equation with , one has
Let be the counting process keeping the record of the number of replacements in and define By analyzing the probabilistic flow at state at time , it can be seen that It then follows that In matrix notation, the above equation can be rewritten as We now introduce the following matrix Laplace transform generating function: Taking the Laplace transform with respect to and the generating function with respect to of both sides of (2.41), one has which can be solved as We note from (2.37) and (2.44) that as it should be.
It may be worth noting that an MMPP is a special case of an MAP, which can be seen in the following manner. Let an MAP be defined on where and . Transitions within is governed by . An entry into is possible only from . When this occurs, the Markov process is immediately replaced at the entering state . The counting process for the number of such replacements then has the Laplace transform generating function given in (2.44) where is replaced by , which coincides with of (2.16) for MMPPs, proving the claim.
2.8. Age-Dependent Counting Process Generated from a Renewal Process (ACPGRP)
An age-dependent counting process generated from a renewal process has been introduced and studied by Sumita and Shanthikumar [12], where items arrive according to an NHPP which is interrupted and reset at random epochs governed by a renewal process. More specifically, let be a renewal process associated with a sequence of i.i.d. nonnegative random variables with common p.d.f. . The age process is then defined by In other words, is the elapsed time since the last renewal. We next consider an NHPP governed by an intensity function . If we define one has Of interest, then, is a counting process characterized by Here where is the set of nonnegative integers, and is the set of nonnegative real numbers.
Since the counting process is governed by the intensity function depending on the age process of the renewal process , it is necessary to analyze the trivariate process . Let the multivariate transform of be defined by It has been shown in Sumita and Shanthikumar [12] that where we define, for with ,
The Laplace transform generating function of defined by is then given by , so that one has from (2.50)
This class of counting processes denoted by ACPGRP may be extended where the underlying renewal process is replaced by an MMPP or an SMMPP. We define the former as a class of age-dependent counting processes governed by an MMPP, denoted by Markov-modulated age-dependent nonhomogeneous Poisson process (MMANHPP), and the latter as a class of age-dependent counting processes governed by an SMMPP, denoted by semi-Markov-modulated age-dependent nonhomogeneous Poisson process (SMMANHPP). The two extended classes are new and become special cases of the unified counting process proposed in this paper as we will see.
All of the counting processes discussed in this section are summarized in Figure 1.
3. Unified Multivariate Counting Process
In this section, we propose a stochastic process representing the unified multivariate counting process discussed in Section 1, which would contain all of the counting processes given in Section 2 as special cases. More specifically, we consider a system where items arrive according to an NHPP. This arrival stream is governed by a finite semi-Markov process on in that the intensity function of the NHPP depends on the current state of the semi-Markov process. That is, when the semi-Markov process is in state with the current dwell time of , items arrive according to a Poisson process with intensity . If the semi-Markov process switches its state from to , the intensity function is interrupted, the intensity function at state is reset to , and the arrival process resumes. Of particular interest would be the multivariate counting processes and with and counting the number of items that have arrived in state by time and the number of transitions of the semi-Markov process from state to state by time respectively. The two counting processes and enable one to introduce a variety of interesting performance indicators as we will see.
Formally, let be a semi-Markov process on governed by a matrix cumulative distribution function (c.d.f.) which is assumed to be absolutely continuous with the matrix probability density function (p.d.f.) It should be noted that, if we define and by then is an ordinary c.d.f. and is the corresponding survival function. The hazard rate functions associated with the semi-Markov process are then defined as
For notational convenience, the transition epochs of the semi-Markov process are denoted by , with . The age process associated with the semi-Markov process is then defined as At time with and , the intensity function of the NHPP is given by . For the cumulative arrival intensity function in state , one has The probability of observing arrivals in state within the current age of can then be obtained as
Of interest are the multivariate counting processes where represents the total number of items that have arrived by time while the semi-Markov process stayed in state , and with denoting the number of transitions of the semi-Markov process from state to state by time . It is obvious that denotes the number of entries into state by time . The initial state is not included in for any . In other words, if , remains until the first return of the semi-Markov process to state . In the next section, we will analyze the dynamic behavior of , yielding the relevant Laplace transform generating functions. In the subsequent section, all of the counting processes discussed in Section 2 would be expressed in terms of and , thereby providing a unified approach for studying various counting processes. The associated asymptotic behavior as would be also discussed.
4. Dynamic Analysis of
The purpose of this section is to examine the dynamic behavior of the multivariate stochastic process introduced in Section 3 by observing its probabilistic flow in the state space. Figure 2 depicts a typical sample path of the multivariate process.
Since is not Markov, we employ the method of supplementary variables. More specifically, the multivariate stochastic process is considered. This multivariate stochastic process is Markov and has the state space , where and are the set of and dimensional nonnegative integer vectors and matrices respectively, is the set of nonnegative real numbers and . Let be the joint probability distribution of defined by
In order to assure the differentiability of with respect to , we assume that has an absolutely continuous initial distribution function with p.d.f. . (If with probability , we consider a sequence of initial distribution functions satisfying as where for and otherwise. The desired results can be obtained through this limiting process.) One can then define By interpreting the probabilistic flow of the multivariate process in its state space, one can establish the following equations: where is the column vector whose th element is equal to with all other elements being , and .
The first term of the right-hand side of (4.3) represents the case that has not left the initial state by time and there have been items arrived during time , provided that and . The second term corresponds to the case that made at least one transition from by time , the multivariate process just entered the state at time , remained in state until time with , and there have been items arrived during the current age , provided that and . For the multivariate process at at time , it must be at followed by a transition from to at time which increases by one, explaining (4.4). Equations (4.5) merely describe the initial condition that .
In what follows, the dynamic behavior of the multivariate process would be captured by establishing the associated Laplace transform generating functions based on (4.3), (4.4) and (4.5). For notational convenience, the following matrix functions are employed: where . We are now in a position to prove the main theorem of this section.Theorem. Let . Then: where .
Proof. First, we assume that has a p.d.f. . Substituting (4.3) and (4.5) into (4.4), one sees that
Consequently, one sees from (4.6) and (4.8) that
where is employed from (3.4). By taking the Laplace transform of both sides of (4.22) with respect to , it follows that
By taking the multivariate generating function of (4.23) with respect to and , it can be seen that
It then follows from (4.10), (4.14), (4.16) and the discrete convolution property that
The last expression can be rewritten in matrix form, and one has
which can be solved for as
where .
Next, we introduce the following double Laplace transform:
and the associated diagonal matrix
By taking the double Laplace transform of (4.3), one sees from (4.7) and (4.28) that
By taking the double generating function, this then leads to
By rewriting the last expression in matrix form, it can be seen that
We now consider the limiting process . For the p.d.f., this limiting process becomes where is the Delta function defined as a unit function for convolution, that is, for any integrable function . Accordingly, it can be seen from (4.8) that . This in turn implies from (4.28) that . Consequently, it follows in matrix form that and . From (4.27), it can be readily seen that , proving (4.19). One also sees from (4.28) that
which proves (4.20), completing the proof.In the next section, it will be shown that all the transform results obtained in Section 2 can be derived from Theorem 4.1.
5. Derivation of the Special Cases from the Unified Counting Process
We are now in a position to demonstrate the fact that the proposed multivariate counting process introduced in Section 3 and analysed in Section 4 indeed unifies the existing counting processes discussed in Section 2. We will do so by deriving the transform results of Section 2 from Theorem 4.1.
5.1. Derivation of Poisson Process
Let be a Poisson process with intensity as discussed in Section 2.1. From (2.5), one sees that is given by
For the unified counting process, we consider a single state Markov-chain in continuous time where only the number of self transitions in is counted. More specifically, let , , , , , and . We note from (3.7) that implies so that from (4.6). This then implies . Similarly, since , one has . It then follows from Theorem 4.1 that Hence, from (5.1) and (5.2), one concludes that .
5.2. Derivation of NHPP
Let be an NHPP of Section 2.2 characterized by a time dependent intensity function . It can be seen from (2.8) that is given by
In order to see that can be viewed as a special case of the proposed multivariate counting process, we first consider a single state semi-Markov process where the dwell time in the state is deterministic given by . The marginal counting process then converges in distribution to as as we show next.
Let , , , , and . It then follows that and therefore from (4.6). This in turn implies that Let be the step function defined by if and otherwise. Since , one sees that Theorem 4.1 together with (5.4) and (5.5) then leads to Now it can be readily seen that in (5.6) converges to in (5.3) as , proving the claim.
5.3. Derivation of MMPP
Let be an MMPP of Section 2.3 characterized by . We show that the Laplace transform generating function given in (2.16) can be derived as a special case of Theorem 4.1.
For the unified multivariate counting process, let , , , , , , and . From (3.3) one sees that . Therefore, one sees from (4.6), (4.9) and (4.10) that and similarly one has, from (4.7), (4.11) and (4.12), It then follows from Theorem 4.1, (5.7)and (5.8) that which coincides with (2.16) as expected.
5.4. Derivation of Renewal Process
In order to demonstrate that a renewal process is a special case of the unified multivariate counting process, we follow the line of the arguments for the case of Poisson processes. Namely, let , , , , , and . From Theorem 4.1, one has which agrees with (2.19).
5.5. Derivation of MRP
Let be an MRP discussed in Section 2.5. We recall that describes the number of entries of the semi-Markov process into state in given that . For the unified multivariate counting process, counts the number of transitions from state to state in . Hence, one has provided that . Accordingly, we set , that is, for all . With , , for all , one has and from (4.6), (4.7) and (4.9) through (4.12), where . Let . It then follows from Theorem 4.1 that It should be noted that, with and , the element of in (5.11) can be written as
We now focus on for . In doing so, let and define It then follows from (5.11) through (5.13) that that is, one has
We recall that , which can be obtained by differencing with respect to at . More formally, one has which is the th column of given in (2.20). By noting that one sees from (5.15) that Since and , it can be seen that Substituting (5.19) into (5.18), one then concludes that This in turn implies that which agrees with of (2.20), completing the derivation.
5.6. Derivation of NESMPS
As in Section 2.6, let the state space of be decomposed into a set of good states and a set of bad states satisfying and . The counting process of Section 2.6 describes the number of entries of into by time . The Laplace transform generating function of the joint probability of , the age process and is given in (2.26).
In the context of the unified multivariate counting process discussed in Section 3, one expects to have In order to prove (5.22) formally, we set From (3.7), (4.6), (4.9) and (4.10), one has so that where is as given in Theorem 4.1. Similarly, it can be seen from (3.7), (4.7), (4.11) and (4.12) that and hence Substituting (5.24) and (5.25) into (4.20), it then follows that
By comparing (2.26) with (5.26), (5.22) holds true if and only if In what follows, we prove that (5.27) indeed holds true. From (5.24), one sees that where and are as given in (2.28). We define the inverse matrix of (5.28) by Since the multiplication of the two matrices in (5.28) and (5.29) yields the identity matrix, it follows that
Solving the above equations for , one has
We next turn our attention to the left-hand side of (5.27). From (2.29), one sees that As before, we define the inverse matrix of (5.35) as Multiplying the two matrices in (5.35) and (5.36) then yields which in turn can be solved for as
Let the left-hand side matrix of (5.27) be described as From (2.30) and (5.36) through (5.41), one sees that From (2.28), one easily sees that , and hence from (5.31). The fact that is straightforward from (5.32). With , one has from (5.33) and from (5.34), completing the proof for (5.27).
5.7. Derivation of MAP
We recall that an MAP is constructed from an absorbing Markov-chain in continuous time on , with being a set of absorbing states, governed by defined in (2.32). A replacement Markov-chain is then generated from , where coincides with within starting from a state in . As soon as reaches state , it is instantaneously replaced at state with probability and the process continues.
In order to deduce an MAP from the unified multivariate counting process as a special case, we start from (4.20) in Theorem 4.1, where the underlying semi-Markov process is now the replacement Marcov chain discussed above. This replacement Marcov chain is defined on governed by with and as in (2.33). We note that from (3.7), (4.6), (4.9) and (4.10), and from (3.7), (4.7), (4.11) and (4.12). Hence, it follows that Let as in (2.35). Since is a Markov chain, the dwell time in state is independent of the next state and is exponentially distributed with parameter . Consequently, one has Substituting (5.45) into (5.44) and noting , it follows that where in (2.34) is employed.
As it stands, the Laplace transform generating function of (5.46) describes the matrix counting process where corresponds to . For of Section 2.7, it is only necessary to count the number of replacements in . Given that state is visited from the current state , this move is direct within with probability , and such a move involves replacement with probability . Accordingly, we set Substitution of (5.47) into (5.46) then leads to which coincides with of (2.44), as expected.
5.8. Derivation of ACPGRP
The age-dependent counting process of Sumita and Shanthikumar [12] has been extended in this paper where the underlying renewal process is replaced by a semi-Markov process with state dependent nonhomogeneous hazard functions. Accordingly, the original model can be treated as a special case of the unified multivariate counting process proposed in this paper by setting , , , , , . With this specification, from Theorem 4.1, one sees that It then follows that which coincides with Equation (2.53).
6. Asymptotic Analysis
Let and be arbitrary subsets of the state space of the underlying semi-Markov process, and define where describes the total number of items arrived in according to the nonhomogeneous Poisson processes within and denotes the number of transitions from any state in to any state in occurred in . Appropriate choice of and would then enable one to analyze processes of interest in a variety of applications. In the variable bit rate coding scheme for video transmission explained in Section 1, for example, one may be interested in the packet arrival stream for a specified mode of the encoder represented by . In a reliability model, the underlying semi-Markov process may describe the state of a production machine. Minimal repair would take place whenever the system state is in at the cost of , while complete overhaul would be forced at the cost of if the machine state enters . The total maintenance cost is then given by . A simplified version of this cost structure has been analyzed by Sumita and Shanthikumar [12] where the underlying semi-Markov process is merely a renewal process with . The purpose of this section is to study a more general cost structure specified by with focus on the Laplace transform generating function and the related moment asymptotic behaviors of and based on Theorem 4.1.
For notational simplicity, we introduce the following vectors and matrices. Let and with their compliments defined respectively by and . The cardinality of a set is denoted by . Submatrices of are denoted by so that one has
with understanding that the states are arranged appropriately.
Let , Throughout the paper, we assume that for . In particular, one has which is stochastic. The Taylor expansion of the Laplace transform is then given by Let be the normalized left eigenvector of associated with eigenvalue so that and .
We recall from Theorem 4.1 that where From (3.6), (3.7), (4.6), (4.9) and (4.10), one sees that Similarly, it follows from (3.6), (3.7), (4.7), (4.11) and (4.12) that The th order partial derivatives of and with respect to at are then given by In matrix form, (6.12) can be written as
Let and , . The Taylor expansion of and is then given by
In order to prove the main results of this section, the following theorem of Keilson [23] plays a key role.
Theorem 6.1 (Keilson [23]). As , one has where
We are now in a position to prove the first key theorem of this section.
Theorem. Let be an initial probability vector of the underlying semi-Markov process. As , one has where
Proof. We first note from (6.8) together with (6.4) that By taking the partial derivatives of (6.19) with respect to at and at respectively, one has Theorem 6.1 of Keilson [23] combined with (6.7), (6.13) then yields the Laplace transform expansions of (6.20), and the theorem follows by taking the inversion of the Laplace transform expansions.The next theorem can be shown in a similar manner by differentiating (6.19) twice with respect to at and at respectively, and proof is omitted.Theorem. As , one has where , and other matrices are as defined in Theorem 6.2.Theorems 6.2 and 6.3 then lead to the following theorem providing the asymptotic expansions of and .Theorem. As , one has where
Proof. It can be readily seen that Substituting the results of Theorems 6.2 and 6.3 into this equation, one sees that where From Theorem 6.1, one has which is of rank one having identical rows. As can be seen from Theorem 6.2, both and satisfy so that and the theorem follows.The asymptotic behavior of can be easily found from (6.2) and Theorem 6.2. The asymptotic expansion of , however, requires a little precaution because it involves the joint expectation of and . More specifically, one has so that In order to evaluate , we note from (6.8) that The asymptotic expansion of can then be obtained as for the previous theorems.Theorem. As , one has where Now the key theorem of this section is given from (6.29), Theorems 6.2, 6.4 and 6.5.Theorem. As , one has where Proof. Equation (6.33) follows trivially from Theorem 6.2. For (6.34), we note from Theorems 6.2, 6.4 and 6.5 together with (6.29) that where From (6.27), the first term on the right-hand side of Equation (6.35) can be rewritten as completing the proof.
7. Dynamic Analysis of a Manufacturing System for Determining Optimal Maintenance Policy
As an application of the unified multivariate counting process, in this section, we consider a manufacturing system with a certain maintenance policy, where the system starts anew at time , and tends to generate product defects more often as time goes by. When the system reaches a certain state, the manufacturing system would be overhauled completely and the system returns to the fresh state. More specifically, let be a semi-Markov process on governed by , describing the system state at time where state is the fresh state and state is the maintenance state. When the system is in state , , product defects are generated according to an NHPP with intensity . It is assumed that the system deteriorates monotonically and accordingly increases as a function of both and . When the system reaches state , the manufacturing operation is stopped and the system is overhauled completely. The maintenance time increases stochastically as a function of . In other words, the further the maintenance is delayed, the longer the maintenance time would tend to be. Upon finishing the overhaul, the system is brought back to the fresh state . Of interest, then, is to determine the optimal maintenance policy concerning how to set .
In order to determine the optimal maintenance policy, it is necessary to define the objective function precisely. Let be the cost associated with each defect and let be the cost for each of the maintenance operation. If we define two counting processes and as the total number of defects generated by time and the number of the maintenance operations occurred by time respectively, the total cost in can be described as Let be the set of natural numbers. The optimal maintenance policy is then determined by
In what follows, we present a numerical example by letting for . For the underlying semi-Markov process, we define the matrix Laplace transform having IFR (Increasing Failure Rate) and DFR (Decreasing Failure Rate) dwell time distributions as described below, where the underlying parameters are set in such a way that the means of IFR dwell times are equal to those of DFR dwell times. By introducing matrices and , for which the details are given in Table 1 along with other parameter values, we define where
Based on Theorem 6.2, the asymptotic behaviors of the mean of and per unit time with maintenance policy are depicted in Figures 3 and 4. One could see that both the mean of and per unit time converges to a positive value as time increases. In order to determine the optimal maintenance policy, for , the corresponding total cost can be computed based on Theorem 6.6. Numerical results are shown in Table 2 and depicted in Figure 5. For the case of IFR, the optimal maintenance policy is at , while for the DFR case, where the running period is taken to be hours.
(a)
(b)
(a)
(b)
(a)
(b)
8. Concluding Remarks
In this paper, a unified multivariate counting process is proposed with nonhomogeneous Poisson processes lying on a finite semi-Markov process. Here the vector process counts the cumulative number of such nonhomogeneous Poisson arrivals at every state and the matrix process counts the cumulative number of state transitions of the semi-Markov process in . This unified multivariate counting process contains many existing counting processes as special cases. The dynamic analysis of the unified multivariate counting process is given, demonstrating the fact that the existing counting processes can be treated as special cases of the unified multivariate counting process. The asymptotic behaviors of the mean and the variance of the unified multivariate counting process are analyzed. As an application, a manufacturing system with certain maintenance policies is considered. The unified multivariate counting process enables one to determine the optimal maintenance policy minimizing the total cost. Numerical examples are given with IFR and DFR dwell times of the underlying semi-Markov process. As for the future agenda, the impact of such distributional properties on the optimal maintenance policy would be pursued theoretically. Other possible theoretical extensions include: (1) analysis of the reward process associated with the unified multivariate counting process; and (2) exploration of further applications in the areas of modern communication networks and credit risk analysis such as CDOs (collateralized debt obligations) for financial engineering.
Acknowledgment
The authors wish to thank two anonymous referees and the associate editor for providing constructive comments and valuable references, which contributed to improve the original version of the paper. This research is supported by MEXT Grand-in-Aid for Scientific Research (C) 17510114.