Optimal Maintenance of a Production System with 𝐿 Intermediate Buffers

Abstract

We consider a production-inventory system that consists of an input-generating installation, a production unit and L intermediate buffers. It is assumed that the installation transfers the raw material 𝑗∈{1,…,𝐿} to buffer 𝐵_𝑗, and the production unit pulls the raw material 𝑗∈{1,...,𝐿} from buffer 𝐵_𝑗. We consider the problem of the optimal preventive maintenance of the installation if the installation deteriorates stochastically with usage and the production unit is always in operative condition. We also consider the problem of the optimal preventive maintenance of the production unit if the production unit deteriorates stochastically with usage and the installation is always in operative condition. Under a suitable cost structure and for given contents of the buffers, it is proved that the average-cost optimal policy for the first (second) problem initiates a preventive maintenance of the installation (production unit) if and only if the degree of deterioration of the installation (production unit) exceeds some critical level. Numerical results are presented for both problems.

1. Introduction

In the present paper, we study two problems, and we generalize the results obtained in two previous papers by Kyriakidis and Dimitrakos [1] and Pavitsos and Kyriakidis [2] that are concerned with the preventive maintenance of a production-inventory system. We consider a manufacturing system (see Figure 1) in which an input-generating installation (𝐼) transfers 𝐿 raw materials to a subsequent production unit (𝑃). We assume that 𝐿 buffers 𝐵₁,…,𝐵_𝐿 have been built between the installation and the production unit. The installation transfers the raw material 𝑗∈{1,…,𝐿} to the buffer 𝐵_𝑗, and the production unit pulls this raw material from the buffer 𝐵_𝑗. The buffers have finite capacities.

Figure 1 

The production-inventory system.

In the first problem it is assumed that the installation deteriorates stochastically over time, and the production unit is always in operative condition. The deteriorating process for the installation is described by some known transition probabilities between different degrees of deterioration. A discrete-time Markov decision model is considered for the optimal preventive maintenance of the installation. The maintenance times are assumed to be geometrically distributed, and the cost structure includes operating costs of the installation, costs for storing the raw materials in the buffers, maintenance costs and costs due to production delay when the installation does not operate or operate partially and the contents of some or all buffers are below some specific levels. It is proved that for fixed contents of the buffers the policy that minimizes the long-run expected average cost per unit time is of control-limit type, that is, it initiates a preventive maintenance of the installation if and only if its degree of deterioration exceeds some critical level. This result generalizes the structural result that was obtained by Kyriakidis and Dimitrakos [1] for the case in which 𝐿=1. In the second problem it is assumed that the production unit deteriorates stochastically over time and the installation is always in operative condition. The deteriorating process for the production unit is described by some known transition probabilities between different degrees of deterioration. A discrete-time Markov decision model is formulated for the optimal preventive maintenance of the production unit. The maintenance times are assumed to be geometrically distributed, and the cost structure includes operating costs of the production unit, costs for the maintenance of the production unit, storage costs, penalty costs, and costs due to the lost production. It is proved that for fixed contents of the buffers the average-cost optimal policy is again of control-limit type, that is, it initiates a preventive maintenance of the production unit if and only if its degree of deterioration exceeds some critical level. This result generalizes the structural result that was obtained by Pavitsos and Kyriakidis [2] for the case in which 𝐿=1.

An example of this system could be a production machine that pulls 𝐿 different parts from 𝐿 buffers and assembles them in order to produce the final product. These parts are transferred by a feeder to the buffers. Note that in the last twenty years a great number of maintenance models for production-inventory systems have been studied (see Van Der Duyn Schouten and Vanneste [3], Meller and Kim [4], Iravani and Duenyas [5], Sloan [6], Yao et al. [7], Rezg et al. [8], Dimitrakos and Kyriakidis [9], Karamatsoukis and Kyriakidis [10], and Hadidi et al. [11]). In these models, the preventive maintenance depends on the working condition of a machine and the level of a subsequent buffer. The first problem that we study in the present paper has its origin in a model introduced by Van Der Duyn Schouten and Vanneste [3]. The states of that model consist of the age of a machine and the content of a subsequent buffer that is fed by the machine. The cost structure included costs due to lost production that were incurred when a repair was performed on the machine and the buffer was empty. The repair times of the machine were assumed to be geometrically distributed. It was proved that, for fixed buffer content, the average-cost optimal policy initiates a preventive maintenance of the machine if and only if its age is greater than or equal to a critical value.

The rest of the paper is organized as follows. In the next section, we describe the problem in which only the installation deteriorates with usage, and we derive the structure of the average-cost optimal policy. In Section 3, we study the case in which only the production unit deteriorates over time, and the structure of the average-cost optimal policy is derived. Numerical results are presented for both problems. In the final section, the main conclusions of the paper are summarized, and we propose topics for future research.

2. The Problem when the Installation Deteriorates Stochastically

We consider a production-inventory system (see Figure 1) which consists of an installation (𝐼) that supplies the buffer 𝐵_𝑗 with the raw material 𝑗∈{1,…,𝐿} and a production unit (𝑃) which pulls 𝑑_𝑗 units of the raw material 𝑗∈{1,…,𝐿} from buffer 𝐵_𝑗 during one unit of time. It is assumed that the production unit is always in operative condition, and that the installation may fail as time evolves. The buffer 𝐵_𝑗, 𝑗=1,2,…,𝐿 has finite capacity which is equal to 𝐾_𝑗 units of raw material 𝑗. As long as the buffer 𝐵_𝑗, 𝑗=1,…,𝐿 is not full and the installation is in operative condition, the installation may transfer 𝑝_𝑗(>𝑑_𝑗) units of the raw material 𝑗∈{1,…,𝐿} to buffer 𝐵_𝑗 during one unit of time and the difference 𝑝_𝑗−𝑑_𝑗 is stored in buffer 𝐵_𝑗. As soon as buffer 𝐵_𝑗, 𝑗=1,…,𝐿 is filled up the installation reduces its speed from 𝑝_𝑗 to 𝑑_𝑗. The numbers 𝑝_𝑗,𝑑_𝑗,𝐾_𝑗, 𝑗=1,…,𝐿 are assumed to be integers.

We suppose that the installation is inspected at discrete, equidistant time epochs 𝜏=0,1,… (say every day), and is classified into one of 𝑚+2 working conditions 0,1,…,𝑚+1 which describe increasing levels of deterioration. Working condition 0 denotes a new installation (or functioning as good as new), while working condition 𝑚+1 means that the installation is in failed (inoperative) condition and it cannot transfer the raw materials to the buffers. The intermediate working conditions 1,…,𝑚 are operative and are ordered ascendingly to reflect their relative degree of deterioration. The transition probability of moving from working condition 𝑖 at time epoch 𝜏 to working condition 𝑟 at time epoch 𝜏+1 is equal to 𝑝_𝑖𝑟. We assume that the probability of eventually reaching the working condition 𝑚+1 from any initial working condition 𝑖 is nonzero. If at a time epoch 𝜏 the installation is found to be at failure state 𝑚+1, a corrective maintenance is mandatory. If at a time epoch 𝜏 the installation is found to be at any working condition 𝑖≤𝑚, a preventive maintenance may be initiated. When a preventive maintenance is performed, the installation does not operate and it does not transfer any raw material to its buffer. It is assumed that the preventive and corrective repair times (expressed in time units) are geometrically distributed with probability of success 𝑎_𝐼 and 𝑏_𝐼, that is, the probability that they will last 𝑡≥1 time units are equal to (1−𝑎_𝐼)^𝑡−1𝑎_𝐼 and (1−𝑏_𝐼)^𝑡−1𝑏_𝐼, respectively. When a preventive or a corrective maintenance is performed and the buffer 𝐵_𝑗, 𝑗∈{1,…,𝐿} contains 𝑥_𝑗 units of raw material 𝑗, the production unit pulls from buffer 𝐵_𝑗 during one unit of time min(𝑥_𝑗,𝑑_𝑗) units of raw material 𝑗. Both maintenances bring the installation to its perfect condition 0.

We introduce the state PM to denote the situation that a preventive maintenance is performed on the installation. Then the state space of the system is the set 𝑆 = {0,…,𝑚+1,𝑃𝑀} × {0,…,𝐾₁} × ⋯ × {0,…,𝐾_𝐿}, where (𝑖,𝑥₁,…,𝑥_𝐿)∈𝑆 is the state in which 𝑖 is the working condition of the installation and 𝑥_𝑗∈{0,…,𝐾_𝑗},𝑗=1,…,𝐿, is the content of buffer 𝐵_𝑗. A policy is any rule for choosing actions at each time epoch 𝜏=0,1,… The possible actions are: action 1 (initiate a preventive maintenance), action 2 (initiate a corrective maintenance), action 𝐽⊆{1,…,𝐿} (transfer raw materials only to those buffers that belong to the nonempty subset 𝐽 of the set {1,…,𝐿}). If at a time epoch 𝜏 the installation is found to be at state PM or state 𝑚+1, the action 1 or the action 2 is compulsory, respectively. If at a time epoch 𝜏 the installation is found to be at working condition 𝑖∈{0,…,𝑚}, then we may either choose either action 1 or action 𝐽⊆{1,…,𝐿}. Hence, the number of possible actions in this case is 2^𝐿 since the number of nonempty subsets of {1,…,𝐿} is ∑^𝐿_𝑖=1^𝐿_𝑖=2^𝐿−1. If at a time epoch 𝜏 action 𝐽 is chosen and 𝑗 belongs to 𝐽 with 𝑥_𝑗<𝐾_𝑗, then the content of buffer 𝐵_𝑗 at next time epoch 𝜏+1 will be min(𝑥_𝑗+𝑝_𝑗−𝑑_𝑗,𝐾_𝑗). This increase of the buffer content will happen even if the working condition of the installation at next time epoch 𝜏+1 is the failure state 𝑚+1. A policy is said to be stationary if at each time epoch 𝜏=0,1,…, it chooses one action which depends only on the current state of the system.

The cost structure of the problem includes operating costs of the installation, storage costs, costs due to the lost production, and maintenance costs. If the working condition of the installation is 𝑖∈{0,…,𝑚} and the buffer 𝐵_𝑗, 𝑗∈{1,…,𝐿} is not full (or full) the cost of transferring 𝑝_𝑗 (or 𝑑_𝑗) units of raw material 𝑗 to buffer 𝐵_𝑗 during one unit of time is equal to 𝑐_𝑗(𝑖) (or 𝑐_𝑗̃(𝑖)). Therefore, if at a time epoch 𝜏 the working condition of the installation is found to be 𝑖∈{0,…,𝑚} and action 𝐽⊆{1,…,𝐿} is chosen, then the operating cost until the next time epoch 𝜏+1 is equal to ∑_𝑗∈𝐽1𝑐_𝑗(𝑖)+∑_𝑗∈𝐽2𝑐_𝑗̃(𝑖), where 𝐽₁∪𝐽₂=𝐽,𝐽₁ corresponds to the buffers that are not full and 𝐽₂ corresponds to the buffers that are full. We assume that the cost of holding a unit of raw material 𝑗∈{1,…,𝐿} in buffer 𝐵_𝑗 for one unit of time is equal to ℎ_𝑗. The cost rates during a preventive and a corrective maintenance of the installation are equal to 𝑐_𝑝 and 𝑐_𝑓, respectively. When a preventive or a corrective maintenance is performed on the installation and all buffers 𝐵₁,…,𝐵_𝐿 are empty (i.e., 𝑥_𝑗=0,𝑗=1,…,𝐿), the production unit does not pull any raw material 𝑗∈{1,…,𝐿} from the buffers. In this case we incur a cost due to production delay that is equal to 𝐶>0 per unit of time. When 𝑥_𝑗≥𝑑_𝑗,𝑗=1,…,𝐿, we do not incur any such cost since all buffers contain enough raw materials to satisfy the demands of the production unit for one unit of time. When the inequality 𝑥_𝑗≥𝑑_𝑗 is satisfied for some (but not for all) 𝑗=1,…,𝐿, the demands for raw materials of the production unit for one unit of time are partially satisfied. Therefore, the productivity of the production unit is reduced in the sense that the time for the production of the final products increases, since for one unit of time, some of the raw materials that are needed for the production of the final products are not available. In this case it seems reasonable to assume that the cost rate due to production delay is equal to 𝐶∑^𝐿_𝑗=1(𝑑_𝑗−𝑥_𝑗)⁺/∑^𝐿_𝑗=1𝑑_𝑗, where (𝑑_𝑗−𝑥_𝑗)⁺=max(𝑑_𝑗−𝑥_𝑗,0) is the unavailable quantity of the raw material 𝑗 during one unit of time. Similarly, if at a time epoch 𝜏 the working condition of the installation is found to be 𝑖∈{1,…,𝑚} and the action 𝐽⊆{1,…,𝐿} is chosen, then the cost due production delay until the time epoch 𝜏+1 is equal to 𝐶∑_𝑗∉𝐽(𝑑_𝑗−𝑥_𝑗)⁺/∑^𝐿_𝑗=1𝑑_𝑗. The following conditions on the cost structure and on the transition probabilities are assumed to be valid.

Condition 1. For 𝑗∈{1,…,𝐿} the sequences {𝑐_𝑗(𝑖)} and {𝑐_𝑗̃(𝑖)}, 0≤𝑖≤𝑚, are nondecreasing with respect to 𝑖. That is, as the working condition of the installation deteriorates, the cost of transferring the raw material 𝑗∈{1,…,𝐿} to buffer 𝐵_𝑗 increases.

Condition 2. For 𝑗∈{1,…,𝐿}, 𝑐_𝑗̃(𝑖)≤𝑐_𝑗(𝑖), 0≤𝑖≤𝑚. That is, the cost of transferring 𝑝_𝑗 units of raw material 𝑗∈{1,…,𝐿} to buffer 𝐵_𝑗 during one unit of time is greater than or equal to the cost of transferring 𝑑_𝑗 units of raw material 𝑗 to buffer 𝐵_𝑗 during one unit of time.

Condition 3. 0<𝑏_𝐼<𝑎_𝐼≤1. That is, the expected time required for a preventive maintenance is smaller than the expected time required for a corrective maintenance.

Condition 4. 𝑐_𝑝≤𝑐_𝑓. That is, the cost rate of a preventive maintenance does not exceed the cost rate of a corrective maintenance.

Condition 5 (An Increasing Failure Rate Assumption). For each 𝑘=0,…,𝑚, the function 𝐷_𝑘(𝑖)=∑^𝑚+1_𝑟=𝑘𝑝_𝑖𝑟 is nondecreasing in 𝑖, 0≤𝑖≤𝑚.
A consequence of this condition is that 𝐼_𝑖≤_𝑠𝑡𝐼_𝑖+1, 0≤𝑖≤𝑚, where 𝐼_𝑖 is a random variable representing the next working condition of the installation if its present working condition is 𝑖. It can be shown (see pages 122-123 in Derman [12]) that this condition is equivalent to the following one:

Condition 6. For any nondecreasing function ℎ(𝑟),0≤𝑟≤𝑚+1, the quantity ∑^𝑚+1_𝑟=0𝑝_𝑖𝑟ℎ(𝑟), 0≤𝑖≤𝑚, is nondecreasing in 𝑖.
We consider a discrete-time Markov decision process in which we aim to find a stationary policy which minimizes the long-run expected average cost per unit time. Note that for 𝐿=1 this problem was studied in Kyriakidis and Dimitrakos [1].

2.1. The Structure of the Optimal Policy

Let 𝛼(0<𝛼<1) be a discount factor. The minimum expected n-step discounted cost 𝑉^𝛼_𝑛(𝑖,𝑥), where (𝑖,𝑥)=(𝑖,𝑥₁,…,𝑥_𝐿) is the initial state, can be found for all 𝑛=1,2,…, recursively, from the following equations (see chapter 1 in Ross [13]): 𝑉^𝛼_𝑛(𝑖,𝑥)=minmin_{{}𝐽⊆1,…,𝐿𝑗∈𝐽}𝑐_𝑗(𝑖)+^𝐿_𝑗=1ℎ_𝑗𝑥_𝑗+∑_𝑗∉𝐽𝑑_𝑗−𝑥_𝑗⁺∑^𝐿_𝑗=1𝑑_𝑗𝐶+𝛼^𝑚+1_𝑟=0𝑝_𝑖𝑟𝑉^𝛼_𝑛−1𝑟,𝑥^′,𝑉^𝛼_𝑛(𝑃𝑀,𝑥),0≤𝑖≤𝑚,0≤𝑥_𝑗<𝐾_𝑗,𝑗=1,…,𝐿,𝑉^𝛼_𝑛(𝑃𝑀,𝑥)=𝑐_𝑝+^𝐿_𝑗=1ℎ_𝑗𝑥_𝑗+∑^𝐿_𝑗=1𝑑_𝑗−𝑥_𝑗⁺∑^𝐿_𝑗=1𝑑_𝑗𝐶+𝛼𝑎_𝐼𝑉^𝛼_𝑛−10,𝑥−𝑑⁺+𝛼1−𝑎_𝐼𝑉^𝛼_𝑛−1𝑃𝑀,𝑥−𝑑⁺,0≤𝑥_𝑗≤𝐾_𝑗,𝑗=1,…,𝐿,𝑉^𝛼_𝑛(𝑚+1,𝑥)=𝑐_𝑓+^𝐿_𝑗=1ℎ_𝑗𝑥_𝑗+∑^𝐿_𝑗=1𝑑_𝑗−𝑥_𝑗⁺∑^𝐿_𝑗=1𝑑_𝑗𝐶+𝛼𝑏_𝐼𝑉^𝛼_𝑛−10,𝑥−𝑑⁺+𝛼1−𝑏_𝐼𝑉^𝛼_𝑛−1𝑚+1,𝑥−𝑑⁺,0≤𝑥_𝑗≤𝐾_𝑗,𝑗=1,…,𝐿, (2.1)(2.2)(2.3) where 𝑥^′ in (2.1) is a vector with 𝐿 components in which the jth component equals to min(𝑥_𝑗+𝑝_𝑗−𝑑_𝑗,𝐾_𝑗), if 𝑗∈𝐽, while, if 𝑗∉𝐽, it is equal to (𝑥_𝑗−𝑑_𝑗)⁺=max(𝑥_𝑗−𝑑_𝑗,0) and (𝑥−𝑑)⁺ in (2.2), and (2.3) is the vector ((𝑥₁−𝑑₁)⁺,…,(𝑥_𝐿−𝑑_𝐿)⁺). The initial condition is 𝑉^𝛼₀(𝑠)=0, 𝑠∈𝑆. Note that, if 𝑥_𝑗=𝐾_𝑗 for some values of 𝑗∈{1,…,𝐿}, (2.1) is valid if 𝑐_𝑗(𝑖) is changed to 𝑐_𝑗̃(𝑖) for these values of 𝑗. Note that the first term in the curly brackets in (2.1) corresponds to the best action among all actions 𝐽⊆{1,…,𝐿}, while the second term corresponds to action 1 (i.e., initiate a preventive maintenance of the installation). The first part of the following lemma is needed to prove that the average-cost optimal policy is of control-limit type for fixed levels of the buffers.

Lemma 2.1. For each 𝑛=0,1,…, we have that (i)𝑉^𝛼_𝑛(𝑖,𝑥)≤𝑉^𝛼_𝑛(𝑖+1,𝑥),0≤𝑖≤𝑚,0≤𝑥_𝑗≤𝐾_𝑗,𝑗=1,…,𝐿, (ii)𝑉^𝛼_𝑛(𝑃𝑀,𝑥)≤𝑉^𝛼_𝑛(𝑚+1,𝑥),0≤𝑥_𝑗≤𝐾_𝑗,𝑗=1,…,𝐿.

Proof. We will prove the lemma by induction on 𝑛. The lemma is valid for 𝑛=0, since 𝑉^𝛼₀(𝑠)=0,𝑠∈𝑆. We assume that it is valid for 𝑛−1(≥0). We will show that it is also valid for 𝑛. First, we prove part (ii) and then part (i).
Part (ii): Let 𝐷=𝑉^𝛼_𝑛−1(𝑚+1,(𝑥−𝑑)⁺−𝑉^𝛼_𝑛−1(0,(𝑥−𝑑)⁺).
For 0≤𝑥_𝑗≤𝐾_𝑗,𝑗=1,…,𝐿, we have that 𝑉^𝛼_𝑛(𝑃𝑀,𝑥)=𝑐_𝑝+^𝐿_𝑗=1ℎ_𝑗𝑥_𝑗+∑^𝐿_𝑗=1𝑑_𝑗−𝑥_𝑗⁺∑^𝐿_𝑗=1𝑑_𝑗𝐶+𝛼𝑎_𝐼𝑉^𝛼_𝑛−10,𝑥−𝑑⁺+𝛼1−𝑎_𝐼𝑉^𝛼_𝑛−1𝑃𝑀,𝑑−𝑥⁺≤𝑐_𝑓+^𝐿_𝑗=1ℎ_𝑗𝑥_𝑗+∑^𝐿_𝑗=1𝑑_𝑗−𝑥_𝑗⁺∑^𝐿_𝑗=1𝑑_𝑗𝐶+𝛼𝑎_𝐼𝑉^𝛼_𝑛−10,𝑥−𝑑⁺+𝛼1−𝑎_𝐼𝑉^𝛼_𝑛−1𝑚+1,𝑥−𝑑⁺=𝑐_𝑓+^𝐿_𝑗=1ℎ_𝑗𝑥_𝑗+∑^𝐿_𝑗=1𝑑_𝑗−𝑥_𝑗⁺∑^𝐿_𝑗=1𝑑_𝑗𝐶+𝛼𝑉^𝛼_𝑛−1𝑚+1,𝑥−𝑑⁺−𝛼𝑎_𝐼𝐷≤𝑐_𝑓+^𝐿_𝑗=1ℎ_𝑗𝑥_𝑗+∑^𝐿_𝑗=1𝑑_𝑗−𝑥_𝑗⁺∑^𝐿_𝑗=1𝑑_𝑗𝐶+𝛼𝑉^𝛼_𝑛−1𝑚+1,𝑥−𝑑⁺−𝛼𝑏_𝐼𝐷=𝑉^𝛼_𝑛(𝑚+1,𝑥). (2.4) The first inequality follows from Condition 4 and from part (ii) of the induction hypothesis. The second inequality follows from Condition 3 and the inequality 𝐷≥0 which is a consequence of part (i) of the induction hypothesis.
Part (i): We have to show that 𝑉^𝛼_𝑛(𝑚,𝑥)≤𝑉^𝛼_𝑛(𝑚+1,𝑥),0≤𝑥_𝑗≤𝐾_𝑗,𝑗=1,…,𝐿,𝑉^𝛼_𝑛(𝑖,𝑥)≤𝑉^𝛼_𝑛(𝑖+1,𝑥),0≤𝑖≤𝑚−1,0≤𝑥_𝑗≤𝐾_𝑗,𝑗=1,…,𝐿. (2.5)(2.6) Inequality (2.5) is easily verified using (2.1) with 𝑖=𝑚 and part (ii) above 𝑉^𝛼_𝑛(𝑚,𝑥)≤𝑉^𝛼_𝑛(𝑃𝑀,𝑥)≤𝑉^𝛼_𝑛(𝑚+1,𝑥). (2.7) For 0≤𝑖≤𝑚−1 and 0≤𝑥_𝑗<𝐾_𝑗,𝑗=1,…,𝐿, we have that 𝑉^𝛼_𝑛(𝑖,𝑥)=minmin_{{}𝐽⊆1,…,𝐿𝑗∈𝐽}𝑐_𝑗(𝑖)+^𝐿_𝑗=1ℎ_𝑗𝑥_𝑗+∑_𝑗∉𝐽𝑑_𝑗−𝑥_𝑗⁺∑^𝐿_𝑗=1𝑑_𝑗𝐶+𝛼^𝑚+1_𝑟=0𝑝_𝑖𝑟𝑉^𝛼_𝑛−1𝑟,𝑥^′,𝑉^𝛼_𝑛(𝑃𝑀,𝑥)≤minmin_{{}𝐽⊆1,…,𝐿𝑗∈𝐽}𝑐_𝑗(𝑖+1)+^𝐿_𝑗=1ℎ_𝑗𝑥_𝑗+∑_𝑗∉𝐽𝑑_𝑗−𝑥_𝑗⁺∑^𝐿_𝑗=1𝑑_𝑗𝐶+𝛼^𝑚+1_𝑟=0𝑝_𝑖+1,𝑟𝑉^𝛼_𝑛−1𝑟,𝑥^′,𝑉^𝛼_𝑛(𝑃𝑀,𝑥)=𝑉^𝛼_𝑛(𝑖+1,𝑥). (2.8) The above inequality follows from Condition 1 and the inequality ^𝑚+1_𝑟=0𝑝_𝑖𝑟𝑉^𝛼_𝑛−1𝑟,𝑥^′≤^𝑚+1_𝑟=0𝑝_𝑖+1,𝑟𝑉^𝛼_𝑛−1𝑟,𝑥^′ (2.9) which is implied by part (i) of the induction hypothesis and Condition 6. Hence (2.6) has been proved for 𝑥_𝑗∈{0,…,𝐾_𝑗−1},𝑗=1,…,𝐿. Similarly, we obtain (2.6) if 𝑥_𝑗=𝐾_𝑗 for some values of 𝑗∈{1,…,𝐿}.

Since the state space 𝑆 is finite, and the state (0,0) is accessible from every other state under any stationary policy, it follows that there exist numbers 𝑣(𝑠), 𝑠∈𝑆 and a constant 𝑔 that satisfy the average-cost optimality equations (see Corollary 2.5 in [13], page 98). For the states (𝑖,𝑥),0≤𝑖≤𝑚,0≤𝑥_𝑗<𝐾_𝑗,𝑗=1,…,𝐿, the optimality equations take the following form: 𝑣(𝑖,𝑥)=minmin_{{}𝐽⊆1,…,𝐿𝑗∈𝐽}𝑐_𝑗(𝑖)+^𝐿_𝑗=1ℎ_𝑗𝑥_𝑗+∑_𝑗∉𝐽𝑑_𝑗−𝑥_𝑗⁺∑^𝐿_𝑗=1𝑑_𝑗𝐶−𝑔+^𝑚+1_𝑟=0𝑝_𝑖𝑟𝑣𝑟,𝑥^′,𝑣(𝑃𝑀,𝑥). (2.10) If 𝑥_𝑗=𝐾_𝑗 for some values of 𝑗∈{1,…,𝐿}, we must change in the above equations 𝑐_𝑗(𝑖) to 𝑐_𝑗̃(𝑖) for these values of 𝑗. In view of part (i) of the above lemma we have the following result.

Corollary 2.2. 𝑣(𝑖,𝑥)≤𝑣(𝑖+1,𝑥),0≤𝑖≤𝑚,0≤𝑥_𝑗≤𝐾_𝑗,𝑗=1,…,𝐿.

The following proposition gives a characterization of the form of the optimal policy.

Proposition 2.3. For fixed contents 𝑥₁,…,𝑥_𝐿 (0≤𝑥_𝑗≤𝐾_𝑗,𝑗=1,…,𝐿) of buffers 𝐵₁,…,𝐵_𝐿, there exists a critical working condition 𝑖^∗(𝑥₁,…,𝑥_𝐿) such that the policy that minimizes the expected long-run average cost per unit time initiates a preventive maintenance of the installation if and only if its working condition 𝑖∈{0,…,𝑚} is greater than or equal to 𝑖^∗(𝑥₁,…,𝑥_𝐿).

Proof. Suppose that for some fixed 𝑥=(𝑥₁,…,𝑥_𝐿) such that 0≤𝑥_𝑗<𝐾_𝑗, 1≤𝑗≤𝐿, the optimal policy initiates a preventive maintenance of the installation at state (𝑖,𝑥) where 𝑖∈{0,…,𝑚−1}. This implies that 𝑣(𝑃𝑀,𝑥)≤min_{{}𝐽⊆1,…,𝐿𝑗∈𝐽}𝑐_𝑗(𝑖)+^𝐿_𝑗=1ℎ_𝑗𝑥_𝑗+∑_𝑗∉𝐽𝑑_𝑗−𝑥_𝑗⁺∑^𝐿_𝑗=1𝑑_𝑗𝐶−𝑔+^𝑚+1_𝑟=0𝑝_𝑖𝑟𝑣𝑟,𝑥^′. (2.11)
To show that the optimal policy prescribes a preventive maintenance on the installation at state (𝑖+1,𝑥) it is enough to show that 𝑣(𝑃𝑀,𝑥)≤min_{{}𝐽⊆1,…,𝐿𝑗∈𝐽}𝑐_𝑗(𝑖+1)+^𝐿_𝑗=1ℎ_𝑗𝑥_𝑗+∑_𝑗∉𝐽𝑑_𝑗−𝑥_𝑗⁺∑^𝐿_𝑗=1𝑑_𝑗𝐶−𝑔+^𝑚+1_𝑟=0𝑝_𝑖+1,𝑟𝑣𝑟,𝑥^′. (2.12) From Conditions 1 and 6 and Corollary 2.2, it follows that the right-hand side of (2.12) is greater than or equal to the right-hand side of (2.11). Hence (2.11) implies (2.12). The same result is obtained similarly if 𝑥_𝑗=𝐾_𝑗 for some values of 𝑗∈{1,…,𝐿}.

Remark 2.4. In the above proposition, if for fixed contents 𝑥₁,…,𝑥_𝐿 of the buffers 𝑖^∗(𝑥₁,…,𝑥_𝐿)=𝑚+1, then the optimal policy never initiates a preventive maintenance of the installation whenever the buffer 𝐵_𝑗, 𝑗=1,…,𝐿, contains 𝑥_𝑗 units or raw material 𝑗.

2.2. Numerical Results

Example 2.5. Suppose that 𝐿=2,  𝑚=5,  𝑎_𝐼=0.6,  𝑏_𝐼=0.4,  𝑐_𝑝=10,  𝑐_𝑓=15,  𝐾₁=5,  𝐾₂=20,  ℎ₁=1,  ℎ₂=1,  𝑝₁=2,  𝑑₁=1,  𝑝₂=2,  𝑑₂=1,  𝑐₁(𝑖)=0.8(𝑖+1),  𝑐₁̃(𝑖) = 0.5(𝑖+1),  𝑐₂(𝑖) = 0.7(𝑖+1),  𝑐₂̃(𝑖) = 0.5(𝑖+1),  0≤𝑖≤𝑚,  and  𝑝_𝑖𝑟=1/(𝑚+2−𝑖),  0≤𝑖≤𝑚,  𝑖≤𝑟≤𝑚+1. It can be readily checked that these probabilities satisfy Condition 5. We computed the optimal policy if 𝐶 is equal to 0.5 or 15.5 by implementing the value-iteration algorithm (see Chapter 3 of Tijms [14]). Our numerical results verify the result of Proposition 2.3. In Table 1 we present the critical numbers 𝑖^∗(𝑥₁,𝑥₂),0≤𝑥₁≤5,0≤𝑥₂≤20. In each cell of this table the first number corresponds to 𝐶=0.5 and the second number corresponds to 𝐶=15.5. The minimum average cost was found to be 7.49 if 𝐶=0.5 which is, as expected, considerably smaller than the minimum average cost if 𝐶=15.5, which was found to be 11.63.

Table 1 

The critical numbers 𝑖^∗(𝑥₁,𝑥₂) for 𝐶=0.5,15.5.

We observe that for 𝑥₁∈{0,…,5} and 𝑥₂∈{0,…,20} the critical number that corresponds to 𝐶=0.5 is smaller than or equal to the critical number that corresponds to 𝐶=15.5. This is intuitively reasonable since if the cost due to production delay takes large values it seems disadvantageous to have all or most of the buffers empty when a maintenance is performed on the installation. Therefore in this case it seems preferable to initiate a preventive maintenance of the installation only if its degree of deterioration is relatively high. We also observe that when 𝐶=15.5 and buffer 𝐵₁ or buffer 𝐵₂ is empty, the optimal policy in most cases never initiates a preventive maintenance of the installation. For example 𝑖^∗(0,3)=6=𝑚+1. It can be also seen from the Table 1 that 𝑖^∗(𝑥₁,𝑥₂) is not a monotone function with respect to 𝑥₁∈{0,…,5} and with respect to 𝑥₂∈{0,…,20} for constant 𝑥₂ and 𝑥₁, respectively. Note that, when 𝑖<𝑖^∗(𝑥₁,𝑥₂),0≤𝑥₁≤5,0≤𝑥₂≤20, the value iteration algorithm gives the optimal action for the operation of the installation. For example, if 𝐶=0.5, 𝑥₁=0, 𝑥₂=18, the optimal action when the working condition of the installation is 3 is to transfer raw material 1 to buffer 1 (i.e., 𝐽={1}). If 𝐶=15.5, 𝑥₁=1, 𝑥₂=1, the optimal action when the working condition of the installation is 2 is to transfer raw material 1 to buffer 1 and raw material 2 to buffer 2 (i.e., 𝐽={1,2}).

Example 2.6. Suppose that 𝐿=2, 𝑚=15, 𝑎_𝐼=0.3, 𝑏_𝐼=0.2, 𝑐_𝑝=10, 𝑐_𝑓=15, ℎ₁=1, ℎ₂=1, 𝐶=80, 𝑝₁=4, 𝑑₁=2, 𝑝₂=3, 𝑑₂=2, 𝑐₁(𝑖)=1.5(𝑖+1), 𝑐₁̃(𝑖)=0.75(𝑖+1), 𝑐₂(𝑖)=2(𝑖+1), 𝑐₂̃(𝑖)=𝑖+1, 0≤𝑖≤𝑚, 𝑝_𝑖𝑟=1/(𝑚+2−𝑖), 0≤𝑖≤𝑚, 𝑖≤𝑟≤𝑚+1. In Table 2 below we present the minimum average cost 𝑔 obtained by the value-iteration algorithm for 𝐾₁∈{1,…,10} and 𝐾₂∈{5,10}.

Table 2 

The minimum average cost as 𝐾₁ or 𝐾₂ varies.

From the above table we see that as 𝐾₁ or 𝐾₂ increases, the minimum average cost decreases. This is intuitively reasonable because in this example it seems favourable to have buffers with large, capacities since the cost rate 𝐶 due to production delay is relatively large while the probabilities 𝑎_𝐼, 𝑏_𝐼 of successful maintenances and the storage cost rates ℎ₁,ℎ₂ are relatively small.

3. The Problem when the Production Unit Deteriorates Stochastically

We consider the same production-inventory system (see Figure 1) as the one introduced in the previous section with the following modifications: (i) the installation is always in operative condition while the production unit may experience a failure as time evolves and (ii) as long as the buffer 𝐵_𝑗, 𝑗=1,…,𝐿, is not empty and the production unit is in operative condition, the production unit may pull the raw material 𝑗 from buffer 𝐵_𝑗 at a constant rate of 𝑑_𝑗(>𝑝_𝑗) units of raw material 𝑗 per unit of time. When the buffer 𝐵_𝑗 is empty and the production unit is in operative condition, the production unit reduces its pull rate from 𝑑_𝑗 to 𝑝_𝑗.

We assume that the production unit is monitored at discrete equidistant time epochs 𝜏=0,1,… (say every day), and is classified into one of 𝑛+2 working conditions 0,…,𝑛+1. We suppose that working condition 𝑖 is better than working condition 𝑖+1. Working condition 0 means that the production unit is new (or functioning as good as new), while working condition 𝑛+1 means that the production unit does not function, and it cannot pull the materials from the buffers. The intermediate working conditions 1,…,𝑛 are operative. If the working condition at time epoch 𝜏 is 𝑖 then the working condition at time epoch 𝜏+1 will be 𝑟 with probability 𝑞_𝑖𝑟. The probability that the deterioration process of the production unit reaches eventually the failure state 𝑛+1 from any initial working condition 𝑖 is assumed to be nonzero. If at a time epoch 𝜏 the production unit is found to be at the failure state 𝑛+1, a corrective maintenance is compulsory. If it is found to be at any working condition 𝑖≤𝑛, a preventive maintenance is optional. The production unit does not operate when it is under preventive maintenance, and it does not pull any raw material from its buffer. When a preventive or a corrective maintenance is performed and the buffer 𝐵_𝑗, 𝑗=1,…,𝐿, contains 𝑥_𝑗 units of raw material j, the installation transfers min(𝑝_𝑗,𝐾_𝑗−𝑥_𝑗) units of raw material 𝑗 to buffer 𝐵_𝑗 during one unit of time. The preventive and corrective maintenance times (expressed in time units) are geometrically distributed with probability of success 𝑎_𝑃 and 𝑏_𝑃, respectively. Both maintenances bring the production unit to its perfect condition 0. The state space of the system is the set 𝑆={0,…,𝑛+1,𝑃𝑀}×{0,…,𝐾₁}×⋯×{0,…,𝐾_𝐿}, where PM represents the situation that the production unit is under a preventive repair. The possible actions are the same as the ones considered in the problem studied in Section 2 with the following modification: action 𝐽⊆{1,…,𝐿} is the action of pulling raw materials only from buffers 𝐵_𝑗, 𝑗∈𝐽. If at a time epoch 𝜏 the production unit is found to be at working condition 𝑖∈{0,…,𝑛}, then we may choose either the action of initiating a preventive maintenance or action 𝐽⊆{1,…,𝐿}.

The cost structure includes operating costs of the production unit, storage costs, maintenance costs, costs due to production delay, and penalty costs. The storage costs ℎ_𝑗, 𝑗∈{1,…,𝐿}, and the maintenance costs 𝑐_𝑝,𝑐_𝑓 are defined exactly in the same way as in the problem studied in the previous section. If the working condition of the production unit is 𝑖∈{0,…,𝑛} and the buffer 𝐵_𝑗, 𝑗∈{1,…,𝐿}, is nonempty (or empty), the cost of pulling 𝑑_𝑗 (or 𝑝_𝑗) units of raw material 𝑗 from buffer 𝐵_𝑗 during one unit of time is equal to 𝑐_𝑗(𝑖) (or 𝑐_𝑗̃(𝑖)). We assume that the cost rate due to production delay as long as a maintenance of the production unit lasts is equal to 𝐶>0. Therefore, if at a time epoch 𝜏 the action 𝐽⊆{1,…,𝐿} is selected and the content of buffer 𝐵_𝑗, 𝑗=1,…,𝐿, is 𝑥_𝑗∈{0,…,𝐾_𝑗}, then the cost due to production delay until time epoch 𝜏+1 is equal to 𝐶(∑_𝑗∉𝐽𝑑_𝑗+∑_𝑗∈𝐽(𝑑_𝑗−𝑝_𝑗−𝑥_𝑗)⁺)/∑^𝐿_𝑗=1𝑑_𝑗, where (𝑑_𝑗−𝑝_𝑗−𝑥_𝑗)⁺=max(𝑑_𝑗−𝑝_𝑗−𝑥_𝑗,0) is the unavailable quantity of raw material 𝑗∈𝐽 during one unit of time. A penalty cost per unit time which is equal to 𝑃_𝑗>0, 𝑗=1,…,𝐿, is also imposed for each unit or raw material 𝑗 that is not stored in buffer 𝐵_𝑗 during a corrective or a preventive maintenance of the production unit when the buffer 𝐵_𝑗 is full. This cost is due to the necessary labor for transferring and storing the raw material in another place until the completion of the maintenance. We assume that Conditions 1–5 on the cost structure that we introduced for the problem studied in the previous section are valid if we replace 𝑏_𝐼 with 𝑏_𝑃,𝑎_𝐼 with 𝑎_𝑃, and 𝑚 with 𝑛. We consider a discrete-time Markov decision process in which we aim to find a stationary policy which minimizes the expected long-run average cost per unit of time. Note that for 𝐿=1, the problem was studied in Pavitsos and Kyriakidis [2].

Since the state space 𝑆 is finite and the state (0,𝐾₁,…,𝐾_𝐿) is accessible from every other state under any stationary policy, it follows that there exist numbers 𝑤(𝑠), 𝑠∈𝑆 and a constant 𝑔 that satisfy the average-cost optimality equations. For the states (𝑖,𝑥), 0≤𝑖≤𝑛, 0<𝑥_𝑗≤𝐾_𝑗, 𝑗=1,…,𝐿, the optimality equations have the following form: 𝑤(𝑖,𝑥)=minmin_{{}𝐽⊆1,…,𝐿}𝐴(𝐽),𝑤(𝑃𝑀,𝑥), (3.1) where 𝐴(𝐽)=_𝑗∈𝐽𝑐_𝑗(𝑖)+^𝐿_𝑗=1ℎ_𝑗𝑥_𝑗+∑_𝑗∉𝐽𝑑_𝑗+∑_𝑗∈𝐽𝑑_𝑗−𝑝_𝑗−𝑥_𝑗⁺∑^𝐿_𝑗=1𝑑_𝑗𝐶+_𝑗∉𝐽𝑃_𝑗𝑝_𝑗+𝑥_𝑗−𝐾_𝑗⁺−𝑔+^𝑚+1_𝑟=0𝑞_𝑖𝑟𝑤𝑟,𝑥^′,𝑤(𝑃𝑀,𝑥)=^𝐿_𝑗=1ℎ_𝑗𝑥_𝑗+𝐶+^𝐿_𝑗=1𝑃_𝑗𝑝_𝑗+𝑥_𝑗−𝐾_𝑗⁺−𝑔+𝑎_𝑃𝑤0,min𝑥₁+𝑝₁,𝐾₁,…,min𝑥_𝐿+𝑝_𝐿,𝐾_𝐿+1−𝑎_𝑃𝑤𝑃𝑀,min𝑥₁+𝑝₁,𝐾₁,…,min𝑥_𝐿+𝑝_𝐿,𝐾_𝐿. (3.2)(3.3) If 𝑥_𝑗=0 for some values of 𝑗∈{1,…,𝐿}, we must change in (3.2) 𝑐_𝑗(𝑖) to 𝑐_𝑗(𝑖) for these values. Note that 𝑥^′ in (3.2) is a vector with 𝐿 components in which the 𝑗th component is equal to (𝑥_𝑗+𝑝_𝑗−𝑑_𝑗)⁺ if 𝑗∈𝐽, while, if 𝑗∉𝐽, it is equal to (𝑥_𝑗−𝑑_𝑗)⁺. It is possible to prove that 𝑤(𝑖,𝑥) ≤ 𝑤(𝑖+1,𝑥),  0≤𝑖≤𝑛,  0≤𝑥_𝑗≤𝐾_𝑗,  𝑗=1,…,𝐿, using the dynamic programming equation for the corresponding finite-horizon problem. The method is exactly the same as that used for the proof of Corollary 2.2 in the previous section and, therefore, we omit the details. An immediate consequence of the above monotonicity result is that the result of Proposition 2.3 is valid for the problem of the optimal preventive maintenance of the production unit.

3.1. Numerical Examples

Example 3.1. Suppose that 𝐿=2, 𝐾₁=7, 𝐾₂=10, 𝑛=10, 𝑎_𝑃=0.6, 𝑏_𝑃=0.4, 𝑐_𝑝=0.4, 𝑐_𝑓=0.8, ℎ₁=1, ℎ₂=2, 𝐶=0.5, 𝑃₁=1, 𝑃₂=1, 𝑝₁=1, 𝑑₁=2, 𝑝₂=1, 𝑑₂=2, 𝑐₁(𝑖)=2(𝑖+1), 𝑐₁̃(𝑖)=1.5(𝑖+1), 𝑐₂(𝑖)=3(𝑖+1), 𝑐₂̃(𝑖)=2.5(𝑖+1), 0≤𝑖≤𝑛, and 𝑝_𝑖𝑟=1/(𝑛+2−𝑖), 0≤𝑖≤𝑛,  𝑖≤𝑟≤𝑛+1. We computed the optimal policy by implementing the value-iteration algorithm. The minimum average cost was found to be 15.67. In Table 3, we present the critical numbers 𝑖^∗(𝑥₁,𝑥₂),  0 ≤ 𝑥₁ ≤ 7,  0 ≤ 𝑥₂ ≤ 10.
We can see from Table 3 that 𝑖^∗(𝑥₁,𝑥₂) is not a monotone function with respect to 𝑥₁∈{0,…,7} or with respect to 𝑥₂∈{0,…,10}, respectively. Note that, when 𝑖<𝑖^∗(𝑥₁,𝑥₂),  0≤𝑥₁≤7,  0≤𝑥₂≤10, the value-iteration algorithm gives the optimal action for the operation of the production unit. For example, if 𝑥₁=0,𝑥₂=9, the optimal action when the working condition of the installation is 𝑖∈{0,1,2,3,4} is to pull raw material 1 from buffer 𝐵₁ and raw material 2 from buffer 𝐵₂ (i.e., 𝐽={1,2}), while, when the working condition is 𝑖∈{5,6,7} the optimal condition is to pull only raw material 2 from buffer 𝐵₂ (i.e., 𝐽={2}).

Table 3 

The critical numbers 𝑖^∗(𝑥₁,𝑥₂),0≤𝑥₁≤7,0≤𝑥₂≤10.

Example 3.2. Suppose that 𝐿=2, 𝐾₁=5, 𝐾₂=15, 𝑛=10, 𝑎=0.6, 𝑏=0.4, 𝑐_𝑝=0.5, 𝑐_𝑓=0.8, 𝐶=0.5, 𝑃₁=1, 𝑃₂=1, 𝑝₁=1, 𝑑₁=2, 𝑝₂=1, 𝑑₂=2, 𝑐₁(𝑖)=2(𝑖+1), 𝑐₁̃(𝑖)=1.5(𝑖+1), 𝑐₂(𝑖)=3(𝑖+1),𝑐₂̃(𝑖)=2.5(𝑖+1), 0≤𝑖≤𝑛, and 𝑝_𝑖𝑟=1/(𝑛+2−𝑖), 0≤𝑖≤𝑛, 𝑖≤𝑟≤𝑛+1. In Figure 2 we present the graph of the minimum average cost 𝑔(ℎ₁) as a function of ℎ₁∈{1,…,10}, if ℎ₂=1, and the graph of the minimum average cost 𝑔(ℎ₂) as a function of ℎ₂∈{1,…,10}, if ℎ₁=1. We observe that 𝑔(ℎ_𝑖), 𝑖=1,2, increases as ℎ_𝑖 increases. The increase of the minimum average cost is more intense when ℎ₂ increases. This can be explained by the fact that the capacity of buffer 𝐵₂ is considerably greater than the capacity of buffer 𝐵₁.

Figure 2 

The minimum average cost 𝑔(ℎ1) and 𝑔(ℎ2).

4. Conclusions and Future Research

We presented two discrete-time Markov decision models for the optimal condition-based preventive maintenance of a production system which consists of two machines and 𝐿 intermediate buffers. The first machine transfers 𝐿 different raw materials to the buffers, and the second machine draws the raw materials from the buffers. The second machine is considered to be a production unit that assembles the raw materials in order to produce the final product. In the first model, it is assumed that only the first machine deteriorates stochastically over time while the production unit is always in operative condition. It is possible to monitor the first machine at discrete equidistant time epochs and to classify it into one working condition that describes its level of deterioration. If the first machine is in failed condition a corrective maintenance must be commenced; otherwise a preventive maintenance may be performed or the action of transferring raw materials to any subset of the set of 𝐿 buffers may be selected. The maintenances bring the first machine to its perfect condition. In the second model it is assumed, that only the production unit deteriorates over time, and the first machine is always in operative machine. It is possible to determine the level of deterioration of the production unit after inspecting it at discrete equidistant time epochs. If the production unit is in failed condition a corrective maintenance must be started; otherwise a preventive maintenance may be initiated or the action of pulling the raw materials from any subset of the set of 𝐿 buffers may be selected. Both maintenances bring the production unit to its perfect condition.

In both models we considered the problem of determining the policy that minimizes the expected long-run average cost per unit time. If the maintenance times are geometrically distributed we proved that, in both models, the optimal policy is of control-limit type, that is, for fixed contents of the buffers it prescribes a preventive maintenance of the first machine or the production unit if and only if its degree of deterioration exceeds some critical level. The proof was achieved through the corresponding finite-horizon problem.

A topic for future research could be a more complicated problem in which the first machine transfers the raw materials to the buffers and the production unit draws them from the buffers in a random manner. Another topic for future research could be the study of the maintenance problems that we would have if the maintenance times are not geometrically distributed but follow some general distributions with suitable conditions.

Acknowledgment

The authors would like to thank an anonymous referee for helpful comments.