Abstract

This paper revisits the classical discrete-time stationary inventory model. A new proof, based on the theory of quasivariational inequality (QVI), of the optimality of (𝑠,𝑆) policy is presented. This proof reveals a number of interesting properties of the optimal cost function. Further, the proof could be used as a tutorial for applications of QVI to inventory control.

1. Introduction

Consider an inventory model which consists in controlling the level of stock of a single product where the demands 𝐷1,𝐷2, for the product in periods 1,2, are independently and identically distributed (i.i.d) random variables with density function 𝜓, and finite mean 𝜇<.

Assume that at the beginning of each period the system is reviewed and we are allowed to increase the level of stock to any level we wish. Orders are assumed to be delivered immediately.

Let 𝑓 be a real-valued function representing the holding and shortage cost with 𝑓(0)=0 and 𝑓(𝑥)>0 for 𝑥0. The cost 𝑐(𝑥) of ordering an amount 𝑥 is given by 𝑐(𝑥)=𝑘+𝑐𝑥,𝑥>0,0,𝑥=0,(1.1) where 𝑐 is the unit cost of the item and 𝑘 is the set-up cost (𝑐>0,𝑘>0). Costs are assumed to be additive and geometrically discounted at a rate 𝛼,0<𝛼<1, and that unmet demand is completely backlogged.

An admissible replenishment policy consists of a sequence (𝑡𝑖,𝜉𝑖),𝑖=1,, where 𝑡𝑖 represents the 𝑖th time of ordering and 𝜉𝑖>0 represents the quantity ordered at time 𝑡𝑖. Write𝒱𝑛={(𝑡𝑖,𝜉𝑖𝒱),𝑖=1,𝑛},=lim𝑛𝒱𝑛=𝒱.(1.2)

Let 𝑥(𝑛) denote the level of stock at time 𝑛,𝑛=0,1,, and let 𝑛=𝜎{𝑥(𝑠),𝑠𝑛} be the 𝜎-algebra generated by the history of the inventory level up to time 𝑛. Assume that for each 𝑛,𝒱𝑛 is 𝑛-measurable. Then for a given initial inventory level 𝑥 and an ordering policy 𝒱, the infinite horizon discounted cost is defined by𝑦(𝑥,𝒱)=𝐸𝒱𝑡=0𝛼𝑡𝑓(𝑥(𝑡))+𝑛𝑖=1𝛼𝑡𝑖𝑐(𝜉𝑖),(1.3)where the expectation is taken with respect to all possible realizations of the process 𝑥(𝑡) under policy 𝒱. Set𝑦(𝑥)=inf𝒱𝑦(𝑥,𝒱).(1.4)The objective is to find an admissible policy 𝒱 such that 𝑦(𝑥,𝒱)=𝑦(𝑥).

Scarf [1] considered a finite horizon version of the problem described in (1.3). He showed using dynamic programming that if the one period expected holding plus shortage cost function is convex, then the optimal policy for period 𝑛 is an (𝑠𝑛,𝑆𝑛) policy. The principal tool used by Scarf was a concept of 𝐾-convexity which he introduced in the same paper. Subsequently, Iglehart [2] extended Scarf's result to the infinite horizon case by showing that the property of 𝐾-convexity holds for the infinite period stationary model. Veinott [3] replaced the requirement of the convexity of the one period expected holding plus shortage cost by a quasiconvexity requirement and added other conditions. Again using dynamic programming, he showed the optimality of an (𝑠,𝑆) policy.

In this paper, we approach the problem of determining the optimal inventory policy as an impulse control problem, the theory which has been developed by Bensoussan and Lions [4]. Under this theory, the Bellman equation of dynamic programming for the inventory problem leads to a set of quasivariational inequalities (QVIs) whose solution leads to the optimal inventory policy. This approach leads to a new proof of the result which does not use 𝐾-convexity and is based on the examination of some properties of an integral equation. Previous applications of QVI to inventory control revolved around diffusion processes from which the machinery needed to prove optimality of (𝑠,𝑆) policy was not simple. This paper we hope can serve as a tutorial of applications of QVI to inventory control. Readers interested in applications of QVI to inventory control may consult [58].

Before we embark on the proof we will first formulate the problem described in (1.3) as a QVI.

Recall that 𝑥(𝑡) refers to the level of stock at time 𝑡, and consider all possible actions at time 𝑡.

(i) If no order is made, then it follows from (1.3) and (1.4) that𝑦(𝑥(𝑡))𝐸[𝑓(𝑥(𝑡)𝐷)]+𝛼𝐸[𝑦(𝑥(𝑡)𝐷)](1.5)or𝑦(𝑥(𝑡))𝛼𝐸[𝑦(𝑥(𝑡)𝐷)]𝐸[𝑓(𝑥(𝑡)𝐷)],(1.6)where 𝐷 refers to the demand in a period.

(ii) If an order of size 𝜉 is made, then the level of stock jumps from 𝑥(𝑡) to 𝑥(𝑡)+𝜉, and𝑦(𝑥(𝑡))𝑘+inf𝜉>0[𝑐𝜉+𝑦(𝑥(𝑡)+𝜉)].(1.7)For 𝑥, define the operators 𝐴 and 𝑀 by(𝐴𝑦)(𝑥)=𝑦(𝑥)𝛼𝐸[𝑦(𝑥𝐷)],(𝑀𝑦)(𝑥)=𝑘+inf𝜉>0[𝑐𝜉+𝑦(𝑥+𝜉)].(1.8)It follows that the problem of finding the optimal solution to (1.3) reduces to solving the following QVI problem:𝐴𝑦𝐹,𝑦𝑀𝑦,(𝐴𝑦𝐹)(𝑦𝑀𝑦)=0,(1.9)where𝐹(𝑥)=𝐸[𝑓(𝑥𝐷)].(1.10)To solve the QVI given in (1.9), we examine an integral equation problem related to the QVI. This is done in Section 2. The properties obtained of the integral equation are then used to show the optimality of (𝑠,𝑆) policy in Section 3.

2. An Integral Equation Problem

Consider the space of continuous functions 𝐶(). Assume that we are given a nonnegative function in 𝐶().

Further, suppose that

(A1)there exists 𝛾,<𝛾<, such that is decreasing on (,𝛾] and nondecreasing on [𝛾,);(A2)(𝑥), as |𝑥|.

For 𝐿 in 𝐶(), define the convolution operator by(𝜓𝐿)(𝑥)=0𝐿(𝑥𝑡)𝜓(𝑡)𝑑𝑡.(2.1)Now, consider the integral equation1𝐿(𝑥)𝛼(𝜓𝐿)(𝑥)=(𝑥),𝑥>𝑠,𝐿(𝑥)=1𝛼(𝑠),𝑥𝑠.(2.2)Here, 𝑠<𝛾, and it is a free parameter.

Under assumption that is in 𝐶(), the integral equation (2.2) has a unique solution in 𝐶() (see [9]). Let 𝐿𝑠 denote this solution. In what follows, there is a list of properties of 𝐿𝑠 which will prove useful in showing the optimality of the (𝑠,𝑆) policy: 𝐿𝑠1(𝑥)=1𝛼(𝑠)𝑥𝑠,(2.3)𝐿𝑠isdecreasingon(𝑠,𝛾],(2.4)𝐿𝑠1(𝑥)1𝛼(𝛾)𝑥in,(2.5)𝐿𝑠(𝑥)as𝑥.(2.6)

Property (2.3) is the boundary condition of (2.2).

Proof of Property (2.4). To show (2.4) argue by contradiction. Assume that 𝐿𝑠 initially does not decrease. In other words, there exists Δ, 𝑠<Δ<𝛾 such that 𝐿𝑠 is nondecreasing on [𝑠,Δ). It follows that for 𝑥 and 𝑡 satisfying 𝑠𝑥Δ and 𝑡0, 𝐿𝑠(𝑥)𝐿𝑠(𝑥𝑡)0; but (2.2) gives(1𝛼)𝐿𝑠(Δ)+𝛼0(𝐿𝑠(Δ)𝐿𝑠(Δ𝑡))𝜓(𝑡)𝑑𝑡=(Δ).(2.7)Therefore, (1𝛼)𝐿𝑠(Δ)(Δ) or 𝐿𝑠(Δ)(1/(1𝛼))(Δ). This leads to 𝐿𝑠(Δ)<(1/(1𝛼))(𝑠) by Assumption (A1). Property (2.3) then implies that 𝐿𝑠(Δ)<𝐿𝑠(𝑠), which leads to a contradiction.
To complete the proof, we again argue by contradiction and assume that there exists 𝜂, and Δ, 𝑠<𝜂<Δ<𝛾, such that 𝐿𝑠 is decreasing on [𝑠,𝜂] and nondecreasing on [𝜂,Δ). Let 𝑥 be such that 𝜂<𝑥<Δ, and 𝐿𝑠(𝑥)<𝐿𝑠(𝑠). We claim that for 𝑡0, 𝐿𝑠(𝑥)𝐿𝑠(𝑥𝑡)𝐿𝑠(𝜂)𝐿𝑠(𝜂𝑡).(2.8)We have by (2.2)(1𝛼)𝐿𝑠(𝑥)+𝛼0(𝐿𝑠(𝑥)𝐿𝑠(𝑥𝑡))𝜓(𝑡)𝑑𝑡=(𝑥).(2.9)Now, use (2.8) and the fact that 𝐿𝑠(𝑥)𝐿𝑠(𝜂) to get from (2.9) that(𝑥)(1𝛼)𝐿𝑠(𝜂)+𝛼0(𝐿𝑠(𝜂)𝐿𝑠(𝜂𝑡))𝜓(𝑡)𝑑𝑡=(𝜂),(2.10)but (𝑥)<(𝜂) by Assumption (A1). This leads to a contradiction. This ends the proof.

Proof of Property (2.5). Assume that Property (2.5) is not true. Using Property (2.4) and the fact that 𝐿𝑠 is continuous, let 𝑥 be the first (smallest) solution of 𝐿𝑠(𝑥)=(1/(1𝛼))(𝛾). Clearly, 𝑥>𝑠, and 𝐿𝑠 attains its minimum at 𝑥 on (,𝑥]. Using (2.2), Assumption (A1), and recalling that 𝜓 is a density function, we get𝐿𝑠(𝑥)=(𝑥)+𝛼0𝐿𝑠(𝑥𝑡)𝜓(𝑡)𝑑𝑡>(𝑥)+𝛼𝐿𝑠(𝑥).(2.11)Therefore, 𝐿𝑠(𝑥)>(1/(1𝛼))(𝛾). This leads to a contradiction. Whence Property (2.4) holds.

Proof of Property (2.6). Using (2.2), we get𝐿𝑠(𝑥)=(𝑥)+𝛼0𝐿𝑠𝛼(𝑥𝑡)𝜓(𝑡)𝑑𝑡(𝑥)+1𝛼(𝛾).(2.12)The last inequality follows from Property (2.5). The result is then immediate from Assumption (A2) by taking the limit as 𝑥. This completes the proof.

We will next present further properties of 𝐿𝑠.

Theorem 2.1. For a given 𝑠<𝛾, there exists an 𝑆(𝑠),𝛾<𝑆(𝑠)<, which minimizes 𝐿𝑠(𝑥) for 𝑥 in .

Proof. The proof follows from Properties (2.3)–(2.6) and the continuity of 𝐿𝑠.

We remark here that 𝑆(𝑠) may not be unique.

For 𝑠<𝛾, define𝐾(𝑠)=𝐿𝑠(𝑠)min𝑥𝐿𝑠(𝑥).(2.13)Clearly, 𝐾 is a well-defined function on (,𝛾) and is nonnegative.

Lemma 2.2. The function 𝐾 is decreasing in 𝑠.

Proof. Let 𝑡<𝑠<𝛾, and for 𝑥 in , define𝐷(𝑥)=𝐿𝑡(𝑥)𝐿𝑠(𝑥).(2.14)It is easy to show that 𝐷 is a solution of (2.2) with the right-hand side changed to 𝑔(𝑥)=(𝑡)(𝑠),𝑥𝑡,(𝑥)(𝑠),𝑡𝑥𝑠,0,𝑥>𝑠.(1) The function 𝑔 is constant on (,𝑡], decreasing on [𝑡,𝑠], and is equal to zero for 𝑥>𝑠. Therefore, a similar argument to that used to show properties (2.4) and (2.5) shows that 𝐷 is decreasing on and is nonnegative. Since 𝑆(𝑠)>𝑠>𝑡, it follows from Theorem 2.1 that𝐿𝑡(𝑡)𝐿𝑠(𝑡)𝐿𝑡(𝑆(𝑠))𝐿𝑠(𝑆(𝑠))𝐿𝑡(𝑆(𝑡))𝐿𝑠(𝑆(𝑠)),(2.15)but 𝐿𝑠(𝑡)=𝐿𝑠(𝑠). Therefore, 𝐿𝑡(𝑡)𝐿𝑡(𝑆(𝑡))𝐿𝑠(𝑠)𝐿𝑠(𝑆(𝑠)), which leads to the required result.

Lemma 2.3. The function 𝐾 is continuous.

Proof. Fix 𝜖>0. Since is continuous, there exists 𝛿>0 such that |(𝑠)(𝑡)|<((1𝛼)/2)𝜖 whenever |𝑠𝑡|<𝛿. Pick 𝑡<𝛾 such that |𝑠𝑡|<𝛿. To make things simple, assume 𝑡<𝑠. It was shown in the proof of Theorem 2.1 that 𝐿𝑡(𝑆(𝑡))𝐿𝑠(𝑆(𝑠))𝐿𝑡(𝑡)𝐿𝑠(𝑠). Now, use the definition of 𝐿𝑡(𝑡) and 𝐿𝑠(𝑠) to get that 𝐿𝑡(𝑆(𝑡))𝐿𝑠(𝑆(𝑠))(1/(1𝛼))((𝑡)(𝑠))<𝜖/2; but|𝐾(𝑡)𝐾(𝑠)||𝐿𝑡(𝑆(𝑡))𝐿𝑠(𝑆(𝑠))|+|𝐿𝑡(𝑡)𝐿𝑠𝜖(𝑠)|<2+𝜖2=𝜖.(2.16)Therefore, 𝐾 is continuous.

Lemma 2.4. (i) 𝐾(𝑠)0as𝑠𝛾.
(ii) 𝐾(𝑠)as𝑠.

Proof. (i) Recall that 𝐾(𝑠)0 and that 𝐿𝑠(𝑥)(1/(1𝛼))(𝛾) by Property (2.5). In particular, 𝐿𝑠(𝑆(𝑠))(1/(1𝛼))(𝛾). It follows that 𝐿𝑠(𝑠)(1/(1𝛼))(𝛾)𝐾(𝑠)0 or (1/(1𝛼))((𝑠)(𝛾))𝐾(𝑠)0. The result is then immediate from the continuity of and Assumption (A2) by letting 𝑠𝛾.
(ii) Define𝐿𝑠(𝑥)=𝑔𝑠(𝑥),(2.17) where 𝑔𝑠𝛼(𝑥)=(𝑥)+11𝛼(𝑠),𝑥>𝑠,1𝛼(𝑠),𝑥𝑠.(2.18) The function 𝐿𝑠 is decreasing on (,𝛾]. Therefore, for 𝑥 in (,𝛾],𝐿𝑠(𝑥)<0𝐿𝑠(𝑥𝑡)𝜓(𝑡)𝑑𝑡.(2.19)Write𝐺𝑠𝐿=𝑠𝐿𝑠.(2.20)It is not difficult to show that 𝐺𝑠 satisfies the following:𝐺𝑠(𝑥)𝛼(𝜓𝐺𝑠𝛼)(𝑥)=𝐿1𝛼(𝑠)𝛼(𝜓𝑠𝐺)(𝑥),𝑥>𝑠,𝑠(𝑥)=0,𝑥𝑠.(2.21)
Again, a similar argument used to prove Property (2.4) can be used to show that 𝐺𝑠 is increasing on (,𝛾). Therefore, 𝐺𝑠(𝛾)𝐺𝑠(𝑠)=0. This in turn leads to 𝐿𝑠(𝛾)𝐿𝑠(𝛾). Now,𝐾(𝑠)=𝐿𝑠(𝑠)𝐿𝑠(𝑆(𝑠))𝐿𝑠(𝑠)𝐿𝑠(𝛾𝐿)𝑠𝐿(𝑠)𝑠(𝛾).(2.22)The right-hand side of (2.22) is equal to (𝑠)(𝛾) with limit as 𝑠. This completes the proof of the lemma.

Now consider the problem of finding a solution 𝑠 to the problem𝐾(𝑠)=𝑘.(2.23)

Theorem 2.5. There exits a unique number 𝑠<𝛾 such that 𝐾(𝑠)=𝑘.

Proof. The proof is immediate from Theorem 2.1 and Lemmas 2.22.4.

3. Optimality of (𝑠,𝑆) Policy

Recall the definitions of the functions 𝑦 and 𝐹 in (1.9) and let𝐿(𝑥)=𝑦(𝑥)+𝑐𝑥,(𝑥)=(1𝛼)𝑐𝑥+𝐹(𝑥)+𝛼𝑐𝜇.(3.1)It is an easy exercise to see that for 𝑥>𝑠,𝐴𝑦=𝐹 is equivalent to 𝐴𝐿=, which is the integral equation (2.2) for 𝑥>𝑠.

Assume that satisfies Assumptions (A1) and (A2) and let 𝑠<0 be the unique solution of (2.23). This value of 𝑠 leads to a value of 𝑆(𝑠) which minimizes 𝐿𝑠 (this may not be unique). Further, let 𝑆 denote the generic value of 𝑆(𝑠). We will next show that the policy which asserts that if the level of stock 𝑥<𝑠, order up to level 𝑆: else do not order, solves the QVI given by (1.9). The proof of optimality relies on the concept of non-𝑘-decreasing functions which may be found in [10, page 137].

Definition 3.1. A function 𝑣 is non-𝑘-decreasing if 𝑥𝑦 implies that𝑣(𝑥)𝑘+𝑣(𝑦).(3.2)

Note that the concept of non-𝑘-decreasing is weaker than the concept of 𝑘-convexity which is a standard tool for showing optimality of (𝑠,𝑆) policy; see [10] for more details.

Our objective is to show that the function 𝐿𝑠 is non-𝑘-decreasing. Note from Properties (2.3)–(2.6) that 𝐿𝑠 is constant on (,𝑠], then decreases at least down to 𝛾, reaches its minimum at some 𝑆, and eventually goes to as 𝑥. Non-𝑘-decreasing means that the function 𝐿𝑠 cannot have a drop bigger than 𝑘 beyond 𝑆. LetΔ=min{𝑥>𝛾,𝐿𝑠(𝑥)=𝐿𝑠(𝑠)}.(3.3)

Note that Δ exists and is unique. Set𝒦(𝑠)=𝐿𝑠(𝑠)min𝑥Δ𝐿𝑠(𝑥).(3.4)

Theorem 3.2. For 𝑠<𝛾, the solution 𝐿𝑠 of (2.2) satisfies 𝐿𝑠(𝑥)𝐿𝑠(𝑦)𝒦(𝑠)𝑥𝑦.(3.5)

Proof. The proof is by contradiction and only a sketch of the proof will be given. Consider the set(𝒦(𝑠))={𝑥Δ,𝐿𝑠(𝑥)𝐿𝑠(𝑦)>𝒦(𝑠),forsome𝑦𝑥}.(3.6)If (𝑠) is empty, there is nothing to prove and theorem is true. Assume that (𝒦(𝑠)) is not empty, in which case it can be shown that there exists a triplet (𝑆1,𝑆2,𝑆3) such that 𝛾<𝑆1<Δ<𝑆2<𝑆3 such that on the interval [𝑠,𝑆3],𝐿𝑠 attains its minimum at 𝑆1, and its maximum at 𝑆2 (as shown in Figure 1) with𝐿𝑠(𝑠)𝐿𝑠(𝑆1)=𝐿𝑠(𝑆2)𝐿𝑠(𝑆3)=𝒦(𝑠).(3.7)We will next show that this cannot happen. Using (2.2), we get𝐿𝑠(𝑆2)=(𝑆2)+𝛼0𝐿𝑠(𝑆2𝑡)𝜓(𝑡)𝑑𝑡,(3.8)𝐿𝑠(𝑆3)=(𝑆3)+𝛼0𝐿𝑠(𝑆3𝑡)𝜓(𝑡)𝑑𝑡.(3.9)It follows that for 𝑡0,𝐿𝑠(𝑆2𝑡)𝐿𝑠(𝑆3𝑡)𝒦(𝑠).(3.10)Using (3.7)–(3.9), we get𝒦(𝑠)=(𝑆2)(𝑆3)+𝛼0(𝐿𝑠(𝑆2𝑡)𝐿𝑠(𝑆3𝑡))𝜓(𝑡)𝑑𝑡(𝑆2)(𝑆3)+𝛼𝒦(𝑠).(3.11)This leads to (1𝛼)𝒦(𝑠)(𝑆2)(𝑆3)<0 since is increasing on (𝛾,) by Assumption (A1). Therefore, we have a contradiction that 𝒦(𝑠)>0. Therefore, 𝑅(𝑠) is empty. This completes the proof.

As a corollary of Theorem 3.2, we have the following.

Corollary 3.3. For 𝑠<𝛾, a solution 𝑆(𝑠) of the equation 𝐿𝑠(𝑠)𝐿𝑠(𝑆(𝑠))=𝑘 is a global minimum of the function 𝐿𝑠.

Note that the proof of Theorem 3.2 revealed that the value of 𝑆 belongs to some interval (𝛾,Δ). Further, the results of the previous section should make a numerical search for the value (𝑠,𝑆) an easy exercise.

Theorem 3.4. The function 𝐿𝑠 defined from the pair (𝑠,𝑆) which solves (2.23) solves (1.9).

Proof. We need to show that 𝑦<𝑀𝑦 for 𝑥𝑠 and 𝐴𝑦𝐹 for 𝑥𝑠. To show 𝐴𝑦𝐹 for 𝑥𝑠, let 𝑥𝑠; therefore, 𝐿𝑠(𝑥)=𝐿𝑠(𝑠), and 𝐴𝑦(𝑥)𝐹(𝑥) is equivalent to 𝐴𝐿𝑠(𝑥); but 𝐿𝑠(𝑠)=(1/(1𝛼))(𝑠). Therefore, 𝐴𝐿𝑠(𝑠)(𝑥) is equivalent to (𝑠)(𝑥), which is true since is decreasing for 𝑥𝛾.
To show that 𝑦𝑀𝑦(𝑥) for 𝑥𝑠, note that𝑀𝑦(𝑥)=𝑘+𝑐(𝑆𝑥)+𝑦(𝑆)for𝑠𝑥𝑆,𝑀𝑦(𝑥)=𝑘+𝑦(𝑥)for𝑥𝑆.(3.12)If 𝑠𝑥𝑆, then 𝑦(𝑥)𝑀𝑦(𝑥) can be written as 𝐿𝑠(𝑥)𝑘+𝐿𝑠(𝑆). This is true since 𝐿𝑠 is non-𝑘-decreasing. This completes the proof.

It is worth noting that Assumption (A1) is equivalent to saying that is quasiconvex. Also, Assumption (A2) can be weakened by replacing it by lim|𝑥|(𝑥)>(𝛾)+𝑘. The limit when 𝑥 can be inferred from (2.22) and the limit when 𝑥 can be obtained from the proof of Property (2.6). The optimality of (𝑠,𝑆) policy remains true.

In this short paper, an alternative proof of the optimality of (𝑠,𝑆) policy was given. The proof also revealed that finding optimal values of (𝑠,𝑆) is a simple exercise in numerical analysis. It is hoped that this new proof will lead to new insights in the examination of some stochastic inventory models.

Acknowledgments

The author would like to thank Michael Johnson for useful discussions on the topic of quasivariational inequalities. He also benefited from the comments of two anonymous referees.