Some Results on Bellman Equations of Optimal Production Control in a Stochastic Manufacturing System

Abstract

The paper studies the production inventory problem of minimizing the expected discounted present value of production cost control in a manufacturing system with degenerate stochastic demand. We establish the existence of a unique solution of the Hamilton-Jacobi-Bellman (HJB) equations associated with this problem. The optimal control is given by a solution to the corresponding HJB equation.

1. Introduction

Many manufacturing enterprisers use a production inventory system to manage fluctuations in consumer demand for the product. Such a system consists of a manufacturing plant and a finished goods warehouse to store those products which are manufactured but not immediately sold. The advantages of having products in inventory are as follows: first, they are immediately available to meet demand; second, by using the warehouse to store excess production during low demand periods to be available for sale during high demand periods. This usually permits the use of a smaller manufacturing plant than would otherwise be necessary, and also reduces the difficulties of managing the system.

We are concerned with the optimization problem to minimize the expected discounted cost control of production planning in a manufacturing systems with degenerate stochastic demand: 𝐽(𝑝)=𝐸[^∞₀𝑒^−𝜌𝑡{ℎ(𝑥_𝑡)+𝑝²_𝑡}𝑑𝑡] (1.1) subject to the dynamics of the state equation which says that the inventory at time 𝑡 is increased by the production rate and decreased by the demand rate can be written according to 𝑑𝑥_𝑡=(𝑝_𝑡−𝑦_𝑡)𝑑𝑡,𝑥₀=𝑥,𝑥>0,0≤𝑝_min≤𝑝_max, (1.2) and the demand equation with the production rate is described by the Brownian motion 𝑑𝑦_𝑡=𝐴𝑦_𝑡𝑑𝑡+𝜎𝑦_𝑡𝑑𝑤_𝑡,𝑦₀=𝑦,𝑦>0, (1.3) in the class 𝒫 of admissible controls of production processes 𝑝_𝑡 with nonnegative constant 𝑝_𝑡≥0 (1.4) defined on a complete probability space (Ω,ℱ,𝑃) endowed with the natural filtration ℱ_𝑡 generated by 𝜎(𝑤_𝑠,𝑠≤𝑡) carrying a one-dimensional standard Brownian motion 𝑤_𝑡, 𝑥_𝑡 is the inventory level for production rate at time 𝑡 (state variable), 𝑦_𝑡 is the demand rate at time 𝑡, 𝑝_𝑡 is the production rate at time 𝑡 (control variable), 𝜌>0 is the constant nonnegative discount rate, 𝐴 is the nonzero constant, 𝜎 is nonzero constant diffusion coefficient, 𝑥₀ is the initial value of inventory level, and 𝑦₀ is the initial value of demand rate.

This optimization control problem of production planning in manufacturing systems has been studied by many authors like Fleming et al. [1], Sethi and Zhang [2], Sprzeuzkouski [3], Hwang et al. [4], Hartl and Sethi [5], and Feichtinger and Hartl [6]. The Bellman equation associated with production inventory control problem is quite different from them and it is treated by Bensoussan et al. [7] for the one-dimensional manufacturing systems with the unbounded control region. Generally speaking, the similar type of linear control problems has been investigated for the stochastic deferential systems with invariant measures like Bensoussan [8], and Borkar [9]. The works of Bensoussan and Frehse [10] Da Prato and Ichikawa [11] on the Bellman equation of ergodic control without convex and polynomial growth hypothesis and the linear quadratic case are done for the linear ergodic control problem. This type of optimization problem has been studied also by Morimoto and Kawaguchi [12] for renewable resources as well as Baten and Sobhan Miah [13] for one-sector neoclassical growth model with the CES function. The optimality can be shown by an extension of the results given in Fujita and Morimoto [14], and for another setting of optimal control in manufacturing systems they are available in Morimoto and Okada [15] and Sethi et al. [16]. These papers treat the cases with bounded control regions. On the contrary, our control region is unbounded as in (1.4).

The purpose of the paper is to give an optimal production cost control by an existence unique solution associated with the two-dimensional HJB equation. We apply the technique of dynamic programming principle [17] for the Riccati-based solution of the reduced (one-dimensional) HJB equation corresponding to production inventory control problem. This paper is organized as follows. In Section 2 by the principle of optimality Bellman [17], we have obtained the HJB equation and then the two-dimensional HJB equation has been reduced to one-dimensional second-order differential equation. We have derived the dynamics of inventory-demand ratio that evolves according to stochastic neoclassical differential equation through Itô’s lemma. We have finally found the Riccati-based solution of production inventory control problem that is satisfied by the value function of this optimization problem. In Section 3 we have established the properties of the value function and have shown the existence of an unique solution associated with the reduced (one-dimensional) Hamilton-Jacobi-Bellman (HJB) equation. Finally in Section 4 we present an application to production control of optimization problem (1.1) subject to (1.2) and (1.3).

2. Riccati-Based Solution of Hamilton-Jacobi Bellman Equation

2.1. The Hamilton-Jacobi-Bellman Equation

Suppose 𝑢(𝑥,𝑦,𝑡)∶𝐑^𝐧×𝐑^𝐧×𝐑→𝐑 is a function whose value is the minimum value of the objective function of the production inventory control problem for the manufacturing system given that we start it at time 𝑡 in state 𝑥, and 𝑦. That is, 𝑢(𝑥,𝑦,𝑡)=inf_𝑝𝐽(𝑝), (2.1) where the value function 𝑢 is finite valued and twice continuously differentiable on (0,∞). We initially assume that 𝑢(𝑥,𝑦,𝑡) exists for all 𝑥, 𝑦, and 𝑡 in the relevant ranges.

Since (1.2) and (1.3) is a scalar equation, the subscript 𝑡 here means only time 𝑡. Thus, 𝑥 and 𝑦 will not cause any confusion and, at the same time, will eliminate the need of writing many parentheses. Thus, 𝑑𝑤_𝑡 is a scalar.

To solve the problem defined by (1.1), (1.2), and (1.3), let 𝑢(𝑥,𝑦,𝑡), known as the value function, be the expected value of the objective function (1.1) from 𝑡 to infinity, when an optimal policy is followed from 𝑡 to infinity, given 𝑥_𝑡=𝑥,𝑦_𝑡=𝑦. Then by the principle of optimality [17], 𝑢(𝑥,𝑦,𝑡)=min_𝑝[{ℎ(𝑥_𝑡)+𝑝²_𝑡}𝑑𝑡+𝑢(𝑥+𝑑𝑥_𝑡,𝑦+𝑑𝑦_𝑡,𝑡+𝑑𝑡)]. (2.2) We assume that 𝑢(𝑥,𝑦,𝑡) is a continuously differentiable of its arguments. By Taylor’s expansion, we have 𝑢(𝑥+𝑑𝑥_𝑡,𝑦+𝑑𝑦_𝑡,𝑡+𝑑𝑡)=−𝜌𝑢(𝑥,𝑦,𝑡)𝑑𝑡+𝑢_𝑦𝑑𝑦_𝑡+𝑢_𝑥𝑑𝑥_𝑡+12𝑢_𝑦𝑦(𝑑𝑦_𝑡)²+higherorderterms. (2.3) From (1.2), we can formally write (𝑑𝑥_𝑡)²=(𝑝)²(𝑑𝑡)²+(𝑦)²(𝑑𝑡)²−2𝑝𝑦(𝑑𝑡)²,(𝑑𝑦_𝑡)²=(𝐴𝑦)²(𝑑𝑡)²+(𝜎𝑦)²(𝑑𝑤_𝑡)²+2(𝐴𝑦)(𝜎𝑦)𝑑𝑤_𝑡𝑑𝑡,𝑑𝑥_𝑡𝑑𝑡=𝑝(𝑑𝑡)²−𝑦(𝑑𝑡)²,𝑑𝑦_𝑡𝑑𝑡=(𝐴𝑦)(𝑑𝑡)²+(𝜎𝑦)𝑑𝑤_𝑡𝑑𝑡. (2.4)(2.5)(2.6)(2.7) The exact meaning of these expressions comes from the theory of stochastic calculus; Arnold [18, chapter 5] and Karatzas and Shreve [19]. For our purposes, it is sufficient to know the multiplication rules of the stochastic calculus: (𝑑𝑤_𝑡)²=𝑑𝑡,𝑑𝑤_𝑡𝑑𝑡=0,𝑑𝑡²=0. (2.8) Substitute (2.3) into (2.2) and use (2.4), (2.5) (2.6), (2.7), and (2.8) to obtain 1𝑢=min[−𝜌𝑢(𝑥,𝑦,𝑡)𝑑𝑡+2𝜎²𝑦²𝑢_𝑦𝑦𝑑𝑡+𝐴𝑦𝑢_𝑦𝑑𝑡−𝑦𝑢_𝑥𝑑𝑡+{𝑝²+𝑝𝑢_𝑥}𝑑𝑡+ℎ(𝑥)𝑑𝑡+∘(𝑑𝑡)]. (2.9) Note that we have suppressed the arguments of the functions involved in (2.3).

Canceling the term 𝑢 on both sides of (2.9), dividing the remainder by 𝑑𝑡, and letting 𝑡→0, we obtain the dynamic programming partial differential equation or Hamilton-Jacobi-Bellman equation 1−𝜌𝑢(𝑥,𝑦)+2𝜎²𝑦²𝑢_𝑦𝑦+𝐴𝑦𝑢_𝑦−𝑦𝑢_𝑥+𝐹^∗(𝑢_𝑥)+ℎ(𝑥)=0,𝑢(0,𝑦)=0,𝑥,𝑦>0, (2.10) where 𝐹^∗(𝑥) is the Legendre transform of 𝐹(𝑥), that is, 𝐹^∗(𝑥)=min_𝑝>0{𝑝²+𝑝𝑥}=−𝑥²/4 and 𝑢_𝑥, 𝑢_𝑦, 𝑢_𝑥𝑥, 𝑢_𝑦𝑦 are partial derivatives of 𝑢(𝑥,𝑦,𝑡) with respect to 𝑥 and 𝑦.

2.2. A Reduction to 1-Dimensional Case

In this subsection, the general (two-dimensional) HJB equation has been reduced to a one-dimensional second-order differential equation. From the two-dimensional state space form (one state 𝑥 for inventory level and the other state 𝑦 for demand rate), it has been reduced to one-dimensional form for (𝑧=𝑥/𝑦) the ratio of inventory to demand.

There exists a 𝑣∈𝐶(0,∞) such that 𝑢(𝑥,𝑦)=𝑦²𝑣(𝑥/𝑦),𝑦>0. Since 𝑢_𝑥=𝑦𝑣^′(𝑥/𝑦),𝑢_𝑦=2𝑦𝑣(𝑥/𝑦)−𝑥𝑣^′(𝑥/𝑦),𝑢_𝑦𝑦=2𝑣(𝑥/𝑦)−2(𝑥/𝑦)𝑣^′(𝑥/𝑦)+(𝑥/𝑦)²𝑣^”(𝑥/𝑦). Setting 𝑧=𝑥/𝑦 and substituting these in (2.10), we have 1−𝜌𝑣(𝑧)+2𝜎²[2𝑣(𝑧)−2𝑣^′(𝑧)𝑧+𝑧²𝑣^”(𝑧)]+2𝐴𝑣(𝑧)−𝐴𝑧𝑣^′(𝑧)−𝑣^′(𝑧)+min_𝑝≥0(𝑝²+𝑝𝑦𝑣^′(𝑧))+ℎ(𝑧)=0. (2.11) Since min_𝑝≥0(𝑝²+𝑝𝑦𝑣^′(𝑧))=𝑦²min_𝑝≥0𝑝((𝑦)²𝑝+(𝑦)𝑣^′(𝑧))=𝑦²min_𝑞≥0(𝑞²+𝑞𝑣^′(𝑧)),min_𝑞≥0(𝑞²+𝑞𝑣^′(𝑧)−𝑣^′(𝑧))=min_𝑘+1≥0((𝑘+1)²+𝑘𝑣^′(𝑧)). (2.12) Then the HJB equation (2.11) becomes −1𝜌𝑣(𝑧)+2𝜎²𝑧²𝑣^”(𝑧)+𝐴𝑧𝑣^′(𝑧)+min_𝑘≥−1((𝑘+1)²+𝑘𝑣^′(𝑧))+ℎ(𝑧)=0,𝑣(0)=0,𝑧>0, (2.13) where 𝜌=−𝜌+𝜎²+2𝐴,𝐴=−(𝐴+𝜎²), and 𝐹^∗(𝑧) is the Legendre transform of 𝐹(𝑧), that is, 𝐹^∗(𝑧)=min_(𝑘+1)≥0{(𝑘+1)²+𝑘𝑧}=−𝑧²/4−𝑧.

The main feature of the HJB equation (2.13) is the vanishing of the coefficient of 𝑢_𝑥𝑥 for 𝑥=0 in partial differential equation terminology, then the equation is degenerate elliptic. Generally speaking, the difficulty stems from the degeneracy in the second-order term of the HJB equation (2.13).

2.3. Value Function

Let us consider the minimum value of the payoff function is a function of this initial point. The value function can be defined as a function whose value is the minimum value of the objective function of the production inventory control problem (1.1) for the manufacturing system, that is, 𝑉(𝑧)=inf𝐸[^∞₀𝑒⁻𝜌^𝑡{ℎ(𝑧_𝑡)+(𝑘_𝑡+1)²̃}𝑑𝑡]=inf𝐽(𝑘_𝑡);(𝑘+1)≥0. (2.14) The value function 𝑉(𝑧) is a solution to the reduced (one-dimensional) HJB equation (2.13) and the solution of this HJB equation is used to test controller for optimality or perhaps to construct a feedback controller. Again the HJB equation (2.13) arises in the production control problem (1.1), (1.2), (1.3) with constraint 0≤𝑝_𝑡≤𝑘_𝑡,∀𝑡≥0. (2.15)

2.4. Stochastic Neoclassical Differential Equation for Dynamics of Inventory-Demand Ratio

As in the certainty optimal production control model, the dynamics of the state equation of inventory level (1.2) can be reduced to a one-dimensional process by working in intensive (per capita) variables. Define 𝑧_𝑡≡𝑥_𝑡𝑦_𝑡𝑘,inventory-demandratio,_𝑡≡𝑝_𝑡𝑦_𝑡,percapitaproduction. (2.16) To determine the stochastic differential for the inventory-demand ratio, 𝑧≡𝑥/𝑦, we apply Itô's lemma as follows: 𝑥𝑧=𝑦≡𝐺(𝑦,𝑡),𝜕𝐺𝑥𝜕𝑦=−𝑦²𝑧=−𝑦,𝜕²𝐺𝜕𝑦²𝑥=2𝑦³𝑧=2𝑦²,𝜕𝐺=𝑥̇𝜕𝑡𝑦=(𝑝−𝑦)𝑦=𝑝𝑦−1=𝑘_𝑡−1. (2.17) From Itô's lemma, 𝑑𝑧=𝜕𝐺𝜕𝑦𝑑𝑦+𝜕𝐺1𝜕𝑡𝑑𝑡+2𝜕²𝐺𝜕𝑦²(𝑑𝑦)². (2.18) From (1.3), we have that (𝑑𝑦)²=𝜎²𝑦²𝑑𝑡. Substituting the above expressions into (2.18), we have that the dynamics of 𝑧_𝑡 to be the inventory-demand ratio at time 𝑡 which evolves according to the stochastic neoclassical differential equation for demand 𝑑𝑧_𝑡𝑧=(−^𝑡𝑦_𝑡)(𝐴𝑦_𝑡𝑑𝑡+𝜎𝑦_𝑡𝑑𝑤_𝑡)+(𝑘_𝑡1−1)𝑑𝑡+22𝑧_𝑡𝑦²_𝑡𝜎²𝑦²_𝑡𝑑𝑡=[−𝐴𝑧_𝑡+(𝑘_𝑡−1)+𝜎²𝑧_𝑡]𝑑𝑡−𝜎𝑧_𝑡𝑑𝑤_𝑡=[(−𝐴+𝜎²)𝑧_𝑡+(𝑘_𝑡−1)]𝑑𝑡−𝜎𝑧_𝑡𝑑𝑤_𝑡≤[𝐴𝑧_𝑡+𝑘_𝑡]𝑑𝑡−𝜎𝑧_𝑡𝑑𝑤_𝑡,𝑧₀=𝑧,𝑧>0. (2.19)

2.5. Riccati-Based Solution

This subsection deals with the Riccati-based solution of the reduced one-dimensional HJB equation (2.13) corresponding to the production inventory control problem (2.14) subject to (2.19) using the dynamic programming principle [17].

To find the Riccati-based solution of HJB equation (2.13), we refer to Da Prato [20] and Da Prato and Ichikawa [11] for the degenerate linear control problems related to Riccati equation in case of convex function like ℎ(𝑧)=𝑧².

By taking the derivative of (2.13) with respect to 𝑘 and setting it to zero, we can minimize the expression inside the bracket of (2.13) (i.e., 𝐹^∗(𝑣^′(𝑧))=min_𝑘≥−1((𝑘+1)²+𝑘𝑣^′(𝑧))) with respect to 𝑘. This procedure yields 𝑘^∗𝑣=−^′(𝑧)2−1. (2.20) Substituting (2.20) into (2.13) yields the equation −1𝜌𝑣(𝑧)+2𝜎²𝑧²𝑣^”(𝑧)+𝐴𝑧𝑣^′(𝑧)−(𝑣^′(𝑧))²4−𝑣^′(𝑧)+𝑧²=0 (2.21) known as the HJB equation. This is a partial differential equation which has a solution form 𝑣(𝑧)=𝑎(𝑡)𝑧²(𝑡). (2.22) Then 𝑣^′(𝑧)=2𝑎(𝑡)𝑧(𝑡),𝑣^”(𝑧)=2𝑎(𝑡). (2.23) Substituting (2.22) and (2.23) into (2.21) yields (1−𝜌𝑎+𝜎²𝑎−𝑎²+2𝑎𝐴)𝑧²−2𝑎𝑧=0. (2.24) Since (2.24) must hold for any value of 𝑧, we must have 𝑎²−𝑎(2𝐴+𝜎²+𝜌)−1=0, (2.25) called a Riccati equation from which we obtain 𝑎=−(2𝐴+𝜎²+𝜌)±(2𝐴+𝜎²+𝜌)²+42=𝐾₁(say). (2.26) So, (2.22) is a solution form of (2.21).

3. Bellman Equations for Discounted Cost Control

3.1. Existence and Uniqueness

To solve the Bellman equation (2.13) let us consider this HJB equation associated with the discounted production control problem in the following form: −1𝜌𝑣(𝑧)+2𝜎²𝑧²𝑣^”(𝑧)+𝐴𝑧𝑣^′(𝑧)+𝐹(𝑣^′(𝑧))+ℎ(𝑧)=0,0<𝜌<1, (3.1) where −𝑧𝐹(𝑧)=4²−𝑧if𝑧≤0,−𝑧if𝑧≥0. (3.2) We make the following assumptions: ℎ∶continuousfunctionon𝐑,ℎ∶non-negative,convexon𝐑, (3.3)ℎ satisfies the polynomial growth condition such that 0≤ℎ(𝑧)≤𝐾(1+|𝑧|^𝑛),𝑧∈𝐑, (3.4) for some 𝐾>0,𝑛≥2.

In order to ensure the integrability of 𝐽(𝑝), we assume that −𝜌+2𝑛𝐴+𝑛(2𝑛−1)𝜎²<0. (3.5) This condition (3.5) is needed for the integrability of 𝑧_𝑡 or 𝐽(𝑝). Under (3.5), we have Lemmas 3.1, 3.2, and Theorem 3.5, which ensures the finiteness of 𝐽(𝑝) and hence the finiteness of an existence unique solution of HJB equation of 𝑣.

First we have established the properties of the value function of the optimal control problem.

Lemma 3.1. Under (3.5) and for each 𝑛∈𝐍₊, there exists 𝐾>0 such that 𝐸[𝑒⁻𝜌_𝑡|𝑧_𝑡|^2𝑛]≤𝐾(1+𝑡). (3.6)

Proof. We have given its proof here to need the same kind of calculations in the future. By Itô’s formula we have 𝑒⁻𝜌_𝑡|𝑧_𝑡|^2𝑛=|𝑧|^2𝑛+^𝑡₀(−𝜌)𝑒⁻𝜌_𝑠|𝑧_𝑠|^2𝑛𝑑𝑠+2𝑛^𝑡₀𝑒⁻𝜌_𝑠|𝑧_𝑠|^2𝑛−1sgn(𝑧_𝑠)𝑑𝑧_𝑠+122𝑛(2𝑛−1)𝜎^2𝑡₀𝑒⁻𝜌_𝑠𝑧_𝑠²|𝑧_𝑠|^2𝑛−2𝑑𝑠=|𝑧|^2𝑛+^𝑡₀𝑒⁻𝜌_𝑠{−𝜌|𝑧_𝑠|^2𝑛+2𝑛𝐴𝑧_𝑠|𝑧_𝑠|^2𝑛−1sgn(𝑧_𝑠)+2𝑛𝑘_𝑠|𝑧_𝑠|^2𝑛−1sgn(𝑧_𝑠)+𝑛(2𝑛−1)𝜎²𝑧_𝑠²|𝑧_𝑠|^2𝑛−2}𝑑𝑠−2𝑛𝜎^𝑡₀𝑒⁻𝜌_𝑠𝑧_𝑠|𝑧_𝑠|^2𝑛−1sgn(𝑧_𝑠)𝑑𝑤_𝑠=|𝑧|^2𝑛+^𝑡₀𝑒⁻𝜌_𝑠{−𝜌+2𝑛𝐴+𝑛(2𝑛−1)𝜎²}|𝑧_𝑠|^2𝑛𝑑𝑠+2𝑛^𝑡₀𝑒⁻𝜌_𝑠𝑘_𝑠|𝑧_𝑠|^2𝑛−1sgn(𝑧_𝑠)𝑑𝑠−2𝑛𝜎^𝑡₀𝑒⁻𝜌_𝑠𝑧_𝑠|𝑧_𝑠|^2𝑛−1sgn(𝑧_𝑠)𝑑𝑤_𝑠. (3.7) Now by (2.15), (3.5) and taking expectation on the both sides, we obtain 𝐸[𝑒⁻𝜌_𝑡|𝑧_𝑡|^2𝑛]≤|𝑧|^2𝑛+2𝑛𝐸[^𝑡₀𝑒⁻𝜌_𝑠|𝑧_𝑠|^2𝑛𝑑𝑠]=|𝑧|^2𝑛+𝐸[^𝑡₀𝑒⁻𝜌_𝑠𝐙(𝑧_𝑠)𝑑𝑠], (3.8) where 𝐙(𝑧)=2𝑛|𝑧|^2𝑛. Obviously, 𝐙(𝑧) is bounded above. Thus we can deduce (3.6).

Lemma 3.2. Under (3.5) and for each 𝑛∈𝐍₊, there exists 𝐾>0 such that 𝜌𝐸[^∞₀𝑒⁻𝜌^𝑡|𝑧_𝑡|^2𝑛𝑑𝑡]≤𝐾(1+|𝑧|^2𝑛+2). (3.9)

Proof. By an application of Itô’s formula to 𝑒⁻𝜌_𝑡|𝑧_𝑡|^2𝑛+2, we have 𝑒⁻𝜌_𝑡|𝑧_𝑡|^2𝑛+2=|𝑧|^2𝑛+2+^𝑡₀(−𝜌)𝑒⁻𝜌_𝑠𝑧_𝑠^2𝑛+2𝑑𝑠+(2𝑛+2)^𝑡₀𝑒⁻𝜌_𝑠|𝑧_𝑠|^2𝑛+1𝑑𝑧_𝑠+12(2𝑛+2)(2𝑛+1)𝜎^2𝑡₀𝑒⁻𝜌^𝑠𝑧_𝑠²𝑧_𝑠^2𝑛𝑑𝑠=|𝑧|^2𝑛+2+^𝑡₀(−𝜌)𝑒⁻𝜌_𝑠𝑧_𝑠^2𝑛+2𝑑𝑠+(2𝑛+2)^𝑡₀𝑒⁻𝜌_𝑠|𝑧_𝑠|^2𝑛+1(𝐴𝑧_𝑠+𝑘_𝑠)𝑑𝑠−2(𝑛+1)𝜎^𝑡₀𝑒⁻𝜌_𝑠𝑧_𝑠|𝑧_𝑠|^2𝑛+1𝑑𝑤_𝑠+12(2𝑛+2)(2𝑛+1)𝜎^2𝑡₀𝑒⁻𝜌_𝑠𝑧_𝑠²𝑧_𝑠^2𝑛𝑑𝑠=|𝑧|^2𝑛+2+^𝑡₀𝑒⁻𝜌_𝑠{−𝜌+2(𝑛+1)𝐴+(𝑛+1)(2𝑛+1)𝜎²}𝑧_𝑠^2𝑛+2𝑑𝑠+2(𝑛+1)^𝑡₀𝑒⁻𝜌^𝑠𝑘_𝑠|𝑧_𝑠|^2𝑛+1𝑑𝑠−2(𝑛+1)𝜎^𝑡₀𝑒⁻𝜌^𝑠𝑧_𝑠|𝑧_𝑠|^2𝑛+1𝑑𝑤_𝑠. (3.10) Now by (2.15), (3.5) and taking expectation on the both sides we obtain 𝐸[𝑒⁻𝜌_𝑡|𝑧_𝑡|^2𝑛+2]≤|𝑧|^2𝑛+2+2(𝑛+1)𝐸[^𝑡₀𝑒⁻𝜌_𝑠|𝑧_𝑠|^2𝑛+1𝑑𝑠]. (3.11) Hence 𝐸[^∞₀𝑒⁻𝜌^𝑠𝐆(𝑧_𝑠)𝑑𝑠]≤|𝑧|^2𝑛+2, (3.12) where 𝐆(𝑧)=2(𝑛+1)|𝑧|^2𝑛+1.
We choose 𝜁<0 such that |𝑧|^2𝑛≤𝐆(𝑧) for all 𝑧≥𝜁. Then 𝐸[^∞₀𝑒⁻𝜌_𝑡|𝑧_𝑡|^2𝑛𝑑𝑡]=^∞₀𝑒⁻𝜌_𝑡𝐸[|𝑧_𝑡|^2𝑛1_{(|𝑧𝑡|<𝜁)}+|𝑧_𝑡|^2𝑛1_{(|𝑧𝑡|≥𝜁)}≤𝑑𝑡]_∞0𝑒⁻𝜌_𝑡[{𝜁^2𝑛+𝐸[𝐆(𝑧_𝑡≤𝜁)]}𝑑𝑡]_2𝑛𝜌+|𝑧|^2𝑛+2. (3.13) Thus we get (3.9) with 𝐾>0 independent of sufficiently small 𝜌.

Proposition 3.3. We assume (3.3), (3.4). Then the value function 𝑣(𝑧)isconvex.

Proof. For any 𝜖>0, there exist 𝑘,𝑘∈𝒫_𝑘 such that 𝐸[^∞₀𝑒⁻𝜌^𝑡{ℎ(𝑧_𝑡)+(𝑘_𝑡+1)²}𝑑𝑡]<𝑣(𝑧)+𝜖,𝐸[^∞₀𝑒⁻𝜌_𝑡{ℎ(𝑧_𝑡)+(𝑘_𝑡+1)²}𝑑𝑡]<𝑣(𝑧)+𝜖, (3.14) where 𝑑𝑧_𝑡=[𝐴𝑧_𝑡+𝑘_𝑡]𝑑𝑡−𝜎𝑧_𝑡𝑑𝑤_𝑡,𝑧₀𝑑𝑧=𝑧∈𝐑,_𝑡=[𝐴𝑧_𝑡+𝑘_𝑡]𝑑𝑡−𝜎𝑧_𝑡𝑑𝑤_𝑡,𝑧₀=𝑧∈𝐑. (3.15) We set 𝑘_𝑡=𝜉𝑘_𝑡+(1−𝜉)𝑘_𝑡,𝑧_𝑡=𝜉𝑧_𝑡+(1−𝜉)𝑧_𝑡,𝑧₀=𝜉𝑧+(1−𝜉),𝑧≡𝑧, (3.16) for 0<𝜉<1. Clearly, 𝑑𝑧_𝑡=[𝐴𝑧_𝑡+𝑘_𝑡]𝑑𝑡−𝜎𝑧_𝑡𝑑𝑤_𝑡. (3.17) Hence, by convexity 𝑣(𝑧)≤𝐸[^∞₀𝑒⁻𝜌_𝑡{ℎ(𝑧_𝑡)+(𝑘_𝑡+1)²}𝑑𝑡]≤𝜉𝐸[^∞₀𝑒⁻𝜌_𝑡{ℎ(𝑧_𝑡)+(𝑘_𝑡+1)²}𝑑𝑡]+(1−𝜉)𝐸[^∞₀𝑒⁻𝜌_𝑡{ℎ(𝑧_𝑡)+(𝑘_𝑡+1)²}𝑑𝑡]≤𝜉(𝑣(𝑧)+𝜖)+(1−𝜉)(𝑣(𝑧)+𝜖). (3.18) Letting 𝜖→0, we get 𝑣(𝑧)=𝑣(𝜉𝑧+(1−𝜉)𝑧)≤𝜉𝑣(𝑧)+(1−𝜉)𝑣(𝑧), (3.19) which completes the convexity of the value function 𝑣(𝑧).

Theorem 3.4. Assume (3.4) and (3.5). Choose 𝑔_𝑟(𝑧)=𝛿+|𝑧|^𝑟 for any 2≤𝑟≤2𝑛, then there exist 𝛿≥1 and 𝜁>0 depending on 𝑀 such that −𝜌𝑔_𝑟1(𝑧)+2𝜎²𝑧²𝑔^”_𝑟(𝑧)+𝐴𝑧𝑔^′_𝑟(𝑧)+max_{|𝑞|≤𝑀}(𝑞²+𝑞𝑔^′_𝑟(𝑧))+𝜁𝑔_𝑟(𝑧)≤0. (3.20) Further 𝐸[^𝜏₀𝑒⁻𝜌_𝑠𝜁𝑔_𝑟(𝑧_𝑠)𝑑𝑠+𝑒⁻𝜌_𝜏𝑔_𝑟(𝑧_𝜏)]≤𝑔_𝑟(𝑧)𝑓𝑜𝑟2≤𝑟≤2𝑛, (3.21) where 𝜏 is any stopping time and 𝑧_𝑡 is the response to (𝑝_𝑡)∈𝒫_𝑀.
Then one has 0≤𝑉(𝑧)≤𝐾(1+|𝑧|^𝑟), (3.22) where 𝐾=𝐾𝛿/𝜁, for some positive constant 𝐾.

Proof. By (3.5), we choose 𝜁∈(0,𝜌) such that −1𝜌+𝑟𝐴+2𝑟(𝑟−1)𝜎²+𝜁<0, (3.23) and then 𝛿≥1 such that 1(−𝜌+𝑟𝐴+2𝑟(𝑟−1)𝜎²+𝜁)|𝑧|^𝑟+𝑀𝑟|𝑧|^𝑟−1+(𝑀²+𝜁𝛿−𝜌𝛿)≤0. (3.24) Then (3.20) is immediate.
Applying Itô’s formula to 𝑒⁻𝜌_𝜏𝑔_𝑟(𝑧_𝜏), we have 𝑒⁻𝜌_𝜏𝑔_𝑟(𝑧_𝜏)=𝑔_𝑟(𝑧)+^𝜏₀(−𝜌)𝑒⁻𝜌_𝑠𝑔_𝑟(𝑧_𝑠)𝑑𝑠+^𝜏₀𝑒⁻𝜌_𝑠𝑔^′_𝑟(𝑧_𝑠)𝑑𝑧_𝑠+12𝜎^2𝜏₀𝑒⁻𝜌_𝑠𝑧_𝑠²𝑔^”_𝑟(𝑧_𝑠)𝑑𝑠=𝑔_𝑟(𝑧)+^𝜏₀𝑒⁻𝜌^𝑠{−𝜌𝑔_𝑟(𝑧_𝑠)+𝐴𝑧_𝑠𝑔^′_𝑟(𝑧_𝑠)+𝑞𝑔^′_𝑟(𝑧_𝑠1)+2𝜎²𝑧²_𝑠𝑔^”_𝑟(𝑧_𝑠)}𝑑𝑠−𝜎^𝜏₀𝑒⁻𝜌^𝑠𝑧_𝑠𝑔^′_𝑟(𝑧_𝑠)𝑑𝑤_𝑠=𝑔_𝑟(𝑧)+^𝜏₀𝑒⁻𝜌_𝑠{−𝜌𝑔_𝑟(𝑧_𝑠)+𝐴𝑧_𝑠𝑔^′_𝑟(𝑧_𝑠1)+2𝜎²𝑧²_𝑠𝑔^”_𝑟(𝑧_𝑠)+max_{|𝑞|≤𝑀}(𝑞²+𝑞𝑔^′_𝑟(𝑧_𝑠))+𝜁𝑔_𝑟(𝑧_𝑠)}𝑑𝑠−max_{|𝑞|≤𝑀}^𝜏_𝑜𝑞²𝑒⁻𝜌^𝑠𝑑𝑠−^𝜏₀𝑒⁻𝜌^𝑠𝜁𝑔_𝑟(𝑧_𝑠)𝑑𝑠−𝜎^𝜏₀𝑒⁻𝜌^𝑠𝑧_𝑠𝑔^′_𝑟(𝑧_𝑠)𝑑𝑤_𝑠. (3.25) Now by (3.20) and taking expectation on the both sides, we obtain 𝐸[^𝜏₀𝑒⁻𝜌_𝑠𝜁𝑔_𝑟(𝑧_𝑠)𝑑𝑠+𝑒⁻𝜌_𝜏𝑔_𝑟(𝑧_𝜏)]≤𝑔_𝑟𝑀(𝑧)−²𝜌, (3.26) from which we deduced (3.21).
The convexity of the value function 𝑉 follows from the same line as (Proposition 3.3). Let (𝑧⁰_𝑡) be the unique solution of 𝑑𝑧⁰_𝑡=𝐴𝑧⁰_𝑡𝑑𝑡−𝜎𝑧⁰_𝑡𝑑𝑤_𝑡,𝑧⁰₀=𝑧. (3.27) Then by (3.4) and (3.21), 𝑉(𝑧)≤𝐸[^∞₀𝑒⁻𝜌^𝑡ℎ(𝑧⁰_𝑡)𝑑𝑡]≤𝐾𝐸[^∞₀𝑒⁻𝜌_𝑡𝑔_𝑟(𝑧⁰_𝑡≤)𝑑𝑡]𝐾𝑔_𝑟(𝑧)𝜁=𝐾(𝛿+|𝑧|^𝑟)𝜁=𝐾𝛿𝜁(1+|𝑧|^𝑟𝛿)≤𝐾(1+|𝑧|^𝑟);2≤𝑟≤2𝑛, (3.28) which implies (3.22) and satisfies (3.4). Hence this completes the proof.

Theorem 3.5. Assume (3.3), (3.4), (3.5). Then there exists a unique solution 𝑣∈𝐶²(𝐑) of (3.1) such that 𝜌𝑣(𝑧)≤𝐾(1+|𝑧|^𝑛+3),𝑧∈𝐑, (3.29) for some constant 𝐾>0. Moreover, 𝑣 admits a representation 𝑣(𝑧)=inf𝐸[^∞₀𝑒⁻𝜌^𝑡{ℎ(𝑧_𝑡)+(𝑘_𝑡+1)²}𝑑𝑡];(𝑘_𝑡+1)≥0. (3.30)

Proof. Since 𝐹^∗(𝑧)=min{(𝑘+1)²+𝑘𝑧} is Lipschitz continuous, this follows from Bensoussan [21] in case of Assumption (3.4) except convexity. For the general case, we take a nondecreasing sequence ℎ_𝑛∈𝐶(𝐑) convergent to ℎ with 0≤ℎ_𝑛≤ℎ. It is well known (Bensoussan [21]) that, for every 𝑛∈𝐍₊, (3.1) has a unique solution 𝑣_𝑛 for ℎ_𝑛 of the form 𝑣_𝑛(𝑧)=inf𝐸[^∞₀𝑒⁻𝜌^𝑡{ℎ_𝑛(𝑧_𝑡)+(𝑘_𝑡+1)²}𝑑𝑡];𝑘_𝑡+1≥0 (3.31) in the class 𝐶²(𝐑) of continuous functions vanishing at infinitely, where 𝑧_𝑡 is a solution of (2.19).
To prove (3.29), we recall (3.30). Hence by (3.4) and Lemma 3.2 we have 0≤𝜌𝑣_𝑛(𝑧)≤𝜌𝐸[^∞₀𝑒⁻𝜌^𝑡{ℎ_𝑛(𝑧_𝑡)+(𝑘_𝑡+1)²≤∣}𝑑𝑡]𝜌𝐸[^∞₀𝑒⁻𝜌_𝑡{ℎ(𝑧_𝑡)+(𝑘_𝑡+1)²∣}𝑑𝑡]≤𝐾(1+𝜌𝐸[^∞₀𝑒⁻𝜌_𝑡{(1+|𝑧_𝑡|^𝑛)+(𝑘_𝑡+1)²}𝑑𝑡])≤𝐾(1+|𝑧|^𝑛+3). (3.32) This implies that 𝑣 satisfies (3.29).
To estimate 𝑣^′_𝑛 on [−𝑠,𝑠] for 𝑠∈𝐑₊, we remember the Taylor expansions of 𝑣∈𝐶²(𝐑): 𝜀𝑣^′𝜀(𝑏)=𝑣(𝑏+𝜀)−𝑣(𝑏)−²2𝑣^”(𝑏+𝜗₁𝑣𝜀),_′(𝑧)=𝑣^′(𝑏)+𝑣^”(𝑏+𝜗₂(𝑧−𝑏))(𝑧−𝑏),𝑥∈𝑉_𝜀,0<𝜗₁,𝜗₂<1, (3.33) where 𝑉_𝜀=𝑉_𝜀(𝑏) is any 𝜀-neighborhood of 𝑏∈[−𝑠,𝑠]. Then wa can obtain the Landau-Kolmogorov inequality: sup_𝑉𝜀|𝑣^′2(𝑧)|≤𝜀sup_𝑉𝜀|𝑣(𝑧)|+3𝜀2sup_𝑉𝜀|𝑣^”(𝑧)|. (3.34) Choosing 0<𝜀<1⋀(1/3(𝑘+2𝐴/𝜎²)), and by (3.1), (3.2), and (3.34) we have sup_𝑉𝜀|𝑣^′_𝑛2(𝑧)|≤𝜀sup_𝑉𝜀|𝑣_𝑛(𝑧)|+3𝜀2sup_𝑉𝜀|𝑣^”_𝑛≤2(𝑧)|𝜀sup_𝑉𝜀|𝑣_𝑛(𝑧)|+3𝜀2sup_𝑉𝜀[2𝜌𝜎²|𝑣_𝑛(𝑧)𝑧²2𝐴|+𝜎²|𝑣^′_𝑛(𝑧)𝑧2|+𝜎²|𝐹(𝑣^′_𝑛(𝑧))𝑧²2𝐴|+𝜎²|ℎ_𝑛(𝑧)𝑧2|]≤(𝜀+3𝜀𝜌𝜎²)sup_𝑉𝜀|𝑣_𝑛2𝐴(𝑧)|+3𝜀(𝑘+𝜎²)sup_𝑉𝜀|𝑣^′_𝑛𝐴(𝑧)|+3𝜀𝜎²sup_𝑉𝜀|ℎ_𝑛(𝑧)|, (3.35) from which sup_𝑉𝜀|𝑣^′_𝑛1(𝑧)|≤1−3𝜀(𝑘+2𝐴/𝜎²)(2𝜀+3𝜀𝜌𝜎²)sup_𝑉𝜀|𝑣_𝑛2𝐴(𝑧)|+3𝜀(𝑘+𝜎²)sup_𝑉𝜀|𝑣^′_𝑛𝐴(𝑧)|+3𝜀𝜎²sup_𝑉𝜀|ℎ_𝑛(𝑧)|. (3.36) Now by (3.34) and (3.4), we have sup_𝑉𝜀|𝑣^′_𝑛𝐾(𝑧)|≤1−3𝜀(𝑘+2𝐴/𝜎²)(2𝜀+3𝜀𝜌𝜎²)(1+(𝑠+1)^𝑛+3). (3.37) Thus, taking the finite covering 𝑉_𝜀(𝑏_𝑖),𝑖=1,2,…,𝑗, of [−𝑠,𝑠], we deduce sup_{−𝑠≤𝑧≤𝑠}|𝑣^′_𝑛(𝑧)|≤^𝑗_𝑖=1sup_{𝑉𝜀(𝑏𝑖)}|𝑣^′_𝑛≤(𝑧)|𝑗𝐾1−3𝜀(𝑘+2𝐴/𝜎²)(2𝜀+3𝜀𝜌𝜎²)(1+(𝑠+1)^𝑛+3), (3.38) and hence sup_𝑛sup_{−𝑠≤𝑧≤𝑠}(|𝑣^′_𝑛(𝑧)|+|𝑣^”_𝑛(𝑧)|)<∞forevery𝑠∈𝐑₊. (3.39) By the Ascoli-Arzelà theorem, we have 𝑣_𝑛→𝑣,𝑣^′_𝑛→𝑣^′uniformlyon[−𝑠,𝑠], (3.40) taking a subsequence if necessary. Passing to the limit, we can obtain (3.1) and (3.30).
Following the inequality (3.29), we have 𝐸[𝑒⁻𝜌_𝑡𝑣(𝑧_𝑡)]→0as𝑡→∞. (3.41) Hence by Itô’s formula to 𝑒⁻𝜌_𝑡𝑣(𝑧_𝑡), for convex function [19, page 219], we have 𝑒⁻𝜌^𝑡𝑣(𝑧_𝑡)=𝑣(𝑧)+^∞₀𝑒⁻𝜌^𝑡{−𝜌𝑣(𝑧)+𝐴𝑧𝑣^′(𝑧)+𝑘_𝑡𝑣^′1(𝑧)+2𝜎²𝑧²𝑣^”−(𝑧)}𝑑𝑡_∞0𝑒⁻𝜌_𝑡𝜎𝑧_𝑡𝑣^′(𝑧)𝑑𝑤_𝑡. (3.42) By virtue of (3.1) and taking expectation on the both sides we have 𝐸[𝑒⁻𝜌^𝑡𝑣(𝑧_𝑡)]=𝑣(𝑧)−𝐸[^∞₀𝑒⁻𝜌^𝑡{ℎ(𝑧_𝑡+(𝑘_𝑡+1)²)}𝑑𝑡]. (3.43) Now by (3.41), we obtain 𝑣(𝑧)=inf𝐸[^∞₀𝑒⁻𝜌^𝑡{ℎ(𝑧_𝑡)+(𝑘_𝑡+1)²}𝑑𝑡];𝑘_𝑡≥−1, (3.44) which is similar to (3.30) and here the infimum is attained by the feedback law 𝛽(𝑣^′(𝑧)) with 𝛽(𝜃)=argmin𝐹(𝜃). The proof is complete.

Corollary 3.6. Assume (3.3), (3.4), and (3.5). Then there exists a unique solution 𝑢∈𝐶²(𝐑) of (2.10) such that 𝜌𝑢(𝑥,𝑦)≤𝐾(1+|𝑥|^𝑛+3),(𝑥,𝑦)∈𝐑, (3.45) for some constant 𝐾>0. Moreover, 𝑢 admits a representation 𝑢(𝑥,𝑦)=inf𝐸[^∞₀𝑒^−𝜌𝑡{ℎ(𝑥_𝑡)+𝑝²_𝑡}𝑑𝑡];𝑝≥0. (3.46)

4. An Application to Production Control

In this section we will study the production control problem to minimize the cost (2.14) over the class 𝒫_𝑘 of all progressively measurable processes 𝑝_𝑡 such that 0≤𝑝_𝑡≤𝑘_𝑡 and lim_𝑡→∞1𝑡𝐸[𝑒⁻𝜌_𝑡|𝑧_𝑡|^𝑛]=0 (4.1) for the response 𝑧_𝑡 to 𝑝_𝑡.

Let us consider the stochastic differential equation 𝑑𝑧^∗_𝑡=[𝐴𝑧^∗_𝑡+𝛽_𝑘(𝑣^′(𝑧^∗_𝑡))]𝑑𝑡−𝜎𝑧^∗_𝑡𝑑𝑤_𝑡,𝑧^∗₀=𝑧,𝑧>0, (4.2) where 𝛽_𝑘(𝑧)=argmin𝐹(𝑧), that is, 𝛽_𝑘−𝑧(𝑧)=𝑘−1if𝑧≤−2𝑘,2−1if−2𝑘<𝑧≤0−1if0<𝑧., (4.3)

We need to establish the following lemmas.

Lemma 4.1. Under (3.5) and for each 𝑛∈𝐍₊, there exists 𝐾>0 such that 𝐸[𝑒⁻𝜌_𝑡|𝑧^∗_𝑡|^2𝑛]≤𝐾(1+𝑡). (4.4)

Proof. Since, we have by Itô’s formula 𝑑(𝑧^∗_𝑡)²=2𝑧^∗_𝑡𝑑𝑧^∗_𝑡+12𝜎²(𝑧^∗_𝑡)²2𝑑𝑡=2𝑧^∗_𝑡[(𝐴𝑧^∗_𝑡+𝛽_𝑘𝑣^′(𝑧^∗_𝑡))𝑑𝑡−𝜎𝑧^∗_𝑡𝑑𝑤_𝑡]+𝜎²(𝑧^∗_𝑡)²𝑧𝑑𝑡,𝑑(_𝑡)²=2𝑧_𝑡𝑑𝑧_𝑡+12𝜎²(𝑧_𝑡)²𝑧2𝑑𝑡≤2_𝑡[𝐴𝑧_𝑡𝑑𝑡−𝜎𝑧_𝑡𝑑𝑤_𝑡]+𝜎²(𝑧_𝑡)²𝑑𝑡. (4.5) Now by the assumptions of comparison theorem we have 𝑏₁(𝑧)=𝐴𝑧+𝛽_𝑘(𝑣^′(𝑧_𝑡))+𝜎²(𝑧)², 𝑏₂(𝑧)=𝐴𝑧+𝜎²(𝑧)², then we have 𝑏₁(𝑧)<𝑏₂(𝑧) and |𝜎𝑧−𝜎𝑧|=|𝜎||𝑧−𝑧|, where 𝜎𝑧=𝜂(𝑧),so∫^𝜖₀𝜂⁻²(𝑧)𝑑𝑥=∫^𝜖₀(𝜎𝑧)⁻²𝑑𝑧=∞. Thus we can see (𝑧^∗_𝑡)²≤(𝑧_𝑡)² by the comparison theorem of Ikeda and Watanabe [22]. Since the explosion time 𝜎=inf{𝑡∶|𝑧^∗_𝑡|=∞}, we have ∞=(𝑧^∗_𝜎)²≤(𝑧_𝜎)². Hence 𝜎=∞.
By the monotonicity of 𝛽(𝑣^′(𝑧)), we have 𝛽_𝑘(𝑣^′(𝑧))<𝛽(𝑣^′(𝑧))for𝑧<𝑧. Then 𝑑𝑧_𝑡=[𝐴𝑧_𝑡+𝛽(𝑣^′(𝑧_𝑡))]𝑑𝑡−𝜎𝑧_𝑡𝑑𝑤_𝑡,𝑧₀=𝑧^∗,𝑑𝑧^∗_𝑡=[𝐴𝑧^∗_𝑡+𝛽(𝑣^′(𝑧^∗_𝑡))]𝑑𝑡−𝜎𝑧^∗_𝑡𝑑𝑤_𝑡,𝑧^∗₀=𝑧^∗,𝑑(𝑧_𝑡−𝑧^∗_𝑡)=[𝐴(𝑧_𝑡−𝑧^∗_𝑡)+𝛽_𝑘(𝑣^′(𝑧_𝑡))−𝛽_𝑘(𝑣^′(𝑧^∗_𝑡))]𝑑𝑡−𝜎(𝑧_𝑡−𝑧^∗_𝑡)𝑑𝑤_𝑡, (4.6) by Itô’s formula, 𝑑(𝑧_𝑡−𝑧^∗_𝑡)²=2(𝑧_𝑡−𝑧^∗_𝑡)𝑑(𝑧_𝑡−𝑧^∗_𝑡1)+2𝜎²(𝑧_𝑡−𝑧^∗_𝑡)²𝑧2𝑑𝑡=2𝐴(_𝑡−𝑧^∗_𝑡)²𝑑𝑡+2(𝑧_𝑡−𝑧^∗_𝑡)(𝛽_𝑘(𝑣^′(𝑧_𝑡))−𝛽_𝑘(𝑣^′(𝑧^∗_𝑡𝑧)))𝑑𝑡−2𝜎(_𝑡−𝑧^∗_𝑡)²𝑑𝑤_𝑡+𝜎²(𝑧_𝑡−𝑧^∗_𝑡)²𝑑𝑡≤(2𝐴+𝜎²)(𝑧_𝑡−𝑧^∗_𝑡)²𝑑𝑡−2𝜎(𝑧_𝑡−𝑧^∗_𝑡)²𝑑𝑤_𝑡. (4.7) Hence (𝑧_𝑡−𝑧^∗_𝑡)²≤(2𝐴+𝜎²)^𝑡₀(𝑧_𝑠−𝑧^∗_𝑠)²𝑑𝑠−2𝜎^𝑡₀(𝑧_𝑠−𝑧^∗_𝑠)²𝑑𝑤_𝑠,𝐸[(𝑧_𝑡−𝑧^∗_𝑡)²]≤(2𝐴+𝜎²)^𝑡₀𝐸[(𝑧_𝑠−𝑧^∗_𝑠)²]𝑑𝑠. (4.8) Set 𝜉(𝑡)=𝐸[(𝑧_𝑡−𝑧^∗_𝑡)²], we obtain 𝜉(𝑡)≤𝐾∫^𝑡₀𝜉(𝑠)𝑑𝑠,∀𝑡≥0 where 𝐾>2𝐴+𝜎². By Gronwall Lemma, we have 𝜉(𝑡)=0a.s.∀𝑡≥0. Therefore, 𝐸[(𝑧_𝑡−𝑧^∗_𝑡)²]=0a.s.∀𝑡≥0, from which we have 𝑧_𝑡≤𝑧^∗_𝑡. So, the uniqueness of (4.2) holds. Thus we conclude that (4.2) admits a unique strong solution (𝑧^∗_𝑡), Ikeda and Watanabe [22, Chapter 4, Theorem 1.1], with 𝐸[|𝑧^∗_𝑡|^2𝑛<∞].
By (3.5) and Itô’s formula, 𝐸[𝑒⁻𝜌_𝑡|𝑧^∗_𝑡|^2𝑛]=|𝑧|^2𝑛+𝐸[^𝑡₀𝑒⁻𝜌_𝑠{−𝜌+2𝑛𝐴+𝑛(2𝑛−1)𝜎²}|𝑧^∗_𝑠|^2𝑛𝑑𝑠]+2𝑛𝐸[^𝑡₀𝑒⁻𝜌_𝑠𝛽_𝑘(𝑣^′(𝑧^∗_𝑠))|𝑧^∗_𝑠|^2𝑛−1sgn(𝑧^∗_𝑠)𝑑𝑠]≤|𝑧|^2𝑛+2𝑛𝐸[^𝑡₀𝑒⁻𝜌_𝑠𝛽_𝑘(𝑣^′(𝑧^∗_𝑠))|𝑧^∗_𝑠|^2𝑛−1sgn(𝑧^∗_𝑠)𝑑𝑠]=|𝑧|^2𝑛+𝐸[^𝑡₀𝑒⁻𝜌_𝑠𝐙^(𝐧)(𝑠)𝑑𝑠], (4.9) where 𝐙^(𝐧)(𝐬)=2𝑛𝛽_𝑘(𝑣^′(𝑧^∗_𝑠))|𝑧^∗_𝑠|^2𝑛−1sgn(𝑧^∗_𝑠).
By (4.3) it is easily seen that 𝑧𝛽_𝑘(𝑣^′(𝑧))≤(𝑘−1)|𝑧| if |𝑧|≥𝑎, for sufficiently large 𝑎>0. Clearly sup_𝑠𝐸[𝑒⁻𝜌_𝑠𝐙^(𝐧)(𝑠)1_{(|𝑧∗𝑠|<𝑎)}]<∞. (4.10) Also 𝐸[𝑒⁻𝜌_𝑠𝐙^(𝐧)(𝑠)1_{(|𝑧∗𝑠|≥𝑎)}]≤𝐸[𝑒⁻𝜌_𝑠(𝑘−1)|𝑧|1_{(|𝑧∗𝑠|≥𝑎)}]. (4.11) By the same line as the proof of Lemma 3.1, we see that the right-hand side is bounded from above. This completes the proof.

Theorem 4.2. One assumes (3.3), (3.4), (3.5). Then the optimal cost control 𝑘^∗_𝑡 is given by 𝑘^∗_𝑡=𝛽_𝑘(𝑣^′(𝑧^∗_𝑡)), (4.12) and the minimum value by 𝐽(𝑘^∗)=𝑣(𝑧), (4.13) where 𝑧^∗_𝑡 is defined by (4.2).

Proof. We first note by (4.3) that 𝐹_𝑘(𝑧)=min{(𝑘+1)²+𝑘𝑧;𝑘+1≥0}, and the minimum is attained by 𝛽_𝑘(𝑧). We apply Itô’s formula for convex functions [19, page 219] to obtain 𝑒⁻𝜌_𝑡𝑣(𝑧^∗_𝑡)=𝑣(𝑧)+^𝑡₀𝑒⁻𝜌_𝑠(−𝜌𝑣(𝑧_𝑠)+𝐴𝑧𝑣^′(𝑧_𝑠)+𝛽_𝑘(𝑣^′(𝑧_𝑠1))+2𝜎²𝑧²𝑣^”(𝑧_𝑠))|_{𝑧=𝑧∗𝑠}−𝑑𝑠_𝑡0𝑒⁻𝜌_𝑠𝜎𝑧^∗_𝑠𝑣^′(𝑧^∗_𝑠)𝑑𝑤_𝑠. (4.14) Taking expectation on the both sides, 𝐸[𝑒⁻𝜌_𝑡𝑣(𝑧^∗_𝑡)]=𝑣(𝑧)+𝐸[^𝑡₀𝑒⁻𝜌_𝑠(−𝜌𝑣(𝑧^∗_𝑠)+𝐴𝑧^∗𝑣^′(𝑧^∗_𝑠)+𝑘^∗_𝑠𝑣^′(𝑧^∗_𝑠1)+2𝜎²𝑧^∗2𝑣^”(𝑧^∗_𝑠))𝑑𝑠]. (4.15) By virtue of (2.13), 𝐸[𝑒⁻𝜌_𝑡𝑣(𝑧^∗_𝑡)]=𝑣(𝑧)−𝐸[^𝑡₀𝑒⁻𝜌_𝑠{ℎ(𝑧^∗_𝑠)+(𝑘+1)²}𝑑𝑠]. (4.16) Choose 𝑛∈𝐍₊ such that 2𝑛>𝑚. By (3.4) and Lemma 3.1, we have 1𝑡𝐸[𝑒⁻𝜌^𝑡𝑣(𝑧^∗_𝑡𝐾)]≤𝑡(1+𝐸[𝑒⁻𝜌^𝑚𝑡|𝑧^∗_𝑡|^𝑚≤𝐾])𝑡(1+𝐸([𝑒^−2𝑛𝜌^𝑡|𝑧^∗_𝑡|^2𝑛])^𝑚/2𝑛)≤𝐾𝑡(1+(𝐾(1+𝑡))^𝑚/2𝑛)<∞, (4.17) which implies that liminf_𝑡→∞1𝑡𝐸[𝑒⁻𝜌_𝑡𝑣(𝑧^∗_𝑡)]=0. (4.18) Hence 𝑧^∗_𝑡 satisfies (4.1). Then we get 𝐸[∫^𝑡₀𝑒⁻𝜌_𝑠{ℎ(𝑧^∗_𝑠)+(𝑘+1)²}𝑑𝑠]≤𝑣(𝑧) from which 𝐽(𝑘^∗)≤𝑣(𝑧). By (3.4), we have 𝐽(𝑘^∗)≤𝑣(𝑧)<∞, hence 𝑘^∗=(𝑘^∗_𝑡)∈𝒫_𝑘.
Clearly 𝐹_𝑘(𝑧)≤(𝑘+1)²+𝑘𝑧 for every 𝑘∈𝒫_𝑘. Again following the same construction of (3.45) and by the HJB equation (2.13), we have 𝐸[𝑒⁻𝜌_𝑡𝑣(𝑧_𝑡)]≥𝑣(𝑧)−𝐸[^𝑡₀𝑒⁻𝜌_𝑠{ℎ(𝑧_𝑠)+(𝑘+1)²}𝑑𝑠],𝑘=(𝑘_𝑡)∈𝒫_𝑘. (4.19) By (4.1) we have 𝐸[^∞₀𝑒⁻𝜌^𝑠{ℎ(𝑧_𝑠)+(𝑘+1)²}𝑑𝑠]≥𝑣(𝑧). (4.20) Thus we deduce 𝐽(𝑘^∗)≥𝑣(𝑧). The proof is complete.

Lemma 4.3. Under (3.5), there exists a unique solution 𝑥^∗_𝑡≥0 of 𝑑𝑥^∗_𝑡=[𝜓_𝑘(𝑢^′(𝑥^∗_𝑡,𝑦_𝑡))−𝑦_𝑡]𝑑𝑡,𝑥^∗₀=𝑥>0, (4.21) where 𝜓_𝑘−𝑥(𝑥)=2𝑥if𝑥≤0,0if0<𝑥,𝜓(𝑥)∶=−2istheminimizerofmin_𝑝≥0{𝑝²+𝑝𝑢_𝑥}. (4.22)

Proof. Set 𝑘(𝑥,𝑦)=𝜓(𝑢_𝑥(𝑥,𝑦))if𝑥,𝑦>0. (4.23) Since 𝑘(𝑥,𝑦) is continuous in 𝑥 and 𝑦, there exists a nonexplosive solution 𝑥^∗_𝑡 of (4.21).
Now we will show 𝑥^∗_𝑡≥0∀𝑡≥0 a.s. Suppose 0<𝑣^′(0+)<∞. Then L’Hospital’s rule gives lim_𝑧→0+𝑧²𝑣^′(𝑧)𝑧=lim_𝑧→0+𝑧²𝑣^”(𝑧)=0. (4.24) Letting 𝑧→0+ in (3.1), we have 𝐹(𝑣^′(0+))=0, and hence 𝑣^′(0+)=−(𝑘+1)²/𝑘≤0. This contradicts the assumption. Thus we get 𝑣^′(0+)=∞, which implies that 𝜓(𝑢_𝑥(0+,𝑦))=0. In case 𝑇=inf{𝑡≥0∶𝑥^∗_𝑡=0}=∞, we have at 𝑡=𝑇, 𝑑𝑥^∗_𝑡𝑑𝑡=𝜓(𝑢_𝑥(𝑥^∗_𝑡,𝑦_𝑡))−𝑦_𝑡=0. (4.25) Therefore 𝑥^∗_𝑡≥0.
To prove uniqueness, let 𝑥^∗_𝑖(𝑡),𝑖=1,2 be two solutions of (4.21). Then 𝑥^∗₁(𝑡)−𝑥^∗₂(𝑡) satisfies 𝑑(𝑥^∗₁(𝑡)−𝑥^∗₂(𝑡))=[(𝜓_𝑝(𝑢_𝑥(𝑥^∗₁(𝑡),𝑦_𝑡))−𝑦_𝑡)−(𝜓_𝑝(𝑢_𝑥(𝑥^∗₂(𝑡),𝑦_𝑡))−𝑦_𝑡)]𝑑𝑡,𝑥^∗₁(0)−𝑥^∗₂(0)=0. (4.26) We have 𝑑(𝑥^∗₁(𝑡)−𝑥^∗₂(𝑡))²=2(𝑥^∗₁(𝑡)−𝑥^∗₂(𝑡))𝑑(𝑥^∗₁(𝑡)−𝑥^∗₂(𝑡))=2(𝑥^∗₁(𝑡)−𝑥^∗₂(𝑡))[𝜓(𝑢_𝑥(𝑥^∗₁(𝑡),𝑦_𝑡))−𝜓(𝑢_𝑥(𝑥^∗₂(𝑡),𝑦_𝑡))]𝑑𝑡. (4.27) Note that the function 𝑥→𝜓(𝑢_𝑥(𝑥,𝑦)) is increasing. Hence (𝑥^∗₁(𝑡)−𝑥^∗₂(𝑡))²≤2^𝑡₀(𝑥^∗₁(𝑠)−𝑥^∗₂(𝑠))²𝑑𝑠. (4.28) By Gronwall's lemma, we have 𝑥^∗₁(𝑡)=𝑥^∗₂(𝑡),∀𝑡>0. (4.29) So, the uniqueness of (4.21) holds. The proof is complete.

Theorem 4.4. Under (3.5), the optimal production inventory cost control 𝑝^∗_𝑡 is given by 𝑝^∗_𝑡=𝜓_𝑝(𝑢^′(𝑥^∗_𝑡,𝑦_𝑡)), (4.30) where 𝑥^∗_𝑡 is defined by (4.21).

Proof. By Lemma 4.3, we observe that (𝑝^∗_𝑡) belongs to 𝒫. Now we apply Itô’s formula 𝑒^−𝜌𝑡𝑢(𝑥^∗_𝑡,𝑦_𝑡)=𝑢(𝑥,𝑦)+^𝑡₀𝑒^−𝜌𝑠{−𝜌𝑢(𝑥,𝑦)+𝑢_𝑥(𝑥,𝑦)(𝑝_𝑠−𝑦_𝑠)+𝐴𝑦𝑢_𝑦+1(𝑥,𝑦)2𝜎²𝑦²𝑢_𝑦𝑦(𝑥,𝑦)}|_{(𝑥=𝑥∗𝑠,𝑦=𝑦𝑠)}+𝑑𝑠_𝑡0𝑒^−𝜌𝑠𝜎𝑦_𝑠𝑢_𝑦(𝑥^∗,𝑦)𝑑𝑤_𝑠. (4.31) By the HJB equation (2.10), we have 𝑒^−𝜌𝑡𝑢(𝑥^∗_𝑡,𝑦_𝑡)=𝑢(𝑥,𝑦)−^𝑡₀𝑒^−𝜌𝑠{ℎ(𝑥^∗)+𝑝_𝑠^∗2}𝑑𝑠+^𝑡₀𝑒^−𝜌𝑠𝜎𝑦_𝑠𝑢_𝑦(𝑥^∗,𝑦)𝑑𝑤_𝑠, (4.32) from which 𝐸[𝑒^{⋀−𝜌(𝑡𝑇)}𝑢(𝑥^∗_𝑡⋀𝑇,𝑦_𝑡⋀𝑇)]+𝐸[^𝑡⋀𝑇₀𝑒^−𝜌𝑠{ℎ(𝑥^∗)+𝑝_𝑠^∗2}𝑑𝑠]=𝑢(𝑥,𝑦). (4.33) Then we obtain by (4.18) 𝐸[^∞₀𝑒^−𝜌𝑠{ℎ(𝑥^∗)+𝑝_𝑠^∗2}𝑑𝑠]=𝑢(𝑥,𝑦). (4.34)
Let 𝑝_𝑡 belong to 𝒫. By the same line as above, we have 𝑒^−𝜌𝑡𝑢(𝑥_𝑡,𝑦_𝑡)=𝑢(𝑥,𝑦)+^𝑡₀𝑒^−𝜌𝑠{−𝜌𝑢(𝑥,𝑦)+𝑢_𝑥(𝑥,𝑦)(𝑝_𝑠−𝑦_𝑠)+𝐴𝑦𝑢_𝑦1(𝑥,𝑦)+2𝜎²𝑦²𝑢_𝑦𝑦(𝑥,𝑦)}|_{(𝑥=𝑥𝑠,𝑦=𝑦𝑠)}+𝑑𝑠_𝑡0𝑒^−𝜌𝑠𝜎𝑦_𝑠𝑢_𝑦(𝑥,𝑦)𝑑𝑤_𝑠. (4.35) Again by the HJB equation (2.10), we can obtain 𝐸[𝑒^−𝜌𝑡𝑢(𝑥_𝑡,𝑦_𝑡)]≥𝑢(𝑥,𝑦)−𝐸[^𝑡₀𝑒^−𝜌𝑠{ℎ(𝑥_𝑠)+𝑝²_𝑠}𝑑𝑠]. (4.36) By the same line as Lemma 4.1 and by (4.1), 𝑢(𝑥,𝑦)≤𝐸[^∞₀𝑒^−𝜌𝑠{ℎ(𝑥_𝑠)+𝑝²_𝑠}𝑑𝑠],forany(𝑝_𝑡)∈𝒫. (4.37) Combining (4.34) with (4.37), we have 𝐽(𝑝^∗)≤𝐽(𝑝). (4.38) Therefore the optimal production cost control 𝑝^∗ which minimizes the production control problem (1.1) subject to (1.2) and (1.3). The proof is complete.

Acknowledgments

The authors wish to acknowledge the support provided by Fundamental Research Grant Scheme, no. 203/PJJAUH/671128, Universiti Sains Malaysia, Pinang, Malaysia, for conducting this research.