Discrete Dynamic Gaming Models in Supply Chain Management and Project ManagementView this Special Issue
Research Article | Open Access
Optimal Control of a Make-to-Stock System with Outsourced Production and Price-Sensitive Demand
We consider a make-to-stock system with controllable demand rate (by varying product selling price) and adjustable service rate (by outsourcing production). With one outsourcing alternative and a choice of either high or low price, the system decides at any point in time whether to produce or even outsource for additional capacity as well as which price to sell the product at. We show in the paper that the optimal control policy is of dynamic threshold type: all decisions are based on the product inventory position which represents the state of the system; there is a state dependent base stock level to decide on production and a higher level on outsourcing; and there is a state dependent threshold which divides the choice of high and low prices.
A case study of Mattel, the world’s largest toy maker, was done by Johnson  with a focus on its production capacity management. In particular, Johnson reported that Mattel owned a state-of-the-art die-cast facility in Penang, Malaysia, that was operating at full capacity to produce die-cast toy vehicles. Due to surge of demand for Hot Wheels, a core line of product, Mattel considered several options to expand production capacity, including one through the Vendor Operations Asia Division to outsource production in Asia-Pacific. VOA added flexibility to Mattel’s in-house manufacturing capability and was one of the company’s most valuable assets. In the meantime Mattel managed demand for the Hot Wheels through a new marketing strategy that changed the assortment mix of cars every two weeks.
In this paper, we consider a single-product make-to-stock system that has the option to increase production capacity by outsourcing to external contract manufacturers. The systems can also manage product demand through adjusting its selling price. For the basic setting of one outsourcing alternative and a choice of either high or low price, the system optimal control problem is to decide at any point in time whether to produce at the in-house facility or to outsource for additional capacity as well as which price to sell the product at. We model the production processes at the in-house and external facilities by exponential times of different means and the demand process by a Poisson process with a price-dependent rate. Thus, mathematically, the problem is optimal control of an make-to-stock queue with discretely adjustable production and demand rates.
With the objective to maximize the total discounted profit, we show in the paper that the optimal control policy is of dynamic threshold type: all decisions are based on the product inventory position which represents the state of the system; there is a state dependent base stock level to decide on production and a higher level on outsourcing; and there is a state dependent threshold which divides the choice of high and low prices. Furthermore, we show that, for a given outsourcing production capacity, all three thresholds of the optimal control policy and the associated optimal profit are decreasing in the outsourcing cost. This implies that outsourcing to lower cost facilities will lead to lower inventory holdings and lower selling price but higher profit.
There is a rich literature on the optimal control of make-to-stock queues, most of which take demand as exogenous and solve for the optimal control of the production rate. Typically, the optimal policy is of base stock type (produce at the maximum rate when the inventory holding falls below certain level, otherwise halt production), which was first proved by Gavish and Graves  and Sobel  for the single product and single machine case. Later works of Zheng and Zipkin , Wein , Veach and Wein , and Bertsimas and Paschalidis  attempt to extend the base stock policy to the multiple product cases. There are also extensions to the case of single product but with multiple demand classes, like Ha [8–10]. Another direction of extension has been to incorporate more detailed modeling of the production facility. For example, Kapuscinski and Tayur  model the production process by a tandem queue, and Feng and Yan  and Feng and Xiao  deal with unreliable production facilities.
Li  and Chen et al. [15, 16] are three works that incorporate controls on both production and demand processes in a make-to-stock queue optimization problem. Li  assumes a continuous spectrum of product selling prices and corresponding demand rates and, hence, manages to derive a concave and differentiable profit function in terms of the production rate and the selling price, which yields a qualitative characterization of the optimal policy. Chen et al.  allow discrete choices of prices and derive an efficient algorithm to compute the optimal policy as well as its qualitative characterization. Chen et al.  consider a make-to-stock manufacturing system with batch production and discrete choices of price and derive the characterization of the optimal control policy. Similar work on a make-to-order queue is done by Ata and Shneorson . Carr and Duenyas  study the optimal control of a mixture of make-to-stock and make-to-order queues. Our work adds in another dimension with the outsourcing option to expand production capacity.
The rest of the paper is organized as follows. Section 2 describes precisely the system model and defines an optimization problem that solves for the optimal policy. Section 3 characterizes the optimal threshold policy, proves its global optimality amongst all nonanticipative control policies, and discusses its relationship to the cost of outsourcing. Section 4 briefly discusses the extension to multiple price choices. Section 5 concludes the paper with a summary of the results and possible extensions in the future research.
To streamline presentation of the paper, we state in the main body of the paper all the results without proofs and collect all the proofs in the appendix.
2. Problem Formulation
The make-to-stock system of concern in the paper has an in-house facility with a production rate and a unit production cost . The system can outsource production to an external facility which can produce at a rate and a per unit cost . We assume that the existing in-house facility has a lower variable production cost than the external facility, that is, , which holds true in the case of Mattel, for example, and is the reason for keeping the in-house facility. We also assume that the production processes at both facilities are random and follow exponential distributions.
The demand process for the product is assumed to be a Poisson process with a price-dependent rate. Specifically, there are two selling prices: high and low , which correspond to two demand rates: and . We assume that and which indicates that the marginal profit gain from switching price from high to low is greater than the in-house production cost . Also to ensure system stability, we assume that .
When a demand arrives, it is filled from the finished goods inventory if possible; otherwise, it is added to a waiting queue which is served in first-come-first-serve order. The finished goods inventory carries a holding cost of per unit product per unit time, and the backordering cost for demand unmet at arrival is per waiting demand per unit time. Define the inventory cost function , where and . We will consider the total discounted profit, assuming a discount factor . To ensure that it is more profitable to produce to fill demand than to backlog forever, we assume that .
We specify a dynamic control policy for the system by , where or 1 representing in-house production is off or on; similarly, or 1 representing outsourced production being off or on, and or representing the price charged at time . A policy is called nonanticipatory if, at all , , , and depend only on information prior to . Let be the collection of all nonanticipatory control policies. Under a given , denote the total demand sold at price up to time by , the total in-house production by , and the total outsourced production by . Then, the product inventory level at time is given by where is the initial inventory at .
Consequently, the total discounted profit under policy is
A policy is said to be optimal if it solves the following optimization problem:
The optimal solution to this semi-Markov decision problem can be characterized by the following Hamilton-Jacobi-Bellman (HJB) equation (cf. Chapter 7 of ):
Since , we have when , and thus, when . In essence, we can envisage an effective production process with rates , , and corresponding to the unit production costs , , and , respectively. As a result, HJB equation (5) can be simplified to
We are especially interested in a class of control policies which are parameterized by three thresholds: , , and , with . An policy decides on production, outsourcing, and pricing in the following manner: (1) when the inventory is above or equal to , there is no production at the in-house facility and no production outsourcing; (2) when it is below and above or equal to , production is on at the in-house facility but there is no outsourcing; (3) when it is below , production is on at the in-house and outsourced to the external facility; (4) the product sale price is set low at when it is above , and the produce selling price is set low at ; otherwise, the price is high at . In Section 3 below, we characterize the best policy and verify that it satisfies the above HJB equation (6) and, thus, is optimal amongst all policies in .
3. Optimality of Policy
The HJB equation (6) can be made more specific when given an policy. For example, for an policy with , it can be simplified to the following equations.
For , for , for , for , and for ,
Let be the profit function of a given policy. The following lemma characterizes some of its limiting behaviors.
Lemma 1. The profit function of an policy has the following limits: (1);(2);(3);(4); and .
Our approach is to first find the best policy and then to prove its global optimality.
Definition 2. An policy is said to be better than an policy if for any initial inventory level and at least for an , the inequality holds strictly. It is said to be the best policy if no other policies are better.
For notational simplicity we also define a first-order difference operator :
The following lemma provides means to compare policies.
Lemma 3. Comparing with a given policy, we have the following results: (1)the policy is better if and only if (2)the policy is better if and only if (3)the policy is better if and only if
The first comparison implies that, under either the policy or the policy, if the marginal profit at inventory level for producing one more unit of product is higher than the in-house production cost, then the policy is better than the policy. Similar implications can be drawn from the other two comparisons.
Lemma 3 leads to the following characterization of the best policy.
Theorem 4. If the policy is the best policy, then (1)at the best base stock level , (2)at the best outsourcing threshold , (3)at the best price switch threshold , Furthermore, we have that is concave in .
Finally, we obtain the main result of the paper based on the characteristics of Theorem 4 as stated above.
Theorem 5. If the policy is the best policy, then its profit function satisfies the HJB equation (6). Hence, the policy is optimal amongst all nonanticipative policies of .
The optimal policy has some important properties which we list below.
Theorem 6. In the best policy, ; and if ; otherwise .
Theorem 6 tells us two results as the following. (i) Under the best thresholds policy, the maximum inventory level is nonnegative; and it is optimal to idle both the in-house and outsourced facilities when the stock level is above the maximum inventory level . (ii) When production is on at both in-house and outsourced facilities and the inventory is increasingly building up, if the marginal profit gain when switching price from high to low is greater than the outsourced production cost , it is more profitable to first stop outsourced production than to first stop in-house production and then decrease price. The converse is true if the marginal profit gain when switching price from high to low is smaller than the outsourced production cost .
Theorem 7. As for fixed sourcing production rate , suppose that the variable cost of outsourced production is negotiable. Let policy be the optimal threshold policy associated with a cost . Then, (1)the optimal thresholds , are piecewise constant, increasing functions of , but is a piecewise constant, decreasing function of .(2)as for the optimal profit function, is decreasing in .
Theorem 7 concludes the following results. (i) A lower variable cost from outsourced production will lead to lower product selling price but higher safety stock level and more outsourced production. And (ii) a lower variable outsourced production cost will lead to higher optimal long-term discounted profit.
4. Extension to Multiple Price Choices
In this section, we briefly discuss the extension of the results in the previous section to multiple price choices. Namely, we have now possible prices to choose from for the selling of the product: , with corresponding demand arrival rates: .
We further assume that the profit rates are also increasingly ordered; that is,
It can be shown that if the profit rates do not follow this order, a dominating subset of the price levels can be chosen to make the other prices unattractive for selection. Readers are referred to Chen, Feng, and Ou  for the analysis on dominating prices.
The price choice model can be optimized by a policy which maximizes the right-hand side of the following Hamilton-Jacobi-Bellman equation: where is the profit function of the policy.
The natural extension of the policy as defined at the end of Section 2 is a -level control policy characterized by parameters.
Consider with and . is the base stock level on and above which there is no production at the in-house facility and also no production outsourcing. And when it is below and above or equal to , production is on at the in-house facility but no outsourcing; when it is below , production is on at the in-house and outsourced to the external facility. The other parameters are price switch thresholds: when the inventory is in the range , (assuming and ), the product is sold at price .
For a given policy, the profit function can be calculated recursively as
Qualitatively, the -level policies possess the following properties similar to those for the two-price case.
Proposition 8. Let be the long run total discounted profit of the -level policy when the initial inventory is ; then (1)the policy is better than the policy if and only if(2)the policy is better than the policy if and only if(3)the policy cannot be better than the policy if ; (4)the policy is better than the policy if and only if or
We also have the following characterization of the best -level policy.
Theorem 9. The best -level policy is characterized by the following relations: (1)the profit function is concave in integer value of ;(2)the best base stock level satisfies and (3)at the optimal threshold , (4)at the best price switch thresholds , ,
The following Theorem 10 characterizes the impact of souring cost on the optimal discounted profit and its optimal thresholds.
Theorem 10. As for fixed sourcing production rate , suppose that the variable cost of outsourced production is negotiable. Let policy be the optimal threshold policy associated with a cost . Then, (1)the optimal thresholds are piecewise constant, increasing functions of , but is a piecewise constant, decreasing function of .(2)the optimal profit function is decreasing in .
5. Concluding Remarks
We have analyzed the optimal control of a single-product make-to-stock system that has the option to increase production capacity by outsourcing to external contract manufacturers and the option to vary product selling prices. Idealizing the system by a simple make-to-stock queue model with discretely adjustable production and demand rates, we obtain a complete characterization of the optimal threshold control policy and prove beneficial impact of low cost outsourced production to the system. We wish to point out that the results can be easily extended multiple outsourcing alternatives and multiple price choices. It should also be possible to extend to the case of multiple demand classes that compete for the same product. More challenging extensions will be on incorporating fixed cost of outsourcing into the model as well as having multiple product classes in the model.
In order to simplify writing, we let
Proof of Lemma 1. First, we attempt to show that . To this end, we let . By definition, function can be derived recursively from the recursion of the policy. In particular, for ,
which leads to
where . These equations establish that if , then and for all , and, further, if , then and for all . Moreover, a limiting behavior of is deduced by the fact that
Next, we are proving a parallel result that
To this end, we consider and define a shift operator such that for any function . The inverse of is expressed formally as . Using we can derive the corresponding characteristic equations for linear recursion (11). Then characteristic equation of recursion (11) is or which has two solutions: It can be checked that .
Then the homogeneous solution to (11) is in the form of where and are constants to be determined at . Note that when , increases to exponentially. However, we do not expect to grow or diminish exponentially when and, hence, postulate that . To determine a particular solution to (11), we suppose it is where , , are to be determined from the following equation:
Comparing the coefficients of constant and linear terms between the two sides yields
Hence, the general solution to (11) becomes
Denoting by the coefficient of for , it is deduced from (A.13) that
Then we get
We can prove (2) and (3) by analogy with this logic and therefore omit its proof.
Now we turn to prove and . Similar to the preceding methodology in the proof of , we can write the general solution of for as the following: where is a constant to be determined at . And for , we get the profit function from (A.13) as follows: where is a constant to be determined at and
Combining and , we get
Proof of Lemma 3. For brevity, we provide only the proof that policy is better than policy if and only if or . The proofs of other two essential and sufficient conditions, (2) and (3), are similar, so including them here is not additionally illustrative.
We first prove inequality implies that policy is better than policy. For that we focus on the case that ; the other cases can be proved in the same fashion.
Let and make use of (7)–(11) to obtain where , .
We know that . While in line with Lemma 1, we get that
Hence, when is a large enough positive integer and , we can approximately write the following equation for :
So, we can verify that the coefficient matrix of the linear equations about variable , is totally nonpositive. Hence, by solving a finite approximate negative system of linear equations, we can corroborate that for all . Thus we get that for all when . At last, there exists an such that because .
Consequently, the policy is better than policy. To this end, we reemploy all above equations but change equation with as follows:
The rest follows exactly the same as above.
We now prove the reverse. And we suppose , or . The above proof steps will lead to (rather than ) for any ; that is, .
Proof of Theorem 4. Based on Lemma 1, we find that (1), (2), and (3) are true. So we only need to prove is concave. For simplicity, we give the proof for the case of . The other cases can be analogously analyzed. To simplify the notation, we drop the superscript * in the proof. Define and , then that is concave is equivalent to for all .
Based on (7)–(11), we obtain the following system of equations:
Evaluating the above for , at and and from their difference, we derive, for ,
Similarly, we derive where , , , , and .
Based on the above equations, we know that and the policy being the best implies , therefore we have .
Equation (A.26) is equivalent to the following equation after taking into account : which gives rise to based on the above result .
Equation (A.25) shows that when , . Thus based on , we have for .
By adding the two sides of (A.26) and (A.27), we obtain Therefore we have .
To consider , we define , and, for ,
Of course, and . Besides that, it is easy to prove that for , and the limit implies that . It can be verified that the coefficient matrix of the linear equation about variable from to is totally nonpositive, where is a large enough positive integer greater than . Thus for all . It implies that for all , and thus, is concave function of .
Proof of Theorem 5. For simplicity, we assume that , and for the other cases, we can argue them in the same fashion. Based on the concavity of the optimal discounted profit function , we know that is a decreasing function of . Hence, for , for , for , and for , which yields Combining the above five inequalities, we can infer that satisfies the Hamilton-Jacobi-Bellman equation (5). It follows, therefore, that the best policy is globally optimal.
Proof of Theorem 6. Given an policy with , we assume that the policy is optimal. For simplicity, we only consider the case , and the other case can be analogously analyzed. While for policy with , we obtain that , where for , satisfies the following equations:
Let where ; then we obtain, through a series of subtractions:
We approximate the above infinite system of linear equations with a finite one involving with being a large enough positive integer greater than . The last equation in the finite system, corresponding to , is where is crossed out. As argued before in the proof of Lemma 3, the coefficient matrix of above linear equations about variable , , is invertible, and, hence, for . And then we get where ; that is,
Furthermore, we have the following equation:
Lemma 1 implies that Because of and , we get that is, . However, this relation invalidates based on the assumption that the policy is optimal.
If , then we have
Based on the concavity of the optimal discounted profit function , we get . Similarly, we get if .
Proof of Theorem 7. We let
where means the optimal thresholds policy corresponding to the sourcing production cost , where sourcing production rate is fixed.
(1) In the proof, we focus on the case that ; the other cases can be proved in the same fashion. As for a fixed sourcing production rate , let satisfy the recursions as follows: