Journal of Optimization

Volume 2013 (2013), Article ID 960879, 12 pages

http://dx.doi.org/10.1155/2013/960879

## Dynamic Optimization Technique for Distribution of Goods with Stochastic Shortages

Department of Mathematics, Faculty of Physical Sciences, University of Benin, P.M.B. 1154, Benin City, Edo State, Nigeria

Received 2 April 2013; Accepted 24 September 2013

Academic Editor: Eric S. Fraga

Copyright © 2013 Charles I. Nkeki. This is an open access article distributed under the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.

#### Abstract

This work considers the distribution of goods with stochastic shortages from factories to stores. It is assumed that in the process of shipping the goods to various stores, some proportion of the goods will be damaged (which will lead to shortage of goods in transit). The cost of the damaged goods is added to the cost of the shipment. A proportion of the total expected cost of the shortage goods is assumed to be recovered and should be deducted from the total cost of the shipment. In order to determine the minimum transportation costs for the operation, we adopt dynamic optimization principles. The optimal transportation cost and optimal control policies of shipping the goods from factories to stores were obtained. We find that the optimal costs of the goods recovered could be determined. It was further found that the optimum costs of distributing the goods with minimum and maximum error bounds coincide only at infinity.

#### 1. Introduction

A distribution company plans to minimize the cost of distributing kinds of products from number of factories to number of stores. For the case of a company distributing a particular product from one factory to all the stores (a single product), that is, at factory , product is distributed to all the stores and factory produce products and are distributed to all the stores, and so on; see Nwozo and Nkeki [1]. It is also expected that the goods that leave the factories will not come back to the factories (in the case of defective, damaged, etc., items). The company considered number of control policies to determine which of them will yield the optimum control policy. They also estimated that certain percentage of the products are to reach their final destination successfully at minimum costs. It is assumed that some proportion of the shortage goods should be recovered. The recovered costs of the shortage goods should be deducted from the total costs continuously over time horizon.

In the related literature, Powell [2] considered the problem of a stochastic fleet assets management problem and used the postdecision state variable implicitly to determine this problem. Closely related to this on the extensive work on stochastic fleet assets management problem is the work by Cheung and Powell [3]. Godfrey and Powell [4] further used postdecision state variable explicitly to determine similar problem on a stochastic fleet management problem. Van Roy et al. [5] proposed the idea of using a parsimonious sufficient statistics in an application of approximate dynamic programming to inventory management. Mulvey and Vladimirou [6] used the stochastic programming technique in financial asset allocation problems for designing low-risk portfolios, and Mulvey and Ruszczynski [7] designed variety of specialized algorithms that help to solve the problems which involved management of homogeneous resources. They further emphasized that for more realistic problems, resources are heterogeneous and that the resulting optimization problems do not generally exhibit pure network structure. Powell et al. [8] described an algorithm for computing parameter values to fit linear and separable concave approximation to the value function to tackle large-scale problems in transportation and logistics. Furthermore, Powell and Topaloglu [9] described a more complicated variation of the algorithm that improved on execution time and memory requirements and such improvement is often critical for practical application to realistic large-scale problems. They further emphasized that the movement of freight over long distances is subject to random delays. Powell and Van Roy [10] studied and applied approximate dynamic programming to high-dimensional resource allocation problems in area of managing a fleet of trucks. Powell [22] worked extensively on an approximate dynamic programming for large-scale asset management problems for both single and multiple assets. Guestrin et al. [11] used an approximate dynamic programming approach to deal with high-dimensional decision spaces. For now, no other applications of approximate dynamic programming have dealt with such large numbers of decision variables (see, Powell and Van Roy [10]). Spivey and Powell [12] studied the dynamic assignment problem, where a resource can serve only one task at a time. They further proposed a general class of dynamic assignment models that involved adaptive and nonmyopic algorithm. Cogill et al. [13] considered the problem of computing decentralized policies for stochastic systems with finite state and action spaces. They presented an algorithm based on linear programming and used this algorithm to obtain a decentralized policy from a function with special structure that approximates the optimal centralized Q-function. Topaloglu and Kunnumkal [14] proposed two approximate dynamic programming methods to optimize the distribution operations of a company manufacturing a certain product at multiple production plants and ship to different customer location for sale. Hauskrecht and Singliar [15] considered real-world distributions and networks which are unreliable and subject to random failures of its components. They investigated MonteCarlo solutions in stochastic networks in which the expected value of resources allocated before and after the occurrence of stochastic failures needs to be optimized. Ozbay et al. [16] proposed mathematical programming models with probabilistic constraints in order to address incident response and resource allocation problems for traffic incident management. They considered the resource allocation problem by assuming that the stochastic distribution of incidents over a network is given and introduced a mathematical model to determine the number of service vehicles allocated to each depot to meet the requirements of the potential incidents by taking into account the stochastic nature of the resource requirement and incident occurrence probabilities. Jacko [17] considered resource allocation problem in telecommunication system. They optimized packet losses by choosing preferred paths in the routing system. They assumed that there are a number of independent competitors with the aim of maximizing the expected return. Elmaghraby and Ramachandra [18] considered the problem of optimally allocating a single resource under uncertainty to the various activities of a project to optimize a certain economic objective composed of resource utilization cost and tardiness cost. Nwozo and Nkeki [19], used DPP to consider the allocation of buses from single station to different routes in Nigeria for profit maximization. Nkeki [20] considered the allocation of vehicles from different (multiple) stations to different routes putting into consideration the rate of breakdown of the vehicles during the allocation but failed to put into consideration the depreciation of the vehicles and effect tax on the returns of the investment. Nwozo and Nkeki [21] studied the allocation of resource to different tasks in a production company so as to maximize returns. They assumed that the company produces the same kinds of products and allocate number of tasks to number of machines. They also assumed that the machines are subject to random breakdown.

This paper discussed the distribution of goods from different factories to stores with stochastic shortage over time horizon.

The highlights of this paper are(a)modeling the distribution of goods from factories to stores in the presence of(i)stochastic shortages of the goods in transit and(ii)deterministic case (i.e., when there are no shortages) was also considered, using a dynamic optimization technique,(b)numerical determination of optimal cost and optimal control policy for the operations was obtained.

The remaining parts of the paper are structured as follows: Section 2 presents the problem formulation and multiperiod expected cost function. In Section 3, we present the dynamic programming formulation of the problem. The optimization of the cost function is determined in this section. Section 4 presents the dynamic programming principles with error bounds. In Section 5, numerical simulations of the problem are discussed. Section 6 concludes the paper.

#### 2. Problem Formulation

The paper studies a problem where there are states and control policies. Let be the set of goods produced in the factories and let be the set of goods that are to be distributed to the stores, where and is the state space of the system. The transition probability matrix corresponding to the control policies is given as The transition costs are given by , , , where is the terminal time period in the planning horizon, represents the state of goods produced in factory at period , represents the state of goods to be distributed to store at period , and the discount factor , . We define the function as the cost of the distribution of the goods corresponding to the control policies , . The function represents the number of different kinds of goods produced in factory at period under policy and represents the number of different kinds of goods distributed to store at period under the policy .

##### 2.1. Multiperiod Expected Cost Function

Suppose that the costs of distributing the goods from factory to store , are at period , where the state of goods in factory is at period and the state of goods to store is at period . Then, the costs over -horizon are Since is the number of different kinds of goods produced in factory at period under policy and and the number of different kinds of goods distributed successfully to store at period under policy , then

The expected minimum discounted cost function obtained under control policy , at period , is given as follows: subject to where is the set of feasible solutions of problem (4). We can express (4) as the expected discounted minimal cost from period onward as an optimization over condition on as follows: subject to the constraints (5).

For a function , if we accumulate the cost of the first -stage and add to it the terminal cost , then (6) becomes subject to the constraints (5).

#### 3. Dynamic Programming Formulation

In this section, we formulate the problem using dynamic programming technique. Since is the state variable at period and the state space, we formulate the problem as a dynamic program. Let the cost of the goods that is to be distributed from factory to store at period under policy be given by , where is the transition probability of the goods from factory to store . Hence, the total expected cost of the goods distributed from factory to store at period is given by Let be the proportion of the total expected cost of goods lost in transit (shortage) from factory to store . Then, (8) becomes Equation (9) represents the total gross shortage cost incurred from the goods lost in transit.

Let be the cost of distributing the goods from the factories to store in period and the expected cost of distributing the goods from the factories to store at period and let be the estimated proportion of the goods that are to be recovered from the lost ones, which are expected to go to the stores at period , so that the net shortage cost incurred from the goods lost in transit will be It then follows that Equation (11) is the transformation equation and is stochastic. We now have the following propositions.

Proposition 1. *If or , then
*

This proposition tells us that the transformation equation (11) is deterministic if or . It implies that all the goods distributed from the factories to stores get to their various destination successfully without shortages.

Proposition 2. *Suppose that there is no recovery of any of the goods lost in transit and ; then must be zero and (11) becomes
*

The optimal policy can be found by computing the value functions through the optimality equation subject to the constraints (5).

Equation (14) can be rewritten as follows: subject to Equivalently, where is the quantity of products that is lost in transit from factory to store at period and . If is a zero vector, it implies that all the quantities of products that left the factories get to the stores successfully without damages or shortage. It can be shown that (6) is equal to (14) (see Powell, [22]). We may use (6) and (14) interchangeably. We now find the best control policy, , which minimizes our problem; that is, we search for We do that by solving the optimality equation subject to Let be the expected cost per stage. Using the mapping and , we have

Theorem 3. *Consider the following.*(i)*Let be the set of all bounded real-valued functions . The mapping is a contraction.*(ii)*The operator has a unique fixed point (given ).*(iii)*For any , .*(iv)*For any , if , then , .*

*Proof. *(See [22]).

We now make the following assumptions.

*Assumption 4. *Consider the following.
(ii) The discount factor .

Theorem 5. *Let the bounded optimal cost function be -dimensional vectors. Then satisfies
*

*Proof. *Let be a positive integer, and policy ; we can decompose the return
into the portion received over the first stages and over the remaining stages:
But
Therefore,
It therefore follows that
By taking the infimum over , we obtain for all and
and by taking the limit as , we obtain
Hence,
This result shows that our optimization problem converges to a fixed point in an infinite horizon. It therefore follows that

#### 4. Dynamic Programming Principles with Error Bounds

In this section, the optimality equation with error bounds is considered. We established the minimum and maximum error bounds of our problem. We show that at infinity the minimum and maximum error bounds cancelled out. Let But It then follows that where and are such that Equation (37) can be improved upon as follows: For a vector , we compute By subtracting (40) and (41), we have is the cost vector associated with the control policy and is the cost per stage vector. It then follows from (37) that for we have Equivalently, for the vector , we have where and . Equation (45) can be improved upon as follows. From (40), we have . It follows that From (46), we obtain Now, combining (46) and (47), we have Substituting (45) into (50), we have After steps, we have As , we have Let ; we have Let in (54); we have From (46), we have that and . Therefore, we have It implies that Similarly, . We therefore have that This is the dynamic programming (value iteration) that is bounded above and below. The value , is the value iteration with minimum error bound and is the value iteration with maximum error bound. Now taking the limit as tends to infinity, we have that ; it now implies that and for ,

#### 5. Numerical Simulation

In this section, we present the numerical simulation of the company problem. The state transition diagram of the distribution of goods from factories to stores, under the control policies , is given by the transition diagram in Figure 1.

The transition probability matrix of distributing the goods from factory to store under the control policies , is presented below:

The initial estimate of the cost of distributing the goods from factory to store in 100000 is given below.

The company estimates of the total goods as shortage in transit and of the total expected shortage to be recovered and should be deducted from the total cost of the shipment. The cost of the damaged goods is added to the cost of the shipment. MATLAB was used to solve the problem. The results are presented in the tables.

The optimal discounted costs of distributing the goods from factories to stores with shortages are presented in Tables 1 to 7 and Figures 2 and 3, while optimal discounted costs of distributing the goods from factories to stores without shortages are presented in Tables 8 to 10 and Figure 5. Table 1 presents the minimum costs of distributing the goods from various factories to all the stores. Here, error bound is not considered. It was obtained that the minimum costs of distributing the goods from factory 1 to all the eight stores are 5,716,200 naira. The minimum costs of distributing the goods from factory 2 to the eight stores are obtained as 6,034,000 naira, from factory 3 to the eight stores are 6,428,400 naira, and so on. It was observed that the costs of distributing the goods from factory 7 to all the stores yield the highest amount. This may be due to longer distance, nature of the roads, or more shortage costs.

Table 2 presents the optimal costs of distributing goods with minimum error bound from the factories to stores and Table 3 presents the optimal costs of distributing the goods with maximum error bound from the factories to stores. It was observed that the minimum costs of distributing the goods with minimum and maximum error bounds coincide only at infinity.

Table 4 shows the optimal costs of distributing the goods without error bound and where there is no recovery of some damaged goods (i.e., when ). It is observed from Tables 1 and 4 that the costs of distributing the goods will increase when the value of decreases and vice versa. This is an expected result. Tables 5 and 6 show the optimal costs of distributing the goods with minimum and maximum error bounds, respectively, and where there is no recovery of some damaged goods.

The additional costs incurred and the corresponding percentage increase in the additional costs incurred when are shown in Table 7.

In Table 7, it was found that the costs of recovered goods in transit from factory 1 to the stores are 228,600 naira with 3.85% when there is no EB and 220000 naira with 3.70% when there is EB from factory 2 to the stores 230,800 with 3.68% when there is no EB, and 240,000 with 3.83% when there is EB and so on.

Figure 2 shows the bar chart representing the minimum costs of distributing the goods from factories to stores by setting , , and . It was observed from Figure 2 that optimum costs can be attained under policy 1. Hence, the optimum policy for the operation is . We therefore deduce from Figure 2 that . Figure 3 shows the bar chart representing the minimum costs of distributing the goods from factories to stores by setting , , and . This is when there are no goods recovered from the expected lost ones in transit. We also found that policy 1 yields the optimum costs. Figure 4 shows the relationship between the minimum and maximum error bound. It was observed that at infinity, the minimum and maximum error bounds coincide. We referred to this point as the point of harmony. Comparing the optimal discounted costs of the distribution of goods in Tables 1, 8, 2, 9, 3, and 10 and Figures 2 and 5, we found that the optimal cost is at minimal when there are no shortages, which is an expected result. It is obvious that the higher the shortages, the higher the cost of the shipment. This may lead to the high cost of the goods on the final consumers.

#### 6. Conclusion

We have considered distribution of goods with stochastic shortages from factories to stores. This paper modeled the distribution of goods from factories to stores in the presence of stochastic shortages of the goods in transit, and the deterministic case (i.e., when there are no shortages) was also considered, using a dynamic optimization technique. Numerical determination of optimal cost and optimal control policy for the operations were obtained. We obtained the optimal transportation cost of shipping goods from factories to stores. The optimal control policies were also obtained. We found that the optimal costs of the goods recovered could be determined. It was found that the optimum costs of distributing the goods with minimum and maximum error bounds coincide only at infinity. We referred to this point as a “point of harmony.”

#### References

- C. R. Nwozo and C. I. Nkeki, “On distribution of goods using dynamic programming principles with error bounds,”
*Journal of Applied Mathematics and Bioinformatics*, vol. 2, no. 1, pp. 1–19, 2012. View at Google Scholar - W. B. Powell, “A comparative review of alternative algorithms for the dynamic vehicle allocation problem,” in
*Vehicle Routing: Methods and Studies*, B. Golden and A. Assad, Eds., pp. 249–292, North-Holland, Amsterdam, The Netherlands, 1988. View at Google Scholar - R. K. Cheung and W. B. Powell, “AN algorithm for multistage dynamic networks with random arc capacities, with an application to dynamic fleet management,”
*Operations Research*, vol. 44, no. 6, pp. 951–963, 1996. View at Google Scholar · View at Scopus - G. A. Godfrey and W. B. Powell, “An adaptive dynamic programming algorithm for dynamic fleet management, I: single period travel times,”
*Transportation Science*, vol. 36, no. 1, pp. 21–39, 2002. View at Publisher · View at Google Scholar · View at Scopus - B. van Roy, D. P. Bertsekas, Y. Lee, and J. N. Tsitsiklis, “A neuro-dynamic programming approach to retailer inventory management,” in
*Proceedings of the 36th IEEE Conference on Decision and Control*, pp. 4052–4057, December 1997. View at Scopus - J. M. Mulvey and H. Vladimirou, “Stochastic network programming for financial planning problems,”
*Management Science*, vol. 38, no. 11, pp. 1642–1664, 1992. View at Publisher · View at Google Scholar - J. M. Mulvey and A. J. Ruszczynski, “A new scenario decomposition method for large-scale stochastic optimization,”
*Operations Research*, vol. 43, no. 3, pp. 477–490, 1995. View at Publisher · View at Google Scholar - W. B. Powell, A. Ruszczynski, and H. Topaloglu, “Learning algorithms for separable approximations of stochastic optimization problems,” Tech. Rep., Department of Operations Research and Financial Engineering, Princeton University, 2002. View at Google Scholar
- W. B. Powell and H. Topaloglu, “Stochastic programming in transportation and logistics,” in
*Handbook in Operation Research and Management Science*, A. Ruszczynski and A. Shapiro, Eds., pp. 249–292, Elsevier, Amsterdam, The Netherlands, 2003, Volume on Stochastic Programming. View at Google Scholar - B. W. Powell and B. van Roy, “Approximate dynamic programming for high-dimensional resource allocation problems,” in
*Handbook of Learning and Approximate Dynamic Programming*, IEEE Press, New York, NY, USA, 2003. View at Google Scholar - C. Guestrin, D. Koller, and R. Parr, “Efficient solution algorithms for factored MDPs,” Tech. Rep., 2003. View at Google Scholar
- M. Z. Spivey and W. B. Powell, “The dynamic assignment problem,”
*Transportation Science*, vol. 38, no. 4, pp. 399–419, 2004. View at Publisher · View at Google Scholar · View at Scopus - R. Cogill, M. Rotkowitz, B. van Roy, and S. Lall,
*An Approximate Dynamic Programming Approach to Decentralized Control of Stochastic Systems*, Stanford University Press, 2004. - H. Topaloglu and S. Kunnumkal, “Approximate dynamic programming methods for an inventory allocation problem under uncertainty,”
*Naval Research Logistics*, vol. 53, no. 8, pp. 822–841, 2006. View at Publisher · View at Google Scholar · View at Scopus - M. Hauskrecht and T. Singliar, “Monte-Carlo optimizations for resource allocation problems in stochastic network systems,” http://arxiv.org/abs/1212.2481.
- K. Ozbay, W. Xiao, C. Iyigun, and M. Baykal-Gursoy, “Probabilities programming models for dispatching and resource allocation in traffic incident management,” 2004, http://ie.rutgers.edu/resource/research_paper/paper_05-025.pdf.
- P. Jacko, “Stochastic programming framework for resource allocation and (nearly-) optimal priority rules,” BCAM-EHU/UPV-Robotiker Joint Workshop, 2009.
- S. E. Elmaghraby and G. Ramachandra, “Optimal resource allocation in activity networks under stochastic conditions,” 2009, http://www.ise.ncsu.edu/people/faculty/docs/elmaghrabyPaper.pdf.
- C. R. Nwozo and C. I. Nkeki, “On dynamic optimization technique for resource allocation problems in a transportation network,”
*Journal of Mathematical Association of Nigeria*, vol. 36, pp. 30–37, 2009. View at Google Scholar - C. I. Nkeki, “The use of dynamic optimization technique for the allocation of buses from different stations to different routes by a transportation company in Nigeria,”
*International Journal of Computational and Applied Mathematics*, vol. 4, pp. 141–152, 2009. View at Google Scholar - C. R. Nwozo and C. I. Nkeki, “On a dynamic optimization technique for resource allocation problems in a production company,”
*American Journal of Operations Research*, vol. 2, pp. 357–363, 2012. View at Publisher · View at Google Scholar - W. B. Powell,
*Approximate Dynamic Programming For Asset Management*, Princeton University Press, Princeton, NJ, USA, 2004.