- About this Journal ·
- Abstracting and Indexing ·
- Advance Access ·
- Aims and Scope ·
- Annual Issues ·
- Article Processing Charges ·
- Articles in Press ·
- Author Guidelines ·
- Bibliographic Information ·
- Citations to this Journal ·
- Contact Information ·
- Editorial Board ·
- Editorial Workflow ·
- Free eTOC Alerts ·
- Publication Ethics ·
- Reviewers Acknowledgment ·
- Submit a Manuscript ·
- Subscription Information ·
- Table of Contents
Discrete Dynamics in Nature and Society
Volume 2014 (2014), Article ID 840725, 11 pages
A Stochastic Dynamic Programming Approach Based on Bounded Rationality and Application to Dynamic Portfolio Choice
Business School, Central South University, Changsha, Hunan 410083, China
Received 14 March 2014; Accepted 5 May 2014; Published 22 May 2014
Academic Editor: Fenghua Wen
Copyright © 2014 Wenjie Bi et al. This is an open access article distributed under the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.
Dynamic portfolio choice is an important problem in finance, but the optimal strategy analysis is difficult when considering multiple stochastic volatility variables such as the stock price, interest rate, and income. Besides, recent research in experimental economics indicates that the agent shows limited attention, considering only the variables with high fluctuations but ignoring those with small ones. By extending the sparse max method, we propose an approach to solve dynamic programming problem with small stochastic volatility and the agent’s bounded rationality. This approach considers the agent’s behavioral factors and avoids effectively the “Curse of Dimensionality” in a dynamic programming problem with more than a few state variables. We then apply it to Merton dynamic portfolio choice model with stochastic volatility and get a tractable solution. Finally, the numerical analysis shows that the bounded rational agent may pay no attention to the varying equity premium and interest rate with small variance.
In reality, how to choose an asset’s portfolio of consumption and investment is one of the most important decisions for many people. In modern portfolio choice field, Merton [1, 2] provides a general framework for understanding the portfolio demand of long-term investors when investment opportunities change over time. In a classical Merton model [1, 2], however, the riskless interest rate, the risky mean rate of return, and the volatility coefficient are usually assumed to be constant. These assumptions are lack of realism, particularly over long time intervals. A large volume of empirical researches in financial market which indicates the assumption that these variables are stochastic volatile and follow a certain stochastic process (e.g., Ornstein-Uhlenbeck process) is more realistic [3, 4]. But when introducing these stochastic variables into the Merton-style portfolio choice model, the problem becomes increasingly complicated and formidable to solve. Also, this will lead to the “Curse of Dimensionality.” Quite a lot of approaches have been developed to deal with this kind of problems, such as martingale methods [5–8] and various approximate numerical algorithms [9–12]. However, these methods have more restrictive assumptions and are too complex to get a tractable solution of strong explanations. Based on the control of small noise, Judd and Guu  proposed a method to solve dynamic programming problems with stochastic disturbance. He makes the simplifying assumption that uncertainty is small and obtains the first- and high-order solutions of complicated dynamic programming model. This method provides a quite suitable solution for dynamic portfolio choice model with stochastic volatility.
On the other hand, a growing body of empirical studies indicate that the agent considers only the variables with high fluctuations but ignores those with small ones [14–16]. Bordalo et al.  showed that the agent rationally chooses to be inattentive to news. Kőszegi and Szeidl  analyzed the monetary policy and found out that when price is changed, the decision makers are usually unaware of it. There are also many literatures showing that the agent pays attention to salient factors. Sims  uses two empirical strategies to analyze how individuals optimize fully with respect to the incentives created by tax policies and shows that tax salience affects agents’ behavioral response. Peng and Xiong  study the allocation of investors’ attention among different information. They find out that investors with limited attention will focus on macroeconomic and industry information rather than that of a specific firm. Seasholes and Wu  demonstrate that attention-grabbing events will attract investors’ attention. In their model, they regard them as the proxy variables and their results empirically indicate that these events have a significant impact on the allocation of investor’s attention. Maćkowiak and Wiederholt  show that decision makers’ attention is usually drawn to salient payoffs.
In recent years, Gabaix  provides a sparse max operator to model dynamic programming with bounded rationality. In the sparse max, the agent pays less or no attention to some features the fluctuations of which are smaller than some thresholds, and he tries to strike a good balance between the utility loss of inattention and the cognitive cost which can be regarded as the loss for taking time to think about the decisions rather than to enjoy oneself. The sparse max seems more realistic than traditional economic models since it has a very robust psychological foundation. Also, it can deal with problems of maximization with constraints easily and get a tractable solution in a parsimonious way.
However, Gabaix  only studies the dynamic programming in a stationary environment without the stochastic volatility terms. But the financial market is strewn with numerous stochastic dynamic programming problems, and these problems are hard to solve due to multitudinous state variables. To address this issue, we extend the sparse max operator and develop a stochastic version of Gabaix’s method. The distinctive feature of this method is that it considers the agent’s behavioral factors (limited attention) and can effectively preclude the “Curse of Dimensionality” for multiple variables. To verify the validity and practicability of our model, we consider the Merton dynamic portfolio choice problem with stochastic volatility variables (e.g., [24, 25]) and get a tractable solution.
The remainder of this paper is organized as follows. Section 2 presents the sparse dynamic programming method proposed by Gabaix . Section 3 extends this model and gives a general principle for solving continuous-time dynamic programming with stochastic variables. In Section 4, we apply our method to Merton dynamic portfolio choice. Finally, we discuss some implications of our findings and suggest topics for future research in Section 5.
2. The Sparse Max Operator without Constraints
We mainly introduce the sparse max operator proposed by Gabaix  in this section. In the traditional version, the agent faces a maximization problem: where , is a utility function and is a constraint. Variable and function have arbitrary dimensions. For any optimal decision, in principle, thousands of considerations are relevant to the agent. Since it would be too burdensome to take all of these variables into account, the agent is used to discarding most of them. At the same time, his attention is allocated purposefully to important variables.
Hence, the agent might sensibly pick a “sparse” representation of the variables; namely, choose the attention vector to replace variable with , where the superscript of represents sparse. The optimal attention vector is obtained by weighing the utility losses for imperfect inattention against the cost savings without thinking too much.
The utility losses from imperfect inattention can be expressed as follows : where is the utility for a sparse agent, , , and is the utility when the agent is fully attentive. denotes the second-order infinitesimal of . , where , , is the standard deviation of , and which indicates by how much a change should change the action for traditional agent. is the second derivative of with respect to . All derivatives above are evaluated at and the default action .
Gabaix  assumes the cognitive cost is , where and parameter is a penalty for lack of sparsity. If , the agent will be a traditional, rational agent.
Based on above analysis, Gabaix  defines the sparse max operator as follows.
Definition 1 (see  Sparse max operator without constraints). The sparse max defined by the following procedure.
Step 1. Choose the attention vector Define as the sparse representation of .
Step 2. Choose the action
and set the resulting utility to be .
Suppose is one-dimensional vector; formula (3) can be transformed into . Gabaix  defines a function to represent the optimal attention vector, namely, and points out when , the function satisfies the sparsity and continuity. When and , we have as shown in Figure 1 .
From Figure 1 we know that the agent will not consider the variable when .
When the vector includes more than one variable and these variables perceived by the agent are uncorrelated, we have through formula (3). To analyze the agent’s inattention expediently, Gabaix  defines the truncation function , so we have . Truncation function has more intuitive economic implications: a one-standard-deviation change of the variable makes the agent change his action by . When is small and satisfies , the agent will not consider this factor. Figure 2 shows the truncation function .
From Figure 2, we know that the agent who seeks “sparsity” should sensibly drop relatively unimportant features. In addition, if the features are larger than that cutoff, they are still dampened: in Figure 2, is below the 45 degree line (for positive , in general, ).
Based on the analysis above, we can use the truncation function to represent the sparse agent’s optimal action.
Remark 2 (see ). If rational optimal action is which is obtained by the Taylor expansion around the default action , then the sparse agent’s optimal action is where is the standard deviation of .
3. A Stochastic Dynamic Programming Approach Based on Sparse Max Operator
In order to effectively deal with stochastic dynamic programming in finance in this section, we extend Gabaix  sparse max operator and propose a bounded rational stochastic dynamic programming model in continuous time.
The general model of stochastic dynamic programming in continuous time is wheredenotes the discount factor, is the utility function, is the decision variable which has an arbitrary dimension, the vector represents important factors which are always considered by the agent, and the vector defined in Section 2 represents factors that that may not be considered by the sparse agent. , are the state transition function of and , respectively. And , represent the stochastic volatility of and , respectively. , are independent standard Brownian motions; namely . We define the value function as .
Assumption 3. The utility function and value function are -order continuously differentiable (, ).
Assumption 4. All state variables are stochastic and they are independent of each other; stochastic volatility of is a function of and while stochastic volatility of is uncorrelated with .
Assumption 5. is one dimensional; that is, only one variable would be always considered by the agent and other variables may not be considered by the agent.
Assumption 6. According to Judd and Guu , we assume the variance of each component of vector is small and independent of one another.
To facilitate analysis, we use to replace , denote the stochastic differential equation of by , , and use the notation as the total derivative with respect to (i.e., the full impact of a change in , including the impact it has on a change in the action ).
Proposition 7. The optimal action in bounded rationality model (7) is where is the standard deviation of .
Proof. See the appendix.
From Proposition 7, we know that, in order to derive the optimal action , we should get the default action which is related to and . The detail process of solving them is described as the following steps, which contain the main results of our method.
Step 1. Solve default action .
By substituting into the basic model (7), we get the default model: This is a general dynamic programming model in continuous time and the state variable is one dimension, so we can get the optimal default action and the value function easily .
Step 2. Solve .
The following Proposition 8 and its proof in the appendix show the result and the process of obtaining .
Proposition 8. The impact of a change in on the value function is
By implicit function theorem, the impact of a change inon the optimal action is , where
Proof. See the appendix.
Now we can get the optimal action based on the two steps above. By the analysis of Proposition 7, we can see that represents the impact of variable on the action . When is smaller than , which means the agent will discard this factor.
4. Application: Dynamic Portfolio Choice
4.1. Merton Portfolio Problem with Stochastic Volatility
In this section, we consider a Merton dynamic portfolio choice problem with stochastic volatility in continuous time . In the traditional version of Merton model , the agent’s optimal problem is where is the discount factor, is the utility function, is the wealth at time , is the standard deviation of , and is the consumption at time . The investment control at time is the fraction of the wealth invested in the risky asset, so is the fraction of the wealth invested on the riskless asset. is the riskless interest rate and is the risky mean rate of return; follows standard Brownian motion. We assume the utility function , where ) is the parameter of risk preference. The goal is to choose consumption and investment control processes to maximize long-term utility.
In model (13), the riskless interest rate and the risky mean rate of return are assumed to be constant . However, this assumption is unrealistic, particularly over long time intervals [27, 28]. Instead, now we assume that these two variables are stochastic and satisfying , , where , represent the long mean of the riskless interest rate and the risky rate of return, respectively, and , are their volatile part. and depend on some “economic factor” ; namely, And satisfies the stochastic differential formula : where and are independent standard Brownian motions. The parameter with allows a correlation between the Brownian motion driving the short rate and its volatility. is the standard deviation of .
The budget equation of model (13) now becomes
We further assume that the stochastic differential equation of follows an Ornstein-Uhlenbeck process [25, 29]: where is the degree of mean reversion in expected excess returns, is the standard deviation of , is a Brownian motion and independent of , and is the coefficient of and . To simplify our model, we assume . By substituting into (17), we obtain . Since , it follows that .
We assume that follows an Ornstein-Uhlenbeck process too, so we obtain where has the same meaning with . is the standard deviation of , and we assume and are small . is a Brownian motion and independent of and . Then we get the following model:
From Section 3, we know that, for the bounded rational agent, the optimal consumption and the optimal fraction of wealth allocated to risky market in model (18) can be expressed as where and are the default actions when , . , are the standard deviation of and , respectively. , , , and are the impact of and on and , respectively. Next we will give the process of solving them using the approach described in Section 3.
Step 1. Solve the default decision and .
By substituting , into the basic model (18), we get the default model: This is the same Merton model as the model in (13); many scholars have solved this problem [1, 2]. We define the value function to be ; then we have  where .
Step 2. Solve , , , and .
Next we will give the results of , , , and . Proposition 9 shows their expressions, and the proof is the solution process.
Proof. See the appendix.
Proposition 9 makes predictions about the sparse agent’s choice. When , the agent is the traditional, perfectly rational agent. And when , it is a policy of a sparse agent. Larger indicates that the agent is less sensitive to fluctuations of both the riskless interest rate and the risky mean rate of return.
4.2. Numerical Example
The purpose of this numerical analysis is to intuitively understand how the boundedly rational agent changes its decisions with the changing of the variances of factors. Firstly, we set , , , and . Let , , , , , , , , , , , and ; then we have , as shown in Figures 3 and 4. In these figures, the horizontal axis is an index of bounded rationality and which is also applied to Figures 5 and 6.
From Figure 3, we know that whatever is, and , which means when the variances of and are big, the agent will consider them in the process of making a decision.
Figure 3(a) shows that if , then the agent reacts like the rational agent: when goes up by 1%, will fall by −2.19% (the agent saves more). For , if goes up by 1%, falls by −1.85%. This result indicates that the greater the cognitive cost about the factor is, the less attention will be paid to this factor by the boundedly rational agent. From Figure 3(b), we can reach a similar conclusion.
Figure 4 also shows the agent will always consider and , that is, and , whatever is. It addition, we can obtain that if , and , which means the rational agent has the same sensitivity about the and when deciding . With the increasing of , the absolute values of and both will decrease, which means that the agent will pay less attention to them. In other words, the impact of and on will decrease for the increasing cognitive cost.
Figure 5(a) shows that when , which means that if the fluctuation is small the agent may discard when he decides the optimal consumption. We can get a similar conclusion from Figure 5(b): when , with .
From Figure 5(a), we know that if , while Figure 3(a) also shows if , which means, for the rational agent, the sensitivity of to has nothing to do with ’s variance. However, the boundedly rational agents have different reactions to as increases, such as when , with in Figure 5(a) while with in Figure 3(a). This disparity indicates that when the cognitive costs are the same and , that is, the agents have the same boundedly rational degree, more volatile factors will be considered while the factor with smaller variance may be neglected.
Additionally, we can know that when , the agent does not react to , namely, (in Figure 5(a)), but will react to a change in (in Figure 5(b)), which is more important: the sensitivity of to remains high even for a high cognitive friction . Note that this “feature by feature” selective attention could not be rationalized by just a fixed cost to consumption, which is not feature dependent. But when , , which indicates that the agent will pay no attention to both and once their thinking costs are beyond some thresholds.
Considering Figure 6(a), we can see that when , , while Figure 4(a) shows whatever is, with which means the smaller the variance of a factor is, the more likely the agent will ignore it. From Figure 6(b), we can also obtain the same conclusion.
Dynamic portfolio choice is an important but complex problem in modern financial field, but extant methods always generate complicated numerical calculations due to numerous state variables. Hence, to address this problem, this paper extends the sparse max operator proposed by Gabaix  and proposes a new approach to deal with dynamic programming under stochastic terms under the assumption of the agent’s limited attention. We apply this method to Merton dynamic portfolio choice problem and find that it effectively simplifies the model’s solution process and avoids the “Curse of Dimensionality.” Finally, numerical example shows that this method has significant economic implications and clearly interprets the agent’s economic behavior when he makes a portfolio choice.
Our study can be extended in several directions. Future research should consider the condition when the stochastic factors are correlated with each other for it is more realistic. Besides, information faced by the agent is always imprecise and incomplete, and the fuzzy set theory is an important approach to deal with this kind of problem [31–33]. Hence, using fuzzy set theory to handle imprecise values in dynamic programming may be another direction for further research.
Proof of Proposition 7. Based on model (7), we define value function
For , we have , where , represent coefficient between and , , and , respectively. Besides, the volatility of a variable that may not be considered by the agent is assumed to be small in Assumption 6; that is, ; so we have
Similarly, we define From the analysis above, we have Then the associated optimal actions can expressed as
First, we will prove at .
From the proof of lemma in Gabaix , we know that where is continuous in and twice differentiable at , with negative semidefinite. In other word, the and differ only by second-order terms in . This basically generalizes the envelope theorem. It implies that at
Differentiating formula (A.2) with respect to gives Differentiating formula (A.7) with respect to and, respectively, we get Similarly, differentiating formula (A.4) with respect to gives Differentiating formula (A.9) with respect to and , respectively, gives
Hence, we have at and at . So
Given , we have . According to (A.11), we obtain , so . Finally where is the standard deviation of .
Proof of Proposition 8. The laws of motion of model (7) are
where . Using Ito formula, Bellman’s equation of model (7) can be expressed as follows:
From the proof of Proposition 7 above, we have , , and . So we obtain
We define function as the derivative of the right side in formula (A.15) with respect to , so satisfies with and we define , where can be derived from the expression of in Step 1 of Section 3. Hence, differentiating formula (A.15) with respect to gives
Now we differentiate at and evaluate at : From (A.17) we get .
And from (A.18) we obtain
According to formula (A.16), we know that the impact of on the optimal action can be expressed as , where
Proof of Proposition 9. Using Ito formula, Bellman’s formula of model (18) is
where , , and represent the coefficient between and , and , and and , respectively. For , , and , we have , , and . According to Assumption 6, the variances of and are so small that we let . Then formula (A.21) becomes as follows:
Differentiating formula (A.22) with respect to and evaluating at , we obtain where which can be obtained from (22); then
Now differentiating (using the total derivative) formula (A.23) with respect to and evaluating at , we obtain
From formula (A.23) we have According to formula (A.24) and the term which can be obtained from (22) we have
Let denote the result of derivation of the right side in formula (A.22) with respect to ; then satisfies with .
Hence, the impact of on is where Since all derivatives are evaluated at , we have , . By substituting (A.28) into (A.27) and using the results of (22), (A.25), (A.26), now we can get Similarly, we have where the concrete expressions of , are referred to formula (22). , can be solved in an analogous way as , .
According to Proposition 7, and can expressed as where
Conflict of Interests
The authors declare that there is no conflict of interests regarding the publication of this paper.
This research is supported by National Natural Science Foundation of China (Grant nos. 71371191, 71210003, and 712221061).
- R. C. Merton, “Lifetime portfolio selection under uncertainty: the continuous-time case,” Review of Economics and Statistics, vol. 51, no. 3, pp. 247–257, 1969.
- R. C. Merton, “Optimum consumption and portfolio rules in a continuous-time model,” Journal of Economic Theory, vol. 3, no. 4, pp. 373–413, 1971.
- F. Wen and X. Yang, “Skewness of return distribution and coefficient of risk premium,” Journal of Systems Science & Complexity, vol. 22, no. 3, pp. 360–371, 2009.
- F. Wen, “Measuring and forecasting volatility in Chinese stock market using HAR-CJ-M model,” Abstract and Applied Analysis, vol. 2013, Article ID 143194, 13 pages, 2013.
- J. C. Cox and C.-F. Huang, “Optimal consumption and portfolio policies when asset prices follow a diffusion process,” Journal of Economic Theory, vol. 49, no. 1, pp. 33–83, 1989.
- I. Karatzas, J. P. Lehoczky, S. P. Sethi, and S. E. Shreve, “Explicit solution of a general consumption/investment problem,” Mathematics of Operations Research, vol. 11, no. 2, pp. 261–294, 1986.
- I. Karatzas, J. P. Lehoczky, and S. E. Shreve, “Optimal portfolio and consumption decisions for a “small investor” on a finite horizon,” SIAM Journal on Control and Optimization, vol. 25, no. 6, pp. 1557–1586, 1987.
- I. Karatzas, J. P. Lehoczky, and S. E. Shreve, “Existence and uniqueness of multi-agent equilibrium in a stochastic, dynamic consumption/investment model,” Mathematics of Operations Research, vol. 15, no. 1, pp. 80–128, 1990.
- R. E. Hall, “The dynamic effects of fiscal policy in an economy with foresight,” The Review of Economic Studies, vol. 38, no. 114, pp. 229–244, 1971.
- M. J. Magill, “A local analysis of -sector capital accumulation under uncertainty,” Journal of Economic Theory, vol. 15, no. 1, pp. 211–219, 1977.
- F. E. Kydland and E. C. Prescott, “Time to build and aggregate fluctuations,” Econometrica, vol. 50, no. 6, pp. 1345–1370, 1982.
- L. J. Christiano, “Linear-quadratic approximation and value-function iteration: a comparison,” Journal of Business & Economic Statistics, vol. 8, no. 1, pp. 99–113, 1990.
- K. L. Judd and S.-M. Guu, “Asymptotic methods for asset market equilibrium analysis,” Economic Theory, vol. 18, no. 1, pp. 127–157, 2001.
- G. A. Miller, “The magical number seven, plus or minus two: some limits on our capacity for processing information,” Psychological Review, vol. 63, no. 2, pp. 81–97, 1956.
- D. Kahneman, Thinking, Fast and Slow, Macmillan, New York, NY, USA, 2011.
- W. Bi, Y. Sun, H. Liu, and X. Chen, “Dynamic nonlinear pricing model based on adaptive and sophisticated learning,” Mathematical Problems in Engineering, vol. 2014, Article ID 791656, 11 pages, 2014.
- P. Bordalo, N. Gennaioli, and A. Shleifer, “Salience theory of choice under risk,” The Quarterly Journal of Economics, vol. 127, no. 3, pp. 1243–1285, 2012.
- B. Kőszegi and A. Szeidl, “A model of focusing in economic choice,” The Quarterly Journal of Economics, vol. 128, no. 1, pp. 53–104, 2013.
- C. A. Sims, “Implications of rational inattention,” Journal of Monetary Economics, vol. 50, no. 3, pp. 665–690, 2003.
- L. Peng and W. Xiong, “Investor attention, overconfidence and category learning,” Journal of Financial Economics, vol. 80, no. 3, pp. 563–602, 2006.
- M. S. Seasholes and G. Wu, “Predictable behavior, profits, and attention,” Journal of Empirical Finance, vol. 14, no. 5, pp. 590–610, 2007.
- B. Maćkowiak and M. Wiederholt, “Optimal sticky prices under rational inattention,” American Economic Review, vol. 99, no. 3, pp. 769–803, 2009.
- X. Gabaix, “A sparsity-based model of bounded rationality, applied to basic consumer and equilibrium theory,” Working Paper, National Bureau of Economic Research, YUN, New York, NY, USA, 2013.
- W. H. Fleming and H. M. Soner, Controlled Markov Processes and Viscosity Solutions, vol. 25, Cambridge University Press, Cambridge, UK, 2006.
- J.-P. Fouque, G. Papanicolaou, and K. R. Sircar, Derivatives in Financial Markets with Stochastic Volatility, Cambridge University Press, Cambridge, UK, 2000.
- M. I. Kamien and N. L. Schwartz, Dynamic Optimization: The Calculus of Variations and Optimal Control in Economics and Management, Courier Dover Publications, Mineola, NY, USA, 1991.
- F. Wen, Z. He, and X. Chen, “Investors' risk preference characteristics and conditional skewness,” Mathematical Problems in Engineering, vol. 2014, Article ID 814965, 14 pages, 2014.
- F. Wen, X. Gong, Y. Chao, and X. Chen, “The effects of prior outcomes on risky choice: evidence from the stock market,” Mathematical Problems in Engineering, vol. 2014, Article ID 272518, 8 pages, 2014.
- W. H. Fleming and T. Pang, “An application of stochastic control theory to financial economics,” SIAM Journal on Control and Optimization, vol. 43, no. 2, pp. 502–531, 2004.
- W. H. Fleming and D. Hernández-Hernández, “An optimal consumption model with stochastic volatility,” Finance and Stochastics, vol. 7, no. 2, pp. 245–262, 2003.
- L. Wang, C. X. Dun, W. J. Bi, and Y. R. Zeng, “An effective and efficient differential evolution algorithm for the integrated stochastic joint replenishment and delivery model,” Knowledge-Based Systems, vol. 36, pp. 104–114, 2012.
- L. Wang, C. X. Dun, C. G. Lee, Q. L. Fu, and Y. R. Zeng, “Model and algorithm for fuzzy joint replenishment and delivery scheduling without explicit membership function,” The International Journal of Advanced Manufacturing Technology, vol. 66, no. 9–12, pp. 1907–1920, 2013.
- L. Wang, H. Qu, Y. Li, and J. He, “Modeling and optimization of stochastic joint replenishment and delivery scheduling problem with uncertain costs,” Discrete Dynamics in Nature and Society, vol. 2013, Article ID 657465, 12 pages, 2013.