Abstract
One of the advantages of stochastic differential equations (SDE) is that they can follow a variety of different trends so that they can establish complex dynamic systems in the economic and financial fields. Although some estimation methods have been proposed to identify the unknown parameters in virtue of the results in the SDE model to speed up the process, these solutions only focus on using explicit approach to solve SDEs, and therefore they are not reliable to deal with data source merged being large and varied. Thus, this study makes progress in creating a new implicit way to fill in the gaps of accurately calibrating the unknown parameters in the SDE model. Essentially, the primary goal of the article is to generate rigid SDE simulation. Meanwhile, the particle swarm optimization method serves a purpose to search and simultaneously obtain the optimal estimation of the model unknown parameters in the complicated experiment of parameter space in an effective way. Finally, in an interest rate term structure model, it is verified that the method effectively deals with parameter estimation in the SDE model.
1. Introduction
Inevitably, a lot of fluctuation dynamics are observed as a result of both internal and external shocks to the system. Stochastic differential equations (SDEs) seem to be a beneficial way to model these fluctuation phenomena by combining deterministic models with a noise term. As a consequence, SDEs have been extensively used to explain uncertainty of complex systems in the area ranging from the subject of biological sciences to the realm of finance and economics [1]. Particularly, SDEs display as a fundamental explanation for volatility coming from an unexpected direction over the short-term interest rate, as well as asset prices in modern finance and economic theory. As is known, one of the main aims of financial modelling is subjected to the accurate calculation in the model. But it is fraught with difficulties and frustrations to estimate those parameters from large samples of financial data. For this reason, it is noted that the description of parameter estimation has become a more eye-catching way in integral research area [2–4].
When it comes to SDEs, it is more difficult to infer unknown parameters compared to their deterministic counterparts. To be exact, under the same parameters and initial conditions, it is available to generate some unideal simulations with SDEs. Therefore, even though many effective methods are supposed to achieve the goal in deterministic models [2], it occurs to be not obvious in SDEs [5]. As a result, this research has focused on establishing efficient methods for SDEs. A big concern has been the slow convergence of Monte Carlo simulations for SDEs. Henceforth, some efficient methods have been produced to succeed in estimation at length like the Taylor approximation, Runge–Kutta, and SDEs generating a large number of simulations [6]. However, it is expected to be a serious barrier because in most cases the search space for the parameters is complex, and the computational time of the inference is expensive.
Currently, three popular methods to infer the process are discussed with the example of the maximum likelihood estimation (MLE) or simulated maximum likelihood (SML) [7, 8], the method of moments [9, 10], and extended Kalman filter [11]. The other most commonly used method is the Bayesian inference method, which will not be discussed in this work. Among them, the method of moments is quite easy to implement application. Although it has been widely used, the major disadvantage has resulted in the characteristic of frequenting approach and large observations. Therefore, if one has a small sample size, the results often suffer from inaccuracy. Meanwhile the MLE approach is not vulnerable to strict limits on set of data and the results of parameters are evaluated by means of reliable analysis of the probability density function (PDF). In effect, an analytical function of closed-form solution is always misconceived for the MLE; therefore, an SML method is developed to obtain the result of PDF by systematic analysis of numerical simulations. Lately, one study examined by lots of scholars that classical statistical models are mostly not precise for the realization of learning-based process, particularly for nonlinear events. Even though the machine-learning techniques provide a better opportunity to undertake the analysis than classical methods, there is little hope of achieving success because of sample size and time consumption [12].
Two kinds of errors can be found in the research of SML. The primary error would refer to the low-rate convergence with the Monte Carlo simulations. To concur this, there is a major concern in variance reduction methods that have been applied to the reduction of stochastic simulations and the bias in the effect of estimation of the moment. Among the method of variance reduction, the existence of the importance sampler and random number generation methods are induced to illustrate the procedure. The second type of error is pointed out to be the difference discretization method based on numerical results and the original SDE. Attempts to solve this dilemma have resulted in the rapid development of the Euler–Maruyama method, which is applied to form the inference of stochastic process. Simple modelling and Gaussian random variable of the numerical solution are the reason for the popularity. But it fails to be stable and achieve a better convergence rate. Hence, it has been advocated that high-order analytic methods and various forms of implicit methods are employed to maintain the stability and enhance the simulation accuracy. Many scholars have found that the convergence rate of SML is proved to be better than the Markov Chain Monte Carlo (MCMC) approach [13]. Also, it can be analyzed by their numerical solutions that implicit SML methods might accurately generate a vast range of estimations on the purpose of following observing data [14].
The other contribution related to the SML method shows the optimal parameters of the model and raises the efficiency. Several approaches to machine learning, the principle of the genetic algorithm (GA) [14, 15] and the application of particle swarm optimization (PSO) algorithm, have mushroomed with rapid growth in the inference of SML parameters [3, 16]. PSO characterized as an artificial intelligence (AI) method is used to make a process of approximation of the minimization problems, which is a sort of nondifferentiable optimization problem in order to arrive at a solution. A number of comparison studies have been conducted to investigate the efficiency of PSO and GA [17–23]. Also, particle swarm optimization provides an important way in fine-tuning the parameters of finance models and deserved popularity in this field [24–29]. Taguchi’s experimental design method has been used to define the user-defined parameters in a comparison study of six algorithms, including the PSO algorithm [30, 31]. It is noted to point out that this method has been turned out to be successfully applied to approximately 700 problems [32]. Empirical studies in recent years have shown that the PSO algorithms achieved high convergence speed for the multiobjective optimization problems [33]. Apart from this, it has been performed to apply to the automatic space exploration on the superscalar computer systems successfully [34, 35]. More notably, it is suggested that PSO has been exercised to greatly improve the implicit SML algorithm through convergence speed and accuracy in consideration of the financial model calibration problems.
To address the issues mentioned above, we design a new research in this study for the application of PSO to infer the unknown parameters in SDE models using the implicit numerical methods for simulating SDEs. The contributions of this work mainly include two parts. The first one is to use implicit methods for simulating stochastic models rather than dominantly using explicit methods in existing research works. This issue is important because the SDE model may have quite a range of stability properties based on the generated parameter samples. The large simulation error may be caused by the stability property of the model rather than the parameter sample. The second contribution is to use efficient PSO algorithm to reduce the computing time. As a heuristic global optimization method, there are still limited research works for applying the PSO algorithm to infer stochastic models. This work will show that the PSO algorithm has greatly improved the efficiency of the implicit SML algorithm with high convergence speed and more accuracy compared to the existing methods.
The remaining sections are organized as follows. Sections 2 and 3 briefly illustrate the research on the interest rate term structure models and method of moments as benchmark for solving models, respectively, while Section 4 will provide our algorithm to generate parameter estimates optimized by PSO for unknown parameter search. Section 5 is presented to be a demonstration of the accuracy and robustness of the proposed algorithm for parameter estimation. Section 6 undertakes an empirical analysis for the application to the US treasury bill data. Finally, in Section 7, conclusions and further research are summarized.
2. Stochastic Model
The standard stochastic differential equation built on the general Brownian motion is analyzed as follows:where is an -adapted -dimensional standard Wiener process in the probability space ; is the drift term with being -dimensional vectors of Borel measurable functions. Also, is the diffusion term with defined on which is a -matrix of Borel measurable functions. The Wiener process has an increment of and follows the Gaussian distribution .
The Euler–Maruyama method is a computer-based implementation consisting of strong convergence estimation of order 0.5 used in computational finance. However, the above method is the real challenge faced by researchers which is that the result attempts to be stable if the numerical simulation of SDEs with a comparatively large diffusion terms has been solved. A possible solution to this problem is presented to take small step sizes to obtain stable simulations at the art of increasingly computing time consumption. In this study, the implicit Milstein method is utilized to obtain the reliability of results of numerical simulation with the characteristics of precision and stabilization.
The Milstein approach is proposed with a higher-order stochastic process of Taylor expansion technique to achieve a better result in order 1.0 strong convergence as shown below:where is the first derivative of . The semi-implicit and fully implicit Milstein methods have a profound impact on the stability and robustness of the system of Milstein scheme only with implicit drift term.
The full implicit Milstein method [36] including implicit drift term and the diffusion term is set up by
In fact, in light of CIR (Cox, Ingersoll, and Ross) model primarily constructing interest rate term structure, the accuracy of inference method has been detected [3]. The dynamic change of the short term of interest rate is a worthwhile field to finance market because bond prices and mortgage contracts are mostly valued in a way of using the term structure of interest rates, option, and derivative [37, 38]. The model of the CIR has been ready to short interest rate with information [39, 40], which is viewed as a linear stochastic differential equation with mean-reversing [41]. Regarding short term of interest rate acting like a square root controlled by diffusion, the formula is a continuous-time system as below:where , represents the speed of adjustment (or mean reversion), is value of the random moving on interest rate of the long term, and is a volatility based on constant measure. It has become apparent that the drift is varying along with the volatility change according to the short-term interest rate level.
According to previous literatures, the explicit Milstein method as described in (2) is better than the Euler method. More emphasis is placed on the comparison with the accuracy of the Milstein method in (2) against the semi-implicit Milstein method in (3) for the purpose of parameter inference of the CIR model. On the premise of Milstein method, the application of the CIR is shown as follows:and, with the linear correlation analysis of the interest rate drift term, a semi-implicit algorithm is included as the following formula:
Figure 1 shows the whole process 5 simulations of the CIR model with result of parameters , , and . Given the relatively small level of volatility, the short-term value remains positive.
3. Method of Moments
For the next sections, the analysis of the parameter of one-dimensional SDE is employed as follows:
If we sample to get observations on some discrete point-in-time , result of the maximum-likelihood (ML) will be estimated based on the maximum likelihood function:
Similarly, minimizing the negative effect of log-likelihood function to estimate is shown as follows:where is set with the density function of the initial value and presents the corresponding value in the case of the transitional probability density function (PDF) at . It must be discussed that, according to the Markovian property in equation (8), the transitional PDF correctly satisfies the Fokker–Planck equation. However, considering the failure to reach the closed-form solution in the Fokker–Planck equation, the final maximum likelihood estimation will make a difference.
However, it is able to approximate a transitional PDF by the method of SDE in (8). For instance, an application of the Euler–Maruyama method can be used to discretize equation (8) given bywhere is time step of the discretization scheme. We can then approximate the transitional PDF of by normal distribution with mean and variance such that
Alternatively, the simplest form on a discrete maximum likelihood function is called the method of moments, which is achieved by the approximated above-mentioned PDF, not the exact transitional PDF in equation (10). In the analysis of this paper, the method is given to be a benchmark in comparison with accuracy and stability of the proposed methods.
As mentioned above, the major focus of the work is placed on the CIR process (5) and to see what happens to the optimal values referring to the parameters , , , and separately as follows [3]:with the optimal value of being such thatis satisfied.
The Milstein variant should make a direct contribution to improving the accuracy with the implementation of the discrete maximum likelihood method. However, it appears that it is not easy to express directly the parameter estimation by the transitional PDF [3, 13]. For that reason, the system of the simulation method of maximum likelihood is identified as the transitional PDF method along with several stochastic simulations.
4. Simulated Maximum Likelihood Method
In this section, a simulated implicit numerical scheme is included to illustrate the growth of the efficiency to estimate parameters by PSO algorithm.
In the application of deterministic models, parameter estimation is successfully achieved through fitting numerical simulations to experimental observations. Unfortunately, the method is far from ready to deal with SDE models because a single SDE model can develop. A case is to discuss the simulated maximum likelihood (SML) method for structuring stochastic models [42, 43]. is given as time sequence observation containing a time period . The joint transitional density function (likelihood function) during the study period is derived as follows:such that parameters in equation (8) need to be determined. By analyzing the density of state, is initially presented withas the transitional density initializing from and running to .
If the framework of financial system is fundamentally prescribed based on the stochastic model (8), we have a stochastic process which exhibits the Markov property [44], and the brief description of transitional density function can be shown as follows:
The equivalence function of the maximum joint transitional density (15) is confirmed to the minimum negative log-likelihood function (10) with the implicit time in the formula.
Since a closed-form solution obtained from the transitional density (17) is impossible to be evaluated, based on a nonparametric kernel density algorithm it can be used as
In replace for the transitional density, in this case are the realizations of at a particular time point following the initial condition , and is obtained by the kernel bandwidth. We also have that is devised as a nonnegative kernel function within a probability unit. Regarding a single variable of SDE models linking with the normal Kernel, the specific bandwidth can be represented as , with as standard deviation of the sample with realization [42]. In the case of multivariate stochastic models, it can be assumed that random variables are independent or alternatively supported by the existence of the theory of multivariate density estimation [45].
After setting up the objective function, another consideration that has been taken into account is to select a method to improve speed of researching the optimal parameters. Numerical solutions tell us that PSO has better computational saving for computational saving purpose; PSO is studied in presence of numerical solutions for nonlinear and unconstrained problems consisting of continuous design variables. However, the computational saving of integer and combined constrained nonlinear program is much lower.
Eberhart and Kennedy [16] have developed a PSO algorithm based on population stochastic optimization scheme; the discovery of PSO approach is inspired by abundant contents of different social behaviours like bird flocking or fish schooling without the idea of evolution operators like crossover and mutation. The information relevant to this method is that potential solutions are referred to as particles swarm across the problem space of current optimum particles. For the further research, a PSO MATLAB toolbox [46] that was downloaded from the MATLAB File Exchange Central is designed to estimate unknown parameter of SDE models. The software package can be successfully made to settle a number of optimization problems. Fortunately, the optimal solution to unknown parameter is identified by SML method referring to the SDEs model (8) by finding the minimum log-likelihood function (10) through the following sampling process and algorithm:(i)Firstly, the process starts with inputting the system states and time points .(ii)Secondly, taking at time () as the starting point, several methods are required to realize realizations of at . The value for random seed is explained specifically by the Gaussian random variables samples.(iii)Thirdly, the study of the nonparametric density estimation (18) with the normal kernel algorithm and multivariate density function have been generally completed to calculate the transitional density (17).(iv)Then, it is necessary to repeat the above step for each time point and describe the log-likelihood function (10).(v)Finally, it offers a conclusion reading the optimal kinetic rates with the process of a particle swarm optimization algorithm on the account of the minimum in (10).
It should be highlighted that equal increments deriving from the Wiener process should be applied to the numerical simulations method to study diverse values of parameter . Also, to minimize variation on estimation, an important research to be summarized is that the condition of the same random seeds or samples mentioned in the second step is applied to estimate numeral values of different candidate parameters.
5. Parameter Estimation of the Interest Rate Models
As mentioned above, we shall now estimate specific parameters from the CIR model. There is no doubt that we can get 20 trajectories from a small step size of () with the help of the semi-implicit Milstein method in a given set of parameters. Great efforts have been made to obtain the estimation of each parameter of the generated trajectories by virtue of the method of moments, the SML method, and the explicit and semi-implicit method of the Milstein scheme. Result can be made by using the PSO algorithm with the population of 40 and generation of 200. The PSO algorithm is implemented 20 times using different random seeds in computation, and thus we achieve 20 estimates of parameters. Hopefully, it is proposed to focus on the mean error and standard deviation (STD) of the estimation.
The above simulation results declare that indeed the method of moments is frequently improper to deal with parameters estimation in model. Because the results of the save parameter have a differential when applying different implementations, for some instances, the parameter relative error will exceed 100%. Worst still, it may have access to a negative coefficients estimation, which happens to be meaningless in finance market, although the noise strength is small. Furthermore, it is important to emphasize that the moment method relies upon observation points correlated with the length and the quantity of various time points of observation. If the length of the data points is large and the chosen observation is big enough, it is proved that the method of moments (see Table 1) can be designed to predict estimates with near acceptable accuracy.
Studying the effect of the noise imprecision of estimation, the volatility of parameter is proposed for each SDE model with 3 differing values. From Tables 2 and 3, the explicit Milstein method and the semi-implicit Milstein method were employed for actual parameter estimations on the basis of negligible minor errors and standard deviations with the evidence of small fluctuations in the SDE model (). In fact, the final result of the SML method linking with the explicit Milstein scheme is considered to be more accurate and reliable. Reasonably, for modelling nonstiff SDEs, the explicit Milstein method is expected to be more precise than the semi-implicit one. If the actual fluctuations in the SDE models are average instead of extreme (), more accurate research results can be attained by the semi-implicit Milstein method compared to the explicit Milstein approach. In terms of application, the stability of the semi-implicit method is slightly prioritized over the precision of the explicit scheme. Particularly, if the noise of interest rate is large enough to estimate (), the acceptable results can be achieved by the semi-implicit method. However, if the noise of interest rate tends to be a large extent, the same method becomes less reliable.
Since then, a good approach has been extensively explored to simulate the SDE models with less step size, and the use of the fully implicit Milstein method should guarantee the numerical simulations to be more stable.
The robustness property of estimates for the stochastic search methods is a critical problem in the current study. For this study, two research tests of variations are conducted, namely, the change of random seeds for solving the stochastic model, which leads to different outcomes, and changes in the implementation of PSO algorithm that employs varying random samples.
To be specific, the first one is partly caused by the convergence property inherent in the procedure of Monte Carlo simulation under certain conditions. A discussion that presented a problem-solving solution is subject to the increase of stochastic simulations. It is defined to fix the course of the interest rate, and 10 sets of random seeds are needed to apply to simulations the stochastic model. The results from Figures 2(a)–2(c) illustrate that the estimations have progressed to be relatively stable when the amount of simulations exceeded 5000. If the number of simulations was extended to , the estimates become highly stable. The results are reflection of the application of GA to estimate the calculated rate with constants given in discrete chemical reaction systems preciously [14].
(a)
(b)
(c)
Secondly, the initial model parameters are proposed to accurately estimate the final results in the PSO algorithm. Prior experiments concluded that a significant influence lies between initial parameters and final estimation when the GA was assigned as the stochastic searching method [14, 15]. Various random seeds have been devoted to the PSO algorithm to initialize parameters of the CIR stochastic model. Figures 3(a)–3(c) contain the implication of the final results for parameter estimates. The information on numerical results with the several relevant experiments describes that the PSO method is reliable to estimate independent initial parameters, posing a definite advantage for the PSO algorithm over the GA in the program of model calibration and estimation of parameters in complicated mathematical models. However, given the opportunity cost, it is suffering to pick the optimal estimation from a series of possible candidates according to the designated standard like imparting the robustness in the mathematical model.
(a)
(b)
(c)
6. Application to US Treasury Bill Data
The instantaneous interest rate of the term structure under the classical single factor model is given aswhere is considered as the standard Wiener process, is deemed as the speed of adjustment parameter, is regarded as the mean interest rate, is considered as the control of volatility, and is considered as the effect of levels, which are devoted to estimate the parameters. According to previous analysis, CIR model has a fine description of a unique and special type of the model in (19) with . A plenty of empirical research results have suggested that should be in preference to estimate rather than being simply imposed into the model.
Concentrating on parameter estimates under stochastic differential equation (SDE) models (19) is complex but needs to be done. The US three-month Treasury Bill rate (The data are available from the Board of Governors of the Federal Reserve System (US), three-month Treasury Bill: Secondary Market Rate [TB3MS], retrieved from FRED, Federal Reserve Bank of St. Louis; https://fred.stlouisfed.org/series/TB3MS) is provided to approximate a reasonable estimate about real short-term interest rates, which is unobservable and instantaneous. For an illustrative purpose, the method of moments is regarded as a benchmark to discuss the approaches. Then the discrete equations that were declared from the single factor model by the Milstein method and semi-Milstein method of SML estimation are suggested as follows:
The analysis of data is carried out on the US Treasury Bill from the first day of January 1996 to the first day of June 2019 with 282 observations in total on the purpose of parameters estimation of the single-factor model (19). An expression of the results of the estimated parameter from the method of moments and explicit Milstein method as well as semi-implicit Milstein method has been presented at Tables 4–6, respectively. Also, the estimation of standard errors by three methods is explained in the relevant tables. As can be seen, a large number of the estimates are confirmed to be similar without difference. With a large sample size, the method of moments is in line with the result of accuracy of other methods. Although the method of moments is still accurate, the results of the standard errors by these two Milstein methods are much smaller. Detailed analysis is designed for these two Milstein methods with a smaller step size during the numerical simulation, whereas a large number of simulations are offered for the statistical inference methods. The sign of the improvement to versify the standard error was not much.
Figure 4 gives a specific description of the course of the monthly interest of the US three-month Treasury Bills. As shown in the graph, the results of mean interest rate fit precisely with the interest rate data. In accordance with the results, it is reported that the scientific semi-implicit Milstein method is greater and more precise than the others. An outstanding phenomenon supports the evidence that the results of parameter of the levels effect by three methods are relatively identical to each other compared to the other parameters estimations. Another important finding broadly supports the analysis that the estimation of standard error of is obviously smaller than initial value of , reflecting that the results of estimation of might be clearly different from 0.5 obtained from the CIR model. These studies further support the idea that the classic single-factor model (19) is better at describing the interest rate than the CIR model. More research confirms that should be received from the estimation of financial data instead of simply being imposed to the interest term structure model.
The performance of the levels effect fits better in the interest term structure model, and the appropriate observations were expressed as follows. It is encouraging to estimate parameters of the SML by semi-explicit Milstein method with the given information of and 5000 simulations in response to be set out to calculate the mean average. The findings from standard deviation range of the actual data were demonstrated in Figure 5, which is acceptable statistically.
7. Conclusions and Future Works
This study provides a new insight into the method of parameters estimation for the SDE models. To undertake a longitudinal analysis of the implicit numerical simulation method in the Monte Carlo simulation along with the PSO method, it can be found that the estimation from the SML by the semi-implicit Milstein method is presented better in efficiency than Euler–Maruyama approach with regard to the convergence and stability. A key discussion on experimental results in moderately stiff SDE model is presented that the semi-implicit Milstein method appears to be more exact than explicit Milstein method. Also, the effects of the PSO method are reliable and almost independent with comparison to the genetic algorithms. For the implementation of the SDE models, especially the interest term structure, actual financial data was explored to estimate the parameters. Therefore, the estimated parameters of the model make a perfect match for actual data.
Compared with the inference of deterministic models, there is more uncertainty in the inference of stochastic models. Generally, it is more difficult for inferring a stochastic model than a deterministic model. Although we have achieved progress in this work for the parameter inference of stochastic model, there are still some limitations of this work. First, the accuracy of the estimated parameters depends on the samples in stochastic simulation. A large number of simulations are needed to ensure the stability of the estimates. It is still an important issue for reducing the computational time for the inference of stochastic models. In addition, it is still a difficult issue to select appropriate estimate from the candidates that all have similar estimation errors. In this work, we use robustness as an additional criterion to select estimates. Other criteria for selecting inference candidates are strongly required. Furthermore, the stochastic differential equations with the generated parameter sample may be highly stiff. In this work, we use the semi-implicit Milstein method to improve the stability property of the explicit Milstein method. It is strongly required to employ numerical methods with better stability properties, such as the fully implicit methods, to simulate stochastic differential equations. All these issues will be interesting topics of our further research.
Data Availability
The data used to support the findings of this study are available from the corresponding author upon request.
Conflicts of Interest
The authors declare that there are no conflicts of interest regarding the publication of this paper.
Acknowledgments
This research was supported by the National Natural Science Foundation of China under Grants 71901222 and 71974204, the Humanities and Social Science Foundation of the Ministry of Education of China under Grant 17YJC630236, and “the Fundamental Research Funds for the Central Universities,” Zhongnan University of Economics and Law, under Grant 2722020JCT008. This work was also supported in part by the National Natural Science Foundation of China under Grant 11571368 and the Australian Research Council (ARC) under Grant DP120104460.