Research Article  Open Access
Some Aspects of Sensitivity Analysis in Variational Data Assimilation for Coupled Dynamical Systems
Abstract
Variational data assimilation (VDA) remains one of the key issues arising in many fields of geosciences including the numerical weather prediction. While the theory of VDA is well established, there are a number of issues with practical implementation that require additional consideration and study. However, the exploration of VDA requires considerable computational resources. For simple enough loworder models, the computational cost is minor and therefore models of this class are used as simple test instruments to emulate more complex systems. In this paper, the sensitivity with respect to variations in the parameters of one of the main components of VDA, the nonlinear forecasting model, is considered. For chaotic atmospheric dynamics, conventional methods of sensitivity analysis provide uninformative results since the envelopes of sensitivity functions grow with time and sensitivity functions themselves demonstrate the oscillating behaviour. The use of sensitivity analysis method, developed on the basis of the theory of shadowing pseudoorbits in dynamical systems, allows us to calculate sensitivity functions correctly. Sensitivity estimates for a simple coupled dynamical system are calculated and presented in the paper. To estimate the influence of model parameter uncertainties on the forecast, the relative error in the energy norm is applied.
1. Introduction
The earth system consists of several interactive dynamical subsystems and each of them covers a broad temporal and spatial spectrum of motions and physical processes. The components of the earth system have many differences in their physical properties, structure, and behavior but are linked together by fluxes of momentum and mass as well as sensible and latent heat. All of these subsystems interact with each other in different ways and can be strongly or weakly coupled. Prediction of future state of the earth system and its components is one of the most important problems of modern science. The most significant progress has been achieved in the forecasting of the atmosphere via numerical models, which describe the dynamical and physical processes in the earth’s gaseous envelope. It is clear that further improvement of forecasts can be pursued via the development of coupled modeling systems that primarily combine the atmosphere and the ocean and describe the interactions between these two systems. Since numerical weather prediction systems calculate a future state of the atmosphere and ocean by integrating a set of partial differential equations that describe the fluid dynamics and thermodynamics, initial conditions that accurately represent the state of the atmosphere and ocean at a certain initial time must be formulated. Numerical weather prediction systems use data assimilation procedures to estimate initial conditions for forecasting models from observations. Data assimilation remains one of the key issues not only in the numerical weather prediction (NWP) but also in other geophysical sciences.
One of the most advanced and effective data assimilation techniques is fourdimensional variational data assimilation (4DVar). In particular, the weather forecasts produced by the ACCESS (Australian Community Climate and Earth System Simulator) at the Bureau of Meteorology use 4DVar in the incremental formulation developed at the Met Office [1, 2]. A comprehensive historical review and current status of 4DVar are presented, for example, in [3–7]. In our view point, some papers and books, such as [8–17], have contributed significantly to the development of mathematical foundation, theory, and practice of 4DVar.
In general, the main objective of 4DVar is to define, as perfectly as possible, the state of a dynamical system by combining, in statistically optimal manner, the observations of state variables of a real physical system together with certain prior information. In NWP this prior information is usually referred to as the background. Mathematically, 4DVar procedures are formulated as an optimization problem, in which the initial condition plays the role of control vector and model equations are considered as constraints. While the theory of variational data assimilation is well established, there are a number of issues with practical implementation that require additional consideration and study. The performance of 4DVar schemes depends on their key information components, such as the available observations, estimates of the observation, and background error covariances that are quantified by the corresponding matrices, as well as the background state. All of those information components strongly impact the accuracy of calculated initial conditions, thus influencing the forecast quality. Thus, it is important to estimate the sensitivity of a certain forecast aspect with respect to variations in the observational data, background information, and error statistics. This problem is usually formulated within the framework of the adjointbased sensitivity analysis (e.g., [18–29]).
However, an adjoint model used to calculate sensitivity functions is derived from a linearized forward propagation model and contains numerous input parameters. Consequently, linearization of strongly nonlinear NWP models and also uncertainties in their numerous parameters generate errors in the initial conditions obtained by data assimilation systems. The influence of linearization and parameter uncertainties on the results of data assimilation can be studied, ideally for each particular NWP model, using sensitivity analysis [30, 31]. Even more problems arise when considering coupled 4DVar data assimilation schemes since the atmosphere and ocean have very different physical properties and timespace spectrum of motions generating initialization shock. Several coupling strategies are being developed for use in NWP systems; however, all of them introduce issues that require additional detailed consideration. For example, the influence of coupling strength on the initial conditions obtained by 4DVAR procedures is one such issue that is important to study and analyze.
Good practice in the development of NWP models and data assimilation systems requires evaluating the confidence in the model. In this context, it is important to estimate the influence of parameter variations on system dynamics and to find those parameters that have the largest impact on system behaviour. Sensitivity analysis, which is an essential element of model building and quality assurance, addresses this very important issue.
The exploration of coupled 4DVar systems, parameter estimation, and sensitivity analysis require considerable computational resources. For simple enough loworder coupled models, the computational cost is minor and, for that reason, models of this class are widely used as simple test instruments to emulate more complex systems. In this paper, we describe a coupled nonlinear dynamical system, which is composed of fast (the “atmosphere”) and slow (the “ocean”) versions of the wellknown Lorenz [32] model. This loworder coupled system allows us to mimic the atmosphereocean system and therefore serves as a key element of a theoretical and computational framework for the study of various aspects of coupled 4DVar procedures [33, 34]. Under certain conditions the Lorenz model exhibits a chaotic behaviour and using conventional methods of sensitivity analysis can be questionable in terms of interpretation of the obtained results [35–37]. The “shadowing” method [36, 37] for estimating the system sensitivity to variations in its parameters allows us to calculate the average along the trajectory sensitivities and therefore to make a clear conclusion with respect to the system sensitivity to its parameters. This method is based on the pseudoorbit shadowing in dynamical systems [38, 39]. Calculated sensitivity coefficients obtained via conventional methods and the “shadowing” approach are presented in the paper.
We also succinctly consider commonly used techniques for sensitivity analysis and parameter estimations of dynamical systems and study the influence of coupling strength parameter on the dynamical behaviour of the coupled system using Lyapunov characteristic exponent analysis. It was found that the coupling strength parameter strongly affects the system dynamics both quantitatively and qualitatively. This fact should be taken into consideration when choosing the coupling strategy in data assimilation systems.
2. LowOrder Coupled Dynamical System
In this section we consider a loworder coupled nonlinear dynamical system obtained by coupling of two versions of the original Lorenz model (L63) [32] with distinct time scales, which differ by a factor (e.g., [33, 34]):where lower case letters represent the fast subsystem and capital letters the slow subsystem, , , and are the parameters of L63 model, is a coupling strength parameter for the and variables, is a coupling strength parameter for , is an “uncentering” parameter, and is a parameter representing the amplitude scale factor. The value indicates that two systems have the same amplitude scale. Thus, the state vector of the coupled model (1a) and (1b) is and the model parameter vector is . Without loss of generality, we can assume that , , and , then and the system (1a) and (1b) can be rewritten in the operator form:where the nonlinear uncoupled operator and linear coupled operator are represented by the following matrices:
The unperturbed parameter values are taken asChosen values of , and correspond to chaotic behaviour of the L63 model. For and the critical value of parameter is 24.74, which means that any value of larger than 24.74 induces chaotic behaviour [32]. The parameter indicates that the slow system is 10 times slower than the fast system.
The coupling strength parameter plays a very important role in qualitative changes in the system dynamics since this parameter controls the interactions between fast and slow subsystems. Qualitative changes in the dynamical properties of a system can be detected by determining and analyzing the system’s spectrum of Lyapunov exponents, which characterize the average rate of exponential divergence (or convergence) of nearby trajectories in the phase space. In the analysis of coupled dynamical systems we are dealing with conditional Lyapunov exponents that are normally used to characterize the synchronization with coupled systems. System (2) has six distinct exponents. If the parameter tends to zero, then system (2) has two positive, two zero, and two negative Lyapunov exponents. The influence of coupling strength parameter on the two largest conditional Lyapunov exponents is illustrated in Figure 1. The numerical experiments demonstrated that those, initially positive, exponents decrease monotonically with an increase in the parameter c. At about they approach the axis and at about negative values. Thus, for , the dynamics of both fast and slow subsystems become phasesynchronous [40]. For , a limit circle dynamical regime is observed since all six exponents become negative.
Apart from Lyapunov exponents, autocorrelation functions (ACFs) enable one to distinguish between regular and chaotic processes and to detect transition from order to chaos. In particular, for chaotic motions, ACF decreases in time, in many cases exponentially, while for regular motions, ACF is unchanged or oscillating. In general, however, the behaviour of ACFs of chaotic oscillations is frequently very complicated and depends on many factors (e.g., [41, 42]). Autocorrelation functions can also be used to define the socalled typical time memory (typical timescale) of a process [43]. If it is positive, ACF is considered to have some degree of persistence: a tendency for a system to remain in the same state from one moment in time to the next. The ACF for a given discrete dynamic variable is defined aswhere the angular brackets denote ensemble averaging. Assuming time series originates from a stationary and ergodic process, ensemble averaging can be replaced by time averaging over a single normal realization:Signal analysis commonly uses the normalized ACF, defined as . ACF plots for realizations of dynamic variables and , and and calculated for different values of the coupling strength parameter are presented in Figures 2 and 3, respectively. For relatively small parameter (), the ACFs for both and variables decrease fairly rapidly to zero, consistently with the chaotic behaviour of the coupled system. However, as expected, the rate of decay of the ACF of the slow variable is less than that of the fast variable . The ACF envelopes for variables and also decay almost exponentially from the maximum to zero. For coupling strength parameter on the interval the ACF of the fast variable becomes smooth and converges to zero. At the same time, the envelopes of the ACFs of variables , , and demonstrate a fairly rapid fall, indicating the chaotic behaviour. As the parameter increases, the ACFs become periodic and their envelopes decay slowly with time, indicating transition to regularity. For calculated ACFs show periodic signal components.
3. The Basics of FourDimensional Variational Data Assimilation
Atmospheric models used for operational NWP are mainly deterministic and derived from a set of multidimensional nonlinear differential equations in partial derivatives, which are the equations of fluid dynamics and thermodynamics that describe atmospheric processes and atmosphereunderlying surface interactions. A generic atmospheric model can be represented by the following continuous autonomous dynamical system:Here is the state vector belonging to a Hilbert space , is a parameter vector, where is a Hilbert space (the space of the model parameters), is a nonlinear vectorvalued function, such that , and x_{0} is a given vector function. This infinitedimensional model has to be truncated by some means to finitedimensional approximate model, for which a solution can be sought numerically. Applying either a projection onto a finite set of basic functions or a discretization in time and space, one can derive the discrete atmospheric model which can be represented as discrete nonlinear dynamical system given by the equationwhere is the dimensional state vector representing the complete set of the model variables that determine the internal state of the atmospheric model at time , is the nonlinear operator that indirectly contains model parameters and propagates the state vector from time to time for , and is the model errors. It is usually assumed that model (8) is “perfect” (), that is the forecast has no errors if the initial condition is perfect. In this case, given the model operator and the initial condition , (8) uniquely specifies the orbit of the dynamical system. Let be the dimensional vector of observations measured at a discrete time , that are linked to the system state via the following equation:where is the nonlinear observation operator that maps the state vector to observation space. It is usually assumed that the observation errors are unbiased, serially uncorrelated, and normally distributed with known covariance matrices .
Suppose that at the initial time the prior (background) model state is known and represents the “best” estimate of the “true” state before any observations are taken. This background state is provided by a previous forecast. It is assumed that has unbiased and normally distributed errors with known covariance matrix :
Given the observations at time , the corresponding observation error covariance matrices (), the background initial state , and the error covariance matrix , the 4DVar data assimilation seeks to minimize, with respect to , a certain cost function expressing the “distance” between observations and corresponding model state using the model equations as constraints:subject to satisfying the set of the “ideal” () model equations with initial state :The 4DVar cost function is usually written as (e.g., [17, 18]):The optimization problem (11) is nonlinear with strong constraints and an iterative minimization algorithm (e.g., gradientbased technique) is required to obtain the solution. The gradient of the cost function (13) is as follows:where is the adjoint of the linearized model operator and is the adjoint of the linearized observation operator . If the model is not “perfect” then we need to take into account the model errors , which are sometimes taken as Gaussian noise:where is a model error covariance matrix. Thus, we obtain the weakly constrained 4DVar data assimilation and the following termshould be added to the righthand side of the cost function (13) [6, 19].
It is important to make the following comments. While NWP has significantly improved over the last several decades, weather forecasts are still intrinsically uncertain. This is because mathematical models used in NWP have several sources of uncertainty and therefore a number of sources of errors. The first one is an intrinsic uncertainty due to chaotic nature of the system. The second one is a structural uncertainty, which is how the model itself represents the physical processes incorporated into it. It is important to underline that our knowledge of the earth system is always imperfect and, therefore, we can only theoretically design the “ideal” model. However, improving model physics demonstrated that even theoretically wellposed models failed to accurately simulate and predict the dynamics of real atmospheric processes. This is because numerical models have a parametric uncertainty (how accurate model parameters are) initial and boundary uncertainty (are initial and boundary conditions known precisely) and, in addition, numerical errors. All of those uncertainties (errors) impair the weather forecast accuracy and limit the time horizon of accurate NWP. Current time horizon of synopticscale NWP is several days.
The climate study has significantly longer time horizon: several decades. The sensitivity analysis of the climate system is associated with stability of characteristics of climatic model attractors with respect to perturbations in model parameters. A priori estimation of the behaviour of state vector, when the perturbations in the climate model parameters tend to zero, is generally an unresolved problem since it is not known whether or not the invariant measure of climate modeling system is continuous with respect to the small perturbations in the differential matrix operator of the numerical model. Indeed, at certain model parameter values, different bifurcation can occur in the system phase space. Therefore, dynamics on the attractor generated by the model may change considerably even for small parameter perturbations.
4. Sensitivity Analysis: Essentials of Conventional Approaches
One of the commonly used measures for estimating the influence of model parameter variations on the state variables is the sensitivity coefficient, which is the derivative of a certain component of a model state vector with respect to some model parameter [30, 31, 44]:where is the infinitesimal perturbation of parameter around some fixed point . Differentiating (8) with respect to , we obtain the set of nonhomogeneous ODEs, the socalled sensitivity equations, which can be written aswhere is the sensitivity vector with respect to parameter , , and is a Jacobian matrix. Thus, to analyze the sensitivity of system (7) with respect to parameter one can solve the following set of differential equations with given initial conditions:Sensitivity equations describe the evolution of sensitivity coefficients along a given trajectory and therefore allow tracing the sensitivity dynamics in time. The procedure for computing sensitivity coefficients includes the following steps.(1)Obtain initial conditions on the system attractor at time by integrating the nonlinear model equations (7) for a long enough time range , starting from random initial conditions.(2)Solve the nonlinear model equations (7) to calculate a trajectory , .(3)Calculate a model Jacobian matrix and a parametric Jacobian matrix.(4)Solve the sensitivity equations (18) with given initial conditions to obtain the desired sensitivity coefficients.
Sensitivity analysis allows us also to explore the sensitivity of a generic objective function (performance measure), which characterizes the dynamical system (7):where is a nonlinear function of the state variables and model parameters . The gradient of the functional with respect to the parameters around the unperturbed state vector quantifies the influence of parameters on the model output results. In particular, the effect of the th parameter can be estimated as follows:where is the variation in parameter . Note thatThis approach is acceptable for loworder models. However, the accuracy of sensitivity estimates strongly depends on choice of the perturbation . By introducing the Gâteaux differential, the sensitivity analysis problem can be considered in the differential formulation eliminating the need to set the value of [30, 31]. The Gâteaux differential for the objective function (20) has the following form:Here is the state vector variation due to the variation in the parameter vector in the direction . Linearizing the nonlinear model (7) around an unperturbed trajectory , we obtain the following system of variational equations, the socalled tangent linear model, for calculating :Then using (24) we can calculate the variation . Since , where is a scalar product, the model sensitivity with respect to parameter variations can be estimated by calculating the components of the gradient . However, this method is computationally ineffective if the number of model parameters is large. The use of adjoint equations allows obtaining the required sensitivity estimates within a single computational experiment (e.g., [6, 30, 31]) since the gradient can be calculated by the following equation:where the vector function is the solution of adjoint modelThis equation is numerically integrated in the inverse time direction. Thus, the algorithm for computing sensitivity functions is as follows.(1)Obtain initial conditions on the system attractor at time by integrating the nonlinear model equations (7) for a long enough time range , starting from random initial conditions.(2)Solve the nonlinear model equations (7) to calculate a trajectory , .(3)Calculate the righthand side of (27) and then integrate numerically this equation in the inverse time direction with the initial conditions .(4)Calculate the gradient (26).
5. Testing the Variational Data Assimilation
We will consider the dynamics of system (2) on its attractor. In order to obtain the model attractor, the numerical integration of (2) is started at with the initial conditions and finished at to guarantee that the calculated model state vector is on the model attractor. The forecast, obtained by NWP models, is substantially determined by initial conditions, which calculated via 4DVar. The accuracy of initial conditions strongly depends on numerous parameters of 4DVar systems. Some of these parameters are uncertain. To estimate the influence of parameter variations on the forecast obtained by model (2), the “true” and “forecast” trajectories were calculated. The “true” trajectory and the “true” state at the verification time are obtained by integrating the model equations over the time interval with unperturbed parameter vector and initial conditions . Then, the forecast trajectory and the forecast state at are obtained by integration of the “forecast” model (2) with initial conditions and a certain perturbed model parameter. Thus, the “forecast” model has the same set of equations as the “true” model; however, some model parameter is slightly changed (i.e., this parameter is known with uncertainty). In order to measure forecast errors the relative error in energy norm is used:
As an example, variations in parameters and are considered and forecast is made for two time periods : 2.5 and 5.0 of nondimensional time units. The forecast error (29) is estimated at time . Table 1 shows the results of forecast verifications. It is obvious that the less the forecast error measure , the higher the forecast skill. Qualitatively, calculated results are consistent with real numerical weather forecasts obtained with complex stateoftheart NWP models: longer leads to lower forecast accuracy and smaller parameter variations (difference between the “true” parameter value and the value used in the “real” model) lead to better forecast accuracy. It can also be observed that parameter influences the forecast accuracy almost twice as much as parameter .

Synthetic data assimilation requires the “true” state , the background (first guess) state , and observations inside the assimilation window as well as error covariance matrices of the prior guess and observations . The length of data assimilation window should be defined as well. The “true” and background trajectories on the data assimilation interval represent some portions of and , respectively. Observations should be provided every 5–10 time steps inside the assimilation window and can be generated by adding the Gaussian random noise (with zero mean and variance ) to the “true” state. Since observation grid and model grid are the same, observation operator is simply an identity mapping. To take into consideration the background covariances, for simplicity, the assumption can be used, where is the variance of background error and is the identity matrix. Under assumption that the observation quality is the same for all variables, the observation covariance matrix can be defined as .
Testing the TL model and its adjoint is required to ensure the convergence of the minimization algorithm in data assimilation procedures. If is a small perturbation of the model state, thenTo verify the applicability of the TL model on the time interval , the relative error should be calculated. The TL model is valid if when . The results of numerical experiments showed that the TL model passed this test with tending towards zero (Table 2). The TL adjoint correctness can be tested by verification of the inner product identityIt was found that this equality is essentially correct: the difference was observed only in the 7th digit, which is consistent with a roundoff error. The second test to verify the adjoint model is the socalled gradient test [45], which aims to compare a finite difference representation of the gradient of 4DVar cost function (13) with the gradient obtained via adjoint model . A linear Taylor approximation of the cost function can be written as Let us introduce the following function:If the gradient is estimated correctly then the function as . The perturbation vector is taken to be [45]where is the norm. Table 3 manifests the success of the gradient test.


6. Sensitivity of the System with respect to Parameters
According to the sensitivity theory [44], general solutions of sensitivity equations for oscillatory nonlinear dynamical systems grow unbounded as time tends to infinity; therefore, sensitivity functions calculated by conventional approaches have a high degree of uncertainty. The reason is that nonlinear dynamical systems that exhibit chaotic behavior are very sensitive to its initial conditions. Thus, the solutions to the linearized Cauchy problem (7) grow exponentially as , where is the leading Lyapunov exponent. As a result, calculated sensitivity coefficients contain a fairly large error [35–37]. To illustrate this point, let us explore the sensitivity of model output to changes in the coupling strength parameter. Let us introduce the following sensitivity coefficients: The corresponding sensitivity equations can be written asSensitivity coefficients can be introduced for any particular model parameter. Since the parameter vector consists of five components (, and ), five sets of sensitivity equations can be derived from the model equations (2). The dynamics of sensitivity coefficients (36) can be traced by solving the sensitivity equations (37) along with the nonlinear model (2).
Sensitivity coefficients (36), calculated on the time interval , are shown in Figure 4. Envelopes of these coefficients grow over time while sensitivity coefficients themselves exhibit oscillating behavior. Figure 5 provides more detailed information on the evolution of sensitivity coefficients (36) calculated on the time interval . It is known that sensitivity coefficient is a measure of the change in state variable due to the variation of the estimated parameter. Unfortunately, obtained sensitivity coefficients are inherently uninformative and misleading. We cannot make a clear conclusion from them about system sensitivity to variations in the parameter . In this regard, the average values of sensitivity functions over a certain period of time can be considered as one of the most important measures of sensitivity, where is a generic objective function (20). However, the gradient cannot be correctly estimated within the framework of conventional methods of sensitivity analysis since for chaotic systems it is observed that [35, 36]This is because the integral does not possess uniform convergence and two limits () would not commute.
Similar results were obtained when we considered the influence of variations in the parameter on the system dynamics. This parameter plays an important role in the formation of system’s dynamical structure and transition to chaotic behavior [32]. Figure 6 shows the differences between components of state vector obtained with and obtained with , where . Even a small perturbation in the parameter generates a tangible difference between corresponding state variables. Let us define the following sensitivity coefficients:The associated system of sensitivity equations can be written as
Envelopes of calculated sensitivity coefficients (40) grow over time and sensitivity coefficients demonstrate the oscillating behavior (Figures 7 and 8). Obtained sensitivity coefficients are also uninformative and inconclusive. The “shadowing” approach for estimating the system sensitivity to variations in its parameters [36, 37] allows us to calculate the average sensitivities and therefore to make a clear conclusion with respect to the system sensitivity to its parameters. A detailed description of two variants of the “shadowing” approach is provided in an appendix, and some results of numerical experiments are presented below.
The main problem arising in the “shadowing” method is to calculate the pseudotrajectory. We consider two sets of numerical experiments: weak coupling () and strong coupling () between fast and slow systems. Fast and slow variables that correspond to the original and pseudoorbits are shown in Figures 9 and 10 when the coupling strength parameter . The Least Square Shadowing variant of the “shadowing” approach was used to calculate pseudotrajectories. The differences between state variables corresponding to the original and pseudotrajectories of the fast and slow systems are plotted in Figure 11. These figures show that the calculated pseudoorbits are close to corresponding true trajectories over a specified time interval, demonstrating the shadowability. The strong coupling does not introduce significant qualitative and quantitative changes in the behavior of pseudotrajectories with respect to the true orbits. The original and pseudo fast and slow variables for are shown in Figures 12 and 13, and the differences between these state variables are presented in Figure 14. Sensitivity estimates with respect to the parameter calculated over the time interval for different values of coupling strength parameter are shown in Table 4. The most sensitive variables are and . The sensitivity of variables , , , and with respect to is significantly less than variables and .

7. Concluding Remarks
We considered a coupled nonlinear dynamical system, which is composed of fast (the “atmosphere”) and slow (the “ocean”) versions of the wellknown Lorenz model. This loworder mathematical tool allows us to mimic the atmosphereocean system and therefore serves as a key part of a theoretical and computational framework for the study of various aspects of coupled 4DVar procedures. Numerical models used to predict the weather are highly nonlinear but tangent linear approximations and their adjoints are used in VDA algorithms. Linear approximation of strongly nonlinear NWP models and also uncertainties in their numerous parameters generate errors in the initial conditions obtained via data assimilation systems. The influence of parameter uncertainties on the results of data assimilation can be studied using sensitivity analysis.
We discussed conventional methods of sensitivity analysis and their inefficiency with respect to calculating sensitivity coefficients for chaotic dynamics. To calculate sensitivity coefficients with acceptable accuracy, the sensitivity analysis method [36, 37], developed on the basis of theory of shadowing of pseudoorbits in dynamical systems [38, 39], was applied. Previously, this method was used to analyze the sensitivity of the periodic van der Pol oscillator, the original Lorenz system, and simplified aeroelastic model that exhibit both periodic and chaotic regimes.
Calculated sensitivity coefficients obtained via conventional methods and the “shadowing” approach are presented and discussed. It was shown that envelopes of sensitivity coefficients obtained by conventional methods grow over time and the coefficients themselves exhibit the oscillating behaviour. Using the “shadowing” method allows us to calculate the average sensitivity functions (coefficients) that can be easily interpreted.
In conclusion, two comments should be highlighted.
(1) The shadowing property of dynamical systems is a fundamental feature of hyperbolic systems that was first discovered by Anosov [46] and Bowen [47]. However, most physical systems are nonhyperbolic. Despite the fact that much of shadowing theory has been developed for hyperbolic systems, there is evidence that nonhyperbolic attractors also have the shadowing property (e.g., [48–51]). In theory this property should be verified for each particular dynamical system, but this is more easily said than done.
(2) The applicability of the shadowing method for sensitivity analysis of modern atmospheric and climate models is a rather complicated problem since these models are quite complex and they contain numerous input parameters. Thus, further research and computational experiments are required. However, we are confident that, by using the basic ideas of the shadowing method, it is possible to better understand the sensitivity analysis of atmospheric models of various levels of complexity.
Appendix
The novel sensitivity analysis method for chaotic dynamical systems developed in [36, 37] is based on the theory of pseudoorbit shadowing in dynamical systems [38, 39], which is one of the most rapidly developing components of the global theory of dynamical systems and classical theory of structural stability [52]. Naturally, pseudo (or approximate) trajectories arise due to the presence of roundoff errors, method errors, and other errors in computer simulation of dynamical systems. Consequently, we will not get an exact trajectory of a system, but we can come very close to an exact solution and the resulting approximate solution will be a pseudotrajectory. The shadowing property (or pseudoorbit tracing property) means that, near an approximate trajectory, there exists the exact trajectory of the system considered, such that it lies uniformly close to a pseudotrajectory. The shadowing theory is well developed for the hyperbolic dynamics, which is characterized by the presence of expanding and contracting directions for derivatives. The study of shadowing problem was originated by Anosov [46] and Bowen [47].
Let be a compact metric space and let be a homeomorphism (a discrete dynamical system on ). A set of points is a pseudotrajectory of ifHere the notation denotes the distance in the phase space between two geometric objects within the brackets.
We say that has the shadowing property if given there is such that for any pseudotrajectory there exists a corresponding trajectory , which traces ; that is
The shadowing lemma for discrete dynamical systems [53] states that, for each , there exists such that each pseudotrajectory can be shadowed.
The definition of pseudotrajectory and shadowing lemma for flows (continuous dynamical systems) [38] are more complicated than for discrete dynamical systems. Let be a flow of a vector field on . A function is a pseudotrajectory of the dynamical system if the inequalities hold for any and . The “continuous” shadowing lemma ensures that, for the vector field generating the flow , the shadowing property holds in a small neighborhood of a compact hyperbolic set for dynamical system .
It is very important to note that the shadowing problem for continuous dynamical systems requires reparameterization of shadowing trajectories. This is the case because for continuous dynamical systems close points of pseudotrajectory and true trajectory do not correspond to the same moments of time. A monotonically increasing homeomorphism such that is called a reparameterization and denoted by Rep. For , is defined as follows [38]:
For simplicity, we will consider a generic autonomous dynamical system with one parameter : The new sensitivity analysis method [36, 37] is based on the “continuous” shadowing lemma and the following two basic assumptions. (a)Model state variables are considered over long time interval , where , and an averaged performance measure is of the most interest for us:(b)The dynamical system under consideration is ergodic.With these assumptions, we can use the arbitrarily chosen trajectory of the system to trace the state variables along the orbit and calculate the performance measure . For example, the arbitrary trajectory can be chosen as a solution of the model equation, such that it is located nearby a certain reference trajectory . Taking into account the shadowing lemma, the closest orbit to satisfies the following constrained minimization problem [37]:where is the parameter that provides the same order of magnitude of the two terms in the integrand and is a time transformation. The second term in the integrand describes reparameterization. Problem (A.7) is called the nonlinear Least Square Shadowing (LSS) problem, and its solution, denoted by and , is a solution of the model equation and time transformation that provides the trajectory to be close to . The performance measure (A.6) averaged over the time interval can be then approximated assince satisfies the model equation at a different . The desired sensitivity estimate can be computed by solving the following linearized LLS problem [37]:The solutions of this problem and relate to the solutions of the nonlinear LSS problem (A.7) as follows: The timedependent parameter is called a timedilation variable and corresponds to the time transformation from the shadowing lemma. Using and , we can estimate the desired sensitivity (the derivative of the linearized performance measure (A.8) with respect to the parameter ):
Several numerical algorithms can be used to solve the linearized LSS problem (A.9). One such method is based on variational calculus, which is used to derive optimality conditions representing a system of linear differential equations that are then discretized and solved numerically to calculate variables and [37]. Let be a uniform discretization time step, then denoting , , , and using the trapezoidal rule to approximate the time derivative of and , we can obtain the following discrete KarushKuhnTucker (KKT) system [37, 54]:whereThe KKT system can be solved using, for example, iterative methods. Gauss elimination approach for solving the KKT system was considered in [37]. The convergence of the LSS method was proved in [55]. Calculated variables and are then used to determine the sensitivity estimate (A.11). The LSS algorithm is summarized as follows.(1)Define a temporal grid , , and then discretize the model equation (A.5) on this grid.(2)Calculate a solution of (A.5) on the time interval .(3)Compute the vectors , , , and .(4)Solve the KKT system to obtain variables and .(5)Compute the gradient components (sensitivity functions) (A.11).Another method that also uses the concept of pseudoorbit shadowing in dynamical systems for sensitivity analysis of chaotic oscillations was developed in [36]. This method is based on inverting the socalled “shadowing operator” that requires calculating the Lyapunov characteristic (covariant) vectors. However, the computational cost needed for the Lyapunov eigenvector decomposition is high when the dynamical system has many positive Lyapunov exponents. To illustrate this approach, suppose that the sensitivity analysis of system (A.5) aims to estimate the following sensitivity coefficient: . Let us introduce the following transform: , where and are true trajectory and pseudoorbit, respectively. The orbit is generated due to the variation in parameter . It can be shown [36] that , where is a “shadow” operator. Thus, to find a pseudotrajectory, we need to solve the equation ; that is, we must numerically invert the operator for a given . To solve this problem, functions and are decomposed into their constituent Lyapunov covariant vectors :Note that each satisfies the following equation:where are the Lyapunov exponents. From (A.14) one can obtainSubstitution of (A.16) into the last term of (A.17) givesFrom (A.15a) and (A.15b) and (A.18) and the relation , we getEquation (A.19) gives the following relationship between and along the orbit:Thus, we can calculate using (A.20) by first decomposing as a sum (A.15b), and then the desired can be obtained from (A.15a). However, if dynamical system has a zero Lyapunov exponent, , then the algorithm described above fails to compute [36]. The problem can be resolved by introducing a timedilation variable that satisfies the following equation:where
In the presence of the variable , the expression for calculating takes the following form: . The supplement affects (A.20) only for :
In general, the procedure for solving a sensitivity analysis problem represents the following set of steps.(1)Obtain initial conditions of the system attractor by integrating the model equations (A.5) from to , starting from random initial conditions.(2)Solve (A.5) to obtain a trajectory , , on the attractor.(3)Compute the Lyapunov exponents and the Lyapunov covariant vectors , .(4)Define and execute the Lyapunov spectrum decomposition of along the trajectory to obtain , .(5)Calculate the timedilation variable using (A.21).(6)Compute along the trajectory .(7)Estimate the sensitivity by averaging over the time interval .Conflict of Interests
The authors declare that there is no conflict of interests regarding the publication of this paper.
References
 F. Rawlins, S. P. Ballard, K. J. Bovis et al., “The met office global fourdimensional variational data assimilation scheme,” Quarterly Journal of the Royal Meteorological Society, vol. 133, no. 623, pp. 347–362, 2007. View at: Publisher Site  Google Scholar
 K. Puri, G. Dietachmayer, P. Steinle et al., “Implementation of the initial ACCESS numerical weather prediction system,” Australian Meteorological and Oceanographic Journal, vol. 63, no. 2, pp. 265–284, 2013. View at: Google Scholar
 R. Daley, Atmospheric Data Analysis, Cambridge University Press, New York, NY, USA, 1991.
 E. Kalnay, Atmospheric Modeling, Data Assimilation and Predictability, Cambridge University Press, New York, NY, USA, 2003.
 F. Rabier, “Overview of global data assimilation developments in numerical weatherprediction centres,” Quarterly Journal of the Royal Meteorological Society, vol. 131, no. 613, pp. 3215–3233, 2005. View at: Publisher Site  Google Scholar
 I. M. Navon, “Data assimilation for numerical weather prediction: a review,” in Data Assimilation for Atmospheric, Oceanic and Hydrologic Applications, S. K. Park and L. Xu, Eds., Springer, New York, NY, USA, 2009. View at: Google Scholar
 D. G. Cacuci, I. M. Navon, and M. IonescuBujor, Computational Methods for Data Evaluation and Assimilation, CRC Press, Boca Raton, Fla, USA, 2013.
 I. Sasaki, “An objective analysis based on variational method,” Journal of the Meteorological Society of Japan, vol. 36, no. 3, pp. 29–30, 1958. View at: Google Scholar
 J. L. Lions, Optimal Control of Systems Governed by Partial Differential Equations, Springer, Berlin, Germany, 1971. View at: MathSciNet
 V. V. Penenko and N. N. Obraztsov, “Variational method for assimilation of meteorological fields,” Meteorology and Hydrology, no. 11, pp. 3–16, 1976. View at: Google Scholar
 F.X. Le Dimet, “Une application des méthodes de controle, optimal 'a l'analyse varationelle,” Rapport Scientifique LAMP 63170, Université Blaise Pascal, ClermontFerrand, France, 1981. View at: Google Scholar
 O. Talagrand, “A study of the dynamics of four—dimensional data assimilation,” Tellus, vol. 33, no. 1, pp. 43–60, 1981. View at: Publisher Site  Google Scholar
 D. G. Cacuci, “Sensitivity theory for nonlinear systems. I. Nonlinear functional analysis approach,” Journal of Mathematical Physics, vol. 22, no. 12, pp. 2794–2802, 1981. View at: Publisher Site  Google Scholar  MathSciNet
 D. G. Cacuci, “Sensitivity theory for nonlinear systems. II. Extensions to additional classes of responses,” Journal of Mathematical Physics, vol. 22, no. 12, pp. 2803–2812, 1981. View at: Publisher Site  Google Scholar  MathSciNet
 J. M. Lewis and J. C. Derber, “The use of adjoint equations to solve a variational adjustment problem with advective constraints,” Tellus A, vol. 37, no. 4, pp. 309–322, 1985. View at: Publisher Site  Google Scholar
 F.X. Le Dimet and O. Talagrand, “Variational algorithms for analysis and assimilation of meteorological observations: theoretical aspects,” Tellus Series A, vol. 38, no. 2, pp. 97–110, 1986. View at: Google Scholar
 O. Talagrand and P. Courtier, “Variational assimilation of meteorological observations with the adjoint vorticity equation. Part 1. Theory,” Quarterly Journal of the Royal Meteorological Society, vol. 113, pp. 1311–1328, 1987. View at: Publisher Site  Google Scholar
 P. Courtier and O. Talagrand, “Variational assimilation of meteorlogical observations with the adjoint vorticity equation. II: numerical results,” Quarterly Journal of the Royal Meteorological Society, vol. 113, no. 478, pp. 1329–1347, 1987. View at: Publisher Site  Google Scholar
 F.X. le Dimet, H.E. Ngodock, B. Luong, and J. Verron, “Sensitivity analysis in variational data assimilation,” Journal of the Meteorological Society of Japan, vol. 75, no. 1, pp. 245–255, 1997. View at: Google Scholar
 N. L. Baker and R. Daley, “Observation and background adjoint sensitivity in the adaptive observationtargeting problem,” Quarterly Journal of the Royal Meteorological Society, vol. 126, no. 565, pp. 1431–1454, 2000. View at: Publisher Site  Google Scholar
 G. Hello and F. Bouttier, “Using adjoint sensitivity as a local structure function in variational data assimilation,” Nonlinear Processes in Geophysics, vol. 8, no. 6, pp. 347–355, 2001. View at: Publisher Site  Google Scholar
 R. H. Langland and N. L. Baker, “Estimation of observation impact using the NRL atmospheric variational data assimilation adjoint system,” Tellus Series A, vol. 56, no. 3, pp. 189–203, 2004. View at: Publisher Site  Google Scholar
 R. M. Errico, “Interpretations of an adjointderived observational impact measure,” Tellus, Series A: Dynamic Meteorology and Oceanography, vol. 59, no. 2, pp. 273–276, 2007. View at: Publisher Site  Google Scholar
 R. Gelaro, Y. Zhu, and R. M. Errico, “Examination of variousorder adjointbased approximations of observation impact,” Meteorologische Zeitschrift, vol. 16, no. 6, pp. 685–692, 2007. View at: Publisher Site  Google Scholar
 D. N. Daescu, “On the sensitivity equations of fourdimensional variational (4DVar) data assimilation,” Monthly Weather Review, vol. 136, no. 8, pp. 3050–3065, 2008. View at: Publisher Site  Google Scholar
 Y. Trémolet, “Computation of observation sensitivity and observation impact in incremental variational data assimilation,” Tellus, Series A: Dynamic Meteorology and Oceanography, vol. 60, no. 5, pp. 964–978, 2008. View at: Publisher Site  Google Scholar
 D. N. Daescu and R. Todling, “Adjoint estimation of the variation in model functional output due to the assimilation of data,” Monthly Weather Review, vol. 137, no. 5, pp. 1705–1716, 2009. View at: Publisher Site  Google Scholar
 R. Gelaro and Y. Zhu, “Examination of observation impacts derived from observing system experiments (OSEs) and adjoint models,” Tellus, Series A: Dynamic Meteorology and Oceanography, vol. 61, no. 2, pp. 179–193, 2009. View at: Publisher Site  Google Scholar
 D. N. Daescu and I. M. Navon, “Sensitivity analysis in nonlinear data assimilation: theoretical aspects and applications,” in Advances Numerical Methods for Complex Environmental Models: Needs and Availability, I. Farago and Z. Zlatev, Eds., pp. 276–300, Benham Science Publishers, 2013. View at: Google Scholar
 D. G. Cacuci, Sensitivity and Uncertainty Analysis, Volume I: Theory, CRC, Boca Raton, Fla, USA, 2003.
 D. G. Cacuci, M. IonescuBujor, and I. M. Navon, Sensitivity and Uncertainty Analysis, CRC, Boca Raton, Fla, USA, 2005. View at: Publisher Site  MathSciNet
 E. N. Lorenz, “Deterministic nonperiodic flow,” Journal of the Atmospheric Sciences, vol. 20, no. 2, pp. 130–141, 1963. View at: Google Scholar
 M. Peña and E. Kalnay, “Separating fast and slow modes in coupled chaotic systems,” Nonlinear Processes in Geophysics, vol. 11, no. 3, pp. 319–327, 2004. View at: Publisher Site  Google Scholar
 L. Siqueira and B. Kirtman, “Predictability of a loworder interactive ensemble,” Nonlinear Processes in Geophysics, vol. 19, no. 2, pp. 273–282, 2012. View at: Publisher Site  Google Scholar
 D. J. Lea, M. R. Allen, and T. W. N. Haine, “Sensitivity analysis of the climate of a chaotic system,” Tellus, vol. 52, no. 5, pp. 523–532, 2000. View at: Publisher Site  Google Scholar
 Q. Wang, “Forward and adjoint sensitivity computation of chaotic dynamical systems,” Journal of Computational Physics, vol. 235, no. 15, pp. 1–13, 2013. View at: Publisher Site  Google Scholar  MathSciNet
 Q. Wang, R. Hu, and P. Blonigan, “Least squares shadowing sensitivity analysis of chaotic limit cycle oscillations,” Journal of Computational Physics, vol. 267, pp. 210–224, 2014. View at: Publisher Site  Google Scholar  MathSciNet
 S. Y. Pilyugin, Shadowing in Dynamical Systems, vol. 1706 of Lecture Notes in Mathematics, Springer, Berlin, Germany, 1999.
 K. J. Palmer, Shadowing in Dynamical Systems. Theory and Applications, Kluwer Academic, Dordrecht, The Netherlands, 2000. View at: Publisher Site  MathSciNet
 M. G. Rosenblum, A. S. Pikovsky, and J. Kurths, “Phase synchronization of chaotic oscillators,” Physical Review Letters, vol. 76, no. 11, pp. 1804–1807, 1996. View at: Publisher Site  Google Scholar
 F. Christiansen, G. Paladin, and H. H. Rugh, “Determination of correlation spectra in chaotic systems,” Physical Review Letters, vol. 65, no. 17, pp. 2087–2090, 1990. View at: Publisher Site  Google Scholar  MathSciNet
 C. Liverani, “Decay of correlations,” Annals of Mathematics, vol. 142, no. 2, pp. 239–301, 1995. View at: Publisher Site  Google Scholar  MathSciNet
 S. Panchev and T. Spassova, “Simple general atmospheric circulation and climate models with memory,” Advances in Atmospheric Sciences, vol. 22, no. 5, pp. 765–769, 2005. View at: Publisher Site  Google Scholar
 E. Rosenwasser and R. Yusupov, Sensitivity of Automatic Control Systems, CRC Press, Boca Raton, Fla, USA, 2000. View at: MathSciNet
 I. M. Navon, X. Zou, J. Derber, and J. Sela, “Variational data assimilation with an adiabatic version of the NMC spectral model,” Monthly Weather Review, vol. 120, no. 7, pp. 1433–1446, 1992. View at: Publisher Site  Google Scholar
 D. V. Anosov, “On class of invariant sets of smooth dynamical systems,” in Proceedings of the 5th International Conference on Nonlinear Oscillations, vol. 2, pp. 39–45, Kiev, Ukraine, 1975. View at: Google Scholar
 R. Bowen, Equilibrium States and the Ergodic Theory of Anosov Diffeomorphisms, vol. 470 of Lecture Notes in Mathematics, Springer, Berlin, Germany, 1975. View at: MathSciNet
 A. Koropecki and E. R. Pujals, “Some consequences of the shadowing property in low dimensions,” Ergodic Theory and Dynamical Systems, vol. 34, no. 4, pp. 1273–1309, 2014. View at: Publisher Site  Google Scholar  MathSciNet
 D. Richeson and J. Wiseman, “Symbolic dynamics for nonhyperbolic systems,” Proceedings of the American Mathematical Society, vol. 138, no. 12, pp. 4373–4385, 2010. View at: Publisher Site  Google Scholar  MathSciNet
 A. A. Petrov and S. Y. Pilyugin, “Shadowing near nonhyperbolic fixed points,” Discrete and Continuous Dynamical Systems, vol. 34, no. 9, pp. 3761–3772, 2013. View at: Google Scholar  MathSciNet
 S. Y. Pilyugin, Spaces of Dynamical Systems, De Gruyter, Berlin, Germany, 2012. View at: Publisher Site  MathSciNet
 A. Katok and B. Hasselblatt, Introduction to the Modern Theory of Dynamical Systems, vol. 54 of Encyclopedia of Mathematics and its Applications, Cambridge University Press, New York, NY, USA, 1995. View at: Publisher Site  MathSciNet
 D. V. Anosov, Geodesic Flows on Closed Riemann Manifolds with Negative Curvature, vol. 90 of Proceedings of the Steklov Institute of Mathematics, 1967.
 S. P. Boyd and L. Vandenberghe, Convex Optimization, Cambridge University Press, Cambridge, UK, 2004. View at: MathSciNet
 Q. Wang, “Convergence of the least squares shadowing method for computing derivative of ergodic averages,” SIAM Journal on Numerical Analysis, vol. 52, no. 1, pp. 156–170, 2014. View at: Publisher Site  Google Scholar  MathSciNet
Copyright
Copyright © 2015 Sergei Soldatenko et al. This is an open access article distributed under the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.