Abstract
The availability of adequate system models to reproduce, as faithfully as possible, the actual behaviour of the experimental systems is of key importance. In marine systems, the changing environmental conditions and the complexity of the infrastructure needed to carry out experimental tests call for mathematical models for accurate simulations. There exist a wide number of techniques to define mathematical models from experimental data. Support Vector Machines (SVMs) have shown a great performance in pattern recognition and classification research areas having an inherent potential ability for linear and nonlinear system identification. In this paper, this ability is demonstrated through the identification of the Nomoto second-order ship model with real experimental data obtained from a zig-zag manoeuvre made by a scale ship. The mathematical model of the ship is identified using Least Squares Support Vector Machines (LS-SVMs) for regression by analysing the rudder angle, surge and sway speed, and yaw rate. The coefficients of the Nomoto model are obtained with a linear kernel function. The model obtained is validated through experimental tests that illustrate the potential of SVM for system identification.
1. Introduction
Nowadays, the breakthroughs in computer science and the many research works on control engineering and autonomous vehicles call for the availability of adequate mathematical models with which new designs and ideas can be tested in simulation to predict the behaviour of the real systems with high accuracy. Furthermore, due to the high cost of practical implementations and tests, it is of higher importance to have available tools and methods to compute these mathematical models for simulation purposes. In this sense, system identification is one of the highlights among the research topics in engineering and one of the most important stages in control of autonomous vehicles. In the present work, we will focus on the computation of a mathematical model that describes the dynamical behaviour of a surface marine vessel by means of experimental data. This model is computed with a Support Vector Machines (SVMs) technique, the Least Squares Support Vector Machines (LS-SVMs). SVM has shown a great performance in pattern recognition and classification research areas, and it has a potential ability for linear and nonlinear system identification that will be shown throughout this work.
The literature on linear and nonlinear system identification is extensive and covers many areas of engineering research. For a short survey on some essential features in the identification area and a classification of methods the reader is referred to [1]. The state-space identification methods are one of the most common and intuitive techniques to determine mathematical models in the engineering practice. The state-space identification methods for linear time-invariant (LTI) systems are well established, [2]. In the nonlinear case, the input and output signals of the system can be used to estimate the state-space representation. Thus, it is of practical interest to be able to determine the description of a nonlinear system from a finite number of input and output measurements, in which the use of Support Vector Machines (SVMs) can provide an interesting new method to solve this problem.
In surface and underwater marine robotics, the systems and vehicles employed can have a high degree of complexity. Moreover, the changing environmental conditions and the complexity of the infrastructure needed to carry out experimental tests call for mathematical models for accurate simulations. System identification of marine vehicles starts in the 70s with the works of [3], where an adaptive autopilot with reference model was presented, and [4], where parametric linear identification techniques were used to define the guidance dynamics of a ship using the maximum likelihood method. We can find in the literature on maritime research several mathematical models to describe the dynamics of surface vessels and underwater vehicles and algorithms to compute these models; see, for example, [5], where several parametric identification algorithms are used to design autopilots for different kinds of ships, [6] in which the hydrodynamic characteristics of a ship are determined by a Kalman Filter (KF) or [7] where an extended Kalman filter (EKF) is used for the identification of a ship dynamics for dynamic positioning. Moreover, different operational conditions that may affect the vessel and that provide different models have been analysed; see [8] where an overview of the motion dynamics topic of surface marine vessels and models is given. The procedure to obtain these models is not an easy and simple task, it usually requires a lot of time on practical tests and an important computational effort. In [9], the authors define a practical ship model and the identification process for simulation, guidance, and control for small and low-cost vehicles. The paper shows the lengthy procedure to obtain a correct identification of the ship model. For some more interesting works, the reader is referred to [10, 11] and the references therein.
Despite this, in most cases and practical situations, it is possible to employ a simple model to reproduce the dynamics of a vessel in standard operational conditions. For this reason, in this paper the Nomoto second-order model is used, [12]. This approach provides a less precise model of the vehicle; however, this model shows very good results in the experimental setup for standard operations, as mentioned above, and it can be obtained with SVM techniques in a fast manner with relatively few data, as it will be shown in Section 4.
Artificial-neural-networks-based techniques have been employed for system identification such as multilayer perceptron (MLP); see, for example, [13]. We can find some works that employ neural networks to define the dynamics of a surface marine vehicle, such as [14–16] or [17]. These techniques have shown to be robust and effective in identification and control, although they also show some inherent drawbacks such as the multiple local minima problem or overfitting, to name but a few. These problems can be avoided using SVM since it is not based on the empirical error implemented in neural networks; rather it is based on the structural risk minimization (SRM); so SVM provides a larger generalisation performance. An estimate of generalisation ability may be obtained from a trained network without using test data, assuming the training data to be representative. Vapnik-Chervonenkis (VC) theory is the basic idea behind SVM which defines a measure of the capacity of a learning machine [18]. The idea is to map input data into a high-dimensional feature Hilbert space using a nonlinear mapping technique, that is, the kernel dot product trick [19] and to carry out linear classification or regression in feature space. The Kernel functions replace a possibly very-high-dimensional Hilbert space without explicitly increasing the feature space [20]. SVM, both for regression and classification, has the ability to simultaneously minimize the estimation error in the training data (the empirical risk) and the model complexity (the structural risk) [21]. Moreover, SVM can be designed to deal with sparse data, where we have many variables but few data. Furthermore the solution of SVM is globally optimal.
In support vector regression (SVR), a nonlinear model is represented by an expansion in terms of nonlinear mappings of the model input, so that they define a feature space that may be of infinite dimension. A convenient feature in SVR is that the optimal model complexity is obtained as part of the solution and does not have to be determined separately. For a survey and a tutorial on support vector regression, the reader is referred to [22] and the references therein. The goal is to find a set of parameters for a given model which are chosen to fit measured values of output and input data.
An inadequate model identification may yield large prediction errors. It is important to remark that even though the problem of system identification is of great importance and the use of SVM has experienced a great boost and diversification in the last few years, not many results are available on the topic of system identification with SVM for regression yet. Exceptions include a series of interesting results such as the work of [23], where the authors make a study of the possible use of SVM for system identification, in particular for parameter estimation in discrete-time linear models and model structure identification for nonlinear models, more specifically Nonlinear Auto-Regressive Moving Average with eXogeneous Input (NARMAX) models. The potential of SVM for system identification is remarked, but some significant issues to be addressed are shown. In [24], an identification method based on SVR is proposed for linear regression models such as finite impulse response (FIR) and Auto-Regressive with eXogeneous input (ARX) models. The paper shows how the method is robust for simulated data and it is even applied to actual data from rolling mill showing a good learning performance. In the work of [25] the application of SVM to time series modelling is considered by means of simulated data from an autocatalytic reactor. In particular, a state-space model is defined for the process with reconstruction methods from nonlinear dynamics. As mentioned above, the identification process in [25], is done by means of simulation data, as most of the papers that deal with identification using SVM. In [26], a SVM with linear-kernel-function-based nonparametric model identification and dynamic matrix control (DMC) technique is introduced. The paper [27] shows how the kernel canonical correlation analysis (KCCA) can be used to construct a state sequence of an unknown nonlinear dynamic system from delay vectors of inputs and outputs. This sequence is then used together with input and output data to identify a nonlinear state-space model. In [28], a design procedure of SVM-based model identification and control strategy is proposed. The proposed design procedure and the SVM-based model identification are illustrated with the control of a simulated continuous-stirred tank reactor. In [29], SVR is applied to the identification of a nonlinear system that corresponds to a Wiener model, that consists of a linear dynamic system with a static nonlinear block. The static nonlinear block is determined by SVR. The model employed is function of the input only, not depending on the measured system outputs. The identification procedure is tested with numerical simulations.
In this work, a variation of SVM, the Least Squares Support Vector Machines (LS-SVMs) [30], is used for system identification with regression. This technique allows a nice simplification of the problem making it more tractable. A study on this subject can be found in [31] where LS-SVM is used for nonlinear system identification for some simple examples of Nonlinear Auto-Regressive with eXogenous input (NARX) input-output models. There also exist some other representative examples that deal with identification using LS-SVM. For example, in [32], the identification of MIMO Hammerstein ARX models based on LS-SVM is studied. The results are compared with two other techniques (the Hermite polynomial expansion and a RBF network) with numerical simulations showing an improvement of the performance with the LS-SVM approach. In [33], a new support vector kernel is defined, the wavelet kernel that satisfies wavelet frames. This kernel is able to approximate arbitrary functions improving the generalisation ability of the SVM. Based on this kernel and on LS-SVM, the LS-WSVM is constructed and it is applied to a nonlinear system identification problem showing better accuracy than a Gaussian kernel for the problem studied. Finally in [34], the LS-SVM is combined with the multiresolution analysis (MRA) to provide a new improved algorithm, the multiresolution least squares support vector machines (MLS-SVMs). This algorithm has the same theoretical framework as MRA but it has a better approximation ability and can gradually approximate the target function at different scales.
With respect to the identification of marine vehicles by using SVM, we can find two interesting references, [35] and [36], where an Abkowitz model for ship manoeuvring is identified by using LS-SVM and -SVM, respectively. The purpose of these works is to determine the hydrodynamic coefficients of a Mariner-class vessel, although the identification of the mathematical model is made with data obtained from simulation and then tested only in simulation. These works do not deal with real data. Furthermore, it is important to point out that most of the works on system identification with SVR deal with simulation data and numerical examples, where the use of SVR is not tested on an experimental setup, as far as the authors know. In striking contrast to what is commonly reported in the literature, in the paper at hand the model identification is done with experimental data obtained from a scale surface marine vessel, and the mathematical model obtained is validated with experimental tests carried out with the scale ship, showing how the model predicts with large accuracy the actual behaviour of the surface vessel. It is important to keep in mind, for the experimental results obtained in this paper, that the analytical properties of SVM can be compromised in stochastic problems because the noise generates additional support vectors. However, if the noise ratio is good and the amplitude is limited, the SVM can solve the problem as if it was deterministic [23].
The key contributions of the present paper are twofold: (i) the mathematical model of a surface marine vessel is obtained with the LS-SVM technique from experimental data collected from a 20/20 degree zig-zag manoeuvre; and (ii) in striking contrast to what is customary in the literature, the prediction ability of the mathematical model computed with LS-SVM regression is tested on an open air environment with the real ship. This allows us to explicitly address the connection between the mathematical model identified and the ship, so that the model can be used to design control strategies to be implemented on the real vehicle and to predict its behaviour.
The document is organized as follows. In Section 2, a brief introduction on LS-SVM is given. The Nomoto second-order model and the training input and output data to be used are defined in Section 3. In Section 4, the Nomoto model computed from the training data is well defined and its behaviour is tested with some turning manoeuvres. Finally, the conclusions and a brief discussion of topics for further research are included in Section 5.
2. Least Squares Support Vector Machines for Regression
In this section, we briefly introduce LS-SVM for system identification. As mentioned in Section 1, for both regression and classification, SVM has the ability to simultaneously minimize the estimation error in the training data (the empirical risk) and the model complexity (the structural risk). SVM can also be designed to deal with sparse data, where we may have many variables but few data. The input data are mapped into a high dimensional feature space using nonlinear mapping techniques, and then linear classification or regression is carried out in feature space.
The use of LS-SVM for regression is very similar to its use for classification. Following the notation and definitions in [22, 30], consider a model in the primal weight space: where is the input data, is the output data, is a bias term, is a matrix of weights, and is the mapping to a high-dimensional space, where can be infinite. The optimization problem in the primal weight space, for a given training set , with as the number of samples, becomes subject to where are error variables, and is the regularisation parameter that determines the deviation tolerated from the desired accuracy which must be positive. The minimization of is closely related to the use of a weight decay term in the training of neural networks, and the second term of the right-hand side of (2) controls the tradeoff between the empirical error and the model complexity. The main differences between the formulation of LS-SVM with the standard SVM are the equality constraints (3) and the squared error term of the second term in the right-hand side of (2), implying a significant simplification of the problem.
The above primal problem cannot be solved when becomes infinite dimensional. Thus, the Lagrangian must be computed and the dual problem derived, yielding where , with , are the Lagrange multipliers. The optimality conditions are defined by computing the derivatives of (4) with respect to , and : for . After elimination of variables and from the above equations, and applying the kernel trick which allows us to work in large-dimensional feature spaces without explicit computations on them [19], the LS-SVM model for function estimation yields where is the kernel function that represents an inner product between its operands. This kernel must be a positive definite kernel and must satisfy the Mercer condition [37].
3. Identification of a Nomoto’s Second-Order Model
In marine systems, as in a multitude of engineering areas, an adequate mathematical model is essential to simulate and predict the behaviour of a real system with the best possible accuracy. If reliable mathematical models are available and they predict the behaviour of the real systems accurately, then control actions can be planned and tested in simulation avoiding the costly, in time and money, practical tests.
A reliable mathematical ship model, like the Abkowitz model [38], requires the identification of a multitude of hydrodynamic parameters. The identification task for this large number of variables can be hard and may take a long time. Moreover, different tests for the different hydrodynamic parameters must be carried out [9, 10].
For the above reason, simpler vehicle models are commonly used so that, although without modelling all the dynamic features of the vehicle, each of these models reproduces the real behaviour of ships with large accuracy in most of practical scenarios. For the problem at hand, we consider a constant surge speed and we determine a mathematical model that defines the ship steering equations. Among these kinds of models, we can find the Nomoto models [12], that are an alternative representation of the model of Davidson and Schiff [39]. We now briefly comment these two models.
Given the linear steering dynamics: the hydrodynamic force and moment can be modelled as in [39]: where is sway force, is moment (rotation about the -axis, with the -axis pointing downwards as is customary in marine systems), is yaw rate, is sway velocity, is nominal surge velocity, is the rudder angle, is moment of inertia about the -axis, is mass, is the -axis coordinate of the centre of gravity, and , and are added inertia hydrodynamic coefficients; for more details, the reader is referred to [10]. See Figure 1 for a definition of the reference frames used in marine systems.
Following the procedure of [10], (8) can be rewritten as where is the state vector, and
From (9), the alternative representation of the second-order model of Nomoto [12] can be defined. This model provides the transfer function that relates the yaw rate to the rudder angle according to or if an integrator (a pole) is included; the transfer function relates the yaw angle to the rudder angle ; that is, where , , and are time constants and is gain. Now rewrite (11) in the time domain: where and represents the third derivative of the yaw angle with respect to time. The parameters of (11) are related to the hydrodynamic derivatives as follows: where , , and , with and , are the elements of matrices , , and .
In addition, we can express the sway velocity in a similar manner: where and are the gain and time constants to describe the sway velocity. In the time domain, (15) yields
The main advantage of the Nomoto second-order model is its simplicity and that the hydrodynamic derivatives do not have to be computed explicitly it is possible to define the parameters of the Nomoto model directly from the experimental data. Therefore, we can compute in a fast and simple manner a ship model for control purposes because it is not the aim of this work to know explicitly the hydrodynamic derivatives.
To proceed with the reconstruction of the ship manoeuvring model, the continuous equations of motion, (13) and (16), are discretized because the experimental data are taken with a given regular interval of time. The discretization is done by the Euler stepping method, so that the above equations can be rewritten as where is the time between two consecutive measurements taken from the IMU (inertial measurement unit) on board the ship, and , , and are the indices of three successive sampling times. Now, it is possible to apply the LS-SVM regression to obtain the parameters of the model of Nomoto by constructing the training data following the equation structure of (17). Thus, the discrete motion equations given by (17) can be rewritten as and in a simplified manner: where and are the output training data, and and are the input training data given by for , and with
The vectors (21) are now computed from LS-SVM regression. It is important to point out that the vectors (21) are separately identified; that is, is defined first, and after that, is defined, so that and are already known for the computation of . The structure of the mathematical model is known in advance; so we can use a linear kernel . The result of applying the LS-SVM technique for the training data gives us for and . Comparing the above equation with (19), after the training process we have if the object function has been well approximated by the LS-SVM algorithm, where the bias term must be equal to or approximately 0. Once the support vectors are defined by the above process, the parameters of the Nomoto model can be obtained immediately from (21).
4. Experimental Results
The vehicle used for the experimental tests is a scale model of an operational vessel, in a scale, see Figure 2. The dimensions of the vessel and the scale model are shown in Table 1.
The scale ship model, henceforth referred to as the ship, is equipped with an electric motor to control the propeller turning speed and a servo to control the rudder angle. The desired rudder angle and surge speed are commanded through a Wi-Fi connection between the ship and the control station at land.
The training data used for system identification are obtained from a 20/20 degree zig-zag manoeuvre with a sample time of 0.2 seconds and a nominal surge speed of 1 m/s. The zig-zag manoeuvre was executed for 100 seconds; so a set of 500 samples is trained. In Figure 3, the commanded rudder angle (dashed line) and the yaw angle (solid line) made by the ship during the 20/20 degree zig-zag manoeuvre are shown. The training data are constructed with the commanded control signal (rudder angle) and the data measured from the IMU on board the ship (yaw rate and sway velocity). From the input and output data, and with (19), the LS-SVM algorithm is now trained to determine the vectors and defined in (21).
Using the data shown in Figure 3, with the regularisation parameter , the Nomoto model parameters obtained from the identification process with LS-SVM regression are , , , , , and . Then, the Nomoto model yields
Figure 4 shows the yaw rate obtained from the IMU on board the ship (solid line) together with the yaw rate obtained in simulation (dotted line) with the Nomoto second-order model previously defined and with the same commanded input signal than the ship. It can be noticed how the behaviour of the model fits the real behaviour of the ship in a nice manner for the training data used in the identification process.
In the same way, in Figure 5 the sway velocity measured from the IMU on board the ship is shown together with the sway velocity obtained from the Nomoto model. Again, the results are very similar showing a good behaviour of the model for the training data. Despite the above, the model for the sway velocity is less precise than the model for the yaw rate because the sway velocity cannot be directly controlled due to the fact that the ship studied is an underactuated vehicle. Moreover, the environmental conditions, like currents and winds, particularly affect the sway velocity. In any case, the model is satisfactory; furthermore, the yaw rate is the component that defines in a higher degree the actual behaviour of the ship.
Once the Nomoto model is well defined and it fits well with the training data, as shown in Figures 4 and 5, the model must be tested with different data and manoeuvres to verify the generalisation performance of the model obtained. In the example shown below, two consecutive turning manoeuvres (evolution circles) are performed with the simulation model and the ship, for the same input signal. The input signal commanded is the rudder angle, that takes the value of 10 degrees during 255 seconds, then a rudder angle of 0 degrees is commanded for 70 seconds, and finally a rudder angle of −10 degrees is commanded during 275 seconds, with a nominal surge speed of 1 m/s.
In Figures 6 and 7, the yaw rate and sway velocity for the ship (solid line) and the Nomoto model (dotted line) are shown, respectively. It is clear that the predicted behaviour is very similar to the actual one of the ship, so that LS-SVM provides a good mathematical model to test different control strategies to be applied on the ship.
We can notice in Figure 6 how the real system deviates slightly in turnings with negative rudder angles because the actual yaw rate behaviour is not exactly symmetric; it is slightly larger for negative rudder angles. Despite this small deviation on the supposed symmetrical behaviour of the ship, the result obtained from the simulated model is very similar to the actual one, and the difference is not significant. This deviation could be produced by environmental conditions, like currents or winds, or by structural characteristics of the ship, like the trimming, that can make the ship to not have exactly a symmetrical behaviour, while this symmetrical behaviour is implicit on the Nomoto models. This possible source of errors in the fidelity of the mathematical model should be further studied to be able to define more complex mathematical models, although in our practical case, as can be deduced from Figures 6 and 7, these errors are negligible.
In Figures 8 and 9, the error between the real and the predicted yaw rate and sway velocity are shown, respectively. We can notice from the yaw rate in Figure 8 how the error, although very small, is larger for negative rudder angles as mentioned above. Despite the commented deviation, the error has a small magnitude and its average value is very close to 0, providing a more than satisfactory prediction of the real dynamical behaviour of the ship. The same comments apply to Figure 9 and the sway velocity instead of the yaw rate.
Therefore, it is clear that the mathematical model defined for a surface marine vehicle with LS-SVM provides a satisfactory result which predicts with large accuracy the dynamics of the experimental system.
Now, it is interesting to compare the result of LS-SVM identification with the result that can be obtained by standard identification methods. The latter model is computed using the identification toolbox of MATLAB called Ident; for the documentation of this toolbox, the reader is referred to [40]. The model computed with MATLAB is defined by
In Figures 10 and 11, the yaw rate and sway velocity, respectively, are shown for the experimental test (solid line), the LS-SVM identification model (dotted line), and the identification model obtained with the Ident tool of MATLAB (dashed line) for the training data, that is, the 20/20 degree zig-zag manoeuvre. We can check from Figures 10 and 11 that for the practical case studied the behaviours obtained with both identification methods are practically the same.
The main difference between both methods lies in the important fact that the SVM approach can deal with a large number of variables and few data, as shown in [35] where multiple hydrodynamic variables are computed from a single manoeuvre in a simulation scenario. In a practical scenario, and with a classical identification methodology, several practical tests should be carried out to define these multiple hydrodynamic variables, with the corresponding time cost. Moreover, if the model structure is not known in advance some approach of black box identification may be used, as in [32] or [29], by means of nonlinear kernels. In these systems, a nonlinear block that is approximated by selection of a basis function is used. In SVM, the nonlinear mappings of this function can be unknown; they may be defined with a nonlinear kernel function. Therefore, this technique has the potential to be implemented for different kinds of marine vehicles in a simple and fast manner. These issues are a very interesting line for future research.
5. Conclusions and Future Work
In this work, the Nomoto second-order model of a ship has been determined through real experimental data obtained from a zig-zag manoeuvre test. This ship model has been identified by analysing the rudder angle, surge and sway speeds, and yaw rate obtained from experimental tests. This identification has been done with a Least Squares Support Vector Machines algorithm. The LS-SVM technique has been widely used for pattern recognition and classification with very good results, and in this work it has been shown that this technique has a high potential to be used in system identification. The model obtained with LS-SVM has been validated through experimental tests showing that the behaviour of the mathematical model is very similar to that of the real ship. Furthermore, the model obtained is suitable to be used for testing control algorithms that can be implemented on the ship. It has been proved that LS-SVM is a powerful tool for system identification that can deal with relative few experimental data and low time cost.
Future work will aim at (i) extending the methodology developed to deal with more complex ship models in which the hydrodynamic coefficients can be well defined and to deal with models whose structure is not known in advance, and (ii) studying the performance of control algorithms for path following and tracking with the ship model defined in comparison with the results obtained for the real vehicle.
Acknowledgment
The authors wish to thank the Spanish Ministry of Science and Innovation (MICINN) for support under Project DPI2009-14552-C02-02.