Abstract
Robust object tracking and maneuver estimation methods play significant role in the design of advanced driver assistant systems and self-driving cars. As an input to situation understanding and awareness, the performance of such algorithms influences the overall effectiveness of motion planning and plays high role in safety. The paper examines the suitability of different probabilistic state estimation methods, namely, the Extended Kalman Filter (EKF) and the more general Particle Filter (PF) with the addition of the Interacting Multiple Model (IMM) approach. These algorithms are not capable of predicting motion for long term in road traffic conditions, though their robustness and model classification capability are essential for the overall system. The performance is evaluated in road traffic scenarios where the tracked object imitates the motion characteristics of a road vehicle and is observed from a stationary sensor. The measurements are generated according to standard automotive radar models. The analysis conducted along two aspects emphasizes the different performance and scaling properties of the examined state estimation algorithms. The presented evaluation framework serves as a customizable method to test and develop advanced autonomous functions.
1. Introduction
Nowadays, highly automated driver assistance systems and autonomous vehicles are in focus of attention and pose many different challenges that need to be solved. The architecture of the motion planning has multiple layers from route planning, through behavioral and tactical planning to local control [1]. The perception subsystem consists of vehicle and environment sensors, data fusion and tracking layers, behavioral classification and prediction tasks [2]. These layers need to work together to fulfill the original task of the vehicle, namely, to reach its destination with the possible minimization of journey time though with respect to traffic rules, passenger comfort, and, most importantly, to safety. The local and short term decisions require the autonomous vehicle to have some ability to reason about the future motion of surrounding vehicles [3]. This leads to the problem of behavior prediction, where the ego vehicle needs to predict the possible future trajectories of the surrounding traffic participants, such as vehicles or pedestrians [4]. Classic object tracking algorithms are not suitable for mid-term motion prediction because they cannot consider the interaction of participants, though their robustness is essential for the proper input generation for these algorithms. The classification ability of multiple model (MM) systems can also enhance the efficiency of the prediction [5].
A maneuvering vehicle can be effectively tracked by using a multimodel state estimator [6]. The interacting multiple model (IMM) estimator is an approximate solution of the general multimodel problem [7, 8], which is computationally tractable with linear scaling in the number of the considered models. The IMM estimator was originally proposed with Kalman filters [9]; however, it can be realized by particle filters [10, 11] or fused with random set filtering methods. In [12, 13] IMM estimation, realized as a particle filter, was presented which is able to work in a cooperative road traffic situation. A random finite set based particle filtering in the IMM framework for cooperative road traffic application was presented in [14]. The choice of coordinate system for maneuvering target tracking affects the structure of the estimation algorithm and the achieved performance [15, 16]. Plenty of research deals with the problem that which object tracking, or in more general, probabilistic state estimation algorithm is suitable for these purposes. One approach is to examine to what extent the process and measurement noise affect the position estimates. The authors of [17] concluded that if the effect of the process noise to measurement noise ratio is above 0.5, the IMM offers better estimates. While the study in [18] states that the performance differences are highly dependent on the maneuvers of the target. This solution assumed a so-called perfect IMM (PIMM), which is a lower bound error estimation, since it never fails to choose the appropriate model. Naturally the PIMM is an ideal model and was generated artificially. Using nonideal IMM, our study examines the effects of the hyperparameter tuning of the IMM, in a road vehicle-like environment, where the model for the sensor imitates the capabilities and performance of radar sensing.
The paper is organized as follows. Section 2 introduces the probabilistic state estimation problem, and the possible solutions, such as Kalman filter, Extended Kalman Filter, Particle Filtering, and IMM methods. Section 3 gives an overview on the scenario and models used for the evaluation of the algorithms, and, finally, Section 4 evaluates the performance of the above-mentioned methods.
2. Problem Statement
In probabilistic estimation the state is considered as a random variable and its distribution is approximated. The approximation can be performed with various precision; however, the underlying mathematical structure, the Bayes theorem, can give a unified description of the different methods. The general problem is to estimate the state given measurements and the model of the considered dynamic system. The model incorporates the time evolution of the system, which is the motion model and the measurement model, which accounts for how we get information from the system through sensors. Formally the estimation at timestep involves the evaluation of the formulawhere the conditional probability is the posterior density of the state in question. The likelihood can be constructed in the knowledge of the measurement model. The denominator equals the marginal Probability density function (PDF) which is called the evidence and serves as a normalizing factor. The prior term can be expressed with the Chapman-Kolmogorov equationwhere is the state transition density constructed from the motion model. The estimation can be carried out in a recursive manner; however, there are no general analytic solution because of the involved integrals. In case of a linear system model and additive Gaussian noise (1) can be shown to reduce to the Kalman filter [19].
Beside the fact that the Kalman filter is the optimal minimum mean squared error filter for linear-Gaussian systems, its usage is limited by the presuppositions of the model. Most real-life systems show nonlinearities and the Gaussian presumption cannot always be held. An approximate method is the Extended Kalman Filter (EKF) which, to handle nonlinearities, linearizes the system equations by first order Taylor expansion around the current state.
A general discrete time state space system model with additive noise has the formwhere is the state transition function, is the measurement function, and and are noise vectors. If the nonlinearities in and are not too strong, the linearization approximates the functions well and the EKF gives good results; convergence however cannot be guaranteed. The structure of the EKF is shown in Algorithm 1.
| Initialize variables: , | |
| for to do | |
| Predict | |
| Compute Jacobian | |
| Predict state | |
| Predict covariance | |
| Update | |
| Compute Jacobian | |
| Residual | |
| Residual covariance | |
| Kalman gain | |
| Update state | |
| Update covariance | |
| end for |
A more general approach to handle nonlinearities or even non-Gaussian PDFs is to use the particle filter (PF), which is an umbrella term. It corresponds to a family of Monte Carlo sampling based sequential algorithms. It is a numerical method that approximates a function, in our context the posterior PDF by particles. The particles are weighted samples, drawn from the distribution; hence, the higher the particle number, the more accurate the approximation. The PDF of a state vector is approximated by particles aswhere is the -th sample, is its weight, and stands for the Dirac measure centred at .
The particle filter can be considered a Bayesian-type filter where the PDFs are numerically handled. At the predict stage samples are drawn from a pool, called the importance density. The choice of the importance density is crucial. An obvious choice is to use the state transitional density whose method is referred to as the bootstrap particle filter. The update stage involves the computation of the likelihoods for every particle which is realized by the evaluation of the PDF associated with the measurement model. The normed likelihood values give the updated particle weights and the point estimate for the current timestep can be obtained by the weighted sum of the particle states. The main steps of the bootstrap particle filter are summarized in Algorithm 2.
| Initialize particles: | |
| for to do | |
| Predict | |
| Draw particles from transition density | |
| Update | |
| Compute particle likelihoods | |
| Output point estimate as weighted sum | |
| Compute normalized weights | |
| Draw from with probability Resample particles | |
| end for |
2.1. Multimodel Estimation
The purpose of multimodel filtering is twofold. On one hand it helps giving a more precise state estimation if the correct system model is used and on the other hand it provides information on the actual mode of operation. To be effective at both at the same time, as will be seen, can be a contradictory requirement.
System model is the collective term for the process or motion model and the measurement or sensor model. The mode of operation or system mode refers to a certain kind of behavior that we identify, e.g., accelerating or turning. A given process model, if general enough, can account for multiple modes of operation. Mode change is the switching between two modes of operation. The mode history is the time sequence of the actual system modes. Mode uncertainty refers to the circumstance that we are not aware of the actual system mode.
One way of dealing with model uncertainties in an estimation problem is to use a number of plausible system model, compare their performances, and choose one result or combine several. The output of a filter associated with a certain mode is referred as the mode-conditioned estimate. There are numerous multiple model algorithms at hand [6]. The static multiple model approach has the supposition that the system does not change its behavior during the observation period and the filter selects the most likely mode and outputs a weighted estimate that is a combination of the individual filters. The dynamic multiple model estimator considers mode switching and uses mode transition probabilities that are predefined parameters. The exact solution to the dynamic multiple model problem, the Generalized Pseudo-Bayesian (GPB) estimator, has complexity that exponentially grows with time because it covers all the possible combinations of mode histories with a filter. Since the exact solution is intractable, one has to approximate and take into account only the last or the last two modes of operation resulting in the first (GPB1) and second order (GPB2) estimators. If mode of operation is considered, the GPB1 runs filters while the GPB2 runs filters, because of deeper memory of the algorithm. Therefore the computational requirement of the GPB1 and GPB2 filters scales are linearly and quadratically respectively in the number of modes.
2.1.1. The Interacting Multiple Model estimator
The interacting multiple model method, proposed in [7], is another type of approximate estimator to the dynamic multiple model problem. It uses only the output of the last step and creates a unique mix from the mode-conditioned estimates for every filter. Being a first order approximation it has a performance near the GPB2 but computationally only intense as the GPB1 [9].
The structure of the IMM estimator is depicted in Figure 1. The inputs to the recursive algorithm at timestep are the mode-conditioned state estimates and the mixing weights as a matrix for . The state estimates consist of a mean value and a covariance matrix, describing a Gaussian distribution. At the mixing stage the input for each filter is computed as a weighted sum of Gaussians:These values and the measurement vector are passed to the filters. Beside the state estimation the filters also produce the residual and the associated covariance matrices. From these a model likelihood is computed asThe mode probabilities are not purely the likelihoods because mode switching dynamics are implemented with the help of the mode transition probability matrix through which the prior values are derived:The updated mode probabilities areAn overall estimate can be computed by weighting each filter output by the mode probabilities:The associated covariance matrix isThe mixing coefficients for the next step in the recursion is given by
2.2. IMM Particle Filter
Originally the IMM algorithm was working with Kalman filters [7]. The realization of the IMM with extended Kalman filters is trivial, because the output of an EKF is consistent with the KF. The same does not hold for the particle filter since its output consists of weighted samples and cannot be directly integrated into the IMM framework. One method is to distribute the particles between the mode-conditioned estimators. The more likely a mode estimate is, the more particles will belong to that estimator. We used another approach, which is closer to the original structure of the algorithm [20, 21].
Every filter has a unique Gaussian input parametrized by . The PF draws samples from this distributionThe weights associated with the samples have equal values: . The particles are then propagated through the motion model:
The likelihood of a particle is
A point estimate is computed from the particles by a weighted sum:with covariance
The particle filter does not produce residual covariance like the EKF. To be integrated in the IMM framework, a model likelihood has to be derived. The estimated measurement isThe model likelihood has an analogous form as in the EKFHere, isWith the mode probability is computed as in (10).
If particle degeneration is an issue, one will perform some form of resampling after the update step [22]. In our case, as we sample from the mixed distribution at every timestep, no particle degeneration is possible; hence, we implemented the PF without resampling. The individual particles are not preserved; only the weighted point estimate, its covariance, and the model likelihood are utilized.
3. Evaluation Framework
To evaluate the filter performances a simulated environment is used. The tracked object is assumed to be a road vehicle with appropriate motion characteristics. The maneuvering vehicle is observed from a stationary sensor at the origin. The measurements are generated according to standard automotive radar models.
3.1. Models
In this study the considered modes of operation are moving with constant velocity and turning with constant speed along a circle or a clothoid segment.
The CV and CT model both use the state vector and the motion model has the form:where either stands for the CV or the CT model. With the given the state vector the CV model iswhich after linearization reads asThe constant turn rate (CT) model is angular velocity dependent:Linearizing (25) around yieldswhere
It would also be possible to use a simple CV motion model with states , system matrixand perform the mixing in the IMM algorithm with state vectors of different length and components. In such case, depending on the actual components of the state vectors, one has to either augment one state vector with the missing element or transform the state vector to the base of the other one. These kinds of computations, detailed in [23, 24], are out of scope of the current study. For this reason the same state vector was used for both the CV and CT motion.
The noise acting on the system is modeled as an additive Gaussian term. For the CV and CT model it is a white noise linear acceleration and white noise linear and angular acceleration, respectively.whereand is a zero mean Gaussian with covarianceIn (36) the diagonal elements stand for the linear and angular acceleration. The factor scales the noise intensity. The PDF associated with the motion model has the form
The measurement vector comes from simulated radar observations with components , where is the bearing angle and is the distance. The measurements originate from the nonlinear sensor model :In linearized form the sensor model reads aswhere .
With the help of (38) the likelihood function is given bywhere the covariance matrix is
3.2. System Setup
As it was pointed out in [25], narrow beam range finders like long range radar and especially lidar have a possibility of failing to detect thin objects, for example, lampposts. An additional problem is that even if the sensor correctly detects an obstacle, a filtering algorithm might not recognize its existence. Sensors that are practically noise free lead to a measurement likelihood function that is spike-like. The support of a likelihood function of this kind, due to limited numerical precision, can be a very small region in the state space. If the estimator algorithm does not provide a prior in that small region, then the likelihood function will be zero out of other priors. At the filtering level the narrow likelihood problem can be handled by applying a Gaussian smoother. Low level sensor fusion techniques and track-before-detect approaches can help to overcome typical sensor drawbacks [26]. At the detector level, new, emerging technologies are expected to overcome this deficiency [27–29].
The vehicle is initialized with state vector . The noise acting on the system is described by . Sensor noise values are chosen according to common automotive radar devices [30].
3.3. Scenario
The vehicle moves along a road segment and its trajectory is segmented into straight lines and circular or clothoid arcs. Two trajectories were designed with different characteristics to help emphasize the nature of the filters. Trajectory1 (T1), as outlined in Table 1, has more turns and Trajectory2 (T2) has more straight parts, as indicated by Table 2. In the clothoid segment in both cases the angular velocity ( and ) changes linearly with time. For T1 starts from zero and increases to 16 deg /s in 20 steps and then decreases to −12 deg /s in 20 steps. The same is true for for T2 except that 10 – 10 steps are used in this case.
The speed is constant throughout the whole simulation. The duration of the simulation is with time step . The steps for and are also .
The filter performances were examined along two parameters, the mode transition probability matrix and the process noise intensity . The performance is also measured along two dimensions: the position error and the estimated mode.
The mode transition matrix is adopted aswhere is the scaling parameter. The values taken by and the process noise intensity are shown in Figure 2.
4. Results
The performance of the filters was measured from two aspects of interest. First, the position error is calculated as the Euclidean distance between the real and the estimated coordinates. Second, as the measure of maneuver detection, the estimated mode is regarded and two measures are defined: from which the first is the ratio of time steps when the correct mode was estimated versus the total time; and the second method is area based and indicates the robustness of the output. The area under the estimated mode probability curve is divided by the length of time that mode was active.
A ratio to measure the effect of relative motion to measurement uncertainty on the position is defined in [17] as the maneuvering index. In our case it can be quantified aswhich takes values between 0.2 and 0.5. According to [17] an IMM is preferred to a single model KF if is higher than 0.5. This limit is challenged in [18] with the conclusion that the IMM can outperform the KF if the model with the lower process noise is active for enough time.
Numerical complexity of the compared algorithms has the following order. The single model EKF has the lowest requirements. The state space is of low dimension, the involved matrix operations can be computed directly, without sophisticated numerical methods. The IMM-EKF needs to run a filter for every considered mode of operation. In addition, the IMM algorithm poses overhead mainly in terms of creating Gaussian mixtures and computing covariance matrices. The IMM-PF is the most computationally expensive. The most demanding steps are sampling, evaluation of the likelihood function, and sample covariance computation. The particle number was held constant 1000 which can be considered as a mediocre quantity. The current implementation used parallelization wherever it was possible at the particle level.
Simulations were conducted for both trajectories. The IMM is implemented with EKF and PF and for comparison single model CV/CT EKF were used. 100 Monte Carlo runs were averaged out for every parameter setting meaning 75000 runs in total. Figures 3, 7, 11, and 13 show IMM filter performance across the whole parameter range. Figures 4, 8, 12, and 14 sum up the efficiency of mode estimation and RMSE for parameter and for every .
Figures 5 and 9 show the results of the 100 MC averaged runs for parameters and . The estimation of one particular run from 100 is presented in Figures 6 and 10.
The estimated mode at each timestep is acquired by selecting the most probable mode:
A general property of the IMM filters in all cases is that the position RMSE is lower for greater values while the dependence on shows variety. The effect of a smaller value is that the diagonal elements in become large and mode transitions are harder to achieve; resulting in a slower multimodel filter and a lower ratio in terms of mode-correctness. In contrary with a higher value mode transitions can be realized with low latency after measurements start to deviate from the expected state. The downside of using values near 0.5 is that the robustness of the mode estimation in terms of the area based measure degrades as the correct mode gets lower confidence. This effect is demonstrated in Figures 4, 8, 12, and 14.
The position RMSE of IMM-EKF exhibits negligible dependence on for both T1 and T2. The accuracy of mode estimation on Figure 7 shows a connection of linear nature between the hyperparameters for T2. In case of T1 the best model estimation efficiency can be achieved with lower noise and higher parameters (Figure 3).
The single model EKF with constant turn rate can be considered as an option in certain situations. If the tracked vehicle performs a great amount of maneuvering, the EKF can outperform the IMM in terms of position RMSE at the cost of providing no mode estimates. The EKF, in scenario T1, estimates the state significantly better than the IMM; on the contrary in T2 the IMM performs comparably while the EKF is worse. The particle filter implementations of the IMM have different characteristics. The position RMSE shows dependence on both hyperparameters. The mode of operation can be estimated with greater efficiency for certain parameters compared to the IMM-EKF. In scenario T1 the dependence of the RMSE on is weak for lower values indicating a greater freedom at the filter design process. The mode can be estimated slightly more efficiently with the IMM-PF than the IMM-EKF in both scenario T1 and T2.
5. Conclusion
The continuously increasing level of automation in road traffic and vehicles poses new requirements towards data processing algorithms [31]. Road vehicles with autonomous functions need the ability to sense the environment and faithfully capture and interpret the traffic situation. Algorithms for advanced driver assistance systems have to run realtime which greatly constrains the level of analysis of acquired data. This study covers performance evaluation of multimodel object tracking and maneuver identification methods of great importance for the automotive industry. The scenario was provided by a simulated environment where the sensor model had the characteristics of common automotive radar which performed measurements on the bearing angle and distance of the tracked object. The implemented estimators are the IMM-EKF, IMM-PF, and the single model EKF. The filters were evaluated along two aspects, the RMSE of the estimated position and the performance of the mode estimates. The varying parameters of the simulation were the mode transition probability matrix and the intensity of the process noise. The robustness of estimates of the actual mode of operation was measured by two methods. The single and multimodel filters were compared in terms of position RMSE. The IMM-KF and IMM-PF have different scaling properties regarding the simulation parameters which imply different design approaches.
It is essential to fine tune or, during operation, reconfigure the filters to help adaption to the actual conditions. The presented work can serve as a guideline creating a customized framework to analyze and help the development of multiple-model state and maneuver estimating algorithms for advanced vehicular functions. In the knowledge of the scaling properties and behavior of the algorithms parameter ranges for the optimal performance can be identified.
List of Notations
| : | Dimension of state space |
| : | Dimension of measurement space |
| : | PDF of state vector |
| : | State vector at time |
| : | Process noise vector |
| : | Measurement vector at time |
| : | Measurement noise vector |
| : | State transition density |
| : | Likelihood function of |
| : | State transition matrix |
| : | Measurement matrix |
| : | Nonlinear state transition function |
| : | Nonlinear measurement function |
| : | Estimated state conditioned on mode |
| : | Estimated covariance conditioned on mode |
| : | Likelihood of mode |
| : | Residual covariance of mode |
| : | Mode transition probability matrix |
| : | Intensity of process noise |
| : | Process noise covariance |
| : | Measurement noise covariance |
| : | Process noise input matrix |
| : | Probability of mode |
| : | Mixing matrix. |
Data Availability
No data were used to support this study. The study used MATLAB simulation to evaluate the methods; the source can be requested from the corresponding author.
Conflicts of Interest
The authors declare that there are no conflicts of interest regarding the publication of this paper.
Acknowledgments
The research reported in this paper was supported by the Higher Education Excellence Program of the Ministry of Human Capacities in the frame of Artificial Intelligence research area of Budapest University of Technology and Economics (BME FIKPMI/FM). The project has also been supported by the European Union, cofinanced by the European Social Fund (EFOP-3.6.3-VEKOP-16-2017-00001: talent management in autonomous vehicle control technologies).