MPP: A Novel Algorithm for Estimating Vehicle Space Headways from a Single Image

Yu, Zhi; Jiang, Qianyin; Li, Xiying

doi:https://doi.org/10.1155/2020/5715686

Journal of Advanced Transportation

On this page

Abstract Introduction Conclusion Data Availability Conflicts of Interest Acknowledgments References Copyright Related Articles

Research Article | Open Access

Volume 2020 | Article ID 5715686 | https://doi.org/10.1155/2020/5715686

MPP: A Novel Algorithm for Estimating Vehicle Space Headways from a Single Image

Zhi Yu,^1,2,3Qianyin Jiang,^1,2,3and Xiying Li^1,2,3

Academic Editor: Juan C. Cano

Received15 Aug 2019

Accepted05 Dec 2019

Published01 Feb 2020

Abstract

Vehicle space headway, also called spacing, is an important and basic traffic parameter. Traditional space headway calculation methods are facing the problems of large errors and high costs. This paper presents a novel algorithm based on measurement point pairs (MPPs) to estimate the real-time microcosmic vehicle space headway from single images in existing traffic surveillance videos and images without any additional equipment. First, the camera is calibrated with road markings to obtain the relationship between the image coordinates and the world coordinates. Second, vehicle pairs of two successive vehicles in the image are established, measurement points on each vehicle are selected by video intelligence analysis technologies, and their world coordinates are calculated by camera calibration results. Finally, the measurement points of the preceding and following vehicles are matched to obtain the MPPs, followed by the calculation of the weighted space headway. By using the measurement point information, one of the most difficult problems in image distance measurement, the lack of height information, is solved. The main factors causing estimation errors are fully addressed and the range and trend of errors under certain conditions are given by virtual simulation. Two real-world experiments are used to prove the accuracy and usability of the MPP in common video scenes: the simulation experiment indicates that the MPP algorithm achieves a high accuracy with estimation error less than ±0.1 m and the relative error within 1.1%; the application experiment shows that the MPP-based calculation is more accurate and stable than the state-of-the-art distance measurement algorithm and that the convenience of the proposed MPP algorithm is higher than that of traditional methods of space headway estimation.

1. Introduction

Vehicle space headway, also called spacing, refers to the distance between successive vehicles in a traffic stream measured from the same point on each vehicle (e.g., front bumper and front axle) [1]. Vehicle space headway is an important and basic traffic parameter and is the key value in describing traffic states, driving behavior and road traffic safety. Therefore, the estimation of space headways is an area of much research in the traffic field because it is necessary to obtain continuous space headway values in a real traffic environment. Because the space headway changes quickly, obtaining a correct and real-time space headway remains a significant problem in current traffic information-collection systems. At present, the space headway estimation algorithms can be divided into three categories.

The first category is time headway-based methods. Generally, it is difficult to estimate the dynamic space headway directly in real-time traffic scenes. The traditional method is to use an indirect estimation method based on the vehicle time headway (TH) [2, 3]. The time headway is the time between successive vehicles as the same point on each vehicle passes a point on a lane or roadway [1]. The time headway is typically acquired by human observation, detectors [3, 4], or video detection [5–7]. The space headway is obtained from the product of the time headway and the instantaneous speed of the vehicle. The time-headway-based method is easily applied; however, errors always exist due to continuous changes in vehicle speeds and frequent lane-changing behaviors. The greater the space headway, the larger the possibility of larger errors. Furthermore, there is a significant amount of work in human observation and high costs in detector installation and maintenance. When video detection methods are used, cameras generally need to be installed at a high height directly above the lanes to avoid intervehicle occlusion.

The second category is sensor-based methods. With the rapid development of information-collection technologies, sensors are increasingly used to measure the space headway on roads. Such sensors mainly include location devices (e.g., GPS [8, 9]) and distance-measuring devices (e.g., LiDAR [10, 11], infrared [12], and radar [13, 14]). The basic principles of these devices are the same: the information on two successive vehicles is collected by the sensors, and the physical sizes of the vehicles and the installation locations of the sensors are combined to obtain the real-time space headway. Due to the low installation density of vehicular sensors and the transient nature of the following relationships between two vehicles in a real traffic environment, sensor-based methods are commonly used only in experimental situations.

The third category is video-based methods. In recent years, video-based space headway estimation methods have become applicable as video surveillance products have become more widely deployed. In ordinary road surveillance videos, Houchin et al. manually labeled the positions of vehicles in video frames when the vehicles stopped and measured the distances between vehicles by using distance-measuring software [3]. Yu and Shi calculated the acceleration, speed, space headway, and other parameters of successive vehicles using a frame-difference method [15]. Lai et al. used image linear transformations to calculate the distances between vehicles and vehicle speeds [16]. Schakel et al. used cameras, pylons, tape measures, and other devices to divide the vehicle distances into eight categories [17]. Dubska et al. [18, 19] estimated the distances on the roads, the length of vehicles, and the speeds of vehicles by vanishing points of the cameras and 3D bounding boxes of the vehicles. In general, the principle of video-based methods is to transform the vehicle distances in images into actual distances. Because imaging is a process of mapping the vehicle perspective from 3D space to 2D space, simple transformations will inevitably lead to calculation errors. Additionally, some researchers estimated the space headway with high-altitude video (commonly captured by unmanned aerial vehicles or high-altitude cameras with approximate aerial views to acquire scenes with a wide range of perspectives). For example, Zheng directly determined the difference between two vehicles’ detected positions [20], which causes large errors because the sizes of the vehicles in the images are too small. The space headway can also be approximately estimated from the reciprocal of the traffic density, which is calculated by dividing the number of vehicles in the video by the lane length or road area [21, 22]. However, this method yields only the macro average space headway and not the precise space headway of each vehicle.

Based on the above, the space headway estimation methods remain limited by difficult measurements, large errors, and high costs. Given the wide installation of surveillance cameras on the main roads of cities and the advanced state of video intelligence analysis technologies that can extract traffic parameters (e.g., traffic flow [23], vehicle speed [18, 24, 25], and vehicle appearance features [26, 27]), traffic states [28], and traffic incidents [29], estimating the space headway from videos may be feasible. In this paper, using existing traffic surveillance videos with no additional equipment or funds, we propose a novel measurement point pair- (MPP-) based algorithm to acquire the real-time space headway between two individual vehicles. Simulations are conducted to analyze the influences of different types of noise and give the range and trend of the calculation error under given conditions. Real-world experiments are performed to evaluate the accuracy and practicability of the presented method in actual traffic scenes and surveillance videos.

The remainder of this paper is organized as follows. Section 2 presents the MPP algorithm. Section 3 describes the simulation and real-world experiments for evaluating the proposed method. Conclusions and future work are presented in Section 4.

2. Measurement Point Pair-Based Algorithm

Figure 1 presents a flowchart of the MPP algorithm. Two stages are included: the off-line stage and the on-line stage. In the off-line stage, camera calibration is the key step in calculating the camera installation parameters and obtaining the relationship between the world coordinates and image coordinates by using markings on the road. The on-line stage comprises three steps. In the first step, vehicle pairs are established. This step autolocates every vehicle in the image and defines every two successive vehicles as a vehicle pair. The second step is measurement point selection. For each vehicle pair, the measurement points are located, and the world coordinates of the points on the preceding and following vehicles are calculated. In the last step, the space headway of each vehicle pair is estimated. First, the measurement point information from the measurement point database is used to calculate the tip position of the vehicle. Second, MPPs are created by choosing different measurement points from the preceding and following vehicles, and the tip distance of each MPP is estimated. Finally, each tip distance is given a weight to calculate the space headway.

2.1. Camera Calibration

The surveillance video-based space-headway-estimating algorithm requires camera parameters. In practice, it is not possible to know all or part of the installation parameters of a traffic surveillance camera. Most traffic surveillance cameras and image-capture devices, such as vehicle image-capture systems and electronic police systems, are fixed, and their installation parameters will not change. Therefore, it is feasible to calibrate the camera manually in advance and to estimate the space headway automatically. Once the camera parameters are calibrated, the space headway can be calculated continuously as long as the position and pose of the camera are unchanged or automatically return to their initial settings after a change or if the changed parameters can be obtained through a cloud platform.

To better record the road situation, most traffic surveillance cameras view straight down the road or at a very small angle to the direction of the road. Kanhere and Birchfield experimentally showed that in this situation [30], camera calibration based on VWL (i.e., a vanishing point in the image and the width and length of a rectangle marked on the road) as in the approach described by He and Yung [31] can achieve a high calibration accuracy and better antinoise performance than other methods. Therefore, method of He and Yung [31] is used in this paper. In this model, xoy is the image coordinate system, and XOY-Z is the world coordinate system.

In practical application, the road surface is defined as the XOY plane. Assuming that the road surface is planar, vehicles travel parallel to the X-axis in the positive X-axis direction, and the tip surface of a vehicle is a vertical plane parallel to the YOZ plane. We also assume that there exists a rectangular marking on the road surface. As shown in Figure 2, rectangle ACDB is the calibration pattern in world coordinates. Edges AB and CD are parallel to the X-axis, and edges AC and BD are parallel to the Y-axis in world coordinates. The markings may be lane markings, a rectangular object of known size, and a small height on the ground, or some other available rectangular patterns. If there is no marking on the road for a camera, we can place a rectangular pattern in a safe area within the video shooting range temporarily before the estimating process and remove it after the calibration. Points a, b, c, and d correspond to points A, B, C, and D in the image coordinates. Variables L and are defined as the length and width, respectively, of rectangle ACDB in world coordinates. Using L, , and the positions of points a, b, c, and d in the image coordinates, we can use He and Yung method to calibrate the camera and obtain the transformation of any point between the image coordinate system and the world coordinate system [31].

2.2. Vehicle Pair Establishment

After the camera calibration, we need to locate in the images every vehicle that needs to be measured to obtain a rough bounding box of the vehicle (for videos, an image is one frame, which may be the frame in which the vehicle arrives at the detection line or a frame extracted by a fixed step). A vehicle autolocalization algorithm is necessary for vehicle measurement point autoestimation. Vehicle localization algorithms are relatively mature, and based on the characteristics of traffic surveillance videos, Sochor et al.’s methods [32] are recommended for vehicle autolocalization in videos and images. After the vehicle localization, two successive vehicles are grouped into a vehicle pair, and the preceding and following vehicles are marked according to their motion direction.

2.3. Measurement Point Selection

The space headway estimation is essentially a problem of image distance measurement. One of the most difficult problems in image distance measurement is the lack of height information of objects in the images. Considering that the vehicle is a rigid body and the structures of the vehicles are roughly the same (consisting of the tip face, windows, body, roof, tail, etc.), we raise the concept of the measurement point to provide auto-obtainable height information for the space headway estimation. A measurement point is defined as a point on the vehicle with a known height, a known distance from the tip of vehicle, and significant vehicle appearance features and semantic attributes. A measurement point database is built off-line before the space headway estimation. The measurement point information that may be used on different types of vehicles is recorded in the measurement point database, including the type of measurement point (e.g., the edges of the license plate, the upper or lower front edges of the vehicle, or the upper or lower edges of the windscreen), the height of the measurement point above the ground, termed h_M, and the distance from the measurement point to the tip of the vehicle on the X-axis, termed d_M. Figure 3 illustrates some of the recommended and commonly used measurement points, and Figure 4 presents the definitions of these variables. If we use point M to present a measurement point in world coordinates, the coordinates of point M are (X_M, Y_M, Z_M), and h_M equals Z_M.

The types of measurement points used can be included or dropped according to the actual situation, such as different camera viewpoints or different image resolutions. Regardless, the following conditions must be satisfied for measurement point establishment:(1)The measurement point variables h_M and d_M must be known.(2)The measurement point must be automatically located in the images by algorithms. For example, measurement points on the edges of license plates can be obtained by methods of license plate localization [33], measurement points on the edges of a windscreen can be obtained by methods of vehicle segmentation [26], and measurement points on the edges of a vehicle can be obtained by methods of vehicle localization [32, 34].(3)The heights of the measurement points are the same for vehicles of the same type.

After the measurement point localization, the image coordinates of the measurement point are obtained. Then, we need to recognize the type, model, and other appearance features of this vehicle; we recommend Li et al.’s method [26] for this purpose. The measurement point information can then be searched in the measurement point database based on the vehicle appearance features, and the height of the measurement point is obtained. Next, He and Yung method [31] is used to calculate the world coordinates of the measurement point using this information. In practical applications, because of the previous development and higher accuracy of license plate localization algorithms (a localization accuracy of 99% is common [33]) compared to other vehicle localization or segmentation algorithms, it is recommended to use the points on the edges of license plates as measurement points for space headway estimation.

2.4. Space Headway Estimation

As shown in Figure 4, variable X_t refers to the coordinate of the tip of a vehicle in world coordinates on the X-axis. For measurement point M on a vehicle, the tip position of the vehicle in world coordinates on the X-axis can be calculated by the following equation:

Errors will inevitably occur in license plate localization (whether autolocated by algorithms or located manually); using multiple measurement points to estimate the space headway can reduce such errors to a certain extent. Therefore, we capture multiple types of measurement points on the two vehicles in a vehicle pair. Here, variable M_pi refers to the ith measurement point on the preceding vehicle, and variable M_fj refers to the jth measurement point on the following vehicle. Using equation (1), the tip position is calculated for each vehicle in a vehicle pair. Variable X_t-pi refers to the tip position of the vehicle in world coordinates on the X-axis calculated by point M_pi, whereas variable X_t-fj indicates the tip position of the vehicle in world coordinates on the X-axis calculated by point M_fj.

Variable MPP_pi-fj refers to an MPP consisting of measurement point M_pi on the preceding vehicle and measurement point M_fj on the following vehicle. According to our assumption described in of Section 2.1, the tip distance of a vehicle pair is the difference on the X-axis of two tip positions of vehicles in world coordinates. Therefore, the tip distance corresponding to MPP_pi-fj is calculated by the following equation:

Therefore, the space headway of the vehicle pair is calculated as follows:where α_pi-fj is the weight of MPP_pi-fj, , and variables m and n are the numbers of measurement points on the preceding and following vehicles, respectively. In practical applications, the chosen measurement points should cover the entire range in the horizontal direction of the segment or the object where the measurement point is located, and the weight α_pi-fj is determined by the confidence of the measurement point autolocalization algorithm.

3. Simulation and Experiment

Three cases are described in this section. First, a virtual simulated environment is designed to study the sensitivity of the proposed method to measurement errors in detail. Second, a real-world simulation experiment is performed to prove the accuracy of the proposed method. Finally, an application experiment is used to assess the applicability of the proposed method in actual surveillance scenes.

3.1. Virtual Simulation

The proposed method is ideal in theory, but errors are caused by many factors in practical applications. Some objective factors include uneven road surfaces and vehicles not driving strictly along the road direction. These errors cannot be corrected after imaging. Subjective factors (or factors generated in the calculation process) include locating errors for the calibration patterns, measurement errors, and locating errors in license plate localization, all of which can be reduced to a minimum by a variety of means. To clearly grasp the effects of these factors on the final space headway estimation results, we constructed a virtual ideal scene to analyze the errors that they cause.

The scene parameters we set are as follows. The world coordinates are given in meters, and the world coordinates of the endpoints of the calibration pattern are (−8, −2, 0), (6, −2, 0), (−8, 2, 0), and (6, 2, 0); that is, the calibration parameter L is 14 m, and is 4 m. The image resolution is 1600 × 1200 pixels. The camera parameters [31] are height of camera h = 8 m, tilt angle t = −10°, rotation angle s = 2°, pan angle p = 87°, and focal length f = 6000 pixels, and the camera viewpoint is the reverse direction of vehicle traveling; that is, the vehicles are traveling toward the camera. There is only one MPP in the simulation. The world coordinates of the measurement point on the preceding vehicle are (4, 0, 0.3), and the distance from the measurement point to the tip of the vehicle on the X-axis is 0.5 m. The world coordinates of the measurement point on the following vehicle are (−2, 0, 0.3), and the distance from the measurement point to the tip of vehicle on the X-axis is 0 m. Hence, the space headway of this vehicle pair is 6.5 m.

Next, we added offsets to different variables to simulate measurement errors to observe the effects on the space headway estimation results. Here, the error of the space headway is defined as the difference between the estimated space headway and the actual space headway. The error is positive when the estimated space headway is larger than the actual space headway and negative otherwise.

3.1.1. Errors in Localizing Calibration Patterns

The calibration pattern position is one of the main factors affecting the calibration results. Here, the localization errors of the calibration pattern endpoints are used as an example to analyze the effects of calibration pattern location errors on the space headway estimation. In this section, points A, B, C, and D are the endpoints of the calibration rectangle in world coordinates, as shown in Figure 2. Figure 5 shows the corresponding endpoints a, b, c, and d in image coordinates. Variables Δx and Δy are used to describe the location offsets of a point on the x- and y-axes, respectively, in image coordinates. Figure 5 illustrates the case of point a, where point a′ refers to the actual location result of point a in image coordinates. In our simulation, we changed the Δx and Δy of each endpoint in turn over the range [−20, 20] pixels. When the locating error of one endpoint was examined, the localization results of the other three endpoints remained unchanged, as did the width and the height of the calibration pattern; that is, the calibration variable L was 14 m, and was 4 m. Finally, the proposed method is used to estimate the space headway; its error distribution is shown in Figure 6, and its error characteristics are shown in Table 1.

(a)

(b)

(c)

(d)

As shown in Figure 6, the error distribution characteristics between points a and c and between points b and d are approximately the same, and the variation of the error is roughly symmetrical along the vertical middle line of the calibration pattern. Moreover, the distributions of the stripes of points a and c are relatively horizontal, whereas those of points b and d are generally inclined along an angle. That is, the errors caused by point a or c largely change in the vertical direction, whereas the errors caused by point b change from the upper right to the lower left and upper left to lower right of point d. The width of the stripe of point a is narrower than that of point c, and the width of point b is narrower than that of point d. Thus, for the same offset in endpoint localization, the rate of change of the error of point a is larger than that of point c, and the rate of change of the error of point b is larger than that of point d.

Additionally, as Table 1 illustrates, the variances of the errors caused by points a and c are 0.0260 m² and 0.0184 m², which are somewhat greater than those of points b and d (0.0083 m² and 0.0051 m², respectively); additionally, the range of errors caused by points a and c is greater than that of points b and d. Thus, the offsets of points a and c are likely to cause greater errors than those of points b and d. In particular, as shown in Figure 6(a), a shift to the upper right corner of the localization result of point a results in a sharp increase in the calculation error. For points a and c, as the offset in the middle-upper direction of the calibration pattern becomes larger, the estimated space headway becomes smaller than the actual space headway; otherwise, the estimated space headway becomes larger than the actual space headway. For points b and d, as the offset becomes closer to the middle of the calibration pattern, the estimated space headway becomes larger than the actual space headway; otherwise, the estimated space headway becomes smaller than the actual space headway. In general, the sensitivity of the calculated localization results to the offset of the endpoints follows the order A > C > B > D.

This simulation leads us to the following conclusion when we are locating endpoints in an image. For more accurate results, the localization accuracy of point a should be as high as possible. Moreover, the localization accuracy must be as high as possible in the vertical direction for points a and c, from the upper right to lower left for point b and from the upper left to lower right for point d.

3.1.2. Errors in Measurement Point Localization

To estimate the space headway in videos and images automatically, a measurement point autolocalization method is generally adopted for an image. The errors in the measurement point localization will have a significant impact on the calculation. In this section, we consider the location errors between the actual localization result and the theoretical positions of the measurement point on the x- and y-axes in image coordinates. These errors are changed on the measurement points of preceding and following vehicles, with a range of change of [−20, 20] pixels. When a location error of a measurement point of a preceding (or following) vehicle is considered, the position of the measurement point on the following (or preceding) vehicle remains unchanged, and the heights of all measurement points remain the same. Finally, the proposed method is used to estimate the space headway; its error distribution is shown in Figure 7, and its error characteristics are shown in Table 2.

(a)

(b)

As shown in Figure 7, the error distribution is approximately horizontal, and the influences of the measurement point localization offsets are relatively even; that is, offsetting the measurement point to the top or bottom of the image has approximately the same effect on the space headway estimation in addition to the sign of the error. Due to the effects of the camera rotation angle and pan angle, the bands have specific angles with the horizontal direction. For the preceding vehicle, the offset of the measurement point position to the top of the image will result in a smaller value of the calculated space headway than the actual value, whereas a downward offset will result in a larger value of the calculated space headway. The situation is reversed for the following vehicle. This is consistent with our intuitive understanding: the preceding vehicle is below the following vehicle in the image, and when the measurement point on the preceding vehicle is shifted upwards in the image or when the measurement point on the following vehicle is shifted downwards in the image, the measured separation is smaller than the real separation between the two vehicles, leading to a smaller calculated result.

Table 2 illustrates that the variance of the error caused by a measurement point on a following vehicle is 1.7 times that of the preceding vehicle. The error caused by a measurement point on the following vehicle is within the range of approximately ±1 m, and that on the preceding vehicle is approximately ±0.8 m. Hence, a measurement point on the following vehicle has a greater effect. Figure 7 also shows that in the designed range, with the same offset, the error caused by the locating offset of the measurement point on the following vehicle is greater than that of the measurement point on the preceding vehicle. In the world coordinate environment, the following vehicle is farther from the camera than the preceding vehicle and thus has a lower resolution than the preceding vehicle in image coordinates; consequently, the actual length represented by one pixel on the following vehicle is longer than that for a pixel on the preceding vehicle. Therefore, in practice, the localization accuracy of a measurement point on a following vehicle needs to be higher.

3.1.3. Selection of Measurement Points

There are typically several measurement points on a vehicle. The measurement errors of the measurement points at different positions and heights will have different effects on the results. In this section, the same measurement offsets are added to measurement points in different positions on the vehicle in world coordinates to observe the space headway error distribution. In the simulation, the range of d_M is [0, 5] m, and the range of h_M is [0, 2] m. Section 2 concludes that for the same offset, the error caused by the locating offset of the measurement point on a following vehicle is greater than that of the measurement point on a preceding vehicle, and the offsets of the measurement point toward the top or bottom of the image have approximately equal effects on the space headway estimation. Therefore, in this section, the offsets of the measurement point on the following vehicle in image coordinates are set to −1 and −20 pixels. That means that the localization result shifts to the top of the image by 5 and 20 pixels, as well as toward the tail direction of the vehicle.

As shown in Figure 8, the error distribution is a stripe with a very small angle vertically; that is, the height of the measurement point has a relatively small effect on the result, while the distance from the measurement point to the tip of the vehicle (variable d_M) contributes more to the result. As the locating offset of the measurement point increases, the angle of the error strip increases, and the widths of the error strips change little. In other words, as the measurement point localization error increases, the influence of the measurement point height on the result increases, while the influence of the measurement point height on the result remains almost unchanged. In Table 3, when the offset increases from −5 to −20 pixels, the space headway results decrease on the whole, but the range of the errors is almost unchanged. We can observe from Figure 8 that as d_M increases, the calculated result is initially larger than the actual result but then gradually decreases and the “zero-error-line” appears. This is because the error components caused by the measurement point locating errors in h_M and d_M cancel each other out, which leads to a “seemingly correct” result. Then, as d_M is gradually increased, the calculation result becomes increasingly smaller than the actual result.

(a)

(b)

In summary, in practical applications, to ensure the accuracy of the measurement point localization, under the premise of ensuring the positioning accuracy, more points near the tip of the vehicle and more lower points should be selected as measurement points, or the weights of the measurement points near the tip of the vehicle and the lower measurement points should be increased to reduce the error.

3.1.4. Errors of the Measurement Point Height Measurement

A measurement point generally corresponds to a physical point on the vehicle body. Due to factors such as the vehicle manufacturing process and tire pressure, the heights of measurement points of the same type on different vehicles of the same model or even measurement points on the same vehicle at different times may not be identical. Additionally, there is a certain error in the height measurement of the measurement points in world coordinates. These factors will lead to errors in the final space headway calculation results. In this section, the variable Δh_M is used to describe the height measurement errors of a measurement point in world coordinates; the range of Δh_M is [−0.2, 0.2] m. When analyzing the height measurement errors of the measurement points, other variables, including the localization of the measurement points in image coordinates, remain unchanged. We then use the proposed method to estimate the space headway; its error distribution is shown in Figure 9, and the error characteristics are shown in Table 4.

(a)

(b)

As illustrated in Figure 9, there is a linear relationship between the measurement point height measurement error and the space headway estimation error. When considering a measurement point on the preceding vehicle, higher-than-actual measurement point height values result in larger-than-actual estimation values. The situation is reversed when measurement point height errors occur on the following vehicle. Figure 10 diagrams the cause of this behavior. Because the measurement point localization result in image coordinates is unchanged during the calculation process, when the height measurement of a measurement point is considered, the measurement point corresponding to the point M actually adopted in the calculation process is point M′, leading to an error of the measurement point locating result in world coordinates on the X-axis. This error is described as ΔX_M, referring to the distance between point M and point M′ on the X-axis in world coordinates. When Δh_M is positive, ΔX_M is positive; that is, the calculated vehicle position is further ahead, and vice versa.

We can also observe that in the same height-measurement-error range, the absolute value of the error caused by the height measurement offset of a measurement point on a following vehicle is larger than that on a preceding vehicle. For example, a height measurement error of −0.2 m on a measurement point causes an error with an absolute value of 1.0729 m when located on the preceding vehicle but an error of 1.2288 m when located on the following vehicle. This discrepancy also reflects the lower resolution of the following vehicle in an image compared to the preceding vehicle; the same height in world coordinates corresponds to fewer pixels on the following vehicle than on the preceding vehicle. However, the height measurement accuracy of a measurement point on the preceding vehicle cannot be neglected; in practice, the height measurements of the measurement points on both vehicles should be as accurate as possible.

3.2. Real-World Simulation Experiment

Considering that in actual surveillance scenes, it is extremely difficult to obtain the precise values of the actual space headway and vehicle measurement values, we design a real-world simulation experiment to simulate common traffic surveillance scenes to verify the accuracy of the proposed MPP method.

At present, the two main types of video road surveillance systems are Vehicle Image Capture (VIC) systems and Road Video Surveillance (RVS) systems. Examples of scenes from these two systems are shown in Figures 11(a) and 11(c). A VIC system is typically a camera that monitors a limited number of lanes (typically 1–3 lanes). When a vehicle passes, the vehicle detector is triggered and transmits a signal to the camera to capture an image. Therefore, the scene in the image is small, the resolution of the vehicle is high, and the parts and features of the vehicle can clearly be seen. The RVS system generally views a large scene and captures continuous images of a section of the road with a long range; the resolutions of the vehicles in the images are low.

(a)

(b)

(c)

(d)

In our experiment, as shown in Figures 11(b) and 11(d), the VIC and RVS scenes were simulated using a NIKON D7200 model camera (all areas of private data are masked). The resolution of these images was 2992 × 2000 pixels. For the VIC scene, the focal length was 35 mm, and the equivalent focal length was 157 mm. In the RVS scene, the focal length was 58 mm, and the equivalent focal length was 87 mm. We tried our best to ensure that the size and depth of the scenes and the ratio of the vehicle size to image in these two scenes are roughly the same as those in the actual surveillance VIC and RVS systems.

During the experiment, the calibration rectangle was demarcated using objects on the ground. The rectangle formed by the marker connection appears in Figure 12 and the image coordinates of them are obtained manually. Distances and heights were measured by a laser rangefinder (model: Suwei S40) or a tape measure, and multiple measurements were averaged to reduce the errors. The measurement results were as follows: the height of the camera h was 5.940 m, the calibration rectangle was 9.679 m × 3.933 m, and the vehicle distance (defined as the distance from the tail of the preceding vehicle to the tip of the following vehicle) was 4.514 m in the VIC scene and 8.855 m in the RVS scene. From the vehicle production information, the length of the preceding vehicle was 4.598 m. Therefore, the real space headway in the VIC scene was 9.112 m, and that in the RSV scene was 13.453 m.

(a)

(b)

As recommended in this paper, the points on the edges of the license plate were used as measurement points. A laser rangefinder was used to measure the heights of the measurement points, and the results were as follows: the heights of the upper and lower edges of the license plate on the preceding vehicle were 0.490 m and 0.331 m, and the corresponding heights on the following vehicle were 0.581 m and 0.440 m. These heights were located manually as well. Ten measurement points were captured on each vehicle, including five on the upper edge and five on the lower edge of each plate. For each scene, each measurement point on the preceding vehicle and every measurement point on the following vehicle form an MPP. Therefore, there are a total of 100 MPPs in each image. Because all measurement points adopted in this experiment were points on the edges of the license plates, their confidences should be identical, that is, α_ij = 0.01. Using equations (1)–(3), we calculated the weighted space headway for the scene; those of the VIC and RVS were 9.014 m and 13.455 m, respectively. The error statistics for this experiment are given in Table 5 and Figure 13. The relative error is the value of the absolute error divided by the actual space headway value.

(a)

(b)

As shown in Figure 13, the error distributions of both scenes exhibit bilateral symmetry similar to mountains with a peak in the middle. In addition, the width of the error range in the RVS scene is much wider than that in the VIC scene, indicating that the error distribution is relatively concentrated in the VIC scene and relatively disperse in the RVS scene. The error variances in Table 5 illustrate this situation as well. These results provide good confirmation of the conclusion of Section 3.1.2: the further the measurement point is from the camera, the greater the space headway error will be for the same measurement point error. Moreover, Table 5 shows that the accuracy in the RVS scene is higher than that in the VIC scene. We believe that this discrepancy does not contradict the conclusion above but arises because the results of the VIC scene overall are smaller than the actual value, as shown in Figure 13. The difference may be caused by the pattern endpoint localization error in the calibration process in the VIC scene.

Nevertheless, Table 5 demonstrates that the proposed method has a high calculation accuracy for space headway estimation in both the VIC and RVS scenes, with a mean absolute error not larger than 0.1 m and a mean relative error not larger than 1.1%. Thus, the calculated results are very close to the actual values. At present, the relative errors of the mainstream image-based vehicle speed estimation algorithms are approximately 1.90%–8.85% [18, 24, 25]. Even if the estimation of the time headway were completely precise, the error of the space headway calculated by the time headway-based method would not be smaller than the error of the vehicle speed estimation. Therefore, the accuracy of our proposed method is higher than that of the traditional method.

3.3. Real-World Application Experiment

The previous section has already proven that our MPP-based algorithm is accurate and has a certain practical application value. Now, we will verify the applicability of the MPP-based algorithm in real-world scenes.

Our experimental data were collected in the northbound direction on Gangwan Avenue in the city of Zhuhai, China. As shown in Figure 14, the cameras of the VIC and RVS systems are installed on the same pole above the road. The northbound direction of Gangwan Avenue consists of three lanes, and there are three VIC cameras and one RVS camera on the pole. The VIC systems are triggered by loop sensors on the road, and each VIC camera captures the vehicles on one lane, while the RVS camera records video of the three lanes in the whole northbound direction. There is a signal control intersection approximately 500 meters upstream of the camera position, and the green light phase is approximately 70 seconds. Therefore, we chose 70 seconds of daytime on August 13, 2016, and the following data during this period of time are collected:(1)The RVS video: the resolution is 1920 × 1080 pixels, and the frame rate is 25 fps.(2)The VIC images of the three lanes: the number of images is 63, and the resolution is 1600 × 1296 pixels.(3)The records of the VIC systems: the number of records is 63. They are corresponding to the VIC images and record the capture time of the vehicle image (the accuracy is 1 second), the speed of the captured vehicle (detected by the loop sensor, the accuracy is 1 km/h), the plate number of the captured vehicle (autorecognized by the VIC system), the lane number the vehicle is driving on, etc.

In the RVS video, we extract the frames when the vehicles reach the loops. Considering that the plate number cannot clearly be seen in the RVS video, we compare the vehicle subimage, vehicle color, vehicle type, and vehicle appearance sequence of the extracted frames with the VIC images and the VIC records manually to identify every vehicle in the extracted frames.

After doing so, at the specific moment when one vehicle reaches the loop, the vehicle is presented by two images: one is the VIC captured image, and the other is the corresponding RVS extracted frame. We call this vehicle the “captured vehicle” (also referring to the preceding vehicle in the vehicle pair), the two images a “captured vehicle image pair,” this moment the “trigger moment,” and this extracted frame the “key frame.” Figure 15 shows a captured vehicle image pair in this case. It should be noted that the following captured vehicle image pairs will be excluded:(1)When the captured vehicle can be seen in the key frames, but the vehicle following it cannot be seen in the key frames(2)When in one captured vehicle image pair, the vehicle following the captured vehicle in the key frame and the vehicle in the VIC image are not the same vehicle.

After filtering, 55 valid captured vehicle image pairs remain. As shown in Figure 16, we use road markings in one of the key images as the calibration rectangle pattern. By consulting the national standard and performing field measurement using the tape measure, we can obtain that the size of the rectangle is 15 m × 3.8 m. In the key frame, we manually select points at the middle of the lower edge of the license plates on the captured vehicles and the following vehicles as the measurement points. Considering that the main types of vehicles on the road during this period are cars and pickup trucks, we set the height of the measurement points to 0.44 m by measuring large numbers of vehicles of the same two types. Then, we use the proposed MPP algorithm to estimate the space headway in each key frame.

To evaluate the performance of our algorithm, we calculated the ground truths and compare our results with the state-of-the-art distance measurement method.

(1)The calculation of ground truth (GT): In this experiment, it is hard to achieve the ground truth of the space headway. Therefore, an indirect calculation method is adopted. Variable TH is defined as the time headway between the captured vehicle and the following vehicle and is calculated by the difference in the capture time of these two vehicles according to the records of the VIC systems. Variable refers to the speed of the following vehicle recorded in the VIC system. We assume that the following vehicles are moving at a uniform speed during the short period from the trigger moment to the moment when it arrives at the loop and that the sensitivity and accuracy of the loop sensor and speed detector are high enough. Therefore, the ground truth value of space headway at the trigger moment is the same as the space headway calculated by the following equation:(2)The calculation of space headway using 3D bounding box (3DBB): We use the distance measurement method based on 3D bounding box in [18] to calculate the vehicle space headway. Vanishing points and vehicle’s contours are required in [18], and we use the methods described in [19, 35] to solve these problems. In addition, we manually locate the preceding and following vehicles in the images, respectively, and eliminate the noise contours near them (such as the contours of the lane lines and road markings). After that, we use the method of [18] to construct 3D bounding boxes of all the preceding and following vehicles in the 55 key frames. As shown in Figure 17, since the camera is located above lane 2, there will be different orientations of bounding boxes when vehicles are on different lanes. For the vehicles in lane 3, the left edges of the vehicles may have higher confidences than the right edges do. The situation is reversed for vehicles in lane 1. Therefore, the space headway by 3D bounding box can be calculated by the following equation:where is the absolute difference between point COR_i and point COR_j along the direction of the car motion, COR_i-p means point COR_i on the preceding vehicles, COR_i-f means point COR_i on the following vehicles, λ is the scene scale described in [18] which is calculated by the length, width, and height of the preceding vehicles in the 55 key frames, and the estimated value of λ is 0.086.

(a)

(b)

Figure 17

3D bounding boxes on vehicles in different lanes. (a) Vehicle in lane 3; (b) vehicle in lane 1. The definitions of some bounding box’s corners in world coordinates are as follows: COR₁ is the right-front-lower corner, COR₂ is the left-front-lower corner, COR₃ is the right-rear-lower corner, and COR₄ is the left-rear-lower corner. As can be seen, for a vehicle’s bounding box in lane 3, edge COR₂COR₄ is visible while edge COR₁COR₃ is not. The situation is reverse when the vehicle is in lane 1. Therefore, the edges of bounding boxes for calculating the length, height, or space headway should be chosen differently for vehicles in different lanes.

Figure 18 shows a comparison of the space headway calculated by different algorithms with the ground truth in the 55 valid captured vehicle image pairs. Considering that the signs of the errors just mean the calculated results are larger or smaller than the ground truth, and that two errors with same absolute values but different signs mean the errors of the same size, we evaluate the absolute errors and relative errors in two ways: considering the signs of the errors and only considering the absolute values of the errors. Table 6 shows the errors between different algorithms.

As shown in Table 6, the proposed MPP algorithm performs better than the 3DBB algorithm in this real-world application experiment with increments of ±1.032 m in absolute error and ±1.266% in relative error. In addition, the upper limit of the errors of MPP is 9.662 m smaller than that of 3DBB. As shown in Figure 18, as the ground truth of space headway increases, the error of the calculated space headway (no matter by which algorithm) tends to increase. We speculate that is because the positions of the preceding vehicles are nearly the same (all at the loop), so when the actual space headway increases, the position of the following vehicle gets farther, meaning a lower resolution of the following vehicles and larger location errors. Among these, the increment of 3DBB is larger, which shows our MPP algorithm is more stable.

From Figure 18 and Table 6, we can also see that generally the space headway obtained by the MPP algorithm is smaller than the actual value, and that the difference between the space headway by MPP and the ground truth is small, with an average difference of ±1.308 m, average relative error of ±2.916%, and a maximum error not larger than ±8.766%. Therefore, the MPP-based algorithm proposed in this paper meets the practical needs.

In general, the proposed MPP algorithm is applicable for real-world traffic surveillance practical scenes, and especially when the actual space headway is not particularly large, the MPP algorithm can get a better calculation result. Furthermore, the proposed MPP algorithm does not require any additional equipment for the roads, the vehicles, or the cameras, and it is thus obviously superior to sensor-based or high-altitude video-based methods in terms of convenience and cost of use.

4. Conclusion and Further Work

This paper presents a precise space headway estimation algorithm from a single image based on MPPs. In this algorithm, considering the characteristics of the vehicle body, we raise the concept of the measurement point and use it to provide height information for the distance measurement. By using the proposed algorithm, we can estimate the real-time space headway of every vehicle using only a single captured or extracted image from existing traffic surveillance videos and thereby support the development of research including that on traffic flow theory, traffic congestion, and traffic safety. Furthermore, we analyzed the factors that cause the estimation errors, and the size and range of the errors were calculated by simulation. On this basis, optimization suggestions are given for the processing of the image calibration, measurement point selection, and measurement point measurement. Moreover, two real-world experiments were conducted, and the results showed that the proposed method achieves high accuracy and has high practicability in space headway estimation in common VIC and RVS scenes of existing traffic surveillance videos. The proposed method is more accurate and stable than the state-of-the-art distance measurement method and is more convenient and less expensive than other common space headway estimation methods.

Future work will be focused on the adaptability of the proposed algorithm to various traffic states, such as free traffic flow and congestion, in which situations of intervehicle occlusion or partial missing measurement points occur frequently.

Data Availability

The experimental data used to support the findings of this study are available from the corresponding author upon request.

Conflicts of Interest

The authors declare that there are no conflicts of interest regarding the publication of this paper.

Acknowledgments

This work was supported by the National Key R&D Program of China (nos. 2018YFB1601101 and 2018YFB1601100) and National Natural Science Foundation of China (no. U1611461).

References

Transportation Research Board, Highway Capacity Manual, Transportation Research Board, Washington, DC, USA, 2010.
B. S. Kerner, “Definitions of the three traffic phases,” in Introduction to Modern Traffic Flow Theory and Control: the Long Road to Three-Phase Traffic Theory, pp. 9–40, Springer-Verlag, Berlin, Germany, 2009.
View at: Publisher Site | Google Scholar
A. Houchin, J. Dong, N. Hawkins, and S. Knickerbocker, “Measurement and analysis of heterogenous vehicle following behavior on urban freeways: time headways and standstill distances,” in Proceedings of the 18th IEEE Conference ITSC, pp. 888–893, Las Palmas, Spain, September 2015.
View at: Publisher Site | Google Scholar
S. H. Ghasemi, M. Jalayer, M. Pour-Rouholamin, A. S. Nowak, and H. G. Zhou, “State-of-the-art model to evaluate space headway based on reliability analysis,” Journal of Transportation Engineering, vol. 142, no. 7, Article ID 04016023, 2016.
View at: Publisher Site | Google Scholar
N. Miller, M. A. Thomas, and J. A. Eichel, “A hidden Markov model for vehicle detection and counting,” in Proceedings of the 12th Conference CRV, pp. 269–276, Halifax, Canada, June 2015.
View at: Publisher Site | Google Scholar
M. Tursun and G. Amrullaa, “A video based real-time vehicle counting system using optimized virtual loop method,” in Proceedings of the 8th International Workshop on Systems, Signal Processing and their Applications (WoSSPA), pp. 75–78, Algiers, Algeria, May 2013.
View at: Publisher Site | Google Scholar
S. M. Abtahi, M. Tamannaei, and H. Haghshenash, “Analysis and modeling time headway distributions under heavy traffic flow conditions in the urban highways: case of Isfahan,” Transport, vol. 26, no. 4, pp. 375–382, 2012.
View at: Publisher Site | Google Scholar
L. Wang, X. Yang, Z. Du, and Z. Zhang, “Study on Chaos model of expected space headway in urban expressway,” in Proceedings of the TRB 2006 Annual Meeting, Washington, DC, USA, January 2016.
View at: Google Scholar
L. Wang, Z. Du, and J. Rong, “Study on expected headway model in urban expressway,” Beijing University of Technology, vol. 31, no. 6, pp. 598–603, 2005.
View at: Google Scholar
X. Zhao, W. Yao, N. Li, and Y. Wang, “Design of leader’s path following system for multi-vehicle autonomous convoy,” in Proceedings of the IEEE International Conference on Unmanned Systems (ICUS), pp. 132–138, Beijing, China, October 2017.
View at: Publisher Site | Google Scholar
H. Zhao, C. Wang, Y. Lin, F. Guillemard, S. Geronimi, and F. Aioun, “On-road vehicle trajectory collection and scene-based lane change analysis: part I,” IEEE Transactions on Intelligent Transportation Systems, vol. 18, no. 1, pp. 192–205, 2016.
View at: Publisher Site | Google Scholar
H. B. Riley and M. Celenk, “IR sensing embedded system development for prototype mobile platform for autonomous convoy,” in Proceedings of the 2nd IEEE 2nd International Conference on Signal and Image Processing (ICSIP), pp. 441–445, Singapore, August 2017.
View at: Publisher Site | Google Scholar
S. B. Maltsev, V. I. Rudik, and V. P. Rukin, “Radar sensor for vehicle cruise control applications,” in Proceedings of the 11th CriMiCo, pp. 626–629, Sevastopol, Ukraine, September 2003.
View at: Google Scholar
M. Zhao, A. Mammeri, and A. Boukerche, “Distance measurement system for smart vehicles,” in Proceedings of the 7th International Conference on New Technologies, Mobility and Security (NTMS), Paris, France, July 2015.
View at: Publisher Site | Google Scholar
S. Yu and Z. Shi, “An improved car-following model considering headway changes with memory,” Physica A: Statistical Mechanics and Its Applications, vol. 421, pp. 1–14, 2015.
View at: Publisher Site | Google Scholar
L. Lai, Y. Hu, and Z. Shi, “An improved optimal velocity model based on space headway and speed of front vehicle,” Northwestern Polytechnical University, vol. 32, no. 1, pp. 118–122, 2014.
View at: Google Scholar
W. J. Schakel, C. M. Gorter, J. C. F. de Winter, and B. van Arem, “Driving characteristics and adaptive cruise control? A naturalistic driving study,” IEEE Intelligent Transportation Systems Magazine, vol. 9, no. 2, pp. 17–24, 2017.
View at: Publisher Site | Google Scholar
M. Dubska, J. Sochor, and A. Herout, “Automatic camera calibration for traffic understanding,” in Proceedings of the British Machine Vision Conference (BMVC), Nottingham, UK, September 2014.
View at: Publisher Site | Google Scholar
M. Dubska, A. Herout, R. Juranek, and J. Sochor, “Fully automatic roadside camera calibration for traffic surveillance,” IEEE Transactions on Intelligent Transportation Systems, vol. 16, no. 3, pp. 1162–1171, 2015.
View at: Publisher Site | Google Scholar
X. Q. Zheng, “Video-based traffic flow parameter measurement and model research,” Dept. Mech. Eng. Sci., Fudan University, Shanghai, China, 2011, Ph.D. dissertation.
View at: Google Scholar
S. Suzuki, M. Tsunoda, M. Taniguchi, and T. Takagi, “Characterization of road traffic flow from measured data of speed and time-headway -relationship between density (k), flow rate (q) and speed (V)-,” in Proceedings of the Annual Conference SICE, pp. 1657–1661, Takamatsu, Japan, September 2007.
View at: Publisher Site | Google Scholar
J. Chung and K. Sohn, “Image-based learning to measure traffic density using a deep convolutional neural network,” IEEE Transactions on Intelligent Transportation Systems, vol. 19, no. 5, pp. 1670–1675, 2018.
View at: Publisher Site | Google Scholar
R. Ke, Z. Li, S. Kim, J. Ash, Z. Cui, and Y. Wang, “Real-time bidirectional traffic flow parameter estimation from aerial videos,” IEEE Transactions on Intelligent Transportation Systems, vol. 18, no. 4, pp. 890–901, 2016.
View at: Publisher Site | Google Scholar
N. K. Kanhere, S. T. Birchfield, and W. A. Sarasua, “Automatic camera calibration using pattern detection for vision-based speed sensing,” Transportation Research Record: Journal of the Transportation Research Board, vol. 2086, no. 1, pp. 30–39, 2008.
View at: Publisher Site | Google Scholar
J. Wang, “Research of vehicle speed detection algorithm in video surveillance,” in Proceedings of the International Conference on Audio, Language and Image Processing (ICALIP), pp. 349–352, Shanghai, China, July 2017.
View at: Publisher Site | Google Scholar
X. Li, M. Yuan, Q. Jiang, and G. Li, “VRID-1: a basic vehicle re-identification dataset for similar vehicles,” in Proceedings of the IEEE 20th International Conference on Intelligent Transportation Systems (ITSC), Yokohama, Japan, October 2017.
View at: Publisher Site | Google Scholar
L. Yang, P. Luo, C. C. Loy, and X. Tang, “A large-scale car dataset for fine-grained categorization and verification,” in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 3973–3981, Boston, MA, USA, June 2015.
View at: Publisher Site | Google Scholar
J. Ren, B. Li, Y. Liu, Y. Chen, L. Xin, and J. Shi, “Detecting and positioning of traffic incidents via video-based analysis of traffic states in a road segment,” IET Intelligent Transport Systems, vol. 10, no. 6, pp. 428–437, 2016.
View at: Publisher Site | Google Scholar
J. Wu, C. Xue, Z. Zhao, and B. Liu, “Effect of camera angle on precision of parameters for traffic accident in video,” in Proceedings of the 3rd International Conference on Digital Manufacturing & Automation (ICDMA), pp. 347–350, Guilin, China, July 2012.
View at: Publisher Site | Google Scholar
N. K. Kanhere and S. T. Birchfield, “A taxonomy and analysis of camera calibration methods for traffic monitoring applications,” IEEE Transactions on Intelligent Transportation Systems, vol. 11, no. 2, pp. 441–452, 2010.
View at: Publisher Site | Google Scholar
X. C. He and N. H. C. Yung, “New method for overcoming ill-conditioning in vanishing-point-based camera calibration,” Optical Engineering, vol. 46, no. 3, pp. 037202–037212, 2007.
View at: Publisher Site | Google Scholar
J. Sochor, J. Spanhel, and A. Herout, “BoxCars: improving fine-grained recognition of vehicles using 3D bounding boxes in traffic surveillance,” IEEE Transactions on Intelligent Transportation Systems, vol. 20, no. 1, pp. 97–108, 2019.
View at: Publisher Site | Google Scholar
S. He, C. Yang, and J.-S. Pan, “The research of Chinese license plates recognition based on CNN and length feature,” in Proceedings of the 29th IEA/AIE, pp. 389–397, Morioka, Japan, August 2016.
View at: Google Scholar
M. Mao, Y. Zhang, B. Wang, H. Liu, and B. Yin, “Vehicle detection through traffic video in congested traffic flow,” in Proceedings of the 6th International Conference on Digital Home (ICDH), pp. 247–256, Guangzhou, China, December 2017.
View at: Publisher Site | Google Scholar
J. Yang, B. Price, S. Cohen, H. Lee, and M. H. Yang, “Object contour detection with a fully convolutional encoder-decoder network,” in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 193–202, Las Vegas, NV, USA, June 2016.
View at: Publisher Site | Google Scholar

Copyright

Copyright © 2020 Zhi Yu et al. This is an open access article distributed under the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.

PDF Download Citation

Download other formats

Order printed copies

Views

935

Downloads

823

Citations