Abstract
Local roads have numerous blind spots caused by complex geometry, obstacles, and narrow width. Thus, conventional proactive countermeasures, such as passive traffic signs and convex mirrors, have not always been effective in preventing local road collisions. In this paper, we present a novel proactive two-step approach for traffic safety on local roads, comprised of detection of pedestrian-to-vehicle and vehicle-to-vehicle collision risks and warning systems. First, using video surveillance and radars to eliminate blind spots, the system detects road objects, predicts their trajectories and reachable areas, and identifies a potential risk situation. Second, it provides road users such as vehicles and pedestrians with warnings through LED variable message signs, which allows them to react effectively in risky situations. We have applied the system to two local road sites in South Korea, including a university campus in Seoul City and an apartment complex in Daejeon City. The detecting system has been validated using a confusion matrix. We have assessed the warning effect through a before-and-after study and found that the proposed system contributed to the improvement of traffic safety at the case study site in that traffic conflicts decreased by 55–62%.
1. Introduction
According to the global status report on road safety published by World Health Organization, the annual number of road traffic fatalities reached 1.35 million in 2016, and there have been approximately 1.3 million every year since [1, 2]. More than 50% of all road traffic fatalities occur among vulnerable road users, including pedestrians, bikers, and motorcyclists. Annually, between 20 and 50 million people have nonfatal injuries, with many developing disabilities as a result. Individuals, their families, and nations as a whole incur significant economic losses due to road traffic accidents. The average cost of road traffic accidents is 3% of a country’s gross domestic product.
For the efficient and effective implementation of traffic safety measures, adequate traffic safety management is essential. Recently, pedestrian safety has emerged as a global public health priority, as pedestrians are exposed to a variety of threats and are prone to sustaining severe injuries in traffic accidents. Since pedestrian traffic is concentrated on local roads, pedestrian safety control on local roads is particularly important. However, due to numerous blind spots on local roads caused by complex geometry, obstacles, and narrow width, it is difficult for road users to accurately estimate their current safety situations. A blind spot is defined as the spatial area outside of a person’s peripheral vision, and it restricts drivers and passengers from seeing objects and other pedestrians moving. This is one of the main reasons that traditional proactive countermeasures, such as traffic signs and convex mirrors, have not always been effective to prevent local road collisions. To overcome this issue, an advanced proactive countermeasure is required that can eliminate blind spots and provide accurate safety information in advance to local road users, including pedestrians.
Proactive near miss accident warning technologies have received much attention in the past decade, and numerous sensor-based algorithms have been developed for a better understanding of current safety situations. These algorithms recognize objects, estimate their trajectories, and assess whether a collision risk exists; based on surrogate safety measures (SSMs) derived from traffic conflicts, they estimate the collision risk of particular traffic scenarios using microscopic traffic parameters such as vehicle speed, acceleration, time headway, and space headway. Depending on how the collision risk is evaluated, the algorithms can be categorized into two groups.
The first group of algorithms uses deep learning methods to detect hazardous situations; after training with prior data categorized as risk situations by SSMs, algorithms predict whether a current circumstance is risky or not. For instance, gated recurrent unit and long-short term memory (LSTM) approaches were used to forecast the likelihood of a collision at a signalized intersection [3, 4]. For unsignalized crosswalks, another LSTM-based collision risk area estimation method was proposed [5].
The second group of algorithms applies SSMs to notice the presence of potentially hazardous scenarios based on the overlap of expected object trajectories. Recent years have witnessed a surge of scholarly works examining SSM practices and/or traffic conflicts, such as [6–8]. SSMs, including time-to-collision (TTC) and post encroachment time, have been extensively utilized to assess traffic safety performance and identify potential accident hazards. For example, Son et al. developed an algorithm to detect the risk of collisions between trucks and pedestrians [9]. Using microscopic simulations, the authors established a road network, and the algorithm was then validated using a confusion matrix. Moreover, Zhang et al. proposed a novel multipedestrian collision risk assessment approach that consists of collision checking, motion prediction, and collision risk assessment modules, and Wu et al. developed a crash warning system for bicycle lane areas at intersections utilizing connected vehicle technology [10, 11]. Ke et al. presented an approach to automatically detect near collisions between vehicles and pedestrians using onboard monocular vision and a moving background [12]. According to the extant literature, collision detection and/or prevention systems are mostly vehicle-integrated; recent studies have reviewed the threat assessment methods for such systems [13–15].
However, to the best of our knowledge, few algorithms have been developed specifically for local roads with blind spots. Moreover, no study has implemented and assessed the safety impacts of their proposed algorithms in real-world practice. In this paper, we present a novel proactive two-step approach for traffic safety on local roads, comprised of detection of pedestrian-to-vehicle and vehicle-to-vehicle collision risks and warning systems. We implemented video and radar equipment to eliminate blind spots at a target local road site. In addition to the detection system, we implemented LED Variable Message Signs (LED-VMS) for providing road users such as vehicles and pedestrians with warning information, which allows them to react effectively in risky situations. We have validated the complete form of a safety countermeasure, including detection and warning systems through implementing it on real-world local road sites in South Korea, including a university campus in Seoul City and an apartment complex in Daejeon City. We have validated the system using a confusion matrix and assessed the traffic safety effect through a before-and-after (BA) study.
The remainder of this paper is structured as follows: the system description section provides an overview of the proposed system’s architecture. Next, we elaborate on the case study sites. The evaluation results section describes the process and outcomes of the analysis. In the last section, we present the conclusion with future research directions.
2. System Description
In this section, we describe how the proposed system works. The system consists of four steps: (1) object detection and perspective transformation; (2) trajectory and reachable area prediction; (3) collision risk determination; and (4) collision risk warning.
2.1. Object Detection and Perspective Transformation
In this step, object detection and perspective transformation are conducted. First, for object detection, we use a video surveillance system and radar devices. We develop the object-detecting algorithm based on You Only Look Once (YOLO) v5, which is the latest deep-learning method and has been used for real-time video-based detection [16, 17]. The developed detecting algorithm is different from the YOLO algorithm in terms of how data is processed and analyzed. The size of input/output image data is not fixed but rather optimized based on hardware usage. In addition, input data are not processed in random order but in a first-in-first-out order so that image analysis proceeds without a bottleneck, as illustrated in Figure 1. Moreover, the data to be analyzed are processed as metadata that is compressed image information. It can improve detection accuracy and computing speed without image quality and pixel loss.
(a)
(b)
Our proposed object-detecting algorithm was compared to the different versions of YOLO, including YOLOv4 TRT, YOLOv4 Darknet, and YOLOv5m. They are considered to be superior to other traditional detecting methods, such as single-shot detection and convolutional neural network algorithms, in terms of high accuracy and computational speed [18–20]. Figure 2 illustrates object-detecting performance for the existing YOLO algorithms and the proposed method with respect to the mean average precision (mAP) and frames per second (FPS), commonly used to evaluate the accuracy and computing speed of detecting methods [21, 22]. The proposed detection algorithm has higher accuracy than YOLOv5m, but their computing speeds are comparable. In terms of both mAP and FPS, the proposed algorithm outperforms the rest detection algorithms. Consequently, based on the aforementioned comparison results, it can be concluded that the proposed system is superior to traditional detecting methods. Moreover, radar devices are employed when video-based detection is unavailable, such as in extreme weather conditions. From the video and radar equipment, the location, speed, and type of the object can be precisely calculated. Second, a perspective transformation is performed. We calculate the objects’ overhead coordinates from the raw data collected from the sensors’ perspective. Therefore, we use a perspective transformation matrix in the Open CV library, which is a computer vision tool [23]. The transformation procedure is depicted in Figure 3.
(a)
(b)
2.2. Trajectory and Reachable Area Prediction
This step is to predict the future locations of the detected objects. First, we identify whether an objective’s prior trajectory is straight or curved, depending on the degree of curvature of the current trajectory. If it is larger than a predetermined threshold [radians], we consider the trajectory to be curved. The time frame index is denoted , and the current frame is The negative and positive values mean the prior and future time points, respectively. The degree of curvature [radians] at the time frame is calculated using the following equation:where is the backward of the time frame index and is a predetermined weight factor such that for all .
If , an object will move straight. The object’s future location is determined based on the expected moving distance calculated by the following equation:where is the expected moving distance [meters] between the object’s center positions at the current and future time points and , respectively; is the estimated current velocity; and is the prediction time frame horizon. If , the future location at the time point is determined based on the location at the time point , , and .
Considering the actual size and uncertain future movement of objects, we set the reachable area of an object at as a rectangle and an ellipse for vehicles and pedestrians, respectively. The vehicles’ rectangular lengths, [meters] and [meters], are determined by the following equation based on a previous study [24]:
A pedestrian’s future location is stochastically defined by an ellipse [25]. The ellipse major axis length, [meters], and minor axis length, [meters], are empirically estimated by pedestrian trajectory data collected from video surveillance systems on local roads in Korea, as illustrated in Figure 4(a) [24]. Pedestrian trajectories were plotted every second for five seconds. It was found that the length of the major axis and minor axis varied depending on . From plotted trajectories, was linearly modeled, as illustrated in Figure 4(b). For , trajectories were plotted mostly within two meters. The estimated ellipse axis lengths are calculated using the following equation [24]:
(a)
(b)
2.3. Collision Risk Determination
If a risk is defined based solely on proximity, we do not account for other factors possibly important for risk identification, such as speed, direction, and future location uncertainty. Thus, we define the collision risk between two objects when their reachable areas overlap, which are obtained based on not only proximity but also the other factors since it has been proven to reduce false negative or positive cases [24]. As illustrated in Figure 5, the proposed system detects pedestrian-to-vehicle and vehicle-to-vehicle collision risks. Pedestrian-to-pedestrian collision risks are excluded since it is associated with relatively low severity, and pedestrians can evade a pedestrian-to-pedestrian collision promptly. Thus, the system does not consider the overlap of pedestrians’ reachable areas.
To reduce the computational burden for calculation between multiple objects on roads, a following three-step algorithm is used, as illustrated in Figure 6. In the first step, we set the maximum detection distance for each object, as shown in Figure 6(a). If a distance from the object (object A in Figure 6(a)) to another is greater than the maximum detection distance (e.g., thirty meters), we exclude the objects. Second, as depicted in Figure 6(b), we exclude situations when the future trajectories of two objects intersect outside local roads or when the time difference between the two objects’ arrival times to the intersecting point exceeds a predetermined threshold (e.g., two seconds). Last, we check if the reachable areas of two objects (not their trajectories) overlap in 0.25-second increments starting from one second earlier than the earlier time between the predicted arrival times of two objects at the trajectory intersection. If there is an overlap three times or more consecutively, we consider the situation a collision risk and define TTC as the time when the two reachable areas first overlap, as shown in Figure 6(c).
2.4. Collision Risk Warning
If a risk situation is identified and its predicted TTC is shorter than a predefined threshold, such as four seconds, a warning message is displayed on the LED-VMS, as depicted in Figure 7. TTC threshold is obtained as the total summation of the perception-reaction time [26], a safety margin time, and the time for the vehicle stopping distance. As shown in Figure 7, a warning message indicating “Stop” is provided. The LED-VMS is normally off, but when a risk situation occurs, it provides warning information until the risk situation is over. The LED-VMS is installed mainly in locations where they can be seen directly in the direction of vehicle and pedestrian routes, allowing drivers and pedestrians to get warning information easily.
3. Case Study Sites
We implemented the proposed system to two local road networks, a university campus in Seoul City and an apartment complex in Daejeon City, South Korea. First, we selected one site on the university campus for system verification, as shown in Figure 8, and we validated the system using a confusion matrix before the evaluation. Second, we selected two pairs of experimental and control sites in the apartment complex in Daejeon City for a BA study. The sites of each pair are associated with similar environmental and traffic conditions [27, 28]. The proposed system was implemented and operated at experimental sites but not at control sites. Since the experimental and control sites are located within the same apartment complex, population characteristics and temporal traffic patterns on local roads can be similar between the experimental and control sites. The experimental and control sites of pair A have parking lots and underground parking entrances, and the experimental and control sites of pair B have roadside parking lots, as shown in Figure 9.
(a)
(b)
(a)
(b)
(c)
(d)
(e)
4. Evaluation Results
4.1. Confusion Matrix
We use a confusion matrix approach to validate the performance of the implemented system, as shown in Table 1. True positive (TP) and true negative (TN) indicate that both predicted and actual situations are risky and not risky, respectively. False positive (FP) indicates that the actual situation is not risky, whereas the predicted situation is. On the other hand, false negative (FN) indicates the opposite situation to FP. Using the four measures, we calculate three metrics that indicate the system’s performance using equations (5)–(7). Accuracy is the most intuitive measure, indicating how precisely a system classifies situations. The true positive rate (TPR), commonly known as recall (or sensitivity), is the proportion of real-risk situations correctly predicted. It indicates how accurately the system classifies risky situations. On the other hand, the false positive rate (FPR) indicates incorrectly predicted no-risk situations among actual no-risk situations. If the FPR is high, it indicates that false alarms can occur despite the absence of potential risk situations.
We collected data on risk and normal situations from the sensors implemented on the university campus in Seoul City. We have tested the model’s performance under two different weather conditions, clear and rainy, and two lighting conditions, day and night. For example, photos of one application site under clear and rainy weather are shown in Figure 10. The data collection period for clear weather was from July 8, 2022, to July 11, 2022, for forty-eight hours. The same amount of time data was collected on rainy weather from August 1, 2022, to August 2, 2022. The sample numbers including both risk and no-risk situations were 162 and 135 for the clear and rainy weather periods, respectively, and the sample numbers for day and night conditions are 235 and 62, respectively. Table 2 presents the system verification results for weather and lighting conditions. Under clear weather, accuracy, TPR, and FPR were found to be 80.9%, 76.1%, and 15.4%, whereas, under rainy weather, they were 80.0%, 77.4%, and 18.3%, respectively.
(a)
(b)
The findings confirmed that the performance of the system particularly for local roads is comparable to that of other systems for other kinds of roads in terms of overall performance [3, 4, 9]. However, there was a difference between day and night. In the nighttime, accuracy and TPR were lower than in the daytime, while FPR was higher than in the daytime. It implies that the lighting condition can affect the system’s performance [29], and improving the performance even at nighttime remains a future work. For the comparison of clear and rainy weather, there was no discernible difference between sunny and wet conditions in terms of either accuracy or TPR. However, the FPR was marginally higher under wet conditions than in dry conditions. This implies that there is a greater likelihood of false alarms occurring while it is raining, since wet circumstances can limit vision and image clarity, hence lowering the precision of object recognition and trajectory prediction [30]. In addition, small errors measured by the sensors (video surveillance camera and radar) can compromise the object detection and trajectory prediction process. Thus, the model could erroneously categorize a no-risk situation as risky.
4.2. Before-and-After Study
Before-and-after safety investigation is an essential element of road safety improvement schemes that strive to quantify the potential benefits of a certain technical approach. Typical BA studies exploit historical collision data to evaluate the changes in collision frequency and/or severity attributable to safety interventions [31]. Several collision-based BA studies have adopted the empirical and the full Bayes methods [32–35].
Nonetheless, due to the rarity and randomness of crashes, statistically reliable safety evaluations require extended periods of collision data before and after the implementation of safety measures, particularly in the case of local road collisions. Therefore, the use of collisions in BA studies may raise a moral conflict of waiting for “collisions” to occur before attempting to prevent them. Moreover, the use of crash data has usually been limited by sample size and under-reporting [36]. For that reason, using crash data is considered a reactive analysis method as opposed to a proactive one.
Hence, in this study, we used traffic conflicts to conduct a BA study. Traffic conflict-based analysis provides insight into the failure mechanism of traffic accidents with a marginal social cost [37]. Additionally, it does not require a long observation period. In July of 2022, we collected traffic conflict data from the surveillance systems installed at two pairs of experimental and control sites; the sample sizes for pairs A and B were 253 and 597, respectively, which are statistically reliable based on the method proposed by [38, 39]. The observed traffic conflict frequencies at individual sites are shown in Figure 11. The figure shows that, for all experimental sites, the conflict frequency was substantially reduced, whereas, for all control sites, the number of traffic conflicts did not differ in the before-and-after period.
(a)
(b)
To statistically analyze the effect, we adopted the odds ratio (OR) method [27, 40–42]. This method is usually employed to systematically compare the effects on experimental sites and control sites. By using the OR method, we can explain before-and-after temporal changes in traffic conflicts and exclude factors unrelated to safety interventions [42]. The OR indicator for pair , , can be calculated as in the following equation:where and represent the number of traffic conflicts in before-and-after experimental periods at the experimental site of pair and and are those numbers for before and after the experimental periods at the control site of pair . In this study, before-and-after experiments correspond to whether or not LED-VMS provided road users with warning information. A value of OR smaller than one shows that the experiment is effective while OR values greater than one show a negative effect. When OR equaled one, all changes are attributable to factors unrelated to the experiment. To intuitively examine the effect, we calculated the treatment effect (TE) for a pair using the following equation:where represents the change in the frequency of traffic conflicts at the experimental site of pair after the implementation of the proposed system, considering the change in the frequency of traffic conflicts at the control site of pair . We can calculate the percentage of reduction in traffic conflicts as . Negative values of TE mean a reduction in conflicts—in other words, a safety improvement. On the contrary, positive values of TE indicate an increase in conflicts and, thus, a safety deterioration. Table 3 shows the values of OR and TE. Both A and B experimental sites showed negative TE.
We integrated the individual ORs using a weighted average to estimate the total OR. In addition, we assessed the statistical significance of the calculated ORs by testing the following hypothesis: OR is equal to one (null hypothesis). We proceeded by converting OR to a logarithmic form. OR is always positive because the number of traffic conflicts cannot be negative. This observation led to the generalization that OR should follow a lognormal distribution. A standard error () of for an individual site pair can be estimated as in the following equation:where is a standard error in site pair . In addition, we assumed that weights are inversely proportional to the variance of individual OR. A weight factor to calculate total OR can be calculated as in the following equation:where is a weight factor in the site pair . The total OR for two site pairs can be calculated as in the following equation:
The test statistic z asymptotically follows a standard normal distribution. It can be calculated as in the following equation:
We reject the null hypothesis if the tail probability of the probability density function is smaller than the significance level of 0.05. The statistical significance result of the safety effect is presented in Table 4. Experimental sites A and B showed 62.4% and 55.0% reductions in traffic conflicts, considering control sites A and B. The number of conflicts decreased by 57.3% overall. All of the results were found to be statistically significant. It is, therefore, implied that the proposed system improved traffic safety at all experimental sites.
In experimental site A, conflict reduction was approximately 62.4%, which is greater than that in experimental site B, where conflict reduction was approximately 55.0%. It suggests that the effect is greater at experimental site A than at experimental site B. This is presumably due to the location of LED-VMS at both sites. At experimental site A, the LED-VMS is located in front of the underground parking exit. However, at experimental site B, the LED-VMS is located on the roadside. Vehicles exiting the underground parking lot in experimental site A can see the LED-VMS signal more clearly than vehicles in experimental site B, which can have an impact on driver reaction time.
5. Conclusions
In this study, we present a novel proactive two-step approach for traffic safety on local roads, comprised of detection and warning. First, using video surveillance and radars to eliminate blind spots, the system detects objects and predicts their trajectories and reachable areas; the system then uses time-to-collision to identify potential risk scenarios based on the overlap of predicted reachable areas. Second, the system provides LED-VMS-based warnings that enable road users, such as drivers and pedestrians, to respond effectively in specific circumstances.
We installed and operated the proposed system on a university campus and an apartment complex in South Korea. First, we validated the system’s performance using a confusion matrix; the system was accurate 80% of the time, and 77% of collision risks were successfully detected by the system, whereas 17% were false alarms. Second, we assessed the traffic safety effect through a BA study. We chose two pairs of experimental and control sites with traffic and geometry comparable. The proposed system significantly reduced traffic conflicts and improved traffic safety, as the results indicated a decrease in traffic conflicts ranging from 55% to 62% across various locations.
The findings from this research are encouraging, and further work needs to be carried out to improve safety on local roads. We are aware that our research may have some limitations. Local roads are difficult to standardize like highways or higher road hierarchies than local roads, and implementing the proposed system on various local road sites requires a high amount of budget. Thus, we do not intend to estimate standard crash modification factors for the proposed system on local roads through a BA study. Instead, we focus on two sites in the context of a case study to derive practical insights from statistically proven site-specific data analysis results. Currently, this research consortium is planning to extend the number of testing sites in the next few years to improve the generality of the proposed system’s potential benefits. In the future, we will, therefore, be able to collect and examine a higher number of samples from newly exploited sites to conduct a meta-analysis for an objective measurement of the integrated quantitative evidence. Second, warning information from LED-VMS alone may not be sufficient for pedestrians using smartphones while walking. Audible alerts would be more suitable for such instances. Thus, we will install multiple heterogeneous warning systems that will be used to provide road users with risk information. In addition, we are planning to further optimize the accuracy and effectiveness of the system by considering conflict severity assessment, to give different levels of warnings (for instance, when the risk is low, only LED-VMS is used, whereas when the risk is high, both LED-VMS and an audible warning are sent). The system can have multiple warning thresholds to classify the level of risk situations.
Data Availability
The data used to support the findings of this study are available from the corresponding author upon request.
Conflicts of Interest
The authors declare that they have no conflicts of interest.
Authors’ Contributions
Jeongin Yun and Aya Selmoune contributed equally to this paper.
Acknowledgments
This work was funded by the Ministry of the Interior and Safety (MOIS), Republic of Korea (grant number 2021-MOIS41-001-00000000-2022 (development of traffic safety risk warning technology for blind spots in living roads)).