Abstract

Motor vehicles are changing the way people live, but they are also putting a huge strain on urban traffic. In the majority of major cities, parking has become the number one problem for car owners to get around. The management efficiency of car parks directly affects the traffic of the whole city. In order to improve the management efficiency of the car park, this paper designs an intelligent parking management system based on ARM and ZigBee wireless sensor network. Firstly, according to the internal environment and economic cost of the car park, ultrasonic sensors are used to monitor whether the parking space is empty or not. The information collected by the ultrasonic sensors is transmitted to the ARM host controller through the ZigBee wireless sensor network, and the ARM host controller determines whether there are free parking spaces based on the collected information. Secondly, Faster R-CNN, a deep learning algorithm, is selected as the license plate recognition model, and the Faster R-CNNN is improved by the residual module. Finally, in order to extend the lifetime of the ZigBee wireless network, the ZigBee routing algorithm is investigated, and an improved routing algorithm based on energy averaging is proposed. The effectiveness of the improved routing algorithm is demonstrated by a simulation analysis through NS2. The test results show that the designed intelligent parking management system is able to complete the functions of parking space detection and license plate recognition normally, thus effectively improving the efficiency of the car park and providing great convenience to motorists.

1. Introduction

With the continuous development of economy and science and technology, people’s quality of life has been continuously improved, which is most intuitively reflected in the number of motor vehicles. In 2009, China surpassed the United States and thus became the world’s number one country in terms of automobile production and sales [1, 2]. Private cars have now become an important part of consumption in the automotive industry [3, 4]. By the end of June 2021, the number of motor vehicles in China reached 390 million, as shown in Figure 1. The dramatic growth in the number of cars has not only brought convenience to people’s travel but also brought various problems to individuals as well as society [5, 6].

At present, the rate of construction of parking infrastructure is much slower than the growth in motor vehicle ownership, resulting in a huge problem in the supply of parking spaces. The parking problem has become the number one problem for car owners when people are travelling by car. The ratio of the number of motor vehicles to the number of parking spaces in large cities is about 1 : 0.8 [7, 8]. In most cities, people need to spend a lot of time searching for a vacant parking space, which not only wastes time but also tends to lead to traffic congestion in the surrounding area. As there is no way to obtain accurate information about available parking spaces in a car park in advance, car owners need to blindly search for available spaces with the naked eye when they arrive on-site [9, 10]. However, the topography of many underground car parks is more complex, and the view inside the car park is not open enough, making it difficult for car owners to find a free space in time. This situation will not only cause confusion inside the car park but also increase the difficulty and cost of car park management.

Both the number of parking spaces and the efficiency of car parks affect the efficiency of the overall transport system [11, 12]. In order to solve the problem of insufficient parking spaces, the government has started to build new parking infrastructure, but it still cannot meet the demand for parking and requires a large amount of financial expenditure. Researchers have tried to use science and technology instead of traditional labour to make parking systems more user-friendly in order to fundamentally improve the efficiency of parking management [13, 14]. Using the latest artificial intelligence and Internet of Things technologies to retrofit car parks, it is possible to share all parking space information on the web, thus making it easier for users to make enquiries.

At present, the application of artificial intelligence technology in vehicle identification has received extensive attention [15]. For example, Geng and Cassandras [16] proposed a vehicle recognition technique based on a BP neural network, which is capable of automatically obtaining license plate information through surveillance video. With the rapid development in the field of deep learning, target recognition using deep learning techniques has achieved satisfactory working results [1724]. Various deep learning-based target recognition applications are increasingly available in the security, military, and human-computer interaction fields. However, deep learning requires a large amount of computational power to support and therefore requires high performance of the algorithm processing equipment. As a result, most deep learning applications are currently networked, meaning that the images captured by the camera are transmitted to a cloud server via the network, and the results are returned to the underlying device after the algorithm has been processed. This underlying device is called an “edge device” [2528].

However, this target recognition system based on the networking method requires a high transmission bandwidth and a large transmission delay and therefore cannot achieve a real-time response. The intelligent parking management system places very stringent requirements on the response latency of target recognition. In addition, the network transmission of image data can easily be intercepted by hackers, leading to user privacy leakage or security risks. In recent years, as large data centres have grown in size, their heat dissipation and security issues have become more prominent. As a result, localised offline working methods are gaining attention, and “edge computing” has become the main tool for offline target recognition [29, 30]. For example, Wang et al. [31] proposed an improved convolutional neural network model that has been successfully applied to a variety of edge computing devices, and the ARM family of processors is well suited for edge computing devices due to its powerful computing capabilities.

Compared with other wireless communication methods, wireless sensor networks have become the mainstream technology for various IoT systems with their advantages of low power consumption, low cost, and high quality [3237]. Wireless sensor networks were born in the late 1990s as a special kind of ad hoc network. A wireless sensor network consists of many sensor nodes. These nodes have functions such as information collection, data processing, wireless communication, and cooperation. These nodes can be randomly or specifically arranged in the area of the monitored environment. All nodes are self-organised through specific protocols and have superb adaptive capabilities. Wireless sensor networks have a very wide range of application prospects and can be used in industrial control, environmental monitoring, intelligent monitoring, and other fields. Currently, the IEEE 802.15.4 (ZigBee) standard is the mainstream technology for wireless sensor networks. ZigBee’s self-organising function and wireless transmission function are very suitable for intelligent parking management systems. This study considers ZigBee technology to be a low-cost, low-rate wireless interconnection standard currently applicable to intelligent parking management systems.

The aim of this research is therefore to design an intelligent parking management system based on ARM and ZigBee wireless sensor network, thus reducing the time taken by users to find parking spaces (increasing the utilisation of parking spaces). In this paper, the Linux embedded operating system was built and ported to the ARM host controller. A ZigBee wireless sensor network was constructed, and a routing improvement algorithm based on energy balancing was proposed. The designed system implements two main functions: (1) parking space detection and (2) license plate recognition. Parking space detection: according to the internal environment of the car park and the economic cost, this paper uses ultrasonic sensors to monitor whether the parking space is empty or not. The information collected by the ultrasonic sensor is transmitted to the ARM main controller via a ZigBee wireless sensor network. License plate recognition: with the help of a computing stick and the ARM host controller, this paper uses deep learning algorithms to process the video captured by the surveillance cameras in the car park in real time and then uses OpenCV tools to carry out plate recognition in order to determine the parking time of each vehicle, thus enabling localised offline computing (without the need for a host computer or cloud platform).

The main innovations and contributions of this paper include the following:(1)By comparing various deep learning algorithms, Faster R-CNN was found to have higher accuracy. Therefore, Faster R-CNN was selected as the license plate recognition model, but its high computation and transmission costs cannot be effectively used in edge computing devices. Therefore, in this paper, the Faster R-CNN is improved by the residual module.(2)In order to extend the lifetime of the ZigBee wireless network, the ZigBee routing algorithm was studied, and an improved routing algorithm based on energy averaging was proposed. Simulation analysis was carried out by NS2 to prove the effectiveness of the improved routing algorithm.

The rest of the paper is organised as follows: In Section 2, the overall system design was studied in detail, while Section 3 provides the license plate recognition model based on deep learning. In Section 4, the ZigBee networking was studied in detail, while Section 5 provides the experimental results and analysis. Finally, the paper is concluded in Section 6.

2. Overall System Design

2.1. General Structure of the System

According to the function of the smart car park and the characteristics of ZigBee technology, the smart car park management system designed in this paper uses the ARM processor OMAP3530 as the main controller. The network transmission part of the system consists of a coordinator node, multiple router nodes, and multiple sensor data collection nodes. The overall structure of the system is shown in Figure 2.

At the entrances and exits of the car park, stop bars and surveillance cameras are placed to control the entry and exit of vehicles. When the vehicle arrives at the entrance, the system identifies the vehicle number and records the corresponding time and plate number. At the same time, the ARM controls the stop bar and releases the vehicle. License plate recognition can improve the efficiency of vehicles passing through entrances and exits. License plate recognition technology is currently used in many car parks. Each parking space detector is connected to a ZigBee node. The data detected by the car space detector is sent from the end node to the ARM main control module via a router. The coordinator does not need to add a parking space detection module.

2.2. Workflow of the Parking Management System

The user can observe the display at the entrance to the car park. If a space is available in the car park, the system displays the serial number and location of the available space. The user then drives the vehicle into the car park. At this point, the camera recognises the number plate and records the relevant data. If there is no space available in the car park, no space is displayed. When the user drives the vehicle to the exit of the car park, the camera recognises the number plate. The system charges the user a fee based on the rate and parking time. After the user has paid the fee, the system raises the stop bar and releases the vehicle. The workflow is shown in Figure 3.

2.3. Design of the Hardware System

The OMAP3530-Mini was chosen as the main controller platform for this system. The OMAP3530 is the latest generation of high-performance ARM processors from TI. The OMAP3530 uses the advanced Cortex A8 architecture and is capable of reaching a main frequency of over 1 GHz. The ARM main controller module uses the OMAP3530 processor as the core and contains modules such as LCD, DDR2, Nand Flash, and ZigBee wireless communication module. The structure of the ARM main controller module is shown in Figure 4.

The wireless communication module consists of a ZigBee wireless sensor network. The ZigBee coordinator node is connected to the ARM host controller module via the RS 232 serial port. The ZigBee coordinator node is responsible for data forwarding between the two modules. The ZigBee node consists of the CC2430 core processor, wireless transceiver module, sensors, I/O interface circuit, and power supply module. The structure is shown in Figure 5.

The core CC2430 processor is responsible for controlling the logic, routing protocols, synchronous positioning, power management, and task management of the entire node. The sensors are responsible for collecting the temperature in the designated monitoring area and completing the data conversion. The wireless transceiver module is responsible for wireless communication with other nodes. The power supply module provides energy for the operation of each module in the node. The wireless transceiver chip used is the Chipcon CC2430, which has a high-performance 2.4 GHz DSSS RF transceiver and an 8051 controller (MCU) inside the CC2430. The main performance parameters of CC2430 are shown in Table 1.

The OMAP3530 and CC2430 require a voltage of 3.3 V, and the other peripheral voltages require 5 V, 3.3 V, and 1.5 V inputs, respectively. Therefore, a DC 5 V switching power supply needs to be converted. The MAX687 is a low-noise linear voltage regulator. The MAX687 will automatically cut off the output voltage when the output voltage falls below 2.96 V. With an output current of up to 1 A, the MAX687 can fully meet the power supply requirements of the OMAP3530 module.

As the level of the standard RS-232 is different from the signal level of the TTL circuit, the MAX32323 chip was used for level conversion. The pin connections between the OMAP3530 and MAX3232 are shown in Table 2.

2.4. Ultrasonic Sensor-Based Parking Space Detection

In an intelligent parking system, it is necessary to detect parking space information and determine whether there are any vehicles in the parking space. This function is the key part of the whole system. At present, the actual common detection technologies are geomagnetic detection technology, infrared detection technology, video image processing detection technology, ultrasonic detection technology, and so on. Ultrasonic sensors have the advantages of low cost, simple maintenance, strong anti-interference ability, and high measurement accuracy; therefore, this paper uses ultrasonic detection technology for real-time detection of car parking status in the car park. As the ultrasonic sensor is affected by the temperature, temperature compensation measures can be taken.

The ultrasonic distance measuring principle used in the intelligent parking system is shown in Figure 6. T indicates the ultrasonic transmitting device, R indicates the ultrasonic receiving device, D indicates the distance of the ultrasonic device from the ground, and H indicates the distance of the ultrasonic device from the roof of the car. The model number of the ultrasonic sensor is HC-SR04. The ultrasonic timing is shown in Figure 7.

Ultrasonic sensors calculate distances from the speed of sound and time. Variations in the speed of sound therefore affect the accuracy of the distance measurement. The speed at which ultrasound travels through the air is temperature-dependent. In order to enhance the accuracy of the measurement and the range of application of the system, temperature compensation is considered. Values of the speed of sound at a variety of temperatures are shown in Table 3. The velocity of sound in air as a function of temperature is shown as follows:where c is the propagation speed of the ultrasound at the current temperature and T is the actual temperature.

As can be seen, temperature has an effect on the speed of sound, and certain measures are required to improve the accuracy of the measurement. Therefore, a temperature compensation module has been added to this paper. The real-time sound velocity is calculated by measuring the current temperature and bringing it into (1). The operating voltage of the temperature sensor DS18B20 is 3.0∼5.5 V, and the temperature measurement range is −55∼ + 125°C, which can fully meet the system requirements. In this paper, the DS18B20 is chosen to provide temperature compensation for the ultrasonic sensor.

3. Deep Learning-Based License Plate Recognition Model

There are two main categories of deep learning-based recognition algorithms [38, 39]: one-stage algorithms and two-stage algorithms. For the one-stage recognition methods, the You Only Look Once (YOLO) and Single Shot Detector (SSD) are mainly used. For the two-stage algorithms, R-CNN, Fast-RCNN, Faster R-CNN, and Mask R-CNN are mainly used. Through the performance comparison of each algorithm model, it is found that Faster R-CNN in networking mode has higher accuracy [40]. However, its high computational and transmission costs prevent it from being effectively used in edge computing devices.

To address the above issues, the Faster R-CNN model is improved. Firstly, the feature extraction network is clipped by the residual module. Secondly, an adjustable candidate region is designed. By setting the size of the convolution sliding window, the region proposal can be obtained more quickly. Finally, the improved Faster R-CNN model was implemented using an ARM host controller and an Intel neural computing stick (an edge computing device) to obtain better licence plate recognition results.

3.1. Structure of the Model

In order to enable Faster R-CNN to run smoothly on edge computing devices, this paper proposes a new lightweight target recognition model based on the Faster R-CNN algorithm, the overall structure of which is shown in Figure 8.

The improved Faster R-CNN model is divided into three parts: feature extraction, region proposal network, and target recognition. The traditional feature extraction part is replaced by a residual network. Due to the excessive scale parameters required by the traditional model, an adjustable region proposal network (ARPN) is proposed in this paper. The ARPN assigns respective sliding windows to the three subchannels (corresponding to the three image feature matrices) to obtain anchor at different target scales [41]. The ARPN can obtain the region proposal more quickly.

3.2. The Model’s Backbone Network

Darknet53 was selected as the feature extraction part of the improved Faster R-CNN model. Darknet53 combines the advantages of Darknet19 as well as other novel residual networks. The backbone network combines multiple 3 × 3 convolutional layers and 1 × 1 convolutional layers. The structure of the backbone network is shown in Table 4.

The proposed model generates three different sets of image feature matrices through the three branches. The convolution kernel extracts shallow features (colour, edges, etc.) at a shallow layer. As the depth of the convolutional layers increases, the model extracts more detailed information. The size of the extracted feature map is inversely proportional to the number of layers.

3.3. Structure of the ARPN

The purpose of the ARPN is to extract a certain number of region proposals from the input image, the structure of which is shown in Figure 9.

The key to generating a high-quality region proposal is to define a reasonable-sized window and anchor for each feature matrix, so a different anchor needs to be assigned to each feature matrix. The specific settings are shown in Table 5.

3.4. Loss Functions

The loss function of the improved Faster R-CNN model is divided into two parts: (1) the ARPN loss function and (2) the target recognition loss function. The ARPN loss function contains both classification loss and regression loss components.

The ARPN loss function is calculated as follows:where M is the learning parameter of the ARPN network, n represents the number of branches, represents the loss weight of the n-th branch, and is the training sample set of the n-th branch.

The target recognition loss function is calculated as follows:where represents the k-th branch; represents the loss weight of the k-th branch; is the training sample set of the k-th branch.

The overall loss function of the improved Faster R-CNN model is calculated as follows:

4. ZigBee Networking

ZigBee nodes in smart parking management systems need to be powered by batteries, so reducing energy consumption is an issue that must be considered in the design of the system; sometimes, a ZigBee node does not need to communicate, but if it is in the communication path between other communicating nodes, the node is still involved in data forwarding and thus consumes energy. In some cases, this problem may also cause network fragmentation or lead to shorter network life cycles. It is therefore essential to investigate and improve the routing of ZigBee networks.

4.1. Principle of ZigBee Routing Algorithm

The routing algorithm primarily provides the best path for the delivery of data. The routing function is implemented in the network layer. Routing algorithms play a crucial role in routing protocols, and the use of different algorithms often yields different pathfinding results. Currently, three routing algorithms are commonly used for the ZigBee protocol [42]: (1) Cluster-Tree algorithm, (2) AODV algorithm, and (3) AODVjr algorithm.

The Cluster-Tree algorithm uses a tree topology, so the routing process is relatively simple. When there is data to be sent, the next hop node is calculated directly by means of a formula. However, the Cluster-Tree algorithm is generally not used when the number of network nodes is high. This is because of the large time delays in data transmission in a tree structure. Compared to the Cluster-Tree algorithm, the AODV algorithm and the AODVjr algorithm have a much improved network communication rate and effectively avoid the routing loop problem. However, each routing node on the communication link must maintain a routing table. Therefore, how to balance the energy consumption of the whole network is an important research direction for routing algorithms.

4.2. Improvement of ZigBee Routing Algorithm

To address the above problems, this paper proposes an algorithm for energy balancing to achieve the goal of extending the survival cycle of the network. The nodes in the network are classified according to the remaining power. For nodes with sufficient node energy, the normal AODV protocol is used for data transmission. When the remaining power of a router node falls below a threshold value, this router node sends a notification to the child nodes. The child node will look for an auxiliary parent node and establish a communication route with it, thus effectively reducing the communication tasks of the original parent node. An example of the energy shunt is shown in Figure 10.

At the beginning of the ZigBee network, each node has an abundance of energy. The data transfer between nodes 2, 3, and 5 and coordinator 0 passes through node 1. At this point, the energy consumption of node 1 is high. After a certain period of time, the energy of node 1 decreases to a certain threshold. Node 1 notifies nodes 2, 3, and 5. Nodes 2, 3, and 5 each initiate the discovery process for the auxiliary parent node. Node 3 finds the auxiliary parent node 7, while node 5 finds the secondary parent node 6. Through the diversion of these two auxiliary parents, the communication burden on node 1 is greatly reduced. The energy consumption of the entire network tends to be balanced, thus extending the survival cycle of the network.

Let the energy of a node be E and the maximum energy be Emax. We classify the residual energy of a node into three classes, as shown in Table 6. Each node is divided into three levels according to its residual energy. When the energy level of a node changes from “A” to “B,” the node sends a notification to all its children.

The child node receives notification from the parent node and initiates the auxiliary parent node discovery process. The child node broadcasts a route sending request group RREQ to the neighbouring nodes (in addition to the parent node). After receiving the RREQ, the neighbouring router node determines whether it satisfies the network depth and energy level conditions. If it satisfies, it responds to the RREP; otherwise, it discards the RREQ.

Assume that there are n paths from the source node to the destination node. The i-th path has m nodes. Eij indicates the remaining energy of the j-th node on the i-th path. E1 indicates that the node is in the “A” zone, E2 indicates that the node is in the “B” zone, and E3 indicates that the node is in the “C” zone. The minimum residual energy of the node on the i-th path is calculated as follows:

Cij denotes the energy consumption required by the jth node on the i-th path to send data, and the total energy consumption of the path is calculated as follows:

5. Experimental Results and Analysis

5.1. Experimental Setup

The designed intelligent parking lot management system was built with hardware such as OMAP3530 processor and Intel neural computing stick. The ARM main controller module has Open CV and NCSDK API installed. The system uses Sony’s 8-megapixel RPi camera based on IMX219. The Intel™ neural computing stick is a VPU model with a USB 3.0 interface. Capture the parking lot surveillance video using the cv2 video capture (URL) function. The surveillance video of some cameras in the parking lot is shown in Figure 11.

5.2. Simulation Analysis of Improved ZigBee Routing Algorithm

Firstly, the improved ZigBee routing algorithm is simulated and analysed. The simulation software version is NS-2.32, and the operating system environment is Ubuntu 10.10. A comparison between the AODV algorithm and the improved ZigBee routing algorithm is carried out in this paper. The number of nodes is set to 11, 31, 51, 71, and 91, respectively. Node 0 is the central coordinator node, the simulation area is a square area, the nodes are spaced 10 m apart, and the simulation time is 1000 seconds. Figure 12 shows the simulated network scenario with 11 nodes.

The data source sends packets at a rate of 10 packet/s, and each packet is 100 bytes. The initial energy of the node is 100 J. The node consumes 0.4 J to send a packet and 0.1 J to receive a packet. Based on the simulation data, a comparison of the survival cycles obtained is shown in Figure 13.

It can be seen that, at smaller network sizes, the survival times of the two routing algorithms are not very different. As the network size increases, the survival time of both routing algorithms decreases. The survival time of the AODV algorithm decreases rapidly, while the improved ZigBee routing algorithm decreases relatively slowly. Compared to AODV, the improved ZigBee routing algorithm with an energy balancing mechanism has a significantly higher survival time. This is because the AODV routing algorithm always looks for the shortest path during routing, causing the energy consumption of key nodes in the path to drop too quickly. However, the improved ZigBee routing algorithm was able to effectively reduce the total energy consumption of the network, thus extending the network survival time. A comparison of routing overheads is shown in Figure 14.

It can be seen that the routing overhead of both routing algorithms grows linearly as the network size increases. The improved ZigBee routing algorithm has a slightly higher overhead than the AODV protocol due to the use of energy-controlled packets to adjust the network topology. However, the increased overhead has little impact on the overall network performance. Although the AODV protocol has lower overhead, the premature failure of its key nodes is difficult to resolve.

5.3. Analysis of Licence Plate Recognition Performance

The video test results of the license plate recognition performance are shown in Figure 15. It can be seen that the improved Faster R-CNN model is able to accurately complete the detection of both close and long-range targets, verifying the strong adaptability to multiple target scales (large and small targets).

To verify the effectiveness of the improved Faster R-CNN model, it was tested against R-CNN and Faster R-CNN. The average accuracies in the three different difficulties are shown in Table 7.

It can be seen that the average accuracy of the improved Faster R-CNN reached 92.84%, 90.19%, and 85.31% in the three different difficulties, respectively. Compared to the original Faster R-CNN, all of them are slightly lower, at 0.86%, 1.72%, and 2.18%, respectively. However, the improved Faster R-CNN after the light weighting process showed a significant improvement in running speed, as shown in Table 8.

It can be seen that the improved Faster R-CNN improves the speed of single image recognition by 4.471 ms and video detection by 1.04 FPS. Collectively, with almost no significant loss in accuracy, the improved Faster R-CNNN effectively improves the speed of license plate recognition with better real-time performance.

6. Conclusion

This paper designs an intelligent car park management system based on ARM and ZigBee wireless sensor network, thus effectively improving the efficiency of the car park. The Linux embedded operating system is built and ported to the ARM host controller. Ultrasonic sensors are used to monitor whether a parking space is empty or not. Localised offline computation was achieved by utilising the Faster R-CNN, a deep learning algorithm, as the licence plate recognition model and using OpenCV tools for licence plate recognition in order to determine the parking time of each vehicle. In addition, the Faster R-CNN was improved with a residual module in order to obtain better real-time performance and security on edge devices. A ZigBee wireless sensor network was constructed, and a routing improvement algorithm based on energy balancing was proposed. ZigBee network simulations and system tests achieved the expected results. Subsequent attempts will be made to use the SSD model for licence plate recognition in order to further improve the accuracy of recognition.

Data Availability

The experimental data used to support the findings of this study are available from the corresponding author upon request.

Conflicts of Interest

The authors declare that they have no conflicts of interest to report regarding the present study.