Abstract
There are many drawbacks such as clustering, background updating, inaccurate testing results, and low anti-interference performance in traditional moving target detection theory. In our study, a background subtraction method to automatically capture the basketball shooting trajectory was used to eliminate the drawbacks of the fixed-point shooting system such as cumbersome installation and time and manpower consumption. It also can improve the accuracy and efficiency of moving target detection. We also synthetically compared to common methods including the optical flow method and interframe difference method. Results showed that the background subtraction method has better accuracy with an accuracy rate over about 90% than the interframe subtraction method (88%) and the optimal flow method (85%) and presents excellent robustness with considering variable speed and nonrigid objects. Meanwhile, the automatic detection system for basketball shooting based on background subtraction is built by coupling background subtraction with detection characteristics. The system detection speed built is further accelerated, and the image denoising is improved. The trajectory error rate is about 0.3, 0.4, and 0.5 for the background subtraction method, interframe subtraction method, and optimal flow method, respectively.
1. Introduction
In recent years, with the development of computer vision technology and video surveillance, the moving target detection technology (MTT), a significant part of them, has gradually attracted researchers’ attention and also has been widely applied in different domains such as defense and security monitoring [1–3]. MTT is a technology which can separate variable parts from a video image on the basis of image segmentation with geometric and statistical features. Results from MTT can provide interesting regions for an object’s identification, track, position, and behavior analysis. MTT has been regarded as a novel technology with high work efficiency and manpower-saving [4]. Multiobject information can be obtained from the video image sequence compared to the stationary image sequence when using MTT to test an object. In a stationary scenario, the image subtraction method has been regarded as one of the best efficient methods because we can easily obtain the background reference model with a net and nonvariable target in the stationary scenario. Meanwhile, a real-time update background reference model used the current frame image so that the background model can adapt scenario variation. As for the moving scenario, both the moving compensatory procedures of background and detection methods applied in the stationary scenario are needed to test the moving target in the image due to its complicated variation [5].
Currently, MTT methods have been improved and/or innovated with the development of computer vision technology [6]. The development regular for MTT can be generalized as follows: (1) the optimal flow method on the basis of optimal flow technology, the interframe difference method on the basis of subtraction image, and the background subtraction method. Background difference and interframe difference have been known as the most popular detection methods in the application visual image procedure system; (2) continuous improvement in traditional detection methods. As for drawbacks for traditional methods, some methods based on improvement make results more stable, more practical, and more real time. On the other hand, combining different MTTs, shortcomings from the individual method can be made up to each other; (3) a new algorithm will appear continuously. With the progress and innovation of digital image procedure technology and computer vision technology, some more efficient, advanced, and more robust algorithms on the basis of old methods have been presented. These methods can adapt environmental variation, noise, and shadow interference. Also, these algorithms can promote the progress of the application of MTT in practice; (4) more robust, easier, real-time detection results and common applications. We should have knowledge that there are still more challenges in applying these MTTs [7]. For example, if we are urging to reach its real-time function, its accuracy may decrease, and vice versa. The reason of cause of this contradictory phenomenon may be owing to common knowledge that high accuracy usually accompanies with a large calculation procedure which caused worse real-time performance; similarly, good real time will decrease its accuracy due to low anti-interference skill. These contradictory attributes of MTTs lead to the individual method making sense only in a particular scenario. Coping with these contradictory problems of MTTs has been a challenge for researchers in the theory research field.
There are an increasing number of MTTs with the improvement of theory on the basis of traditional MTTs. For instance, the optimal flow method [8], interframe difference method [9], and background subtraction method [10] have been widely used in different research fields due to their respective advantage. The optimal flow method tests objects more accurately and is reasonable for testing a moving target. The most typical algorithm of optimal flow is the Lucas–Kanade (L–K) and Horm–Schunck (H–S) algorithm. With the increasing calculation skill of computers in recent years, a large number of optimal flow technologies have been appeared. For example, Liu et al. [11] presented an optimal flow feature-based robust gait-characterizing method, and they obtained a reasonable conclusion. Zhu et al. [12] provided an L–K improvement-based method and applied it into mileage calculation. Lee et al. [13] measured cable elongation at break based on the improved L–K optical flow method. Interframe subtraction can test and segment a moving target according to video sport information between neighboring frame images because image information between neighboring frames from a video image contains much moving target information. However, we should acknowledge that the original interframe method still has drawbacks. For example, when grayscale and texture in neighboring frames are similar with each other, this method can only obtain the target’s edge contour, target moving information, however, which cannot be completely detected using the original method. Moreover, when the target moves at a high speed, the background occlusion variable area will become large between neighboring frames, which will lead to misjudgment of the occlusion background as the target, which further impacts, to some extent, the feature parameter extraction of target and moving target segmentation. Therefore, many researchers improved the original interframe subtraction method. For instance, Yuan Hang and Wang [14] used 3 interframe subtractions to investigate a moving target. Zheng et al. [15] detected and tracked the human body by combining 3 interframe subtractions with the mean-shift method. The theory of the background subtraction method is similar with the interframe mentioned above; the difference between them is that the background subtraction method need not use neighboring frames, but builds a background reference model which is subtracted from the current image’s frame to detect the moving target. In other words, the selection of the background reference model is vital for completely segmenting moving regions; meanwhile, due to its real-time performance in the video image sequence, the background subtraction method is the most popular method to detect moving target information.
After basketing, the capture of the trajectory brings many difficulties due to the particularity of its movement characteristics. In our study, we used the background subtraction method coupled with an automatic capture system to detect the basketball trajectory for the offsetting efficiency mentioned above; meanwhile, we also applied the optimal flow method and interframe approach to detect basketball trajectory information to compare them with the background subtraction method. The basketball shooting trajectory is parabolic, and the background subtraction method can meet the conditions of basketball trajectory capture in any scene. Traditional methods of capturing the trajectory of basketball shots usually lead to unsatisfactory effects due to improper sample selection. Therefore, in our study, the Gaussian mixture model background difference method is used to improve the traditional method and the accuracy of capturing motion trajectories in complex scenes. The method can effectively improve the accuracy of the automatic capture of basketball shooting trajectories and also the adaptability to complex scenes and improve the limitations of the capture method.
2. Theory of the Algorithm of Background Subtraction, Interframe, and Optimal Flow Methods
2.1. Background Subtraction Method
The core theory of background subtraction is matching the current frame with the reference image from the background model and then calculating the similarity value between the image point and that in the background model. The mathematical expression is as follows:where point (x,y) is the pixel value in any position, x=0,….., H-1,y=0,….,V-1, where H and V represent the horizontal and vertical resolution, respectively, , is the pixel value in the current frame and the pixel value in the background model in the point (x, y), respectively; and T represents the segmentation threshold. When is 1, it means presports attractions are in point (x,y) and 0 means at the background point. If the grayscale value of the background reference point is over that of an unknown point, then it can be regarded as presports attraction, and the background point otherwise.
In order to extract quickly and accurately the position and outlook feature of the basketball from the video/image sequence, a binarization procedure is used for the grayscale image. In this procedure, the selection of segmentation threshold is first in the process of binarization. Currently, there are many methods to accomplish the selection of threshold ( Table 1).
2.1.1. Background Model
As we all know, the scenario environment will change with time moving, so it is necessary to the real-time update background model. In recent years, a lot of methods have been used to update the model (see Table 2). These methods were usually improved and optimized on the basis of original theory.
After constructing the background model and then subtracting each pixel in the video image sequence from the background model built, if the pixel value exists in the same location between the image sequence and background model, the pixel point is regarded as a background point, and as a moving target otherwise. In our design procedure, we first build the background model based on the former m-1 frame from image sequence information and then subtract the current image from the image sequence in the background model, and finally, final image information over the threshold can be obtained. However, the image sequence over the threshold is not complete due to noise; therefore, we need to eliminate its noise using morphology theory. The detailed procedure is shown in Figure 1.
2.2. Interframe Subtraction Algorithm
The interframe subtraction method subtracts pixels between continuous 2 or 3 frames from the video image sequence and then compares those with the threshold preset to extract moving regions from image information. In common situations, excepting for an interested moving target, other objects are static in scenarios of the image sequence. Therefore, variation of parts in the image is only caused by moving parts. However, in fact, the moving target usually exists in a complicated environment, where much noise exists; therefore, we need to eliminate noise like the background subtraction method. The process of the method is shown in Figure 2.
In the detection procedure, is obtained by subtracting (m−1)-th from the m-th frame according to equation (2), and then, if its value is over the threshold, the image value is 1; otherwise, it is 0. The details of the mathematical expression are as follows:
There are many studies combining the background with interframe to detect moving target information.
2.3. Optimal Flow Method
Chen et al. [19] proposed the optimal flow concept in the 1950s; they projected an object in a three-dimensional space on a two-dimensional plane; once the object moved, an optimal flow field was formed in the scenario, where the location of the target can be judged by comparing the variation of pixels between neighboring frames. If the grayscale value of image sequence between neighboring frames in the projected image is unchanged and the image in each frame sequence is continuous, it satisfies the following equation:where represents the grayscale value at position (x, y) and time t.
Taylor’s expansion of equation (3) iswhere , , and represent the partial differential forms of the grayscale value in position (x, y) at time t, respectively. Equation (4) is the moving status of the target we have aimed. The method is applied in many fields by researchers. Zhou et al. [20] deeply analyzed and improved the problem of detecting moving targets by the L–K optical flow method based on the optical flow algorithm.
Some hypotheses for the optimal flow method should be pointed: (1) constant brightness: the brightness value (pixel gray value) of a pixel changed with time, and the color of adjacent frames can remain unchanged. This is the basic setting of the optical flow method; all optical flow methods must meet it; (2) continuous time: continuous “small movement,” time changes will not cause drastic changes in the location. In this way, the gray value change caused by the position change between adjacent frames can be used to obtain the deviation of the gray value to the position. All optical flow methods must meet it; and (3) spatial consistency: the pixels of the same subimage have the same motion.
Of course, we should point these three methods have their own advantages and disadvantages (see Table 3). Comparing the other two methods, the background subtraction method is priorly suggested by a large number of scholars. In our study, we compared the three methods’ performance in basketball shooting trajectory detection to judge the best detection methods for basketball shooting trajectory.
3. Characteristics of Basketball Shooting
In order to detect the basketball shooting trajectory accurately, we first need to know about the feature of basketball shooting; after identifying the characteristics of shooting, the position of the capture device can be further set. The feature can be concluded as follows:(1)The shot will spin after the shot or when the board is scratched(2)The direction of shooting on the board may be right, middle, left, and line of shooting in any positions(3)Under conditions of the shooting goal, the diameter of basketball changes from big to small(4)When the basketball touches the hoop or backboard, the ball will be shaped
3.1. Device Installation
The selection of a capture device is extremely important to accurately detect the basketball shooting trajectory. There are a lot of devices to choose for obtaining data such as cameras and sensor devices. In our study, we chose sensors which were set on the athlete body and board to obtain data of the basketball shooting trajectory. Combining the comparison results with the basketball sports environment and sports characteristics, we chose the sexual sensor to complete the design. It is highly adaptable for environmental factors and low cost, suitable for the basketball environment. Because of its dependence on smaller environmental factors, its robustness is also more stable. Therefore, we used this sensor to complete the design. Among inertial sensors, MEMS sensors have the advantages of wireless transmission, low cost, superior trajectory capture effect, and convenient operation, which are widely used. Therefore, in this design, the MEMS inertial sex sensor completes our design. There are 3 parts in the trajectory capture framework: the trajectory collector, repeater, and server. According to the abovementioned framework, a fixed trajectory collector was installed on the basketball stadium, collecting the trajectory of the basketball movement to complete the capture work. To ensure the stability of the process of the trajectory capture, we designed the repeater. The repeater is responsible for the track processing and forwarding of trace data. In the design process of this repeater, charging is used for power supply to the ARM1176-S core processor, Bluetooth communication transmitter set up, and wireless network communication interface.
Samsung ARM1176 core is used to process the data received by the repeater. The trajectory collector, repeater, and server were connected to form a network of shooting trajectory capture equipment. After this network is set up and fixed, adopting a one-to-many topology, the entire network is divided into 2 layers to ensure the real time and continuity of the capture process.
3.2. Data Preprocessing
In this design process, in order to ensure the real-time capture of trajectory images, inertial sensors were used to capture the trajectory of the shooting on the basis of shooting action. In addition, to install a fixed collector on the backboard, a miniature inertial sensor was installed on the body to ensure the accuracy of trajectory acquisition. According to human kinematics, the corresponding human skeleton model was established on the basis of shooting motion capture. Based on the waist and legs, all the human body postures of the joint take the root node as the origin, and the inertial sensor was placed on the athlete’s forearm, and the corresponding position and relative posture of the human body were used to capture the shooting action. Reading the sensor data of the human body model through the abovementioned settings and according to the movement data of these nodes, the collection time of the backboard collector was adjusted. After a person wears the sensor, in order to ensure the accuracy of motion capture, it is adjusted through the server interface. After setting data, collection results were uploaded in the inertial sensor, driving the repeater, transferring sensor information, and exercise information into the txt file format. The abovementioned data were stored in the server for processing the data.
In the process of basketball sports collection, even the standardized sports process and sophisticated collection equipment cannot directly obtain the data of the shooting trajectory. The data obtained in the abovementioned steps mainly include the real data of human body movement, the data generated by the gravity and material force of basketball, the noise in the sports environment, and the transmission sensor’s inherent zero drift random noise. Methods of noise extraction mainly include field average and median value filter. In our study, we used the field average method to eliminate noise.
Finally, after the preliminary work was carried out, the basketball shooting trajectory can be finished by using three moving target detection methods (background subtraction, interframe subtraction, and optimal flow). The details of the workflow are presented in Figure 3.
3.3. Construction of the Automatic Identification System
In our study, we combined the automatic identification algorithm with the MATLAB application to design the automatic detection system for basketball shooting. Firstly, we identified hardware and software equipment of the system using the environment of software and hardware of the system and then constructed an interactive interface using the GUI tool of MATLAB. Lastly, we evaluated the performance of the automatic identification system constructed using twenty data groups.
Hardware devices mainly include industry cameras and computers. The software environment mainly includes the development environment and operating environment. These are all built through the MATLAB environment.
4. Results and Discussion
4.1. Accuracy Evaluation of the System
We evaluated the accuracy of the system through the shooting goal which contains false detection rate, missed detection rate, and accuracy rate. We totally shoot the basketball 20 times; the results can be found in Table 4. Then, we evaluated the accuracy of the system we built through comparing with the actual condition. Results from Figure 4 show that only the background subtraction model’s accuracy rate reached 90%; both the other two models’ accuracy rates are over 85% (88% for the interframe subtraction model and 85% for the optimal flow model). Therefore, we can infer the ranking of performance from high to low in the basketball shooting trajectory in the background subtraction model, interframe subtraction model, and optimal flow model.
4.2. Simulation Evaluation
In order to ensure the effectiveness of our design and whether it can solve the related problems about the original method, the realization environment is constructed, and the research on its capture effect is completed. A form of comparative experiment is used to verify the capture accuracy of the designed automatic capture method and the original trajectory capture method.
1000 images are acquired through image video sequence acquisition. Region segmentation and feature reorganization of the image and video sequence were performed; the intensity of noise interference received is −8 dB in the progress of image acquisition. To carry out the basketball flight trajectory tracking simulation experiment, we first collected the basketball trajectory data and then eliminated their noise. Lastly, juxtaposed experiment was conducted with trajectory error (Figure 5).
Results from Figure 5 show that the best performance between the 3 models was of the background subtraction model with mean 0.3 error, followed by the interframe subtraction model (0.4) and optimal flow model (0.5).
Computer vision technology is hybrid by image procession, artificial intelligence, and machine vision. In our study, we designed an automatic detection system for basketball trajectory using computer vision technology. However, we should acknowledge that our system still needed to improve due to time and other objective factors; for example, there are false detection and missed detection in our experiment. We only consider one basketball in our experiments; a multitarget problem is needed to consider. DSP and FPGA technology should be considered into the automatic detection system to reduce costs in future work.
5. Conclusions
In our study, we constructed an automatic identification system on the basis of the GUI tool of MATLAB application for basketball shooting trajectory detection based on the background subtraction method. Meanwhile, we also compared the model with the other two models, the interframe subtraction model and optimal flow model. We first performed a comparative experiment: the basketball was practically shot 20 times, and the shooting number and the shooting number of shooting goal were recorded. Industry cameras, sensors, and computers were used to collect data about basketball shooting. The basketball shooting trajectory was judged using the automatic detection system on the basis of the background subtraction method compared to the practical experiment. Moreover, we performed a simulation experiment in order to meet the validity and accuracy of the automatic detection system we have built. Accuracy evaluation with false detection rate, missed rate, and accuracy rate was compared to practical experiment of 20 times shooting. In our simulation experiment, we evaluated the trajectory error rate on the basis of trajectory data sequence from 1000 images which were used to extract the gray pixel feature value. The results show that compared to the other two methods (interframe subtraction method and optimal flow method), the background subtraction method has better accuracy (average 95%) and real-time performance for trajectory detection, is more robust for detecting the target with uncertain moving speed, and is nonrigid. In the simulation experiment, the average trajectory error rate reached 0.3, 0.4, and 0.5 for the background subtraction method, interframe subtraction method, and optimal slow method, respectively.
Data Availability
The experimental data used to support the findings of this study are available from the corresponding author upon request.
Conflicts of Interest
The authors declare no conflicts of interest regarding the present study.