Complexity

Volume 2018, Article ID 8730281, 11 pages

https://doi.org/10.1155/2018/8730281

## Large-Screen Interactive Imaging System with Switching Federated Filter Method Based on 3D Sensor

^{1}School of Mechanical and Electric Engineering, Soochow University, Suzhou, China^{2}Key Laboratory of Intelligent Perception and Systems for High-Dimensional Information of Ministry of Education (Nanjing University of Science and Technology), Nanjing, China^{3}Collaborative Innovation Center of Industrial Energy-Saving and Power Quality Control (Anhui University), Hefei, China

Correspondence should be addressed to Lei Yu; nc.ude.adus@iel_uy

Received 8 August 2018; Revised 5 November 2018; Accepted 22 November 2018; Published 4 December 2018

Academic Editor: Carlos F. Aguilar-Ibáñez

Copyright © 2018 Lei Yu and Junyi Hou. This is an open access article distributed under the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.

#### Abstract

Large-screen human-computer interaction technology is reflected in all aspects of daily life. The dynamic gesture tracking algorithm commonly used in recent large-screen interactive technologies demonstrates compelling results but suffers from accuracy and real-time problems. This paper systematically addresses these issues by a switching federated filter method that combines particle filtering and Mean Shifting algorithms based on a 3D sensor. Compared with several algorithms, the results show that the one-hand and two-hand large-screen gesture tracking based on the switched federated filtering algorithm work well, and there is no tracking failure and loss of target. Therefore, the switching federated tracking positioning algorithm can be well applied to the design of human-computer interaction system and provide new ideas for future human-computer interaction.

#### 1. Introduction

The definition of human-computer interaction technology is based on a certain software program through the corresponding input and output devices, and the organic combination of computer and human operations is employed to realize the related technology of interactive communication [1–17]. The current large-screen human-computer interaction technology is mainly divided into the following categories: (1) infrared frame touch interaction; (2) capacitive touch-based electronic screen touch interaction; (3) speech recognition interaction; (4) computer vision interaction. These types of technologies have their own advantages and disadvantages in terms of cost, accuracy, interactivity, etc. [3–7]. Touch interactions based on infrared frame and capacitive touch-based electronic screens are all based on interactive hardware fixed at a certain place. Therefore, not only are such man-machine interaction operations costly, but also the range and accuracy of the interaction are affected by the hardware constraints [4–8]. Also, interactions based on speech recognition are often limited by complex noisy environments which are mostly used in the small-scale civil-computer interaction field and therefore have a smaller application area. The human-machine interaction system based on machine vision is mainly based on sensors and other devices to collect images or signals and then completes the signal preprocessing operations on the image signals collected by the sensors or digital signals through an independently designed human-computer interaction software. The targets to be tracked are segmented from the background, and a series of operations such as target tracking and motion recognition are used to complete the corresponding human-computer interaction [4, 7–11]. Therefore, the cost of human-computer interaction system designed based on computer vision is relatively low, and the interaction effect can be well obtained.

Hand gesture recognition methods based on computer vision are mainly divided into two categories: one is an analysis method based on a 3D model, and the other is based on a 2D image. The analysis method with the 3D model needs to establish a parametric model that describes gestures. Because it can provide 3D data, a more accurate gesture model can be established. However, the method has many parameters and high computational complexity, and it is difficult to achieve real-time results in current technologies. Also, with the two-dimensional image method, it is mainly to analyze the performance of the image and extract the effective hand features for identification. Because of the loss of the third 3D space information, the gesture model cannot be effectively established for the described features. Besides, its parameters are less, so it can meet the requirements of real-time processing. These two methods can hardly guarantee the balance between system parameters and real-time performance.

This paper proposes a large-screen interactive imaging system with switching federated filter method based on 3D sensor and validates the tracking results with the independently developed gesture position tracking platform. The main innovations of this paper are as follows: (1) An improved switching federated filter algorithm combining particle filtering and Mean Shift is introduced into the field of large-screen gesture tracking to track dynamic gestures; (2) the self-developed gesture tracking platform and 3D interactive software are combined to observe the gesture tracking effects; (3) the large-screen single-hand and two-hand gesture tracking based on the switching federated filtering algorithm work well and there is no phenomenon of tracking failure and losing the target.

The structure of this paper is as follows: An improved switching federated filter algorithm combining particle filtering and Mean Shift is employed to track and locate the dynamic gestures in Section 2. Section 3 presents the dynamic gesture tracking and positioning experiment verification based on switching federated filter method. Finally, conclusion is given in Section 4.

#### 2. Dynamic Gesture Tracking Algorithm with Switching Federated Filter

Common gesture tracking algorithms cannot handle the dynamic gesture tracking problem in complex environments. This paper has developed an improved switching federated filter algorithm that combines particle filtering and Mean Shift algorithm. In the case of slow movement of the human hand, after the uniform displacement of the particles, the particles drift to the dynamic gesture area except for a small number of particles. The gesture position can then be obtained without subsequent prediction of the particles, so that the running time of algorithm can be saved. The average value of the particles will drift when the movement of the hand is fast or there is occlusion. If the region of the gesture cannot be detected, the particle will return to the condition before the drift, waiting for the algorithm to perform corresponding processing on the next frame. Therefore, how to select the corresponding filtering algorithm under different conditions is the key problem. In this paper, the switching system scheme is introduced into the filter for the first time, and applied to the large-screen interactive imaging system.

A switching system consists of a series of sequential or discrete differential equations subsystems and switching rules or a switching strategy [4, 18–34]. If the entire filtering process is regarded as a hybrid dynamic system, then each filter algorithm can be regarded as a subsystem of the system. The proposed system is constructed by coupling the switching system with arbitrary switching rules and using two subsystems composed of arbitrary switching filter system. In the case when the movement of the human hands is fast, the filter subsystems of particle filter algorithm will be chosen; when the human hand moves slowly, the filter subsystems of Mean Shift algorithm will be selected. The meaning of the switching signal iswhere denotes that the -th subsystem is switched on at and the -th subsystem is switched off at . Thus, in finite time, the switching sequence is finite, and there exists no state transition during the switching moment, where is the initial time and is the -th switching time. When , the trajectory of the switched nonlinear system is produced by the -th subsystem, defining , as the minimum interval time of the -th subsystem.

The current common particle filter implementation framework is based on resampling and sequential importance sampling, which can be called sampling importance resampling filter [35–37]. Therefore, it mainly consists of three basic operation steps to form an iterative cycle as follows:

*Step 1 (sampling). *Based on the Bayesian posteriori estimation and state transfer equation at the previous moment to achieve the purpose of updating the particle state, the predicted distribution (i.e., prior distribution) can be achieved.

*Step 2 (weight update). *Based on the latest observed information , employ the likelihood function to perform a corresponding weight update operation for the weights:where represents the weight of the particle, represents the importance distribution function, and represents the posterior probability density.

*Step 3 (resampling). *Based on the principle of identity distribution, the resampling operation is performed on the updated particle combinations after the weights update operation is completed. A new set of particles with most of the particle weights can then be obtained. The number of times that particles weights with nonbiased weight values are sampled is expected to be

Meanwhile, the Mean Shift algorithm as the other filter subsystems is a process that uses nonparametric density estimation, which was used to perform iterative search based on feature spatial gradient directions and then obtain sample data with local density maximum [12–14]. Compared with other tracking methods based on optimized matching search, the advantage of the Mean Shift filter algorithm is that the method does not need to know the characteristics of the feature space in advance; the only requirement is the relevant sample data. The filtering is performed according to these sample points. Therefore, this kind of algorithm has less computational complexity. The implementation of the algorithm is not difficult, and the algorithm has strong real-time performance to meet the systems with high real-time requirements. In the Mean Shift algorithm, the most critical thing is to calculate the offset mean for each point and then update the position of the point based on the newly calculated offset mean.

For n sample points , in a given d-dimensional space , the basic form of the Mean Shift vector for point x is

where is a high-dimensional sphere with a radius of h. is defined as

The form of Mean Shift has a fundamental problem: in the region of ,, each point has the same effect on . In fact, this effect is related to the distance between and each point. The importance of different samples is different. Aiming at the abovementioned considerations, the kernel function and the sample weight are added to the basic Mean Shift vector form, and the following modified Mean Shift vector form is obtained:

Among them,

is a unit of kernel function. is a positive definite symmetric matrix, called the bandwidth matrix, which is a diagonal matrix. is the weight of each sample. The form of the diagonal matrix is

The above Mean Shift vector can be rewritten as

Mean Shift vector is a normalized probability density gradient. In the Mean Shift algorithm, the probability density is actually used to obtain the local optimal solution of the probability density. For a probability density function , the known d-dimensional space of n sampling points , the kernel function of is estimated:

where is a weight assigned to the sample point and is a kernel function. The estimate of the gradient of the probability density function is

Let and ; there are

Among them, the second square brackets mean the Mean Shift vector, which is proportional to the probability density gradient. Mean Shift vector correction results are as follows:

Considering , then the abovementioned formula becomes

The process after the integration of the specific algorithm is as follows.

*Step 1. *In the initial frame, all the particles are distributed in the gesture area according to the Gaussian distribution.

*Step 2. *In the next frame, all particles are mean shifted, the Pasteurian after shifting is taken as the weight of the particle, the weights are normalized, and the number of effective particles is calculated.

*Step 3. *Determine whether the number of effective particles is greater than the threshold .

*Step 4. *If the effective number of particles is greater than the set threshold , the target area is in the particle distribution area. Then, all the particles are sorted by weight from the highest to the lowest. Pick a particle whose weight is greater than the threshold to calculate the location of the gesture area.

*Step 5. *Define the seed as the center of the region calculated in tep 4, the area growth is segmented, the ellipse model of the gesture is established with the separated area, and resample the significance of the particles.

*Step 6. *If the effective number of particles is less than the set threshold , particles are spread by using a four-week Gaussian distribution.

*Step 7. *Use Mean Shift algorithm on all particles, the Pasteurian after shifting is taken as the weight of the particle, the weights are normalized, and the number of the effective particles is calculated.

*Step 8. *If the effective number of particles is greater than the set threshold , go to tep 4.

*Step 9. *If the effective number of particles is less than the set threshold , determine whether the particle weight has increased.

*Step 10. *If the weight of some particle increased, it means that a small part of the particles have spread to the gesture area. Resample the significance of the particles and then go to tep 7. If the particle weight do not increase, and the particles spread to the surrounding less than 3 times, then go to tep 6. If they have been spread 3 times, it is considered that there is no target area in the frame, the particles are restored to the state before being shifted, and then go to the next frame.

The algorithm flowchart is shown in Figure 1.