Mathematical Problems in Engineering

Volume 2018, Article ID 7974340, 13 pages

https://doi.org/10.1155/2018/7974340

## Semantic Segmentation of Human Model Using Heat Kernel and Geodesic Distance

Correspondence should be addressed to Huanyu Yang; moc.361@77uynauhy

Received 14 September 2017; Accepted 18 January 2018; Published 20 February 2018

Academic Editor: Simone Bianco

Copyright © 2018 Huanyu Yang et al. This is an open access article distributed under the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.

#### Abstract

A novel approach of 3D human model segmentation is proposed, which is based on heat kernel signature and geodesic distance. Through calculating the heat kernel signature of the point clouds of human body model, the local maxima of thermal energy distribution of the model is found, and the set of feature points of the model is obtained. Heat kernel signature has affine invariability which can be used to extract the correct feature points of the human model in different postures. We adopt the method of geodesic distance to realize the hierarchical segmentation of human model after obtaining the semantic feature points of human model. The experimental results show that the method can overcome the defect of geodesic distance feature extraction. The human body models with different postures can be obtained with the model segmentation results of human semantic characteristics.

#### 1. Introduction

The 3D human model is widely used in anthropometry, clothing design, virtual human animation, game, and so on. The corresponding 3D human point cloud model is obtained by the 3D scanner, and then the data is represented as a grid or a surface by method for 3D reconstruction. Model segmentation is the basis of shape analysis. Segmentation of human model is different from other 3D models because of the particularity of the human body; the segmentation of the human model should accord with the human body semantic knowledge. In order to obtain the semantic knowledge of the human body, such as the definition of semantic knowledge related to hand, arm, head, legs, and trunk, it is necessary to obtain the model’s feature points automatically, such as head vertex, neck point, and perineum point.

Geodesic distance is a widely used method during detecting human feature points. However, facing the model topology changes, such as the case of hands in hands, geodesic distance method is unable to detect the feature points of the hand. The method to determine the characteristics of human joints according to the proportion of the human body anthropometric value may lead to deviation because the human body model does not meet the standard ratio. Heat kernel signature is a method based on attribute representation model of Laplace-Beltrami operator and heat kernel function. It limits the variables in the time domain which can fully express geometric characteristics of the model. Also it has affine invariance and deformation invariance.

In this paper, we propose a segmentation method based on heat kernel signature and geodesic distance. Our key idea is to discover semantic feature points by computing the heat kernel signature of human model. We search the partial maxima of the thermal energy distribution of the model and obtained a set of feature points of the model by calculating the heat kernel signature of the point cloud of human body model. We adopt the method of geodesic distance to realize the hierarchical segmentation of human model after obtaining the semantic feature points of human model. The experimental results show that the algorithm can be used to solve the feature points that geodesic distance method cannot defect. And the human body model with different postures can get the segmentation result which accords with the human semantic feature.

The paper is organized as follows. In Section 2, we review previous works on shape segmentation and 3D human segmentation, as well as the application of heat kernel descriptors and geodesic distances. The details of heat kernel function and heat kernel signature are given in Section 3. Model landmarks definition is described in Section 4. Our algorithm is presented in Section 5. Then the experiment and the result are presented in Section 6. Eventually, in Section 7, the conclusion and the possible future work are given.

#### 2. Related Work

##### 2.1. Shape Segmentation

A large amount of work has been done on shape segmentation in the context of shape analysis. According to the similarity of image segmentation, the 3D model segmentation algorithm is based on the ideas and terminology of image segmentation algorithm in a great extent. At present, the main method of point cloud segmentation is as follows.

*(**1) Method Based on Edge Detection*. Edge based detection is a feature-based approach. The geometric features such as the characteristic curve or normal vector and curvature are used as the boundary conditions of the segmentation by calculating to obtain the feature points. The effect of segmentation is largely determined by the results of boundary detection. It is easy to be affected by the noise point, which leads to poor positioning accuracy. Woo et al. [1] used octree to organize grid structure. Also the normal deviation is used as the basis of grid subdivision and feature extraction. Guillaume et al. [2] proposed a method that is similar to curvature region to construct adjacency graph and ensured the effective segmentation of the patches at the “edge.” To achieve the segmentation of the model by calculating the thermonuclear energy point and heat dissipation point of each point of model, Rodrigues et al. [3] proposed a triangular mesh segmentation method based on contour. The main idea is to find the contours of each region first and then determine and collect all of their intrinsic triangles.

*(**2) Method Based on Region Growing*. The segmentation algorithm based on region growing includes two approaches: bottom-to-up and top-to-down. The bottom-to-up algorithm is commonly referred to as the region growing algorithm. Select the seed point first and spread out from seed until there is no continuous set of points in the neighborhood, judging whether the surrounding neighborhood points belong to a surface. Finally, combine the neighborhood composition regions. The key of the algorithm is regional growth strategy. The top-to-down algorithm is also called hierarchical decomposition algorithm. Supposing that all the points belong to the same patch, the decomposition results of different levels of detail are obtained by using the hierarchical subdivision of octree [4] and hierarchical directional bounding box tree (OBBTree) [5].

*(**3) Method Based on Clustering*. The segmentation algorithm based on clustering regards the segmentation of point cloud model as the classification process of data points with certain characteristic parameters. The coordinates of the point cloud data, normal vector, curvature value, heat kernel function value, and so on may be characteristic parameters of clustering analysis. -means clustering method is used to realize the segmentation of point cloud model in [6]. Katz and Tal [7] use fuzzy clustering method to achieve the segmentation of point cloud model. Zhang et al. [8] adopt the method of hierarchical clustering to achieve the segmentation of point cloud model. Hilaga et al. [9] achieve the segmentation of the model through clustering the curvature of the grid. Skraba et al. [10] calculated the model heat kernel function value using the method of persistent clustering to cluster analysis, which obtained mesh segmentation algorithm of multiscale isometric invariant. Chen et al. [11] obtained the heat kernel signature value of 3D human body model vertex by calculating, then segmenting the model initially by setting segmentation threshold. Gadi [12] realized the grid segmentation of the point cloud model by using minimum rules and spectral clustering.

*(**4) Method Based on Topological Structure*. It can be boiled down to Reeb graph [13, 14], level set method [15], shock graph [16], Laplace-Beltrami operator [17], and so on. based on the shape segmentation method of geometry and topology information. Although they can accurately calculate the global shape features, they cannot effectively solve the problem of preserving local details. Yao et al. [15] proposed a partitioning method for 3D printing model in the framework of multiphase level set. Firstly, six indexes are constructed to evaluate the partition quality: stress load, surface detail, interface aspect, packing size, printing ability, and assembly ability. Then according to the analysis index, we use the level set method to improve the quality of the partition on this basis.

*(**5) Method Based on Online Learning*. In recent years, online learning methods have also been applied in model segmentation. Zhang et al. [18] put forward online learning method to realize shape classification. Several online random forests are used as the training items for weak learning through online multiclass linear programming enhancement algorithm. The training model is regarded as a data item and graph cut optimization algorithm is used to realize the segmentation of 3D model. Shu et al. [19] achieved unsupervised 3D shape segmentation and collaborative segmentation through the depth of learning methods.

In addition to the above several methods, we have the classic model segmentation method as well as personal rank algorithm [21], hierarchical mesh segmentation based on fitting primitive [22], normalized segmentation [23], and so on. In recent years, the research on cosegmentation [24–26] has gradually become the hotspot of model segmentation. But in this paper, we do not relate to the content of this research.

##### 2.2. 3D Human Segmentation

The characteristics of human body are similar to those of other kinds of 3D models. Some of the characteristics of the human body are related to changes in the partial geometric quantities (such as normal vector and curvature). However, unlike other types of models, some human characteristics are used in the measurement of garment CAD. Allen et al. [27] adopt the feature points of template matching and positioning; it does not have generality because its accuracy depends on the similarity between the input model and the template. Wuhrer et al. [28] extend the literature [27] method, using a large number of labeled 3D human models to construct a statistical model and detect the feature points of human body of the model. Kalogerakis et al. [29] used conditional random fields for sample learning to labeling and shape segmentation. In the research of human inspect feature points, many existing researches adopt template matching method where its accuracy depends largely on the similarity between the input model and the template. However, the algorithm proposed by researchers in shape feature analysis is suitable for any model without considering human specific domain knowledge. So the segmentation results may be inconsistent with the human structure hierarchy. Reeb graph is a typical shape feature description algorithm that has been used for 3D human body segmentation and feature points extraction [13, 30]. Werghi et al. [13] proposed a 3D human body segmentation algorithm based on Reeb graph, constructing the Reeb graph by selecting the geodesic distance function as the Morse function. The method has robustness to changes in body shape and attitude. But the drawback is the large amount of calculation and the low algorithm efficiency. According to the characteristics of the key points such as axillary points, locating the segmentation position is based on feature method [7, 31–33]. It can achieve the fast segmentation of point cloud model. EU virtual human project [34] uses multiscale fuzzy morphological analysis to locate and segment human body feature points. However, the concave convex geometric features make it difficult to ensure that the segmentation results are consistent with the semantic features of the human body.

##### 2.3. Application of Heat Kernel Signature and Geodesic Distances

HKS (heat kernel signature) is a method of using spectral theory to characterize the properties attribute of the model. The descriptor is derived from the heat transfer theory. It can fully express the geometric characteristics of the model and has affine invariance and deformation invariance. Heat kernel signature is widely used in 3D point cloud segmentation, retrieval, model reconstruction, and so on. In Section 3, the details of heat kernel function and heat kernel signature are given.

In the research field of 3D point cloud model retrieval, it is used for multiscale matching of nonrigid shapes because the heat kernel signature has deformation invariance [35, 36]. Ovsjanikov et al. [37] used HKS to achieve a single point correspondence between the models. Gȩbal et al. [38] used the Laplace-Beltrami operator to construct an ADF (Auto Diffusion Function) similar to HKS to realize the shape analysis of 3D point cloud model. Lovato et al. [20] used ADF to achieve the feature points analysis of the 3D human point cloud model upon the method of [38]. In the research field of 3D point cloud model segmentation, Benjamin et al. [39] calculate the continuous heat kernel value of 3D model. Greedy algorithm is used to analyze the area of heat kernel energy storage, which is the large difference in heat kernel value of the point and its neighbor point. The method of relative entropy analysis is used to obtain the area of thermonuclear diffusion region, which is the area of thermonuclear value with uniform distribution to obtain the final fragmentation of 3D model. Skraba et al. [10] used the method of persistent clustering to analyze the model. Heat kernel function value was calculated to obtain the multiscale invariant mesh segmentation algorithm.

In the process of selecting feature points, the widely used method is geodesic distance. The fuzzy hierarchical segmentation algorithm proposed by Katz and Tal [7] adopts geodesic distance to detect feature points. The proposed branch segmentation algorithm in [31], multidimensional scaling, and geodesic distance have been used to detect feature points. The method of using geodesic distance to obtain the feature points of human body is relatively simple, but there are problems too, for example, the precision analysis of geodesic distance and topology change of human body model. If the two hands are together, the geodesic distance method cannot get the characteristic points of the hand.

#### 3. Heat Kernel Function and Heat Kernel Signature

##### 3.1. Definition of Heat Kernel Signature

Heat kernel operator and heat kernel function are from the thermal diffusion theory of Riemann manifold. The heat equation is used to describe the variation of heat distribution with time. For compact Riemann manifolds , the thermal diffusion process can be described by the heat equation [35].where is the Laplacian operator over a Riemannian manifold , is the heat distribution at a point and at time , and is thermal diffusivity (typically ). Given has boundary, it should meet the Dirichlet boundary condition, for all point at time , .

Given an initial heat distribution function , where is the heat distribution at time , when approaches to zero, , is called heat kernel operator. and satisfy . These two operators have the same feature vector; their eigenvalues are and , respectively. Therefore, for any manifold , there is a function such that the following equation holds.which satisfies the smallest function called heat kernel function. Heat kernel equation is the fundamental solution to the heat equation on a specified domain with appropriate boundary conditions; its physical significance is the amount of heat transferred from to at time . Because the conduction of heat is associated with surface shape of the model, heat kernel function built over the surface of model is relevant to the model’s geometric properties. It is also one of the main tools in the study of the properties of the Laplace operator over Riemannian manifold . As a heat kernel function over a compact Riemannian manifold , the eigen decomposition of it is expressed aswhere is the eigenvalue. And is the feature vector of Laplace-Beltrami operator, respectively.

Paper [35] puts forward a method to simplify heat kernel function, that is, to build HKS according to heat kernel function and to confine the vector of heat kernel function in a specified time domain and sample it over the vertex on the model to build a heat kernel function vector. This vector is called HKS (heat kernel signature).

For vertex over Riemannian manifold , its HKS can be described as

Heat kernel takes up most of heat kernel function’s merit property. But in order to reduce the complexity of heat kernel and reduce redundant information, it converts two vectors in heat kernel function into one, making its HKS more concise and easy to calculate and compare.

##### 3.2. Properties of Heat Kernel

Heat kernel has many good geometric properties, such as invariant isometric transformation, affine invariant, multiple scale, and stability. The following are some of the main properties of heat kernel.

###### 3.2.1. Invariant Isometric Transformation

Heat kernel is invariant under isometric transformations. Given as the isometric transformation from Riemannian manifold to , where all and all ,With isometric transformation between manifold and manifold , the property of heat kernel remains invariant; therefore, heat kernel is only determined by the inner geometric property of manifold.

###### 3.2.2. Affine Invariant

From (5) you can see that metric spaces and are isometric isomorphism. According to Matsu-Magnolia theorem, isometric isomorphism on normed vector spaces whose coefficients are real numbers must be affine transformations. Therefore, heat kernel has in addition to offset thermonuclear metric invariance, with affine invariance.

Due to the above two characteristics of heat kernel, it has nothing to do with attitude of the model. Therefore, heat kernel is widely used in model retrieval [20], model block with independent attitude [10, 39], skeleton extraction of the model with independent attitude [37], and other fields.

###### 3.2.3. Multiple Scale Property

The vertex of model on heat kernel is a function that associated with diffusion time . When the value of time is very small, the value of heat kernel mainly depends on the geometrical characteristics of the small adjacent area. With the growth of time, the scope of the adjacent region will grow. That means, when the value of is small, heat kernel reflects the characteristics of the partial geometric model. When the value of is large, it reflects global geometric properties.

The ADF [38] method was used to extract the feature points of human model in literature [20]. The ADF method is developed by the heat kernel (HKS). It is also associated with the diffusion function of time similar to heat kernel. The influence of the value of on the extraction of human feature points is analyzed in literature [20]. When the value is larger, because the ADF value reflects the global characteristics of the model, the detection of the human body is less, but the accuracy is high. When the value is small, because the ADF value reflects the partial characteristics of the model, the detection of the human body is more, but the accuracy is lower.

In the analysis of algorithm experimental results of Section 5.1, in order to consider the partial characteristics of the human model and the influence of the global characteristics on the feature points, we carried out a number of values to and the corresponding analysis.

###### 3.2.4. Stability

Stability means heat kernel function is stable over the noise of model, insensible to weak disruption. Heat kernel equates to the average length of the whole path from to with time . When there is noise over the model, only those passing the noise would be affected; therefore, heat kernel is relatively stable.

Both [10, 39] analyzed the stability of heat kernel function, adopted heat kernel function to divide the model, and introduced noise to the model. Basically, it would not affect the result of destructing the model.

##### 3.3. Calculation of Heat Kernel Signature

Heat kernel function has the same eigenvector and eigenvalue as , the operator of Laplace-Beltrami. Therefore, the solution of heat kernel signature becomes the solution of the discretization and feature system of the Laplace-Beltrami operator. HKS algorithm is detailed as follows.

*Step 1 (the discretization of the Laplace-Beltrami operator). *Let manifold be the triangle net discretization, shown as . is the vertex on the model; is the edge of the model. The discrete Laplace-Beltrami operators are usually shown as follows:where is the set of the neighboring vertexes of vertex and is the edge weight of and . As heat diffuses along the net according to the change of time, a suitable edge weight can control the pace of heat diffusion well and also can counteract the effects that irregular sampling caused to the discretization. Equation (4) can be expressed in the form of a matrix as follows.If is the uniform operator of Laplacian, the settings for cotangent weighting method [40] applied in this article are the following:where and are the two triangles which share the edges and is the area of the triangle . and are the diagonals of the two triangles which share the edges .

*Step 2 (calculation of eigenvalue and eigenvector ). *Characteristic system of satisfies the following equation: where is the diagonal matrix of ; the diagonal elements are eigenvalue ; is the matrix of ; the eigenvectors of are .

*Step 3 (calculation of the heat kernel signature value). *Equation (4) can be expressed in the form of a matrix as follows:where , . The heat kernel signature of each matrix of the model can be calculated through (10).

#### 4. Model Landmarks Definition

##### 4.1. Definition of Human Body Model Semantic Feature Point

According to different data processing needs, human body model has different semantic feature point definitions. Inside the human body model, human body level structures are basically decided by the semantic feature points of body parts such as perineum point and waist line point. Human body model semantic feature point definitions are shown in Table 1. The corresponding position on the human body model of each feature point is displayed in Figure 1.