Advances in Fuzzy Systems

Volume 2015, Article ID 404510, 7 pages

http://dx.doi.org/10.1155/2015/404510

## Quantitative Analyses and Development of a -Incrementation Algorithm for FCM with Tsallis Entropy Maximization

Department of Electrical and Computer Engineering, Gifu National College of Technology, 2236-2 Kamimakuwa, Motosu-shi, Gifu 501-0495, Japan

Received 4 March 2015; Revised 16 July 2015; Accepted 2 August 2015

Academic Editor: Ning Xiong

Copyright © 2015 Makoto Yasuda. This is an open access article distributed under the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.

#### Abstract

Tsallis entropy is a -parameter extension of Shannon entropy. By extremizing the Tsallis entropy within the framework of fuzzy -means clustering (FCM), a membership function similar to the statistical mechanical distribution function is obtained. The Tsallis entropy-based DA-FCM algorithm was developed by combining it with the deterministic annealing (DA) method. One of the challenges of this method is to determine an appropriate initial annealing temperature and a value, according to the data distribution. This is complex, because the membership function changes its shape by decreasing the temperature or by increasing . Quantitative relationships between the temperature and are examined, and the results show that, in order to change equally, inverse changes must be made to the temperature and . Accordingly, in this paper, we propose and investigate two kinds of combinatorial methods for -incrementation and the reduction of temperature for use in the Tsallis entropy-based FCM. In the proposed methods, is defined as a function of the temperature. Experiments are performed using Fisher’s iris dataset, and the proposed methods are confirmed to determine an appropriate value in many cases.

#### 1. Introduction

Statistical mechanics investigates the macroscopic properties of a physical system consisting of multiple elements. In recent years, a popular area of research has been the application of statistical mechanical models or tools to information science.

There exists a strong relationship between the membership functions of fuzzy -means clustering (FCM) [1] and the maximum entropy or entropy regularization methods [2, 3] and the statistical mechanical distribution functions. In other words, FCM, when regularized or maximized with a Shannon-like entropy, yields a membership function that is similar to the Boltzmann (or Gibbs) distribution function [2, 4], and when regularized or maximized with fuzzy-like entropy [5], FCM yields a membership function similar to the Fermi-Dirac distribution function [6]. These membership functions are suitable for annealing methods, because they contain a parameter corresponding to a system temperature. The advantage of using entropy maximization methods is that fuzzy clustering can be interpreted and analyzed from both statistical physical and information-processing points of view.

Tsallis [7] achieved a nonextensive extension of Boltzmann-Gibbs statistics by postulating a generalized form of entropy with a generalization parameter , which, in the limit as goes to , approaches the Shannon entropy. Tsallis entropy is applicable to numerous fields, including physics, chemistry, bioscience, networks, and computer science, and it has proved to be useful [8–10]. For example, Tsallis entropy can be applicable for attribute selection in network intrusion detection [11]. It also can be utilized as an optimization function of thresholding image segmentation [12]. In [13, 14], Menard et al. discussed fuzzy clustering in the framework of nonextensive thermostatistics. By taking the possibilistic constraint into account, the possibilistic membership function was derived, and its properties were considered from various viewpoints.

On the other hand, based on the Tsallis entropy, another form of entropy (or a measure of fuzziness) for a membership function can be defined. A form of the membership function can then be derived by extremizing (maximizing) this entropy within the framework of FCM [15]. In comparison with the conventional entropy maximization methods [2, 3], this method yields superior results [15].

Deterministic annealing (DA) [4] is a deterministic variant of simulated annealing, and it can be applied to clustering [16]. By applying DA to FCM using Tsallis entropy, a DA-FCM algorithm using Tsallis entropy has been developed [15]. As for another application example of DA, in [17], the -parameterized DA expectation maximization algorithm is proposed.

One of the important characteristics of the membership function of this method is that centers of clusters are given as a weighted function of the membership function to the power of . We also note that it changes its shape in a similar way by decreasing the system temperature (or annealing) or by increasing . However, it remains unknown how appropriate value and initial annealing temperature should be determined according to the data distribution.

The purpose of the present study is to overcome the above problem, which involves quantitative analyses of the relationships between the temperature and , and to develop -incrementation algorithms by integrating and the temperature.

The analyses show that the temperature and affect almost inversely. Based on these results, we developed two kinds of -incrementation algorithms for Tsallis entropy-based FCM, in which is defined as a function of the temperature. These algorithms are compared with the conventional Tsallis entropy-based DA-FCM method.

In the first algorithm, is increased so as to maintain similar shapes of with the conventional -reduction method. In the second algorithm, is defined as an inverse of a decreasing pseudo-temperature.

Experiments are performed using Fisher’s iris dataset [18], and it was confirmed that, in many cases, appropriate value is determined automatically from the temperature. Furthermore, the proposed methods improve the accuracy of classification and are superior to the conventional method.

However, it was also found that the number of computation iterations depends on , and sometimes it becomes greater than that of the conventional method; this suggests that should be optimized to some extent.

#### 2. Entropy Maximization Method

Let () be a data set in -dimensional real space, which is to be divided into clusters. In addition, let () be the centers of the clusters, and let () be the membership functions. Furthermore, letbe the FCM objective function that is to be minimized.

##### 2.1. Entropy Maximization for FCM

The Tsallis entropy is defined aswhere is the total number of microscopic possibilities of the system.

Based on (2), the entropy (or a measure of the fuzziness) of a membership function is defined as follows:

The objective function can be written as

Under the normalization constraint ofthe Tsallis entropy functional is given bywhere and are the Lagrange multipliers and must be determined so as to satisfy (5).

By extremizing (6) with respect to , the stationary condition yields the following membership function:where

In the same way, the center of the cluster is given by

and satisfywhich leads toBy analogy with statistical mechanics, this relationship makes it possible to regard as the internal energy and as an artificial system temperature [19].

#### 3. Dependencies of on Temperature and

In (9), works as a weight value to each , and it determines . In this paper, for simplicity, is set to be . This makes the denominator of (7) become the sum of the same forms of its numerator. In Figures 1(a) and 1(b), the numerator of is plotted as a function of , parameterized by and , respectively. In these figures, in order to examine the shape of (the subscript is omitted in this formula) as a function of the distance between the center of the cluster and various data points, is considered to be a continuous variable .