Research Article | Open Access
Shigeru Furuichi, "Characterizations of Generalized Entropy Functions by Functional Equations", Advances in Mathematical Physics, vol. 2011, Article ID 126108, 12 pages, 2011. https://doi.org/10.1155/2011/126108
Characterizations of Generalized Entropy Functions by Functional Equations
We will show that a two-parameter extended entropy function is characterized by a functional equation. As a corollary of this result, we obtain that Tsallis entropy function is characterized by a functional equation, which is a different form that used by Suyari and Tsukada, 2009, that is, in a proposition 2.1 in the present paper. We give an interpretation of the functional equation in our main theorem.
Recently, generalized entropies have been studied from the mathematical point of view. The typical generalizations of Shannon entropy  are Rényi entropy  and Tsallis entropy . The recent comprehensive book  and the review  support to understand the Tsallis statistics for the readers. Rényi entropy and Tsallis entropy are defined by
Throughout this paper, we call a parametric extended entropy, such as Rényi entropy and Tsallis entropy, a generalized entropy. If we take in (1.2), we have the so-called binary entropy . Also we take in (1.2), and we have the Shannon's entropy function . In this paper, we treat the entropy function with two parameters. We note that we can produce the relative entropic function by the use of the Shannon's entropy function .
We note that Rényi entropy has the additivity but Tsallis entropy has the nonadditivity where means that and are independent random variables. Therefore, we have a definitive difference for these entropies although we have the simple relation between them where -exponential function is defined if . Note that we have .
Tsallis entropy is rewritten by where -logarithmic function (which is an inverse function of ) is defined by which converges to in the limit .
Since Shannon entropy can be regarded as the expectation value for each value , we may consider that Tsallis entropy can be regarded as the -expectation value for each value , as an analogy to the Shannon entropy, where -expectation value is defined by
However, the -expectation value lacks the fundamental property such as , so that it was considered to be inadequate to adopt as a generalized definition of the usual expectation value. Then the normalized -expectation value was introduced and by using this, the normalized Tsallis entropy was defined by
We easily find that we have the following nonadditivity relation for the normalized Tsallis entropy:
As for the details on the mathematical properties of the normalized Tsallis entropy, see , for example. See also  for the role of Tsallis entropy and the normalized Tsallis entropy in statistical physics. The difference between two non-additivity relations (1.4) and (1.11) is the signature of the coefficient in the third term of the right-hand sides.
We note that Tsallis entropy is also rewritten by so that we may regard it as the expectation value such as where means the usual expectation value . However, if we adopt this formulation in the definition of Tsallis conditional entropy, we do not have an important property such as a chain rule (see  for details). Therefore, we often adopt the formulation using the -expectation value.
As a further generalization, a two-parameter extended entropy was recently introduced in [11, 12] and systematically studied with the generalized exponential function and the generalized logarithmic function . In the present paper, we treat a two-parameter extended entropy defined in the following form:
for two positive numbers and . This form can be obtained by putting and in (1.13), and it coincides with the two-parameter extended entropy studied in . In addition, the two-parameter extended entropy (1.14) was axiomatically characterized in . Furthermore, a two-parameter extended relative entropy was also axiomatically characterized in .
In the paper , a characterization of Tsallis entropy function was proven by using the functional equation. In the present paper, we will show that the two-parameter extended entropy function can be characterized by the simple functional equation.
2. A Review of the Characterization of Tsallis Entropy Function by the Functional Equation
Proposition 2.1 (see ). If the differentiable nonnegative function with positive parameter satisfies the following functional equation: then the function is uniquely given by where is a nonnegative constant depending only on the parameter .
Proof. If we put in (2.1), then we have . From here, we assume that . We also put then we have
Putting in (2.3), we have
Substituting into , we have
By repeating similar substitutions, we have
Then, we have
due to . Differentiating (2.3) by , we have
Putting in the above equation, we have
By integrating (2.3) from to 1 with respect to and performing the conversion of the variables, we have By differentiating the above equation with respect to , we have Taking the limit in the above, we have thanks to (2.7). From (2.9) and (2.12), we have the following differential equation: This differential equation has the following general solution: where is an integral constant depending on . From , we have . Thus, we have Finally, we have From , we have .
If we take the limit as in Proposition 2.1, we have the following corollary.
Corollary 2.2 (see ). If the differentiable nonnegative function satisfies the following functional equation: then the function is uniquely given by where is a nonnegative constant.
3. Main Results
In this section, we give a characterization of a two-parameter extended entropy function by the functional equation. Before we give our main theorem, we review the following result given by Kannappan [18, 19].
Equation (3.3) was used to prove the uniqueness theorem for two-parameter extended entropy in . As for (3.3), a tree-graphical interpretation was given in . The condition (3.1) can be read as the independent case () in (3.3).
If is an entropic function, then it vanishes at 0 or 1, since the entropy has no informational quantity for the deterministic cases, then the above identity is reduced in the following:
In the following theorem, we adopt a simpler condition than (3.1).
Theorem 3.2. If the differentiable nonnegative function with two positive parameters satisfies the following functional equation: then the function is uniquely given by where and are nonnegative constants depending only on the parameters (and ).
Proof. If we put , then we have due to . By differentiating (3.7) with respect to , we have
Putting in (3.9), we have the following differential equation:
where we put . Equation (3.10) can be deformed as follows:
that is, we have
Integrating both sides on the above equation with respect to , we have
where is a integral constant depending on and . Therefore, we have
By , we have . Thus, we have
Also by , we have .
As for the case of , we can prove by the similar way.
Remark 3.3. We can derive (3.6) from our condition (3.7). Firstly, we easily have from our condition equation (3.7). In addition, we have for ,
Thus, we may interpret that our condition (3.7) contains an essential part of the two-parameter generalized Shannon additivity.
Note that we can reproduce the two-parameter entropic function by the use of as with for simplicity. This leads to two-parameter extended relative entropy  See also  on the first appearance of the Tsallis relative entopy (generalized Kullback-Leibler information).
If we take or in Theorem 3.2, we have the following corollary.
Corollary 3.4. If the differentiable nonnegative function with a positive parameter satisfies the following functional equation: then the function is uniquely given by where is a nonnegative constant depending only on the parameter .
Here, we give an interpretation of the functional equation (3.19) from the view of Tsallis statistics.
Remark 3.5. We assume that we have the following two functional equations for , :
These equations lead to the following equations for :
where and . Taking the summation on and in both sides, we have
under the condition . If the function is given by (3.20), then two above functional equations coincide with two nonadditivity relations given in (1.4) and (1.11).
On the other hand, we have the following equation from (23) and (3.21): By a similar way to the proof of Theorem 3.2, we can show that the functional equation (3.25) uniquely determines the function by the form given in (3.20). Therefore, we can conclude that two functional equations (23) and (3.21), which correspond to the non-additivity relations (1.4) and (1.11), also characterize Tsallis entropy function.
If we again take the limit as in Corollary 3.4, we have the following corollary.
Corollary 3.6. If the differentiable nonnegative function satisfies the following functional equation: then the function is uniquely given by where is a nonnegative constant.
As we have seen, the two-parameter extended entropy function can be uniquely determined by a simple functional equation. Also an interpretation related to a tree-graphical structure was given as a remark.
Recently, the extensive behaviours of generalized entropies were studied in [21–23]. Our condition given in (3.7) may be seen as extensive form. However, I have not yet found any relation between our functional (3.7) and the extensive behaviours of the generalized entropies. This problem is not the purpose of the present paper, but it is quite interesting to study this problem as a future work.
This paper is dedicated to Professor Kenjiro Yanagi on his 60th birthday. The author would like to thank the anonymous reviewers for providing valuable comments to improve the paper. The author was partially supported by the Japanese Ministry of Education, Science, Sports and Culture, Grant-in-Aid for Encouragement of Young Scientists (B) 20740067.
- C. E. Shannon, “A mathematical theory of communication,” The Bell System Technical Journal, vol. 27, p. 379–423, 623–656, 1948.
- A. Rényi, “On measures of entropy and information,” in Proceedings of the 4th Berkeley Symposium on Mathematical Statistics and Probability, vol. 1, pp. 547–561, University California Press, Berkeley, Calif, USA, 1961.
- C. Tsallis, “Possible generalization of Boltzmann-Gibbs statistics,” Journal of Statistical Physics, vol. 52, no. 1-2, pp. 479–487, 1988.
- C. Tsallis, Introduction to Nonextensive Statistical Mechanics, Springer, New York, NY, USA, 2009.
- C. Tsallis, D. Prato, and A. R. Plastino, “Nonextensive statistical mechanics: some links with astronomical phenomena,” Astrophysics and Space Science, vol. 290, pp. 259–274, 2004.
- H. Suyari, “Generalization of Shannon-Khinchin axioms to nonextensive systems and the uniqueness theorem for the nonextensive entropy,” IEEE Transactions on Information Theory, vol. 50, no. 8, pp. 1783–1787, 2004.
- S. Furuichi, “On uniqueness theorems for Tsallis entropy and Tsallis relative entropy,” IEEE Transactions on Information Theory, vol. 51, no. 10, pp. 3638–3645, 2005.
- H. Suyari, “Nonextensive entropies derived from form invariance of pseudoadditivity,” Physical Review E, vol. 65, no. 6, p. 066118, 2002.
- C. Tsallis, R. S. Mendes, and A. R. Plastino, “The role of constraints within generalized nonextensive statistics,” Physica A, vol. 261, pp. 534–554, 1998.
- S. Furuichi, “Information theoretical properties of Tsallis entropies,” Journal of Mathematical Physics, vol. 47, no. 2, p. 023302, 2006.
- G. Kaniadakis, M. Lissia, and A. M. Scarfone, “Deformed logarithms and entropies,” Physica A, vol. 340, no. 1–3, pp. 41–49, 2004.
- G. Kaniadakis, M. Lissia, and A. M. Scarfone, “Two-parameter deformations of logarithm, exponential, and entropy: a consistent framework for generalized statistical mechanics,” Physical Review E, vol. 71, no. 4, p. 046128, 2005.
- E. P. Borges and I. Roditi, “A family of nonextensive entropies,” Physics Letters A, vol. 246, no. 5, pp. 399–402, 1998.
- T. Wada and H. Suyari, “A two-parameter generalization of Shannon-Khinchin axioms and the uniqueness theorem,” Physics Letters A, vol. 368, no. 3-4, pp. 199–205, 2007.
- S. Furuichi, “An axiomatic characterization of a two-parameter extended relative entropy,” Journal of Mathematical Physics, vol. 51, no. 12, p. 123302, 2010.
- H. Suyari and M. Tsukada, “Tsallis differential entropy and divergences derived from the generalized Shannon-Khinchin axioms,” in Proceedings of the IEEE International Symposium on Information Theory (ISIT '09), pp. 149–153, Seoul, Korea, 2009.
- Y. Horibe, “Entropy of terminal distributions and the Fibonacci trees,” The Fibonacci Quarterly, vol. 26, no. 2, pp. 135–140, 1988.
- P. l. Kannappan, “An application of a differential equation in information theory,” Glasnik Matematicki, vol. 14, no. 2, pp. 269–274, 1979.
- P. l. Kannappan, Functional Equations and Inequalities with Applications, Springer, New York, NY, USA, 2009.
- L. Borland, A. R. Plastino, and C. Tsallis, “Information gain within nonextensive thermostatistics,” Journal of Mathematical Physics, vol. 39, no. 12, pp. 6490–6501, 1998.
- C. Tsallis, M. Gell-Mann, and Y. Sato, “Asymptotically scale-invariant occupancy of phase space makes the entropy extensive,” Proceedings of the National Academy of Sciences of the United States of America, vol. 102, no. 43, pp. 15377–15382, 2005.
- C. Tsallis, “On the extensivity of the entropy , the q-generalized central limit theorem and the q-triplet,” Progress of Theoretical Physics, no. 162, pp. 1–9, 2006.
- C. Zander and A. R. Plastino, “Composite systems with extensive (power-law) entropies,” Physica A, vol. 364, pp. 145–156, 2006.
Copyright © 2011 Shigeru Furuichi. This is an open access article distributed under the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.