- About this Journal ·
- Abstracting and Indexing ·
- Aims and Scope ·
- Annual Issues ·
- Article Processing Charges ·
- Articles in Press ·
- Author Guidelines ·
- Bibliographic Information ·
- Citations to this Journal ·
- Contact Information ·
- Editorial Board ·
- Editorial Workflow ·
- Free eTOC Alerts ·
- Publication Ethics ·
- Reviewers Acknowledgment ·
- Submit a Manuscript ·
- Subscription Information ·
- Table of Contents
Abstract and Applied Analysis
Volume 2014 (2014), Article ID 505184, 8 pages
A Generalization of the Havrda-Charvat and Tsallis Entropy and Its Axiomatic Characterization
1Department of Mathematics, College of Natural Sciences, Arba Minch University, Arab Minch, Ethiopia
2Department of Applied Sciences, Maharishi Markandeshwar University, Solan, Himachal Pradesh 173229, India
Received 3 September 2013; Revised 20 December 2013; Accepted 20 December 2013; Published 19 February 2014
Academic Editor: Chengjian Zhang
Copyright © 2014 Satish Kumar and Gurdas Ram. This is an open access article distributed under the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.
In this communication, we characterize a measure of information of types , , and by taking certain axioms parallel to those considered earlier by Havrda and Charvat along with the recursive relation (; , , ), ; , , ) (()), ; ,()), ; , ), , , , . Some properties of this measure are also studied. This measure includes Shannon’s information measure as a special case.
Shannon’s measure of entropy for a discrete probability distribution given by has been characterized in several ways (see Aczél and Daróczy ). Out of the many ways of characterization the two elegant approaches are to be found in the work of (i) Faddeev , who uses branching property namely, for the above distribution , as the basic postulate, and (ii) Chaundy and McLeod , who studied the functional equation Both of the above-mentioned approaches have been extensively exploited and generalized. The most general form of (4) has been studied by Sharma and Taneja , who considered the functional equation We define the information measure as for a complete probability distribution .
Measure (6) reduces to entropy of type (or ) when (or ) given by When , measure (7) reduces to Shannon’s entropy , namely, The measure (7) was characterized by many authors by different approaches. Havrda and Charvát  characterized (7) by an axiomatic approach. Daróczy  studied (7) by a functional equation. A joint characterization of the measures (7) and (8) has been done by author in two different ways. Firstly by a generalized functional equation having four different functions and secondly by an axiomatic approach. Later on Tsallis  gave the applications of (7) in Physics.
To characterize strongly interacting statistical systems within a thermodynamical framework—complex system in particular—it might be necessary to introduce generalized entropies. A series of such entropies have been proposed in the past, mainly to accommodate important empirical distribution functions to a maximum ignorance principles. The understanding of the fundamental origin of these entropies and its deeper relations to complex systems is limited. Authors  explore this question from first principle. Authors  observed that the 4th Khinchin axiom is violated by strongly interacting system in general and by assuming the first three Khinchin axioms derived a unique entropy and also classified the known entropies with in equivalence classes.
For statistical system that violates the four Shannon-Khinchin axioms, entropy takes a more general form than the Boltzmann-Gibbs entropy. The framework of superstatistics allows one to formulate a maximum entropy principle with these generalized entropies, making them useful for understanding distribution functions of non-Markovian or nonergodic complex systems. For such systems where the composability axiom is violated there exist only two ways to implement the maximum entropy principle; one is using the escort probabilities and the other is not. The two ways are connected through a duality. Authors  showed that this duality fixes a unique escort probability and derived a complete theory of the generalized logarithms and also gave the relationship between the functional forms of generalized logarithms and the asymptotic scaling behavior of the entropy.
Suyari  has proposed a generalization of Shannon-Khinchin axioms, which determines a class of entropies containing the well-known Tsallis and Havrda-Charvat entropies. Authors  showed that the class of entropy functions determined by Suyari’s axioms is wider than the one proposed by Suyari and generalized Suyari’s axioms characterizing recently introduced class of entropies obtained by averaging pseudoadditive information content.
In this communication, we characterized the measure (6) by taking certain axioms parallel to those considered earlier by Havrda and Charvát  along with the recursive relation (9). Some properties of this measure are also studied.
The measure (6) satisfies a recursive relation as follows: where , and .
Proof. which proves (9).
2. Set of Axioms
For characterizing a measure of information of types , and associated with a probability distribution , , , we introduce the following axioms:(1)is continuous in the region (2); (3); (4) for every ;(5)for every , where and , .
Theorem 1. If , then the axioms (1)–(5) determine a measure given by
where and .
Before proving the theorem we prove some intermediate results based on the above axioms.
Lemma 2. If , and , then
Proof. To prove the lemma, we proceed by induction. For , the desired statement holds (cf. axiom (4)). Let us suppose that the result is true for numbers less than or equal to , we will prove it for . We have
One more application of induction premise yields For , (18) reduces to Similarly for , (18) reduces to Expression (17) together with (19) and (20) gives the desired result.
Lemma 3. If , , , , and , then
Proof. Proof of this lemma directly follows from Lemma 2.
Lemma 4. If , then where , , and
Proof. Replacing in Lemma 3 by and putting , , , where and are positive integer, we have
Putting in (24) and using (by axiom (2)), we get
which is (22).
Comparing the right hand sides of (24) and (25), we get Equation (27) together with (22) gives Putting in (28) and using , we get That is, , where is an arbitrary constant.
For , we get .
Thus, we have Similarly, which is (23).
Now (22) together with (23) gives
Proof of the Theorem. We prove the theorem for rationals and then the continuity axiom extends the result for reals. For this let and ’s be positive integers such that and if we put , then an application of Lemma 3 gives
Equation (34) together with (23) and (32) gives
which is (15).
This completes the proof of the theorem.
3. Properties of Entropy of Types and
The measure , where , , is a probability distribution, as characterized in the preceding section and satisfies certain properties, which are given in the following theorems:
Theorem 5. The measure is nonnegative for , .
Case 1. ; , ; Since, and , we get
Case 2. Similarly for and , we get Therefore from Case 1, Case 2, and axiom (2), we get This completes the proof of theorem.
Definition 6. We will use the following definition of a convex function.
A function over the points in a convex set is convex if for all and The function is convex if (40) holds with in place of .
Theorem 7. The measure is convex function of the probability distribution , , , when either and or and .
Proof. Let there be distributions
associated with the random variable .
Consider numbers such that and and define where Obviously, and thus is a bonafide distribution of .
Let and , then we have , that is, , for , .
By symmetry in , and the above result is true for and .
Theorem 8. The measure satisfies the following relations:(i)Generalized-Additive: where (ii)Subadditive: for , the measure is subadditive; that is, where , and are complete probability distributions.
Proof of (ii). From part (i), we have As , for , This proves the subadditivity.
In addition to well-known information measure of Shannon, Renyi’s, Havrda-Charvat, Vajda , Darcózy, we have characterized a measure which we call , and information measure. We have given some basic axioms and properties with recursive relation. The Shannon’s  measure included in the , and information measure for the limiting case and ; and . This measure is generalization of Havrda-Charvat entropy.
Conflict of Interests
The authors declare that there is no conflict of interests regarding the publication of this paper.
- J. Aczél and Z. Daróczy, On Measures of Information and Their Characterization, Academic Press, New York, NY, USA, 1975.
- D. K. Faddeev, “On the concept of entropy of a finite probabilistic scheme,” Uspekhi Matematicheskikh Nauk, vol. 11, no. 1(67), pp. 227–231, 1956.
- T. W. Chaundy and J. B. McLeod, “On a functional equation,” Proceedings of the Edinburgh Mathematical Society. Series II, vol. 12, no. 43, pp. 6–7, 1960.
- B. D. Sharma and I. J. Taneja, “Functional measures in information theory,” Funkcialaj Ekvacioj, vol. 17, pp. 181–191, 1974.
- C. E. Shannon, “A mathematical theory of communication,” The Bell System Technical Journal, vol. 27, pp. 379–423, 623–636, 1948.
- J. Havrda and F. Charvát, “Quantification method of classification processes. Concept of structural -entropy,” Kybernetika, vol. 3, pp. 30–35, 1967.
- Z. Daróczy, “Generalized information functions,” Information and Computation, vol. 16, pp. 36–51, 1970.
- C. Tsallis, “Possible generalization of Boltzmann-Gibbs statistics,” Journal of Statistical Physics, vol. 52, no. 1-2, pp. 479–487, 1988.
- R. Hanel and S. Thurner, “A comprehensive classification of complex statistical systems and an ab-initio derivation of their entropy and distribution functions,” Europhysics Letters, vol. 93, no. 2, Article ID 20006, 2011.
- R. Hanel, S. Thurner, and M. Gell-Mann, “Generalized entropies and logarithms and their duality relations,” Proceedings of the National Academy of Sciences of the United States of America, vol. 109, no. 47, pp. 19151–19154, 2012.
- H. Suyari, “Generalization of Shannon-Khinchin axioms to nonextensive systems and the uniqueness theorem for the nonextensive entropy,” IEEE Transactions on Information Theory, vol. 50, no. 8, pp. 1783–1787, 2004.
- V. M. IIic, M. S. Stankovic, and E. H. Mulalic, “Comments on Generalization of Shannon-Khinchin axioms to nonextensive systems and the uniqueness theorem for nonextensive entropy,” IEEE Transactions on Information Theory, vol. 59, no. 10, pp. 6950–6952, 2013.
- I. Vajda, “Axioms for -entropy of a generalized probability scheme,” Kybernetika, vol. 2, pp. 105–112, 1968.