New Inequalities between Information Measures of Network Information Content

Abstract

We refine a classical logarithmic inequality using a discrete case of Bernoulli inequality, and then we refine furthermore two information inequalities between information measures for graphs, based on information functionals, presented by Dehmer and Mowshowitz in (2010) as Theorems 4.7 and 4.8. The inequalities refer to entropy-based measures of network information content and have a great impact for information processing in complex networks (a subarea of research in modeling of complex systems).

1. Introduction

We first present the classical Shannon’s entropy; see [1, 2]. Let 𝑋 be a discrete random variable taking values in 𝐴, and let 𝑝(𝑥_𝑖)=𝑃𝑟(𝑋=𝑥_𝑖) be the probability mass function of 𝑋; then, the Shannon entropy of 𝑋 is defined by 𝐻(𝑋)∶=−_{𝑥𝑖∈𝐴}𝑝𝑥_𝑖log𝑝𝑥_𝑖. (1) We continue with some basic theoretical information about graphs; see [3, 4]. We consider 𝐺=(𝑉,𝐸),|𝑉|<∞ to be a finite undirected graph, where 𝐸⊆^𝑉₂. 𝐺 is connected if for any arbitrary vertices 𝑣_𝑖,𝑣_𝑗 exists an undirected path from 𝑣_𝑖 to 𝑣_𝑗. Furthermore, we will consider 𝒢_𝑈𝐶 to be the set of finite undirected and connected graphs. This is usually used in strategies for improving transport efficiency in Network Information Content, including designing efficient routing strategies and making appropriate adjustments to the underlying network structure [5].

Now, taking into account the work presented in [6, 7], we define the concept of information functional. Let 𝐺=(𝑉,𝐸)∈𝒢_𝑈𝐶, and let 𝑆 initially be an abstract set; then, 𝑓∶𝑆→𝑅⁺ is called the information functional of 𝐺 (we assume that 𝑓 is monotonous). The abstract set 𝑆 can be, for example, the vertex sets, a set of paths, certain subgraphs, and so forth. The set 𝑆 is used to define the functional 𝑓 that captures the structural information of graph 𝐺 and has to be defined concretely.

Using different information functionals to measure the entropy of a graph, we obtain different probability distributions; so, the resulting graph entropies are also different. Further, we define, as in [6, 7], the entropy of a graph 𝐺=(𝑉,𝐸)∈𝒢_𝑈𝐶 with arbitrary vertex labels 𝑣_𝑖, using an arbitrary information functional 𝑓, to be 𝐼^𝑓(𝐺)∶=−^|𝑉|_𝑖=1𝑓𝑣_𝑖∑^|𝑉|_𝑗=1𝑓𝑣_𝑗𝑓𝑣log^𝑖∑^|𝑉|_𝑗=1𝑓𝑣_𝑗, (2) and considering a vertex 𝑣_𝑖∈𝑉, 𝑝^𝑓(𝑣_𝑖)=𝑓(𝑣_𝑖)/∑^|𝑉|_𝑗=1𝑓(𝑣_𝑗) interpreted as vertex probabilities (∑^|𝑉|_𝑖=1𝑝^𝑓(𝑣_𝑖)=1), we conclude that 𝐼^𝑓(𝐺)=−^|𝑉|_𝑖=1𝑝^𝑓𝑣_𝑖log𝑝^𝑓𝑣_𝑖. (3)

2. Refinement of a Classical Logarithmic Inequality

It is well known from the literature that log_𝑏(𝑥)≤1(𝑏)()ln𝑥−1,∀𝑥>0. (4) Trying to refine this inequality, we consider the classical discrete Bernoulli inequality, as follows.

Theorem 1 (Bernoulli inequality). Let 𝑎_𝑖≥0,𝑥_𝑖>−1,𝑖=1,…,𝑛 and ∑^𝑛_𝑖=1𝑎_𝑖≤1. Then, ^𝑛_𝑖=11+𝑥_𝑖^𝑎𝑖≤1+^𝑛_𝑖=1𝑎_𝑖𝑥_𝑖, (5) if 𝑎_𝑖≥1 or 𝑎_𝑖≤0, and if 𝑥_𝑖>0, or −1<𝑥_𝑖<0,𝑖=1,…,𝑛, then ^𝑛_𝑖=11+𝑥_𝑖^𝑎𝑖≥1+^𝑛_𝑖=1𝑎_𝑖𝑥_𝑖. (6) Now, using a factorization of a real number, one concludes as follows.

Theorem 2. Let 𝑥>0 be a real number; then, log_𝑏(𝑥)≤1ln(𝑏)^𝑛_𝑖=1𝑥_𝑖≤1(𝑏)(),ln𝑥−1 (7) where 𝑛≥2 and 𝑥=∏^𝑛_𝑖=1(𝑥_𝑖+1), with 𝑥_𝑖>0 if 𝑥>1, 𝑥_𝑖=0 if 𝑥=1, and 0>𝑥_𝑖>−1 if 1>𝑥>0, ∀1≤𝑖≤𝑛.

Proof. The inequality becomes an identity for 𝑥=1; that is, 𝑥_𝑖=0,∀1≤𝑖≤𝑛. We will treat the other two cases, when 𝑥>1 and when 1>𝑥>0 in the same manner, as follows: log_𝑏(𝑥)=log_𝑏^𝑛_𝑖=1𝑥_𝑖+1=^𝑛_𝑖=1log_𝑏𝑥_𝑖+1, (8) for which applying the classical logarithmic inequality for 𝑥_𝑖+1>0,∀1≤𝑖≤𝑛 yields log_𝑏(𝑥)≤1ln(𝑏)^𝑛_𝑖=1𝑥_𝑖, (9) and making use of the second part of the discrete Bernoulli inequality, previously presented, we obtain 1ln(𝑏)^𝑛_𝑖=1𝑥_𝑖≤1ln(𝑏)^𝑛_𝑖=1𝑥_𝑖=1+1−1(𝑏)(),ln𝑥−1 (10) and we are done.

3. Refinement of Dehmer and Mowshowitz Inequalities between Information Measures

In the work of Dehmer and Mowshowitz [6], by Theorems4.7 and 4.8 are presented two information inequalities derived assuming only the characteristic properties of the functions involved. Here, we refine those results by using the previous theorem, as follows.

Theorem 3. Let 𝑓 and 𝑓^∗ be information functionals. Then (all logarithms are in base 2), 𝐼^𝑓(𝐺)≥−^|𝑉|_𝑘=1𝑝^𝑓𝑣_𝑘log𝑝^𝑓∗𝑣_𝑘−1ln(2)^𝑛_𝑖=1𝑝^𝑘_𝑖^|𝑉|_𝑘=1𝑝^𝑓𝑣_𝑘≥−^|𝑉|_𝑘=1𝑝^𝑓𝑣_𝑘log𝑝^𝑓∗𝑣_𝑘−1ln(2)^|𝑉|_𝑘=1𝑝^𝑓(𝑣_𝑘)²−𝑝^𝑓𝑣_𝑘𝑝^𝑓∗𝑣_𝑘𝑝^𝑓∗𝑣_𝑘,𝐼^𝑓∗(𝐺)≤1ln(2)^𝑛_𝑖=1𝑝^𝑘_𝑖^|𝑉|_𝑘=1𝑝^𝑓∗𝑣_𝑘−^|𝑉|_𝑘=1𝑝^𝑓∗𝑣_𝑘log𝑝^𝑓𝑣_𝑘≤1ln(2)^|𝑉|_𝑘=1𝑝^𝑓𝑣_𝑘−𝑝^𝑓∗𝑣_𝑘−ln(2)𝑝^𝑓∗𝑣_𝑘log𝑝^𝑓𝑣_𝑘, (11) where 𝑛≥2 and 𝑝^𝑓(𝑣_𝑘)/𝑝^𝑓∗(𝑣_𝑘)=∏^𝑛_𝑖=1(𝑝^𝑘_𝑖+1),∀1≤𝑘≤|𝑉|, with ∀1≤𝑘≤|𝑉|, 𝑝^𝑘_𝑖>0 if 𝑝^𝑓(𝑣_𝑘)>𝑝^𝑓∗(𝑣_𝑘), 𝑝^𝑘_𝑖=0 if 𝑝^𝑓(𝑣_𝑘)=𝑝^𝑓∗(𝑣_𝑘), and 0>𝑝^𝑘_𝑖>−1 if 𝑝^𝑓(𝑣_𝑘)<𝑝^𝑓∗(𝑣_𝑘), ∀1≤𝑖≤𝑛.

Proof. Applying Theorem 2 for 𝑥=𝑝^𝑓(𝑣_𝑘)/𝑝^𝑓∗(𝑣_𝑘)>0 yields that log𝑝^𝑓𝑣_𝑘−log𝑝^𝑓∗𝑣_𝑘≤1ln(2)^𝑛_𝑖=1𝑝^𝑘_𝑖≤1(2)𝑝ln^𝑓𝑣_𝑘−𝑝^𝑓∗𝑣_𝑘𝑝^𝑓∗𝑣_𝑘,∀1≤𝑘≤|𝑉|. (12) By multiplying this inequality first with −𝑝^𝑓(𝑣_𝑘) and then with 𝑝^𝑓∗(𝑣_𝑘), we obtain −𝑝^𝑓𝑣_𝑘log𝑝^𝑓𝑣_𝑘1≥−ln(2)^𝑛_𝑖=1𝑝^𝑘_𝑖⋅𝑝^𝑓𝑣_𝑘−𝑝^𝑓𝑣_𝑘log𝑝^𝑓∗𝑣_𝑘1≥−(2)𝑝ln^𝑓(𝑣_𝑘)²−𝑝^𝑓𝑣_𝑘𝑝^𝑓∗𝑣_𝑘𝑝^𝑓∗𝑣_𝑘−𝑝^𝑓𝑣_𝑘log𝑝^𝑓∗𝑣_𝑘,−𝑝^𝑓∗𝑣_𝑘log𝑝^𝑓∗𝑣_𝑘≤1ln(2)^𝑛_𝑖=1𝑝^𝑘_𝑖⋅𝑝^𝑓∗𝑣_𝑘−𝑝^𝑓∗𝑣_𝑘log𝑝^𝑓𝑣_𝑘≤1(2)𝑝ln^𝑓𝑣_𝑘−𝑝^𝑓∗𝑣_𝑘−𝑝^𝑓∗𝑣_𝑘log𝑝^𝑓𝑣_𝑘,∀1≤𝑘≤|𝑉|. (13) Finally, summating after 𝑘 yields the wanted results.

4. Conclusions

As a main result of this paper, we improved two inequalities between entropy-based measures, this giving us a way of bounding measures of network properties. A good example for this purpose is presented into [6], where, using concrete information functionals, the corresponding entropies for the graphs in a set of 2265 nonisomorphic chemical graphs, that is, in MS2265, are computed. Here, new stronger lower and, respectively, upper bounds for entropy-based measures are given and thus, by knowing the limitations of such measures, we narrow down the set of expected values. The utility of inequalities between measures of structural information content is found in problem solving through problem transformation, as is also spotted in [6].

Acknowledgment

This work was supported and founded by the project “ERRIC-Empowering Romanian Research on Intelligent Information Technologies/FP7-REGPOT-2010-1”, ID: 264207.