MIClique: An Algorithm to Identify Differentially Coexpressed Disease Gene Subset from Microarray Data

Table 1

Concepts of entropy and MI defined by Shannon's theory of information.

Concepts of Shannon's theory of information	Descriptions
𝐻(𝑋)=−∑𝑥𝑝(𝑥)log2𝑝(𝑥)	The uncertainty of a random variable X is measured by its entropy 𝐻(𝑋); 𝑝(𝑥) is the probability density of 𝑋
𝐻(𝑋∣𝑌)=−∑𝑥𝑝(𝑥∣𝑦)log2𝑝(𝑥∣𝑦)	The uncertainty of a random variably 𝑋 given knowledge of another random variable 𝑌 is measured by the conditional entropy 𝐻(𝑋∣𝑌)
𝐻(𝑋,𝑌)=−∑𝑥∑𝑦𝑝(𝑥,𝑦)log2𝑝(𝑥,𝑦)	The uncertainty of a pair of random variables 𝑋, 𝑌 is measured by the entropy 𝐻(𝑋∣𝑌)
𝐻(𝑋,𝑌)=𝐻(𝑋)+𝐻(𝑌∣𝑋)=𝐻(𝑌)+𝐻(𝑋∣𝑌)
MI(𝑋;𝑌)=∑𝑥∑𝑦𝑝(𝑥,𝑦)log2𝑝(𝑥,𝑦)𝑝(𝑥)𝑝(𝑦)	Given two random variables 𝑋 and 𝑌, the amount of information that each one of them provides about the other is the mutual information MI(𝑋;𝑌)
MI(𝑋;𝑌)=𝐻(𝑋)+𝐻(𝑌)−𝐻(𝑋,𝑌)