Uniqueness of the Level Two Bayesian Network Representing a Probability Distribution

Abstract

Bayesian Networks are graphic probabilistic models through which we can acquire, capitalize on, and exploit knowledge. they are becoming an important tool for research and applications in artificial intelligence and many other fields in the last decade. This paper presents Bayesian networks and discusses the inference problem in such models. It proposes a statement of the problem and the proposed method to compute probability distributions. It also uses D-separation for simplifying the computation of probabilities in Bayesian networks. Given a Bayesian network over a family 𝐼 of random variables, this paper presents a result on the computation of the probability distribution of a subset 𝑆 of 𝐼 using separately a computation algorithm and D-separation properties. It also shows the uniqueness of the obtained result.

1. Introduction

Bayesian networks are graphical models for probabilistic relationships among a set of variables. They have been used in many fields due to their simplicity and soundness. They are used to model, represent, and explain a domain, and they allow us to update our beliefs about some variables when some other variables are observed, which is known as inference in Bayesian networks.

Given a Bayesian network [1] relative to the set 𝑋_𝐼=(𝑋_𝑖)_𝑖∈𝐼 of random variables, we are interested in computing the joint probability distribution of a nonempty subset 𝑆 (called target) of 𝐼.

The computation of the probability distribution of 𝑋_𝑆 requires marginalizing out a set of variables of 𝑆=𝐼−𝑆 from the joint distribution 𝑃_𝐼 corresponding to the Bayesian network.

In large Bayesian networks, the computation of probability distributions and conditional probability distributions may require summations relative to very large subsets of 𝐼. Consequently there is a need to order and segment, if possible, these computations into several computations that are less intensive and more accessible to a parallel treatment. These segmentations are related to the graphic properties of the Bayesian networks.

This paper describes the computation of 𝑃_𝑆 using a specific order described by a proposed algorithm, and the segmentations of 𝑃_𝑆.

We consider discrete random variables, but the results presented here can be generalized to continuous random variables with the density of 𝑋_𝑖 relative to a finite measure 𝜇_𝑖 (the summations will be replaced by integrations relative to those measures 𝜇_𝑖).

The paper is organized as follows. Section 2 introduces Bayesian networks and Level two Bayesian networks. We, then, present in Section 3 the inference problem in Bayesian networks and the proposed computation algorithm. Section 4 outlines D-Separation in Bayesian networks and describes graphical partitions that will allow the segmentations of the computations of probability distributions. Section 5 proves the uniqueness of the results obtained in Sections 3 and 4.

2. Bayesian Networks and level two Bayesian networks

2.1. Bayesian Networks

A Bayesian network (BN) is a family of random variables (𝑋_𝑖)_𝑖∈𝐼 such that: (i)the set 𝐼 is finite and endowed with a structure of a directed acyclic graph (DAG), 𝐺, where, for each 𝑖: (a)𝑝(𝑖) is the set of parents of 𝑖 (𝑝(𝑖)={𝑗;(𝑗,𝑖)∈𝐺}) (b)𝑒(𝑖) is the set of children of 𝑖 (𝑒(𝑖)={𝑗;(𝑖,𝑗)∈𝐺}) (c)𝑑(𝑖) is the set of descendants of 𝑖 (𝑑(𝑖)={𝑘;∃(ℓ₀,…,ℓ_𝑛)ℓ₀=𝑖,ℓ_𝑛=𝑘,∀𝑠∈{1,…,𝑛}(ℓ_𝑠−1,ℓ_𝑠)∈𝐺}); (ii)for each 𝑖, 𝑋_𝑖 is independent of (𝑋_𝑗)_{𝑗∈𝐼−[𝑝(𝑖)∪𝑑(𝑖)]} conditional on 𝑋_𝑝(𝑖) (for more details see, e.g., [1–5]).

We know that this is equivalent to the equality 𝑃_𝐼𝑥_𝐼=_𝑖∈𝐼𝑃_{𝑖/𝑝(𝑖)}𝑥_𝑖/𝑥_𝑝(𝑖), (2.1) where 𝑃_𝐼 is the joint probability distribution of 𝑋_𝐼=(𝑋_𝑖)_𝑖∈𝐼 and 𝑃_{𝑖/𝑝(𝑖)} is the probability distribution of 𝑋_𝑖 conditional on 𝑋_𝑝(𝑖)=(𝑋_𝑗)_{𝑗∈𝑝(𝑖)}.

The joint probability distribution corresponding to the BN in Figure 1 can be written as𝑃_𝐼𝑥₁,𝑥₂,𝑥₃,𝑥₄,𝑥₅,𝑥₆,𝑥₇,𝑥₈,𝑥₉,𝑥₁₀=𝑃₁𝑥₁𝑃₂𝑥₂𝑃_3/1,2𝑥₃/𝑥₁,𝑥₂𝑃₄𝑥₄𝑃₆𝑥₆×𝑃_5/3,4𝑥₅/𝑥₃,𝑥₄𝑃_7/5,6𝑥₇/𝑥₅,𝑥₆𝑃_8/3,7𝑥₈/𝑥₃,𝑥₇𝑃_9/8𝑥₉/𝑥₈𝑃_10/8𝑥₁₀/𝑥₈. (2.2)

Figure 1 

An example of a Bayesian network.

2.2. level two Bayesian networks

We consider the probability distribution 𝑃_𝐼 of a family (𝑋_𝑖)_𝑖∈𝐼 of random variables in a finite space Ω_𝐼=∏_𝑖∈𝐼Ω_𝑖. Let ℐ be a partition of 𝐼 and let us consider a DAG 𝒢 on ℐ.

We say that there is a link from 𝐽^′ to 𝐽^′′ (where 𝐽^′ and 𝐽^′′ are atoms of the partition ℐ) if (𝐽^′,𝐽^′′)∈𝒢. If 𝐽∈ℐ, we denote by 𝑝(𝐽) the set of parents of 𝐽, that is, the set of 𝐽^′ such that (𝐽^′,𝐽)∈𝒢.

The probability 𝑃_𝐼 is defined by the Level Two Bayesian network (BN2), on 𝐼, (ℐ,𝒢,(𝑃_{𝐽/𝑝(𝐽)})_𝐽∈ℐ), if for each 𝐽∈ℐ, we have the conditional probability 𝑃_{𝐽/𝑝(𝐽)}, or the probability of 𝑋_𝐽 conditioned by 𝑋_𝑝(𝐽) (which, if 𝑝(𝐽)=∅, is the marginal probability 𝑃_𝐽), so that 𝑃_𝐼𝑥_𝐼=_𝐽∈ℐ𝑃_{𝐽/𝑝(𝐽)}𝑥_𝐽/𝑥_𝑝(𝐽). (2.3)

The probability distribution 𝑃_𝐼 associated to the BN of level 2 in Figure 2 can be written as𝑃_𝐼𝑥_𝐼=𝑃₁𝑥₁𝑃_2/1𝑥₂/𝑥₁𝑃_3,4/2𝑥₃,𝑥₄/𝑥₂𝑃_5/3,4𝑥₅/𝑥₃,𝑋₄𝑃_6/1,3,4𝑥₆/𝑥₁,𝑥₃,𝑥₄. (2.4)

Figure 2 

Level two Bayesian network.

2.3. Close Descendants

We define the set of close descendants of a node 𝐽 (denoted cd(𝐽)) as the set of vertices containing the children of 𝐽 and the vertices located on a path between 𝐽 and one of its children.

In the example below (Figure 2), we have cd(1)={2,{3,4},6}={2,3,4,6}. (2.5)

2.4. Initial Subset

For each subset 𝑆, we denote by 𝑆⁺ the initial subset defined by 𝑆, that is, the set consisting of 𝑆 itself and the 𝑆^′ such that there is a path in 𝒢 from 𝑆^′ to 𝑆.

We can identify this subset with the union of all 𝑆^′ such that 𝑆^′ is an ancestor of 𝑆

For each 𝑆, the Initial subset 𝑆⁺ is a BN, in other words the restriction of a BN to an initial subset is a BN 𝑃_𝑆⁺=_𝑠∈𝑆+𝑃_{𝑠/𝑝(𝑠)}. (2.6)

In the example above (Figure 2), we have{6}⁺={1,2,{3,4}}={1,2,3,4}. (2.7)

3. Inference in Bayesian Networks

Consider the BN in Figure 1.

Suppose we are interested in computing the distribution of 𝑆=𝐼−{3}, in other words all variables are in the target 𝑆 except 𝑋₃.

By marginalizing out the variables 𝑋₃ from the joint probability distribution 𝑃_𝐼, the target distribution 𝑃_𝑆 can be written as 𝑃_𝑆𝑥₁,𝑥₂,𝑥₄,𝑥₅,𝑥₆,𝑥₇,𝑥₈,𝑥₉,𝑥₁₀=𝑃₁𝑥₁𝑃₂𝑥₂𝑃₄𝑥₄𝑃₆𝑥₆𝑃_7/5,6𝑥₇/𝑥₅,𝑥₆𝑃_9/8𝑥₉/𝑥₈×𝑃_10/8𝑥₁₀/𝑥_8𝑥3𝑃_3/1,2𝑥₃/𝑥₁,𝑥₂𝑃_5/3,4𝑥₅/𝑥₃,𝑥₄𝑃_8/3,7𝑥₈/𝑥₃,𝑥₇=𝑃₁𝑥₁𝑃₂𝑥₂𝑃₄𝑥₄𝑃₆𝑥₆𝑃_7/5,6𝑥₇/𝑥₅,𝑥₆𝑃_9/8𝑥₉/𝑥₈𝑃_10/8𝑥₁₀/𝑥₈𝛼𝑥₁,𝑥₂,𝑥₄,𝑥₅,𝑥₇. (3.1)

𝛼 is a function that depends on 𝑋₁,𝑋₂,𝑋₄,𝑋₅, and 𝑋₇ but has nothing to do with 𝑃_1,2,4,5,7 joint probability distribution of 𝑋₁,𝑋₂,𝑋₄,𝑋₅,𝑋₇.

By doing this we loose the structure of the BN.

If we do the marginalization as follow we obtain (according to Bayes' theorem): 𝑃_𝑆𝑥₁,𝑥₂,𝑥₄,𝑥₅,𝑥₆,𝑥₇,𝑋₈,𝑋₉,𝑋₁₀=𝑃₁𝑥₁𝑃₂𝑥₂𝑃₄𝑥₄𝑃₆𝑥₆𝑃_9/8𝑥₉/𝑥₈𝑃_10/8𝑥₁₀/𝑥₈×_𝑥3𝑃_3/1,2𝑥₃/𝑥₁,𝑥₂𝑃_5/3,4𝑥₅/𝑥₃,𝑥₄𝑃_7/5,6𝑥₇/𝑥₅,𝑥₆𝑃_8/3,7𝑥₈/𝑥₃,𝑥₇=𝑃₁𝑥₁𝑃₂𝑥₂𝑃₄𝑥₄𝑃₆𝑥₆𝑃_9/8𝑥₉/𝑥₈𝑃_10/8𝑥₁₀/𝑥₈𝑃_{5,7,8/1,2,4,6}𝑥₅,𝑥₇,𝑥₈/𝑥₁,𝑥₂,𝑥₄,𝑥₆. (3.2)

In other words 𝛼(𝑥₁,𝑥₂,𝑥₄,𝑥₅,𝑥₇)𝑃_7/5,6(𝑥₇/𝑥₅,𝑥₆)=𝑃_{5,7,8/1,2,4,6}(𝑥₅,𝑥₇,𝑥₈/𝑥₁,𝑥₂,𝑥₄,𝑥₆).

Which provides 𝑆 with a structure of a level two Bayesian network shown in Figure 3.

Figure 3 

Level two Bayesian network on 𝑆.

The variables used in the marginalization above, to keep a structure of a BN2, is the the set of close descendants defined above.

More general, if we have to sum out more than one variable there is a need to order the variables first. The aim of the inference will be to find the marginalization, or elimination, ordering for the arbitrary set of variables not in the target. This aim is shared by other node elimination algorithms like “variables elimination” [6], “bucket elimination” [7].

The main idea of all these algorithms is to find the best way to sum over a set of variables from a list of factors one by one. An ordering of these variables is required as an input. The computation depends on the ordering elimination; different elimination ordering produce different factors.

The algorithm we proposed to solve this problem is called the “Successive Restrictions Algorithm” (SRA) [8]. SRA is a goal-oriented algorithm that tries to find an efficient marginalization (elimination) ordering for an arbitrary joint distribution.

The general idea of the algorithm of successive restrictions is to manage the succession of summations on all random variables out of the target 𝑆 in order to keep on it a structure less constraining than the Bayesian network, but which allows saving in memory; that is the structure of Bayesian Network of Level Two. This was possible using the notion of close descendants.

The principle of the algorithm was presented in details in [8].

4. D-Separation and Computations in a Bayesian Network

We have introduced an algorithm which makes possible the computation of the probability distribution of a subset of random variables (𝑋_𝑠)_𝑠∈𝑆 of the initial graph. It is also possible to use the SRA to compute any probability distribution of a set of variables 𝑋_𝐴 conditionally to another subset 𝑋_𝐵 (𝑃_𝐴∣𝐵).

This algorithm tries to achieve the target distribution by finding a marginalization ordering that takes into account the computational constraints of the application. It may happen that, in certain simple cases, the SRA would be less powerful than the traditional methods [6, 9–13], but it has the advantages of adapting to any subset of nodes of the initial graph, and also to present in each stage interpretable result in terms of conditional probabilities, and thus technically usable.

In addition to the SRA we propose, especially for large Bayesian networks, to segment the computations into several less heavier computations that could be carried independently. These segmentations are possible using the D-separation.

4.1. D-Separations and Classical Results

Consider a DAG (𝐼,𝐺). A chain is a sequence (𝑥₀,…,𝑥_𝑛) of elements of 𝐼 such that for all 𝑖≥1, (𝑥_𝑖−1,𝑥_𝑖)∈𝐺 or (𝑥_𝑖,𝑥_𝑖−1)∈𝐺.

𝑥₁,…,𝑥_𝑛−1 are called intermediate nodes on this chain.

On an intermediate node 𝑥_𝑖 a chain can have three connexions as illustrated in Figure 4.

(a)

(b)

(c)

Figure 4 

(a) A converging connexion. (b) A diverging connexion. (c) A serial connexion.

Let (𝐼,𝐺) be a DAG, 𝑆⊆𝐼, 𝑎 and 𝑏 be distinct nodes in 𝑆. A chain between 𝑎 and 𝑏 is d-separated by 𝑆 if there is an intermediate node 𝑥 satisfying one of the two properties: (i)𝑥∈𝑆 and the connection is, on 𝑥, serial or diverging, (ii)𝑥∉𝑆⁺ and the connection is converging on 𝑥.

In other words, A chain is not d-separated by 𝑆 if it is in a converging connection at each intermediate node of 𝑆, and in a serial or a diverging connection at each intermediate node that has no descendants in 𝑆.

Classic Result  1
If 𝐴 and 𝐵 are d-separated by 𝑆. then the variables 𝑋_𝐴 and 𝑋_𝐵 are independent conditional on 𝑋_𝑆.

4.2. Notions

Given a subset 𝐶 of 𝐼. Markov Blanket. 𝐹(𝐶) of a subset 𝐶 is made of the parents of 𝐶, the children of 𝐶 and the variables sharing a child with 𝐶.Markov Boundary of 𝐶. 𝑀(𝐶)=𝐶∪𝐹(𝐶)Close Descent of 𝐶. 𝑇(𝐶) is the set of all close descendants of the elements in 𝐶 other that those in 𝐶 itself.Exterior Roots of 𝐶. 𝑅(𝐶) is the set of the parents of the elements of 𝐶∪𝑇(𝐶) other that those in 𝐶∪𝑇(𝐶) itself.

As we can see on the following example:

𝐹(𝐶)={1,10,14,12,9}, 𝑀(𝐶)={3,7,1,10,14,12,9}, 𝑇(3,7)={10,14,12}, 𝑅(3,7)={1,9}.

Classic Result  2
𝑋_𝐶 is d-separated from the rest of the variables conditional on the Markov blanket of 𝐶. The proof of these two results and other results related to D-separation can be found in [11].

4.3. Moral and Hypermoral Graphs

Another classic graphic property used with some inference algorithms that we can find in the literature is the notion of the Moral graph.

Moral Graph
Given a DAG (𝐼,𝐺)
its associated moral graph is the undirected graph (𝐼,𝐺_𝑚), where 𝐺_𝑚 is the set
(i)of pairs {𝑖,𝑗} such that (𝑖,𝑗)∈𝐺 or (𝑗,𝑖)∈𝐺, (ii)of pairs {𝑖,𝑗} such that 𝑖 and 𝑗 have a child in common.

In a similar way, we define what we call the hypermoral graph defined as follow:

Hypermoral Graph
Given a DAG (𝐼,𝐺)
its associated hypermoral graph is the undirected graph (𝐼,𝐺_hm), where 𝐺_hm is the set:
(i)of pairs {𝑖,𝑗} such that (𝑖,𝑗)∈𝐺 or (𝑗,𝑖)∈𝐺, (ii)of pairs {𝑖,𝑗} such that 𝑖 and 𝑗 have a close descendant in common.

In Figure 5, there is a link between 4 and 7 in the hypermoral graph because they share 9 as a close descendant.

(a)

(b)

Figure 5 

Hypermoral and Moral Graphs.

4.4. Moral and Hypermoral Partitions

The moral graph helps defining the moral partition as follow.

(i) We call a 𝑆-moral partition the partition of 𝑆⁺−𝑆, denoted 𝒫^𝑚_𝑆⁺_−𝑆, defined by the equivalence relation ∼_𝑆⁺_−𝑆, where 𝑥∼_𝑆⁺_−𝑆𝑦 means “ there exists a chain from 𝑥 to 𝑦, in 𝑆⁺−𝑆, not blocked by 𝑆.”

In an equivalent way “there exists a chain, in the moral graph 𝐺_𝑚, connecting 𝑥 to 𝑦 without an intermediate node in 𝑆.”

In a similar way we define the hypermoral partition.

(ii) We call an 𝑆-hypermoral the partition of 𝑆⁺−𝑆, denoted 𝒫^hm_𝑆⁺_−𝑆, defined by the equivalence relation ≈_𝑆⁺_−𝑆, where 𝑥≈_𝑆⁺_−𝑆𝑦 means “there exists a chain, in the hypermoral graph 𝐺_hm, connecting 𝑥 to 𝑦 without an intermediate node in 𝑆”. As Illustrated in Figure 6.

Figure 6 

(a) 𝒫hm𝑆 The 𝑆-hypermoral partition. (b) 𝒫𝑚𝑆 The 𝑆-moral partition.

4.5. Results

The following results show the possibility of segmenting the computation of the probability distribution 𝑃_𝑆.

Theorem 4.1. Let (𝐼,𝐺,𝑃_𝐼) be a BN, and let 𝑆 be a subset of 𝐼. Let 𝒫^hm_𝑆⁺_−𝑆 be the 𝑆-hypermoral partition of 𝑆⁺−𝑆 and let 𝐾 be the set of elements of 𝑆 which are not close descendants of any element of 𝑆⁺−𝑆, that is, 𝐾={𝑘∈𝑆,∀𝑦∈𝑆⁺−𝑆,𝑘∉cd(𝑦)}. Then, 𝑃_𝑆=_𝑘∈𝐾𝑃_{𝑘/𝑝(𝑘)𝐶∈𝒫hm𝑆+−𝑆}𝑃_{𝑇(𝐶)/𝑅(𝐶)}, (4.1) where 𝑃_{𝑇(𝐶)/𝑅(𝐶)}=_{𝐶(𝐶)𝑖∈𝐶∪𝑇}𝑃_{𝑖/𝑝(𝑖)}𝑥_𝑖/𝑥_𝑝(𝑖). (4.2)

The proof of the theorem can be found in [14].

Theorem 4.2. The set 𝑄_𝑠 of singletons {𝑘}, where 𝑘∈𝐾 (if 𝐾≠∅), and of subsets 𝑇(𝐶)∩𝑆⁺, where 𝐶∈𝒫^hm_𝑆⁺_−𝑆, constitutes a partition of 𝑆. As Illustrated in Figure 7.

(a)

(b)

Figure 7 

(a) 𝒫hm𝑆 the 𝑆-Hypermoral Partition. (b) BN2 Representing 𝑃𝑆.

5. Unique Partition

We have seen in the last two sections that the application of the SRA for the computation of 𝑃_𝑆 provides a structure of BN2, and that the use of D-Separations properties allows the segmentation of the computation of 𝑃_𝑆 and provides also a structure of a level two Bayesian network on 𝑆. In fact the two obtained structures are same, this results is giving by the following theorem.

Theorem 5.1. The following two sets. (1)The subsets 𝑇(𝐶)∩𝑆⁺, where 𝐶∈𝒫^hm_𝑆⁺_−𝑆. (2)The set 𝑄_𝑠 of singletons {𝑘}, where 𝑘∈𝐾 (if 𝐾≠∅) constitute a unique partition defining a BN2 on 𝑆.

Interpretation
This theorem indicates that the level two Bayesian network, characterizing the probability distribution 𝑃_𝑆, obtained by application of the SRA, is unique independently of the choices done while running the algorithm. This unique partition is constituted of sets of the two types 1 and 2 mentioned above.

Proof. Let us show that the partition of the target 𝑆 consists of the subsets of types 1 and 2 as mentioned above in the theorem, in other words consists of 𝑇(𝐶)∩𝑆⁺ where 𝐶∈𝒫^hm_𝑆⁺_−𝑆, and {𝑘} for all 𝑘∈𝐾.
As 𝑆⁺ is a BN-containing 𝑆, without a loss of generality, we can limit ourselves to the case where 𝑆⁺=𝐼.
The application of the SRA for the computation of 𝑃_𝑆 requires marginalizing out the set of variables of 𝑆=𝐼−𝑆 following a specific order. Let try to show that the obtained partition by application of the SRA is same mentioned above.
Let us proof this result by induction on the cardinality of 𝑆.
Let us assume that Card(𝑆)=1. In this case 𝑆 has only one element, 𝑆={𝑖}.
On one side, marginalizing out the variable (á) 𝑖, by application of the SRA, creates a new node 𝐸_𝑖 that contains the close descendants of 𝑖 (i.e., 𝐸_𝑖=cd(𝑖)).
The BN2 resulting from this marginalization is formed of the new node 𝐸_𝑖 along with all other remaining nodes, in other words all the {𝑘} such that 𝑘∈𝐼−(𝑖₁∪cd(𝑖₁))=𝑆−𝐸_𝑖, which is shown in Figure 8.
On the other side, since Card(𝑆)=1, 𝒫^hm_𝑆 is constituted of a unique equivalent class, 𝐶={𝑖}, so, by definition, the partition of 𝑆 is constituted of
(1)𝑇(𝐶)∩𝑆⁺=cd(𝑖)∩𝐼=cd(𝑖), (2)all other nodes, in other words the nodes 𝑘∈𝐾 such that 𝐾=𝑘∈𝑆,∀𝑦∈𝑆⁺−𝑆,𝑘∉cd𝑦=𝐼−(𝑖∪cd(𝑖))=𝑆−cd(𝑖)=𝑆−𝑇(𝐶). (5.1)This shows the result in this first case.
Let us suppose now that Card(𝑆)=𝑛, and (𝑖₁,…,𝑖_𝑛) as an hierarchical order on 𝑆.
We are going to sum out following the inverse order of the given hierarchical order.
Let us assume that the result is right till step ℓ (in other words marginalizing out 𝑖_𝑛−ℓ+1) and let's proof the result for step ℓ+1 (in other words marginalizing out 𝑖_𝑛−ℓ).
What justify the proof by induction is that once the marginalizing out 𝑖_𝑛,…,𝑖_𝑛−ℓ+1 these last elements will not interfere in the next steps of marginalizing out 𝑖_𝑛−ℓ,…,𝑖₁.
The result is right till step ℓ means that we dispose on 𝐴=𝑆∪{𝑖₁,…,𝑖_𝑛−ℓ}, of a partition that contains the subsets {𝑇(𝐶)∩𝑆⁺} for 𝐶∈𝒫^hm_𝐴⁺_−𝐴, and the nodes {𝑘} such that 𝑘∈𝐼−⋃_{𝐶∈𝒫hm𝐴+−𝐴}(𝐶∪𝑇(𝐶)), (i.e., 𝑘∈𝐴−⋃_{𝐶∈𝒫hm𝐴+−𝐴}𝑇(𝐶)), where 𝒫^hm_𝐴⁺_−𝐴 is the 𝑆-hypermoral partition associated to 𝐴.
Showing the result for ℓ+1 means to find a partition of 𝑆∪{𝑖₁,…,𝑖_{𝑛−ℓ−1}} constituted of subsets of type 1 et 2 mentioned above.
On one hand, let's first try to find the partition obtained by application of the SRA. We know that, the marginalization of 𝑖_𝑛−ℓ creates a new node, 𝐸_{𝑖𝑛−ℓ}, which contains the close descendants of 𝑖_𝑛−ℓ in the BN2 on 𝐴 (𝐸_{𝑖𝑛−ℓ}=cd(𝑖_𝑛−ℓ)), shown by Figure 9.
So if we write 𝐽={𝑘₁,…,𝑘_𝑟}, the set of 𝐾 such that for all 𝑗∈{1,…,𝑟}, 𝑘_𝑗∩𝐸_{𝑖𝑛−ℓ}≠∅ and 𝐿={𝐶₁,…,𝐶_𝑚}, the set of 𝒫^hm_𝐴⁺_−𝐴 such that for all 𝑠∈{1,…,𝑚}, 𝑇(𝐶_𝑠)∩cd(𝑖_𝑛−ℓ)≠∅.
In this case, the partition of 𝐵=𝐴−{𝑖_𝑛−ℓ}=𝑆∪{𝑖₁,…,𝑖_{𝑛−ℓ−1}} is constituted of the new node 𝐸_{𝑖𝑛−ℓ}=cd(𝑖_𝑛−ℓ)=𝐿∪𝐽, and all the other nodes that are left isolated, in other words all the nodes 𝑘∈(𝐾−𝐽) and all the {𝑇(𝐶)∩𝐴⁺} where 𝐶∈(𝒫^hm_𝐴⁺_−𝐴−𝐿).
If we write 𝐾^′=(𝒫^hm_𝐴⁺_−𝐴−𝐿)∪(𝐾−𝐽) so the partition of 𝐵 by application of the SRA is composed of the following two types of sets:
(1)𝑇(𝐶)∩𝐴⁺=cd(𝑖_𝑛−ℓ)=𝐸_{𝑖𝑛−ℓ}=𝐿∪𝐽, (2)all other isolated nodes, that is, the nodes of 𝐾^′.
On the other hand, let's now try to determine the partition of 𝐵=𝐴−{𝑖_𝑛−ℓ} using the D-Separation properties.
We have on 𝐴 a partition composed of nodes {𝑇(𝐶)∩𝐴⁺} where 𝐶∈𝒫^hm_𝐴⁺_−𝐴, and all nodes {𝑘} such that 𝑘∈(𝐴−⋃_{𝐶∈𝒫hm𝐴+−𝐴}𝑇(𝐶)).
Since 𝐵=𝐴−{𝑖_𝑛−ℓ}, and no descendant of 𝑖_𝑛−ℓ is in 𝑖₁,…,𝑖_{𝑛−ℓ−1}, (by definition of the hierarchical order), so only one equivalent subset is associated, 𝒫^hm_𝐵⁺_−𝐵={{𝑖_𝑛−ℓ}}, it results by definition, the partition of 𝐵=𝑆∪{𝑖₁,…,𝑖_{𝑛−ℓ−1}} is composed of the following types of subsets:
(1)𝑇(𝐶)∩𝐵⁺=𝑇({𝑖_𝑛−ℓ})=cd(𝑖_𝑛−ℓ)=𝐸_{𝑖𝑛−ℓ}=𝐿∪𝐽, (2)all other isolated nodes 𝑅 such that 𝑅=𝑟∈𝐵,∀𝑠∈𝐵⁺−𝐵,𝑟∉cd(𝑠). (5.2)
So we have 𝑅=𝐵−(𝑖_𝑛−ℓ∪cd(𝑖_𝑛−ℓ))=𝐴−cd(𝑖_𝑛−ℓ)=𝐴−𝐸_{𝑖𝑛−ℓ}=𝐴−(𝐿∪𝐽)=𝐾^′.
This shows that this partition is same partition obtained by application of the SRA.

(a)

(b)

Figure 8 

(a): fraction of a BN before marginalizing out 𝑖. (b) fraction of a BN2, on 𝑆, obtained after summing out 𝑖.

(a)

(b)

Figure 9 

(a): fraction of a BN2 on 𝐴. (b): BN2 obtained ater summing out 𝑖𝑛−ℓ.