The Manifestation of Stopping Sets and Absorbing Sets as Deviations on the Computation Trees of LDPC Codes

Abstract

The error mechanisms of iterative message-passing decoders for low-density parity-check codes are studied. A tutorial review is given of the various graphical structures, including trapping sets, stopping sets, and absorbing sets that are frequently used to characterize the errors observed in simulations of iterative decoding of low-density parity-check codes. The connections between trapping sets and deviations on computation trees are explored in depth using the notion of problematic trapping sets in order to bridge the experimental and analytic approaches to these error mechanisms. A new iterative algorithm for finding low-weight problematic trapping sets is presented and shown to be capable of identifying many trapping sets that are frequently observed during iterative decoding of low-density parity-check codes on the additive white Gaussian noise channel. Finally, a new method is given for characterizing the weight of deviations that result from problematic trapping sets.

1. Introduction

Prior to 1993, channel codes were typically designed with the goal of maximizing the minimum distance of the code [1, 2]. The combination of a code with a large minimum distance and a decoder that minimizes the probability of codeword error often resulted in good asymptotic performance. The discovery of turbo codes [3] and the rediscovery of low-density parity-check (LDPC) codes [4] revealed that codes with relatively poor minimum distance properties could achieve near-capacity performance at bit error rates of 𝑃_𝑏<10⁻⁶. This resulted in a reduced emphasis on maximizing minimum distance when design codes for use on the additive white Gaussian noise (AWGN) channel. As with many other classes of codes, there are no practical bounds for the decoders used for turbo codes and LDPC codes and simulations are required to accurately determine the performance at practical operating points.

With the discovery of turbo codes and the various subsequent iterative decoders, the phenomenon of the error floor has become prominent in practical code design. The term error floor refers to the situation where the error rate at the output of the decoder suddenly starts to decrease at a slower rate as a function of increasing signal-to-noise ratio (SNR); that is, the performance curve flattens out. The error floors that occur with iterative decoding of turbo codes and LDPC codes are problematic in practical systems because it is difficult to predict the specific operating point at which they occur, and thus design engineers risk using codes that may have unknown error floors that limit the performance of the system. In the case of iterative decoding of turbo codes, it has been shown that the error floor is usually the result of the overall turbo code having low weight codewords that begin to limit the performance of the code after some SNR is reached [5]. The minimum distance of a turbo code can be increased, and hence the likelihood of an error floor sufficiently mitigated, through the use of various interleaver designs [6, 7].

Low-density parity-check codes with iterative decoding are also known to exhibit error floors [8, 9], albeit at much lower bit error rates than the error floors of turbo codes of similar block length. In many cases, because of the exceptionally low bit error rates at which the error floors of LDPC codes appear, it is not practical to use Monte Carlo simulations to demonstrate the existence of these error floors. The inability to run conventional computer simulations down to the error floor combined with the lack of practical upper bounds for LDPC codes has inhibited the deployment of LDPC codes in high-throughput applications that require near error-free performance with bit error rates of 𝑃_𝑏<10⁻¹⁵.

LDPC codes are most commonly decoded using iterative message-passing decoders such as the min-sum decoder [10] and the sum-product decoder [11] due to their excellent performance and low implementation complexity. Many attempts have been made to estimate the performance of these decoders by characterizing their error mechanisms. Three of the most well-known error mechanisms are stopping sets [12], trapping sets [13], and absorbing sets [8]. Unfortunately, none of these mechanisms leads to strict upper bounds on the performance of iterative decoding over the AWGN channel, and thus they are of limited use in determining error floors. Wiberg showed that deviations on the computation trees of LDPC codes can be used to compute tight upper bounds on the performance of LDPCs with iterative decoders [10]; however, it is computationally intractable to do so after even a small number of decoder iterations. A practical method for determining the error floor of LDPCs with iterative decoding has yet to be discovered.

This paper attempts to make progress on this problem by integrating the precise, but computational intractable, work of Wiberg with the experimental studies of the error mechanisms observed when iteratively decoding LDPCs. The paper begins with a tutorial review of the existing methods for analyzing the performance of iterative message-passing decoders. Then, the notion of a problematic trapping set is introduced and its relationship to deviations is examined in detail, with the goal of determining what makes deviations either more or less problematic during iterative message-passing decoding. Finally, an iterative method is given for finding problematic trapping sets using the weights of deviations on the computation trees.

2. Background

The following model for channel coding is used throughout this paper. First, a vector 𝐮∈𝔽^𝐾₂ of 𝐾 information bits is generated by a binary source. The binary source is assumed to be memoryless, which is often the result of source coding (data compression), and therefore all information sequences in 𝔽^𝐾₂ are equally probable. A binary 𝐾×𝑁 generator matrix 𝐺 may be used by the channel encoder to map the information bits 𝐮 to a length 𝑁 codeword 𝐜∈𝔽^𝑁₂, via the mapping 𝐜=𝐮𝐺. Here, it is assumed that the matrix 𝐺 is full-rank, and thus the rate of the code is 𝑅=𝐾/𝑁.

Before a codeword 𝐜∈𝐶 is transmitted over the channel, it is modulated via the transformation 𝑥_𝑖=𝑚(𝑐_𝑖)=2𝑐_𝑖−1, (1)

for all 𝑖=0,…,𝑁−1. The received signal vector 𝐲∈ℝ^𝑁 is given by 𝐲=𝐱+𝐧, (2)

where 𝐧∈ℝ^𝑁 is the Gaussian noise vector. The log-likelihood ratio (LLR) vector, often used for soft-decision decoding, is given by 𝜆_𝑖=𝑃_𝑌∣𝑋(𝑦_𝑖∣−1)𝑃_𝑌∣𝑋(𝑦_𝑖,∣1) (3)

for all 𝑖=1,…,𝑁. This reduces to 𝜆_𝑖=(−2/𝜎²)𝑦_𝑖 when 𝐧 is a vector of AWGN. An estimate 𝐜̂ of the transmitted codeword 𝐜 is derived from the received vector 𝐲 at the channel decoder. Finally, the information bits 𝐮 extracted from 𝐜̂ are passed to the sink.

As mentioned earlier, each of the information sequences 𝐮∈𝔽^𝐾₂ is equiprobable. Since there is a one-to-one mapping between information sequences and codewords, all codewords in the code 𝐶 are equiprobable as well. Therefore, 𝑃(𝐜_𝑖)=𝑃(𝐜_𝑗) for all 𝐜_𝑖,𝐜_𝑗∈𝐶, where 𝑃(𝐜_𝑖) is the probability that codeword 𝐜_𝑖 is transmitted. When considering the performance of linear codes, equiprobable codewords allow for the assumption that the all-zeros codeword was transmitted. The all-zeros codeword assumption is used throughout this paper.

From the generator matrix 𝐺, it is possible to derive an (𝑁−𝐾)×𝑁 parity-check matrix 𝐻 for the code. A parity-check matrix of a code 𝐶 is any matrix 𝐻, such that 𝐻𝐜^𝑇=𝟎 for all 𝐜∈𝐶. LDPC codes are often defined by their parity-check matrix 𝐻. In particular, LDPC codes are a class of codes with sparse parity-check matrices. A sparse parity-check matrix is any binary matrix that contains more binary 0s than binary 1s. A (𝑑_𝑉,𝑑_𝐹)-regular LDPC code is one that has a fixed number 𝑑_𝑉 of binary 1s in each column of the parity-check matrix and some fixed number 𝑑_𝐹 of binary 1s in each row of the parity-check matrix. An example of a (2,3)-regular LDPC code of length 𝑁=6 and dimension 𝐾=3 is given by the parity-check matrix 𝐻_(2,3)=111000110100001011000111. (4) The parity-check matrix of a length 𝑁, dimension 𝐾 code must contain at least (𝑁−𝐾) rows, since the kernel of 𝐺 has dimension (𝑁−𝐾). However, it is possible for the parity-check matrix to contain more than (𝑁−𝐾) rows. Therefore, the number of rows in the parity-check matrix is denoted by 𝑀, where 𝑀≥(𝑁−𝐾).

A Tanner graph is a bipartite graphical representation of a low-density parity-check matrix. To construct a Tanner graph from a parity-check matrix, each column 𝑖 in the parity-check matrix is assigned to a corresponding variable node 𝑣_𝑖 in the Tanner graph, and each row 𝑗 is assigned to a corresponding check node 𝑓_𝑗 in the Tanner graph. The set of all variable nodes is 𝑉, and the set of all check nodes is 𝐹. There is an edge 𝑒_𝑖,𝑗 between variable node 𝑣_𝑖 and check node 𝑓_𝑗 in the Tanner graph if and only if the entry in 𝐻 at the intersection of the 𝑗th row and 𝑖th column is a binary 1. The Tanner graph 𝑇=(𝑉⋃𝐹,𝐸) is thus defined by the set of variable nodes 𝑉, the set of check nodes 𝐹, and the set of edges 𝐸. The Tanner graph corresponding to the parity-check matrix 𝐻_(2,3) (given by (4)) is shown in Figure 1.

Figure 1 

Tanner graph of a (2,3)-regular LDPC code.

Note that in the Tanner graphs of irregular LDPC codes variable nodes and check nodes do not all have the same number of incident edges. The number of check nodes that a specific variable node 𝑣_𝑖 is connected to is denoted by 𝑑_𝑣𝑖, and the number of variable nodes that a specific check node 𝑓_𝑖 is connected to is denoted 𝑑_𝑓𝑖.

2.1. Iterative Message-Passing Decoding

The min-sum (MS) and sum-product (SP) decoders are low-complexity, sub-optimal iterative decoders that can be used to decode low-density parity-check codes. Given a particular parity-check matrix, the MS decoder operates by passing messages between the check nodes and the variable nodes along the edges of the Tanner graph of the code.

Before introducing the decoders, some additional notation is necessary. The set of neighbors of check node 𝑓_𝑖 in the Tanner graph is denoted 𝑁(𝑓_𝑖)={𝑣_𝑗∣ℎ_𝑖,𝑗=1}, and similarly the set of neighbors of variable node 𝑣_𝑖 in the Tanner graph is denoted 𝑁(𝑣_𝑖)={𝑓_𝑗∣ℎ_𝑗,𝑖=1}. To denote the set of neighbors of check node 𝑓_𝑖 excluding variable node 𝑣_𝑗, the notation 𝑁(𝑓_𝑖)⧵𝑣_𝑗 is used. Similarly, the set of neighbors of variable node 𝑣_𝑗 excluding check node 𝑓_𝑖 is denoted 𝑁(𝑣_𝑗)⧵𝑓_𝑖. During decoding, messages are passed between neighboring check nodes and variable nodes along the edges of the Tanner graph. Messages from check node 𝑓_𝑖 to variable node 𝑣_𝑗∈𝑁(𝑓_𝑖) are denoted by 𝑚_{𝑓𝑖→𝑣𝑗}, and messages from variable node 𝑣_𝑖 to check node 𝑓_𝑗∈𝑁(𝑣_𝑖) are denoted 𝑚_{𝑣𝑖→𝑓𝑗}. Given the transmitted codeword 𝐱, the channel output 𝐲 available at the receiver, and a maximum number of iterations ℓ_max, the steps for MS and SP decoding are given in the following algorithm

Algorithm 1 (Min-Sum/Sum-Product Decoding). Step 1 (Initialization). Set the number of iterations to ℓ=0. For all messages 𝑚_{𝑓𝑖→𝑣𝑗}, set 𝑚_{𝑣𝑖→𝑓𝑗}=𝜆_𝑖=𝑃_𝑌∣𝑋(𝑦_𝑖∣−1)𝑃_𝑌∣𝑋(𝑦_𝑖=∣1)−2𝜎²𝑦_𝑖. (5)Step 2 (Check Node Update). Set ℓ=ℓ+1. For all messages 𝑚_{𝑣𝑖→𝑓𝑗}, setMin-Sum: 𝑚_{𝑓𝑖→𝑣𝑗}=(_{𝑣𝑘∈𝑁(𝑓𝑖)⧵𝑣𝑗}sgn(𝑚_{𝑣𝑘→𝑓𝑖}))(min_{𝑣𝑘∈𝑁(𝑓𝑖)⧵𝑣𝑗}∣𝑚_{𝑣𝑘→𝑓𝑖}). (6)Sum-Product:𝑚_{𝑓𝑖→𝑣𝑗}=2⋅tanh⁻¹(_{𝑣𝑘∈𝑁(𝑓𝑖)⧵𝑣𝑗}𝑚tanh(^{𝑣𝑘→𝑓𝑖}2)). (7)Step 3 (Variable Node Update). For all messages 𝑚_{𝑣𝑖→𝑓𝑗}, set 𝑚_{𝑣𝑖→𝑓𝑗}=𝜆_𝑖+_{𝑓𝑘∈𝑁(𝑣𝑖)⧵𝑓𝑗}𝑚_{𝑓𝑘→𝑣𝑖}. (8)Step 4 (Check Stop Criteria). For all 𝑚_𝑣𝑖, set 𝑚_𝑣𝑖=𝜆_𝑖+_{𝑓𝑘∈𝑁(𝑣𝑖)}𝑚_{𝑓𝑘→𝑣𝑖}. (9) For all 𝑐_𝑖̂, set 𝑐_𝑖̂=0if𝑚_𝑣𝑖>0,1if𝑚_𝑣𝑖<0, (10) with 𝑃(𝑐_𝑖̂=0∣𝑚_𝑣𝑖=0)=𝑃(𝑐_𝑖̂=1∣𝑚_𝑣𝑖=0)=0.5.If 𝐻𝐜̂^𝑇=𝟎 or ℓ≥ℓ_max, stop decoding, else return to Step 2.
One of the primary strengths of the min-sum and sum-product decoders is the relatively small number of operations performed during each iteration. During each iteration, the messages 𝑚_{𝑣𝑖→𝑓𝑗} and 𝑚_{𝑓𝑗→𝑣𝑖} must be computed for each binary 1 in the parity-check matrix. For a (𝑑_𝑉,𝑑_𝐹)-regular LDPC code, there are (𝑁×𝑑_𝑉)=(𝑀×𝑑_𝐹) binary 1s in the parity-check matrix. When the degree of the nodes and the number of iterations is fixed, the complexity of MS decoding scales linearly with the length 𝑁 of the code.
In practice, the min-sum and sum-product decoders do not always output a codeword. It has been shown that when the MS decoder does not output a codeword after a large number (>200) of iterations has been performed, the output often cycles in a repeating sequence of two or more noncodeword outputs [14]. In Sections 2.2 through Section 2.4, three different characterizations are given for the noncodeword outputs of iterative message-passing decoders.

2.2. Stopping Sets

The notion of stopping sets was first introduced by Forney et al. [15] in 2001. Two years later, a formal definition of stopping sets was given by Di et al. [12]. They demonstrated that the bit and word error probabilities of iteratively decoded LDPC codes on the binary erasure channel (BEC) can be determined exactly from the stopping sets of the parity-check matrix.

Definition 1 (stopping sets [12]). A stopping set 𝒮 is a subset of the set of variable nodes 𝑉, such that any check node connected to a variable node contained in 𝒮 is connected to at least two variable nodes in 𝒮.

A small example of a stopping set is given in Figure 2. Consider the subset 𝒮={𝑣₁,𝑣₂,𝑣₃,𝑣₄} of the set of variable nodes 𝑉. There are five check nodes {𝑓₁,𝑓₂,𝑓₃,𝑓₄,𝑓₅} connected to the set 𝒮, and each of them is connected to 𝒮 at least two times. Note that only 𝑓₂ is connected to the set 𝒮 an odd number of times; If each of the check nodes is connected to 𝒮 an even number of times, 𝒮 corresponds to a codeword support set where all bits in 𝒮 can be flipped without changing the overall parity of any of the check nodes.

Figure 2 

Example of a stopping set in the Tanner graph of an LDPC code.

The intuition behind stopping sets begins with an understanding of iterative message-passing decoders. Information given to a specific variable node from a neighboring check node is derived from all other variable nodes connected to that check node. Consider two variable nodes 𝑣_𝑖,𝑣_𝑗∈𝑁(𝑓_𝑘), where both variable nodes contain an erasure. In this case, each of the sets 𝑁(𝑓_𝑘)⧵𝑣_𝑖 and 𝑁(𝑓_𝑘)⧵𝑣_𝑗 contains at least one erasure, thus making it impossible for the check node 𝑓_𝑘 to determine the parity of either set. For this reason, none of the check nodes connected to a stopping set is capable of resolving erasures, if each variable node contained in the stopping set begins with an erasure from the channel.

Work relating linear programming (LP) pseudocodewords to stopping sets for the binary erasure channel [15], and both the binary symmetric channel (BSC) and the additive white Gaussian noise channel [16], has revealed a relationship between linear programming pseudocodewords and the size of stopping sets. Although stopping sets have a strong relationship with LP pseudocodewords, the performance of neither the MS decoder or the SP decoder on the BSC and AWGN channels can be predicted using stopping sets alone.

2.3. Trapping Sets

Trapping sets, also referred to as near-codewords, were first introduced by MacKay and Postol [13] to provide an explanation for the weaknesses of algebraically constructed low-density parity-check codes. They define trapping sets as follows.

Definition 2 (trapping sets [13]). Consider a length 𝑁 code with parity-check matrix 𝐻, and let 𝒯⊆{1,…,𝑁} be a set containing |𝒯|=𝑡 coordinates. Consider a length-𝑁 binary vector 𝐲 with 1s in the coordinates of 𝒯 and 0s elsewhere. If the syndrome 𝐬=𝐻𝐲 has Hamming weight 𝑤_𝑡, the set 𝒯 is referred to as a (𝑡,𝑤_𝑡) trapping set.

Consider the trapping set shown in Figure 3, where the set 𝒯={1,2,3,4} corresponds with a set of variable nodes {𝑣₁,𝑣₂,𝑣₃,𝑣₄} in the Tanner graph of the parity-check matrix 𝐻. There are four variable nodes in the set, so 𝑡=4, and if all variable nodes are set to a binary 1, only check nodes 𝑓₂ and 𝑓₃ are connected to an odd number of binary 1s, so the syndrome 𝐬 has Hamming weight equal to 2. Therefore, according to Definition 2, this set of variable nodes defines a (4,2) trapping set.

Figure 3 

Example of a (4,2) trapping set in the Tanner graph of an LDPC code.

It is important to note that any set of variable nodes can be considered a trapping set defined by some set of parameters, and the significance of trapping sets varies greatly depending on the parameters (𝑡,𝑤_𝑡). In much the same way that low-weight codewords are problematic to decoding, erroneous channel information is more likely to affect the majority of variable nodes in a trapping set which has low-weight 𝑡. Richardson [17] shows that trapping sets with small weight 𝑡 and a small number of unsatisfied check nodes 𝑤_𝑡 are more likely to cause errors. When a trapping set has small 𝑤_𝑡, the extrinsic information being passed into 𝒯 can not overcome the intrinsic information reinforced within 𝒯.

In [17], trapping sets are examined for different decoders on the binary erasure channel, binary symmetric channel, and the additive white Gaussian noise channel. Whereas stopping sets can be used to precisely determine the probability of error on the BEC, trapping sets appear to cause errors on the AWGN channel. Richardson [17] uses the parameters and multiplicity of various problematic trapping sets to estimate the error floor of LDPC codes at bit error rates where simulations are not feasible. Unfortunately, the somewhat vague definition of problematic trapping sets makes it difficult to use them for performance analysis.

2.4. Absorbing Sets

In an attempt to clarify the ambiguity of problematic trapping sets, Zhang et al. introduced the notion of absorbing sets [8]. They define absorbing sets as follows.

Definition 3 (Absorbing Sets [8]). Let 𝒜⊆𝑉 be a set containing |𝒜|=𝑎 variable nodes. Also, let 𝑂(𝒜)⊆𝐹 be a set of check nodes such that |𝑂(𝒜)|=𝑤_𝑎, and each check node in the set 𝑂(𝒜) has an odd number of edges connected to 𝒜. If each variable node in 𝒜 is connected to strictly more check nodes in 𝐹⧵𝑂(𝒜) than in 𝑂(𝒜), the set 𝒜 is referred to as a (𝑎,𝑤_𝑎) absorbing set. A fully absorbing set also satisfies the condition that each variable node in 𝑉 is connected to more check nodes in 𝐹⧵𝑂(𝒜) than in 𝑂(𝒜).

Note that an (𝑎,𝑤_𝑎) absorbing set is also an (𝑎,𝑤_𝑎) trapping set, but the converse is not always true. Figure 4 shows an example of a (4,1) absorbing set. The set of variable nodes in this absorbing set is 𝒜={𝑣₁,𝑣₂,𝑣₃,𝑣₄}, and the set of unsatisfied check nodes is 𝑂(𝒜)={𝑓₄}. The variable node 𝑣₂ is not connected to 𝑓₄, and the variable nodes 𝑣₁, 𝑣₃, and 𝑣₄ are each connected to 𝑓₄ and at least two other check nodes in 𝐹. Therefore, each of the variable nodes in 𝒜 is connected to more satisfied check nodes than unsatisfied check nodes. Also, note that the (4,2) trapping set in Figure 3 is not an absorbing set because variable nodes 𝑣₂ and 𝑣₄ are connected to two unsatisfied check nodes and only one satisfied check node.

Figure 4 

Example of a (4,1) absorbing set in the Tanner graph of an LDPC code.

Simulations show that the majority of errors encountered in the error floor region during sum-product decoding of the IEEE 802.3 an low-density parity-check code could be attributed to absorbing sets [8, 9]. Although absorbing sets appear to be useful for estimating the performance of iterative message-passing decoding, they do not lead to strict upper bounds. For upper bounds, it is possible to use the concept of deviations on the computation tree.

2.5. Computation Trees and Deviations

In his 1996 dissertation, Wiberg [10] presented groundbreaking analytical results with respect to iterative decoding of low-density parity-check codes. He provided extensive analysis of both the MS and SP decoders by introducing a model of iterative decoding known as the computation tree. Wiberg showed that the MS decoder minimizes the probability of word error when decoding a code whose Tanner graph is a tree, while for the same type of code the SP decoder minimizes the probability of bit error.

In addition to introducing computation trees, Wiberg also introduced the concept of deviations. Wiberg proved that deviations on the computation tree with negative cost are required in order for errors to occur during MS and SP decoding. Because of the importance of computation trees and deviations in understanding finite tree-based decoding, they are examined in detail in this section.

Consider a low-density parity-check code represented by a Tanner graph 𝑇=(𝑉⋃𝐹,𝐸). A computation tree rooted at variable node 𝑣_𝑖 after ℓ iterations is denoted 𝑅_𝑣^(ℓ)_𝑖. In order to construct a computation tree from the Tanner graph, a variable node 𝑣_𝑖 is placed at the top level (root) of a descending tree. To construct the next level in the tree directly below 𝑣_𝑖, each of 𝑣_𝑖’s neighbors in 𝑁(𝑣_𝑖) is added to this level and connected to 𝑣_𝑖. This process continues level-by-level, where nodes in the previous level are used to determine nodes on next level, while maintaining that each node in the computation tree has the same set of neighbors as its corresponding node in the Tanner graph. For example, if variable node 𝑣_𝑗 on the last completed level is connected to check node 𝑓_𝑘 on the level above it, then all check nodes in 𝑁(𝑣_𝑗)⧵𝑓_𝑘 must appear on the next level and be connected to 𝑣_𝑗, thereby ensuring that 𝑣_𝑗 is connected to exactly one copy of each check node in 𝑁(𝑣_𝑗).

Figure 5 gives an example of a Tanner graph, and its corresponding computation tree rooted at 𝑣₁ after two iterations. Nodes at the bottom level of the computation tree are referred to as leaf nodes. Notice that the leaf nodes are the only nodes in the computation tree that are not connected to a copy of each of their neighbors in the original Tanner graph.

(a) Tanner graph

(b) Computation tree

Figure 5 

Computation tree of a simple repetition code after ℓ=2 iterations.

Computation trees are precise models for analyzing the performance and behavior of min-sum and sum-product decoding for a finite set of iterations. Each of these decoders can be precisely modeled after ℓ iterations by constructing 𝑁 different computation trees that contain 2ℓ+1 levels of nodes including the root node. The 𝑁 computation trees are each rooted at a different variable node from the original Tanner graph. Then, for every variable node 𝑣_𝑖 in each computation tree, the LLR cost 𝛾_𝑖 is assigned to that variable node. At this point, MS or SP decoding operations can be performed from the leaf nodes up to the root node. The final cost at each of the root nodes determines the binary estimate of the transmitted codeword computed by the decoder. Because the MS and SP decoders are optimal on Tanner graphs that are trees, the MS and SP decoders are optimal on each of the computation trees derived from the Tanner graph. MS chooses the least cost valid configuration on the tree, where a valid configuration refers to any assignment of binary numbers to the variable nodes such that each check node is adjacent to an even number of variable nodes assigned to a binary 1. The SP decoder, on the other hand, chooses the value at the root node that has the highest probability over all valid configurations.

Although the computation tree model is precise, after a small number of iterations it becomes impractical to analyze the performance of specific codes by considering all valid configurations on the computation tree. The number of valid configurations on the computation tree can be computed by treating the computation tree as a Tanner graph. In order to define a Tanner graph given the computation tree, treat all check nodes and variable nodes in the computation tree separately. For example, if multiple copies of variable node 𝑣₁ are distributed throughout the computation tree, each copy is treated as a distinct variable node. After regarding each variable node in the computation tree as distinct, one can show that each check node on the computation tree corresponds to a linearly independent parity-check equation. If there are |𝑅_𝑣^(ℓ)_𝑖(𝑉)| variable nodes and |𝑅_𝑣^(ℓ)_𝑖(𝐹)| check nodes on a computation tree rooted at variable node 𝑣_𝑖 after ℓ iterations, then there are a total of 2^{|𝑅𝑣𝑖(ℓ)(𝑉)|−|𝑅𝑣𝑖(ℓ)(𝐹)|} valid configurations on the tree. On a (𝑑_𝑉,𝑑_𝐹)-regular LDPC code, the number of variable nodes after ℓ iterations is given by|𝑅_𝑣^(ℓ)_𝑖(𝑉)|=1+^ℓ−1_𝑖=0𝑑_𝑉(𝑑_𝐹−1)(((𝑑_𝑉−1)(𝑑_𝐹−1))^𝑖), (11) and the number of check nodes is given by |𝑅_𝑣^(ℓ)_𝑖(𝐹)|=^ℓ−1_𝑖=0𝑑_𝑉(((𝑑_𝑉−1)(𝑑_𝐹−1))^𝑖). (12)

To illustrate the growth rate in the number of valid configurations on the computation tree, consider an LDPC code where each variable node has degree 𝑑_𝑉=3 and each check node has degree 𝑑_𝐹=6. These commonly used code parameters result in what are known as a (3,6)-regular LDPC codes. Table 1 shows the number of variable nodes given by |𝑅_𝑣^(ℓ)_𝑖(𝑉)|=1+^ℓ−1_𝑖=015(10^𝑖), (13)

Table 1 

The number of nodes and valid configurations on the computation tree of a (3,6)-regular LDPC code.

the number of checks nodes given by |𝑅_𝑣^(ℓ)_𝑖(𝐹)|=^ℓ−1_𝑖=03⋅10^𝑖, (14)

and the corresponding number of valid configurations on the computation tree after 1, 2, and 3 iterations. Note that the growth rate is not affected by the block length of the code.

Table 1 illustrates the computational complexity associated with considering each valid configuration on the computation tree. In light of this, Wiberg [10] derived a simplified bound on the performance of MS decoding operating on a particular computation tree. In order to obtain this bound, Wiberg introduced the concept of deviations on the computation tree.

Definition 4 (Deviation [10]). A deviation is any set of variable nodes on the computation tree satisfying the following three conditions.(1)Each check node in the computation tree is adjacent to either two or zero variable nodes in the deviation set.(2)A deviation set contains the root node of the computation tree.(3)No proper and nonempty subset of variable nodes in the deviation from a valid configuration on the computation tree.

Figure 6 shows an example of a deviation on the computation tree given in Figure 5(b). The larger blue variable nodes are contained in the deviation, whereas the smaller red nodes are not.

Figure 6 

Example of a deviation on the computation tree.

Wiberg uses the set of deviations on the computation tree to derive an upper bound on the performance of the min-sum decoder. It is necessary, but not sufficient, for at least one deviation 𝛿 in the set of all deviations Δ to have negative cost in order for an error to occur at the root node. The cost of the deviation, denoted by 𝐺(𝛿), can be found by summing the LLR cost of each of the nodes in the support of the deviation. The cost of a deviation is given by 𝐺(𝛿)=_{𝑣𝑖∈𝛿}𝛾_𝑖, (15)

where copies of 𝑣_𝑖∈𝛿 are counted as many times as they appear in the deviation. A necessary, but not sufficient, condition for an error to occur on the computation tree rooted at variable node 𝑣_𝑖 is given by [10] min_𝛿∈Δ𝐺(𝛿)<0. (16)

Using this condition, a bound can be derived on the probability that the minimum-cost configuration on the computation tree contains a binary 1 at the root node. This bound is 𝑃(𝑣_𝑖=1)≤𝑃(min_𝛿∈Δ≤𝐺(𝛿)<0),_𝛿∈Δ𝑃(𝐺(𝛿)<0), (17)

which can be further loosened to𝑃(𝑣_𝑖=1)≤_𝛿∈Δ𝑃(𝐺(𝛿)<0), (18) by using the union bound.

Wiberg [10] shows that the bound given by (18) can be used to predict the performance of min-sum decoding of infinite-length codes after a specific number of iterations. Wiberg begins by assuming that the computation trees have no repeated nodes. This assumption simplifies the weight enumerators of the deviations for regular LDPC codes. Wiberg also shows that (18) can be used to bound MS decoder performance when there are multiple copies of each variable node in the tree. Thus, in theory, Wiberg’s deviation bound can be used to bound the performance of MS decoding of finite length codes. The following proposition shows that the number of deviations grows exponentially with 𝑑_𝑉, thus making it computationally intractable to enumerate the deviations even after a small number of iterations.

Proposition 1. Let 𝑅_𝑣^(ℓ)_𝑖 be the computation tree of a (𝑑_𝑉,𝑑_𝐹)-regular LDPC code, rooted at variable node 𝑣_𝑖 after ℓ iterations. Then, the number of deviations that exist on 𝑅_𝑣^(ℓ)_𝑖 is (𝑑_𝐹−1)^{∑ℓ𝑖=1𝑑𝑉(𝑑𝑉−1)𝑖−1}. (19)

Proof. By the definition of a deviation, we must assign the root node 𝑣_𝑖 to a binary 1. Each of the 𝑑_𝑉 check nodes immediately below 𝑣_𝑖 must assign exactly one of the their (𝑑_𝐹−1) child variable nodes to a binary 1. Thus, there are a total of (𝑑_𝐹−1)^𝑑𝑉 deviations after one iteration. In addition, there are exactly 𝑑_𝑉 leaf nodes in the support of each deviation after one iteration.
Each of the previous 𝑑_𝑉 leaf nodes gets connected to (𝑑_𝑉−1) check nodes after two iterations. Each of these check nodes assigns one of their (𝑑_𝐹−1) child variable nodes to a binary 1. Therefore, for each deviation after one iteration there are (𝑑_𝐹−1)^{𝑑𝑉(𝑑𝑉−1)} different deviations after two iterations. This brings the total number of deviations to (𝑑_𝐹−1)^𝑑𝑉(𝑑_𝐹−1)^{𝑑𝑉(𝑑𝑉−1)}=(𝑑_𝐹−1)^(𝑑𝑉)2 after two iterations. The total number of leaf nodes in the support of the deviation after two iterations is 𝑑_𝑉(𝑑_𝑉−1).
Following this pattern, the 𝑑_𝑉(𝑑_𝑉−1) variable nodes in support of the deviation after two iterations branchs out to 𝑑_𝑉(𝑑_𝑉−1)² check nodes. There are (𝑑_𝐹−1)^{𝑑𝑉(𝑑𝑉−1)2} ways of assigning the leaf nodes to the support of the previous deviation. This brings the total number of deviations to (𝑑_𝐹−1)^(𝑑𝑉)2(𝑑_𝐹−1)^{𝑑𝑉(𝑑𝑉−1)2}=(𝑑_𝐹−1)^{(𝑑𝑉)3−(𝑑𝑉)2+𝑑𝑉} after three iterations.
After ℓ iterations, the 𝑑_𝑉(𝑑_𝑉−1)^ℓ−2 old leaf nodes in the support of the deviation branch out to 𝑑_𝑉(𝑑_𝑉−1)^ℓ−1 new leaf nodes in the support of the deviation. There are (𝑑_𝐹−1)^{𝑑𝑉(𝑑𝑉−1)ℓ−1} ways of assigning the support to the previous deviation, and the total number of deviations after ℓ iterations is^ℓ_𝑖=1(𝑑_𝐹−1)^{𝑑𝑉(𝑑𝑉−1)𝑖−1}=(𝑑_𝐹−1)^{∑ℓ𝑖=1𝑑𝑉(𝑑𝑉−1)𝑖−1}. (20)

The number of deviations on the computation tree of a (3,6)-regular low-density parity-check code is given in Table 2 for iterations 1 through 5. Even after only a small number of iterations, it becomes impractical to enumerate each of the deviations in order to compute the upper bound on the probability of bit error of the root variable node of the computation tree.

Table 2 

Number of deviations at iterations 1–5 for a (3,6)-regular LDPC code.

Using computation trees, Wiberg provided a precise model of the behavior of the min-sum and sum-product decoders. Unfortunately, the size of the computation trees and the number of configurations on them grows too large for practical analysis. Deviations provide a simplified approach to the analysis of computation trees, but the number of deviations also grows exponentially with the number of iterations.

3. Stopping Sets, Absorbing Sets, and Resulting Deviations

Deviations can be used to define a necessary condition for an error to occur during iterative decoding. The condition simply states that there must be at least one deviation with cost less than zero, assuming that the all-zeros codeword was sent. However, this condition says nothing about which deviations are more or less likely to cause errors. What is known is that at high SNRs low-weight, deviations are much more likely to cause errors than high-weight deviations. Thus it is reasonable to expect that low-weight stopping sets and low-weight deviations coincide over the BEC channel, since low-weight stopping sets are precisely the cause of errors for iterative decoding over the BEC [15]. For the same reason, one can expect that trapping sets, or more specifically absorbing sets, coincide with low-weight deviations over the AWGN channel, since they have been frequently observed to cause errors at high SNR during iterative decoding of LDPC codes over the AWGN channel [8, 17]. Connections between stopping/absorbing sets and deviations and their effect on decoding performance are examined in this section.

3.1. Stopping Sets as Deviations

Stopping sets consist of a subset 𝒮 of the variable nodes, such that each check node connected to an element in 𝒮 is connected at least twice. According to the definition of a deviation given in Definition 4, each check node connected to the variable nodes in a deviation is connected exactly two times. These two properties can be used to study the relationship between stopping sets and their corresponding deviations.

First, consider a computation tree where each of the variable nodes begins with an assignment of a binary 0. Then, assign all copies of variable nodes in 𝒮 to a binary 1. If 𝒮 does not correspond with a codeword in 𝐶, the resulting configuration on the computation tree will not be a valid configuration. For example, consider check node 𝑓₂ in Figure 2. Each time, check node 𝑓₂ appears in the computation tree, it will be connected to three variable nodes assigned to a binary 1, including the parent variable node and two child variable nodes. If one of the child variable nodes of 𝑓₂ along with all of its descendants is set to a binary 0, check node 𝑓₂ will be satisfied. If this is done for every unsatisfied check node in the computation tree, a deviation is created that contains only variable nodes in 𝒮. Thus, this method allows one to create a deviation from a stopping set.

Using the method previously described, a deviation can be constructed using only the variable nodes contained in a stopping set. This is illustrated in Figure 7(a), where a portion of the deviation defined by the set 𝒮 from Figure 2 is given. Since only one of the child variable nodes can be included in the deviation, it is sufficient to randomly include 𝑣₂ and exclude 𝑣₄, since both nodes are included in 𝒮. The effect of this deviation is now examined over the BEC and the BSC. Stopping sets are known to cause errors over the BEC. The reason for this is illustrated by Figure 7(b). When each variable node in 𝒮 is received as an erasure 𝜀, it only takes one erasure below 𝑓₂ to cause an erasure message to be sent from 𝑓₂ to 𝑣₁. However, even though two erasures are connected below 𝑓₂ in Figure 7(b), the same erasure message is sent up to 𝑣₁. This example illustrates that a deviation containing all erasures is sufficient for an erasure output at the root node. Thus there is a strong connection between stopping sets and deviations over the BEC, since both cause errors using iterative decoding and deviations can be created from stopping sets.

(a) Deviation

(b) Costs on the BEC

(c) Costs on the BSC

Figure 7 

A small portion of a deviation on a computation tree illustrating the impact of stopping set deviations on the BEC and BSC channels.

The reason stopping sets will not cause errors as frequently over the BSC is illustrated in Figure 7(c). A deviation containing only the variable nodes in a stopping set exists on a computation tree rooted at any one of the variable nodes in 𝒮 regardless of the channel. However, the impact of deviations is not the same across all channels. For example, if two messages of −1 (representing binary 1s) are being sent from 𝑣₂ and 𝑣₄, the message sent from 𝑓₂ up to 𝑣₁ is a +1 (representing a binary 0). This example shows that while a deviation created from a stopping set is sufficient to cause erasure outputs from the decoder over the BEC, this same deviation may not cause errors over the BSC.

From Figure 7, it is clear that deviations created from stopping sets have a different effect on iterative decoding over the binary erasure channel and the binary symmetric channel. While it is not as clear how deviations created from stopping sets will effect decoding over the AWGN channel, some similarities can be drawn between the BEC and the AWGN. In terms of channel LLR costs, an erasure over the BEC behaves like a LLR cost of zero over the AWGN. A real-valued LLR interpretation of the BEC channel can be created using real-valued costs of −1.0,+1.0, and 0.0 to represent a binary 1, 0, and an erasure 𝜀, respectively. Binary information is transmitted as 𝑥=−1.0 and 𝑥=+1.0 over the AWGN channel, and the probability 𝑃(𝑦=+1.0∣𝑥=−1.0)=𝑃(𝑦=−1.0∣𝑥=+1.0)<𝑃(𝑦=0.0∣𝑥=−1.0)=𝑃(𝑦=0.0∣𝑥=+1.0), regardless of the channel SNR. Thus, it is reasonable to suspect that the AWGN channel behaves more like the BEC than the BSC, especially at high channel SNR.

3.2. Absorbing Sets as Deviations

Absorbing sets project to deviations on the computation tree in a different way than stopping sets. Since each check node connected to a stopping set 𝒮 is connected at least twice, a deviation can easily be defined using only nodes in 𝒮 on any computation tree rooted at a node in 𝒮. Unlike stopping sets, absorbing sets can have only a single connection to a check node. When an absorbing set has a single connection to a check node, it is not possible to form a deviation on any computation tree using only the variable nodes in 𝒜, unless there is a stopping set 𝒮⊆𝒜. If 𝒮⊆𝒜, a deviation can be formed on the computation tree by simply avoiding variable nodes in 𝒜 that are not in 𝒮.

For an absorbing set 𝒜 that does not contain a stopping set, it is of interest to know how the absorbing set manifests itself as a deviation on the computation tree rooted at one of the variable nodes in 𝒜. This manifestation takes the form of a deviation with as many variable nodes in 𝒜 as possible. Because it is known that each variable node is connected to strictly more satisfied check nodes than unsatisfied check nodes, it is possible to compute a bound on the number of variable nodes in a deviation 𝛿 that are contained in 𝒜 for regular LDPC codes.

Consider an absorbing set 𝒜 on the Tanner graph of a (𝑑_𝑉,𝑑_𝐹)-regular low-density parity-check code. Let each variable node 𝑣_𝑖∈𝒜 be connected to at least 𝑑_𝒜>𝑑_𝑉/2 satisfied check nodes. A computation tree rooted at a variable node 𝑣_𝑖∈𝒜 after ℓ iterations is given by 𝑅^ℓ_𝑣𝑖. A deviation on this computation tree can be constructed by selecting the nodes in the deviation level-by-level. The deviation construction begins by including the root node 𝑣_𝑖 at level ℓ=0. At the next level, one variable node connected to each of the check nodes in 𝑁(𝑣_𝑖) must be included in 𝛿. When possible, variable nodes in 𝒜 will always be included in 𝛿. Therefore, after ℓ=1 there are at least 1+𝑑_𝒜 variable nodes in 𝛿 that are also in 𝒜, and at most 𝑑_𝑉−𝑑_𝒜 variable nodes in 𝛿 that are not in 𝒜. Continuing to level ℓ=2 in the computation tree, each of the 𝑑_𝒜 variable nodes in 𝒜 at level ℓ=1 in 𝛿 connects to 𝑑_𝒜−1 new variable nodes in 𝒜 and 𝑑_𝑉−𝑑_𝒜 new variable nodes not in 𝒜. Each of the 𝑑_𝑉−𝑑_𝒜 variable nodes at level ℓ=1 in 𝛿 that are not in 𝒜 connects to 𝑑_𝑉−1 variable nodes that are not it 𝒜. After ℓ=2, the number of variable nodes in 𝛿 that are also in 𝒜 is|𝛿_𝒜|≥1+𝑑_𝒜^ℓ_𝑖=1(𝑑_𝒜−1)^𝑖−1. (21) Similarly, the total number of variable nodes in 𝛿 is|𝛿|=1+𝑑_𝑉^ℓ_𝑖=1(𝑑_𝑉−1)^𝑖−1. (22) Therefore, the number of variable nodes in 𝛿 that are not in 𝒜 is |𝛿|−|𝛿_𝒜|.

It is clear from (21) and (22) that the lower bound on the portion of variable nodes in 𝒜 within the deviation 𝛿 given by |𝛿_𝒜|/|𝛿| approaches zero as ℓ approaches infinity. Thus, the bound does not appear to be an accurate method for calculating the true portion. The bound given by (21) is computed under the worst-case scenario that, after a deviation reaches a check node with only one connection to the set 𝒜, the descendants of that check node contained in the deviation do not contain any more variable nodes in 𝒜. On a connected Tanner graph with no nodes of degree one and 𝑑_𝒜≠𝑑_𝑉, this assumption is most likely never true, since nodes in 𝒜 will eventually (with increasing ℓ) be included in the descendants of the failed check node, and consequently the variable nodes will also be included in any deviation which is a manifestation of 𝒜.

For finite-length, (𝑑_𝑉,𝑑_𝐹)-regular low-density parity check codes, it is possible to determine the exact number of variable nodes |𝛿_𝒜| after a given number of iterations ℓ. In order to find |𝛿_𝒜|, each variable node 𝑣_𝑖∈𝒜 is assigned a LLR cost of 𝜆_𝑖=0.0 and each variable node 𝑣_𝑗∈𝑉⧵𝒜 is assigned a LLR cost of 𝜆_𝑗=+1.0. Then, using the resulting LLR cost vector, MS decoding is performed for ℓ iterations. The final cost 𝑚_𝑖 for any variable node 𝑣_𝑖∈𝒜, is the number of nodes in the minimum-cost deviation on the computation tree rooted at 𝑣_𝑖 after ℓ iterations. Deviations for (𝑑_𝑉,𝑑_𝐹)-regular LDPC codes contain a fixed number of variable nodes determined by (22), and the only way to reduce the cost of the deviation is to include variable nodes from the set 𝒜. Thus, the minimum-cost deviation on the computation tree of a (𝑑_𝑉,𝑑_𝐹)-regular LDPC code will include the maximum number of variable nodes in 𝒜 that is possible, and the MS decoder will output the cost of this deviation. It is important to note that the cost of the deviation returned by the MS decoder corresponds to the number of variable nodes in the deviation that are not in 𝒜. Therefore, in order to determine the number of variable nodes in the deviation that are in 𝒜, it is necessary to subtract this MS decoder cost from the result given by (22).

Example 1. Consider the length 𝑁=20, dimension 𝐾=10, (3,6)-regular low-density parity-check code defined by the parity check matrix𝐻=01000010010010010001001001000010100100100100100100100000110000110000010001101000001010001001001000010100010010010100000100010011000100010010100100001010000010101000010101000010010010001010000011000100. (23)A (3,1) fully absorbing set is defined by the set of variable nodes 𝒜₁={𝑣₃,𝑣₇,𝑣₁₆}, where columns 1 through 20 of 𝐻 corresponds to the set of variable nodes {𝑣₀,𝑣₁,…,𝑣₁₉}. After 50 iterations, each of the deviations on each of the 𝑁 computation trees contains |𝛿|≈3.378×10¹⁵ variable nodes, as computed by (22). After setting the LLR costs for variable nodes in 𝒜₁ to 𝜆=0.0, and all other LLR cost to 𝜆=+1.0, the output of the MS decoder after 50 iterations for variable node 𝑣₇ is a cost of 𝑚_𝑣7=3.044×10¹⁴. Therefore, the minimum-weight deviation contains 3.073×10¹⁵=3.378×10¹⁵−3.044×10¹⁴ variable nodes also contained in the absorbing set 𝒜₁. Thus, a deviation can be formed on the computation tree with 91% of its variable nodes coming from 𝒜₁. In contrast, the bound given by (21) for (3,6)-regular LDPC codes after 50 iterations with 𝑑_𝒜=2 only guarantees that 101 variable nodes from 𝒜₁ will be included in the minimum weight deviation. This huge disparity between the bound and the actual result given by MS decoding reveals the effects of the assumptions used to derive (21). It is worth noting that the proportion of variable nodes in 𝒜₁ in the minimum weight deviation after 51 and 52 iterations were also 91%, so this proportion appears to stabilize after a sufficient number of iterations.
Using this same method on the (3,3) fully absorbing set 𝒜₂={𝑣₀,𝑣₇,𝑣₁₈}, it is found that the minimum weight deviation after 50 iterations contains 2.696×10¹⁵ copies of the variable nodes in 𝒜₂, equivalent to 78% of the total number of variable nodes in the deviation. By comparing the properties of the minimum-weight deviations resulting from 𝒜₁ and 𝒜₂, one might expect that 𝒜₁ is more likely to cause errors than 𝒜₂. This is because 91% of the cost of the deviation resulting from 𝒜₁ is determined by the variable nodes in 𝒜₁, compared to only 78% for 𝒜₂. Simulation results in Section 4 show that 𝒜₁ causes errors much more frequently than 𝒜₂.

4. Finding Problematic Trapping Sets

Any set of nodes can be interpreted as a trapping set, including stopping sets, absorbing sets, and fully absorbing sets. This is because trapping sets are only defined by the number of variable nodes in the set and the number of failed check nodes. In order to simplify analysis, the trapping sets studied in this paper are restricted to the study of problematic trapping sets.

Definition 5 (Problematic Trapping Set). A problematic trapping set is a trapping set such that the number of failed check nodes connected to the trapping set is less than or equal to the number variable nodes contained in the trapping set.

Because trapping sets with small weight and a small number of failed check nodes are often the cause of errors at high SNR [17], it is unlikely that the restriction to problematic trapping sets will eliminate error patterns of interest. In Section 3, it was shown that MS decoding can be used to determine the proportion of variable nodes that are both inside and outside an absorbing set. The same idea is used in this section to find problematic trapping sets using MS decoding. The iterative method given by Algorithm 1 operates by forcing the trapping set to contain one variable node, and then adding more variable nodes one-by-one that decrease the cost of the minimum-cost deviation the most.

for 𝑖fixed=1,…,𝑁   Set 𝑚min=∞.   Set 𝜒={𝑖fixed}.   while 𝑚min>0.0     for 𝑖=1,…,𝑁     -Set 𝜒=𝜒⋃{𝑖}.     -Set 𝜆𝑘=0.0 for all 𝑘∈𝜒.     -Set 𝜆𝑘=1.0 for all 𝑘∈𝑉⧵𝜒.     -Perform MS Decoding for ℓ iterations.      if min𝑗=1,…,𝑁𝑚𝑣𝑗<𝑚min        -Set 𝑚min=min𝑗=1,…,𝑁𝑚𝑣𝑗.        -Set 𝑗min=argmin𝑗=1,…,𝑁𝑚𝑣𝑗.       end      -Set 𝜒=𝜒⋂({𝑉⧵{𝑖}}⋃{𝑖fixed}).      end     -Set 𝜒=𝜒⋃{𝑗min}.     -Create a binary vector v with 𝑣𝑘=1 if 𝑘∈𝜒, and       𝑣𝑘=0 if 𝑘∈𝑉⧵𝜒.     -Compute the integar syndrome 𝐬int=𝐻𝐯𝑇.     -Compute the binary syndrome 𝐬bin=𝐻𝐯𝑇 with       Hamming weight 𝑤𝑠.     -Compute the integar vector 𝐳=𝐻𝑇𝐬bin      if min𝑘=1,…,𝑀𝑠int,𝑘≥2        𝜒 is a (|𝜒|,𝑤𝑠) Stopping Set.     end     𝜒 is a (|𝜒|,𝑤𝑠) Trapping Set.     if 𝑧𝑘<⌊𝑑𝑣𝑘2⌋ for all 𝑘∈𝜒          𝜒 is a (|𝜒|,𝑤𝑠) Absorbing Set.     end     if  𝑧𝑘<⌊𝑑𝑣𝑘2⌋ for all 𝑘=1,…,𝑁         𝜒 is a (|𝜒|,𝑤𝑠) Fully Absorbing Set.     end   end end

Algorithm 1 

Iterative problematic trapping set finder.

Once Algorithm 1 reaches cost 𝑚_min=0.0, it is possible to discover more problematic trapping sets by removing nodes one-by-one from the set 𝒳 that result in the smallest increase in the cost 𝑚_min. In order to examine the efficacy of Algorithm 1, the length 𝑁=20, dimension 𝐾=10, (3,6)-regular low-density parity-check code given in Example 1 was used. This code was chosen because the Hamming weight of all of its codewords could be enumerated, and thus the minimum distance of the code could be easily computed. Applying Alogorithm 1 to this code resulted in a total of 37 trapping sets with parameters shown in Table 3. Note that the proportion of nodes in each set that are included in the minimum weight deviation is given by “Dev. %”. This code was found to have minimum distance equal to 4, so only stopping/trapping/absorbing sets of weight less than or equal to 3 are tabulated.

Table 3 

Stopping/trapping/absorbing sets of a length 𝑁=20, dimension 𝐾=10, (3,6)-regular LDPC code with weight of 3 or less found using Algorithm 1, and the number of times they were observed after 1000 iterations of SP decoding at SNR = 8.0 dB.

In order to determine how effective Algorithm 1 is at locating problematic trapping sets, the same length 𝑁=20, dimension 𝐾=10, (3,6)-regular low-density parity-check code was simulated using sum-product decoding over the additive white Gaussian noise channel with an SNR of 𝐸_𝑏/𝑁₀ = 8.0 dB. A total of 1500 noncodeword outputs with weight less than or equal to 3 were observed during SP decoding after 1000 iterations. It is important to note that the noncodewords outputs were not simply the last quantized output given after 1000 iterations. The output of SP decoding typically changes after each iteration when it does not converge to a codeword. For this reason, the noncodeword outputs were computed by averaging the cost 𝑚_𝑣𝑖 for each variable node 𝑣_𝑖 from 𝑖=1,…,𝑁 over the last 200 iterations to compute a final output cost. This is similar to the method used in [14] for characterizing the changing outputs of the MS decoder.

Table 3 shows the number of observed output errors, and compares them to the problematic trapping sets found using Algorithm 1. Approximately 82% of the observed errors corresponded to one of the problematic trapping sets found using Algorithm 1. Also, the number of times a particular problematic trapping set is observed indicates how problematic the set is to the SP decoder. The single most problematic set, resulting in over 60% of the output errors, was the (3,1) set that satisfies the definitions of a stopping set, absorbing set, and fully absorbing set. Errors falling into the “Other” category were highly variable, and no specific output pattern in this set accounted for more than 6 of the total observed errors.

The problematic trapping sets with the highest proportion of nodes within their corresponding deviation were the (3,1) trapping sets. Not surprisingly, the (3,1) stopping set has the highest proportion of variable nodes in its deviation. While the proportions were noticeably different between different-sized sets, the difference was minimal within sets of the same size. Furthermore, it is difficult to make any connections between proportions within sets of the same size and their corresponding probability of causing an error. One possible reason for this might be the overlap between the different problematic trapping sets, and between the problematic trapping sets and codewords. For example, the reason that the (2,2) fully absorbing set {𝑣₆,𝑣₁₂} did not appear in the simulations might be because two of its variable nodes overlap with the exceptionally problematic (3,1) stopping set, and thus any significant channel noise received by variable nodes 𝑣₆ and 𝑣₁₂ may be highly likely to cause the (3,1) stopping set to be output by the decoder.

It is worth noting that the average value of received information within the absorbing sets of weight less than or equal to three was 𝑦=0.102115. Recall that a binary 0 is modulated to 𝑥=−1.0, so the mean value of the noise within the absorbing sets was +1.102115. This cost further justifies the earlier assertion that the AWGN channel behaves more like the BEC channel at high SNR than the BSC channel, since an erasure over the BEC can be interpreted as noise of +1.0 and a bit flip over the BSC can be interpreted as noise of +2.0.

Algorithm 1 was able to find 82% of the most problematic errors with weight less than 𝑑_min. In order to test the algorithm on an LDPC code with longer block length, a length 𝑁=200, 𝐾=100, (3,6)-regular LDPC code was used. The resulting output of Algorithm 1 was 1019 absorbing sets, 941 fully absorbing sets, and 1 stopping set, and the sizes of the sets ranged from 3 to 9. Only sets containing less than 10 variable nodes were considered problematic, since the code is known to contain a codeword of weight 10. Simulations were performed using SP decoding at SNR = 5.0 dB, at which the bit error rate of the code is 𝑃_𝑏=4.8×10⁻⁹. Overall, 40 noncodeword errors were observed, of which 20 were error patterns of weight less than 10. Of these, all were absorbing sets and 19 were fully absorbing sets. Unlike the results given for the length 𝑁=20 code, only two of the 20 absorbing sets was found by Algorithm 1. However, the two that were found were the two smallest absorbing sets, including (5,3) and (6,4) fully absorbing sets.

As expected, the number of trapping sets grows very large when increasing the size and dimension of the code. Algorithm 1 is capable of locating the majority of problematic trapping sets for a small length 𝑁=20 LDPC code, but for the larger length 𝑁=200 LDPC code it was only able to identify the two smallest sets observed during simulations. This is likely due to the fact that the code had not yet reached its error floor, as evidenced by the fact that half of the observed error patterns had weight greater than or equal to the weight of a known codeword. Because error floors occur at such low bit error rates for large LDPC codes, it is difficult to observe problematic trapping sets using simulations. Thus, the effectiveness of Algorithm 1 at identifying problematic trapping sets remains unknown for large codes with error floors beyond the reach of simulations.

5. The Weight of Deviations Induced byProblematic Trapping Sets

In [17], Richardson characterizes trapping sets by their size and the number of associated failed check nodes. To find the impact of the trapping sets with respect to probability of error, Richardson uses simulations that force the noise in the trapping set and push the received information away from modulated 0s and towards modulated 1s. The result is an estimate of the probability of error caused by trapping sets at high SNR. In this section, a new method is used to examine the probability of error associated with trapping sets. Instead of using simulations to estimate the probability of error, deviations induced by the trapping set are created to analyze the probability of error. Since bounds on the probability of error can be derived from deviations, if one could prove that minimum-weight deviations were induced by problematic trappings sets and then computed the weights of the deviations, it may be possible determine the probability of error associated with trapping sets without having to rely on simulations.

It may seem surprising that almost all problematic trapping sets listed in Table 3 result in a deviation where the variable nodes within the trappings set make up the majority of the variable nodes within that deviation. For example, consider the fully absorbing set given by the nodes {𝑣₁₁,𝑣₁₉}. While there are only two variable nodes contained in the absorbing set, they make up 76% of the variable nodes in a deviation that exists on the computation tree rooted at either variable node within the set. This implies that there might be a way of cleverly designing deviations which contain a disproportionately large number of certain variable nodes. A deviation rooted at variable node 𝑣₁₉ after 4 iterations is shown in Figure 8. This deviation was designed to include more copies of variable nodes 𝑣₁₁ and 𝑣₁₉ than other variable nodes. Overall, the number of copies of each variable node in the deviation is #𝑣₁₁=16, #𝑣₁₉=13, #𝑣₆=9, #𝑣₁₂=5, and #𝑣₁₅=3. Although the overall configuration contains 5 different variable nodes, almost 2/3 of them are copies of 𝑣₁₁ or 𝑣₁₉. Now, consider the subgraph of the Tanner graph defined by these 5 variable nodes, shown in Figure 9. This subgraph defines a (5,1) stopping set. It was shown in Section 3.1 that, because it is a stopping set, the deviation in Figure 8 could continue to grow indefinitely without the need for variable nodes outside the stopping set.

Figure 8 

Deviation designed to maximize the number of copies of 𝑣11 and 𝑣19.

Figure 9 

Subgraph of the Tanner graph.

To construct the deviation in Figure 8, decisions were made at check nodes 𝑓₀ and 𝑓₆. Those decisions are expressed in the directed bipartite graph shown in Figure 10. The number of decisions for each check node is equivalent to the number of edges incident to the check node. Check nodes 𝑓₁, 𝑓₄, 𝑓₅, and 𝑓₉ each have two copies in the directed graph to preserve the fact that they have bidirectional edges. However, check nodes 𝑓₀ and 𝑓₆ do not simply have bidirectional edges connecting them to each of their incident variable nodes, as demonstrated by the deviation in Figure 8. This comes from the fact that check nodes with degree higher than 2 are still only connected to 2 variable nodes within the deviation. Thus, at each check node with degree greater than 2, a decision is made as to which variable nodes it includes in the deviation.

Figure 10 

Subgraph of the Tanner graph with repeated check nodes and directed edges.

Using the adjacency matrix of the directed bipartite graph, it is possible to compute the number of copies, or the multiplicity, of each node at each level in the deviation. The method for computing the multiplicity of nodes in the deviation is similar to the method given in [18] for computing the multiplicity of nodes on computation trees. For the directed Tanner graph given in Figure 10, the adjacency matrix is given by𝐴=000100000000101000000010111000𝟎0000010000000100000100000000001110001010000001000000010001000000100100000100100000001𝟎01000000010001010000010001000000100. (24)

The columns and rows of 𝐴 correspond, in the same order, to the nodes 𝑣₆, 𝑣₁₁, 𝑣₁₂, 𝑣₁₅, 𝑣₁₉, 𝑓₀₁, 𝑓₀₂, 𝑓₀₃, 𝑓₀₄, 𝑓₁₁, 𝑓₁₂, 𝑓₄₁, 𝑓₄₂, 𝑓₅₁, 𝑓₅₂, 𝑓₆₁, 𝑓₆₂, 𝑓₆₃, 𝑓₉₁, and 𝑓₉₂. Beginning with a vector 𝐦₀ which has a 1 in the position of the root node and 0s elsewhere, the number of nodes on each level 𝑖 of the deviation can be calculated recursively using𝐦_𝑖=𝐴𝐦_𝑖−1,if𝑖iseven,𝐴𝐦_𝑖−1−𝐵𝐦_𝑖−2,if𝑖isodd, (25) where 𝐵 is used to subtract the parent check nodes at each check node level in the deviation. The matrix 𝐵 for the directed Tanner graph in Figure 10 is given by𝟎𝐵=00010000000011100110011001100000011100110. (26)

Using the recursive formula given by (25), the number of variable nodes at different levels in the computation tree is given in Table 4. The deviation at level 𝑖=100 exists on the computation tree rooted at variable node 𝑣₁₉ after ℓ=50 iterations. This deviation contains a total of 3.38×10¹⁵ variable nodes, over 63% of which are copies of 𝑣₁₁ and 𝑣₁₉. Using a similar method to that given in [19], the effective weight of the deviation can be calculated. First, let 𝑎_𝑣𝑖 be the number of copies of variable node 𝑣_𝑖 within the deviation, and let 𝑏 be the total number of variable nodes in the deviation. The cost of the deviation is modeled by the normal distribution 𝒩(_{𝑣𝑖∈𝒮}𝑎_𝑣𝑖𝑏,_{𝑣𝑖∈𝒮}(𝑎_𝑣𝑖𝑏𝜎)²), (27)

Table 4 

The multiplicity of variable nodes contained in the deviation created using the directed graph given in Figure 10.

where 𝜎² is the variance of the AWGN noise. This random variable can be rescaled resulting in the distribution ∑𝒩((^{𝑣𝑖∈𝒮}𝑎_𝑣𝑖∑_{𝑣𝑖∈𝒮}(𝑎_𝑣𝑖)²)²∑,(^{𝑣𝑖∈𝒮}𝑎_𝑣𝑖∑_{𝑣𝑖∈𝒮}(𝑎_𝑣𝑖)²)²𝜎²), (28)

where the mean(∑_{𝑣𝑖∈𝒮}𝑎_𝑣𝑖∑_{𝑣𝑖∈𝒮}(𝑎_𝑣𝑖)²)², (29) is the weight of the deviation. Note that if all 𝑎_𝑣𝑖 were equal, the weight of the set 𝒮 would be equal to the number of nodes it contains, which is consistent with the notion of Hamming weight when 𝒮 is equal to a codeword. From Table 4, the weight of the deviation created from the directed Tanner graph in Figure 10 after ℓ=50 iterations is 3.8784. This is less than the Hamming weight of the minimum distance codeword in the code, which has weight 4.0. Thus, the minimum weight of deviations on the computation tree rooted at 𝑣₁₉ is probably less than the minimum distance of the code.

6. Conclusion

Practical methods for predicting and understanding the performance of low-density parity-check codes with iterative decoders are needed in order to avoid the use of codes with error floors. Trapping sets, which include absorbing sets and stopping sets, provide insight into the error mechanisms of iterative decoders but are too imprecise to be used to make design decisions with respect to error floors. Deviations on computation trees are precise and can be used to compute strict upper bounds on the performance of MS and SP decoding, but computing these bounds quickly becomes computationally intractable. The paper examined the connections between trapping sets and their corresponding deviations through the notion of problematic trapping sets in an attempt to find a practical and precise method for predicting the performance of LDPC codes with iterative decoding.

It was shown that the variable nodes in a stopping set can be used to define a deviation, while trapping sets and absorbing sets only define a deviation if a subset of their variable nodes forms a stopping set. When trapping sets and absorbing sets do not include a stopping set, it is necessary to include additional variable nodes in order to construct a corresponding deviation. The number and proportion of variable nodes outside the set that are needed to construct the deviation can be found experimentally using a modification of the MS decoder. This modified MS algorithm leads to an iterative method for identifying low-weight problematic trapping sets in an LDPC code. Simulation results demonstrate that this method is capable of finding many of the low-weight trapping sets that determine the performance of LDPC codes at moderate SNRs. The efficacy of this algorithm is limited by computational constraints.

Finally, an analytical approach for determining the weight of deviations induced by trapping sets on the computation tree was introduced. This approach involves determining the minimum-weight stopping set that contains a given trapping set, and then determining a directed Tanner graph from the stopping set that favors certain variable nodes within the trapping set. It was then shown that the effective weight of the deviation can be found using a recursive method for computing the multiplicity of variable nodes within the deviation, In one example, it was proven that a deviation exists on the computation tree with weight less than the Hamming weight of the code. This result shows that trapping sets probably result in a necessary condition for an error to occur during iterative decoding, and in certain cases, this condition is satisfied with probability higher than the probability of an ML codeword error.

Acknowledgments

This paper was funded in part by AFOSR Contract FA9550-06-1-0375 and Department of Education Grant no. P200A070344.