Abstract

In conventional stochastic computation, all the input streams are Bernoulli sequences (BSs), which may result in large random error. To reduce random error and improve computational accuracy, some other sequences have been reported as alternatives to BSs. However, these sequences only apply to the specific stochastic circuits, have difficulties in hardware generation, or have length constraints. To this end, new sequences without these disadvantages should be considered. This paper proposes the random error analysis method for stochastic computation based on autocorrelation sequence (AS), which is more general than the conventional one based on BS. The analysis results show that we can use the proper ASs as input streams of stochastic circuits to reduce random error. On the basis of that conclusion, we propose the random error reduction scheme based on maximal concentrated autocorrelation sequence (MCAS) and BS, both of which are ASs. MCAS and BS are applicable to any combinational stochastic circuit, are easily generated by hardware, and have no length constraints, which avoid the disadvantages of sequences in the previous work. Moreover, we apply the proposed random error reduction scheme into several typical stochastic circuits as case studies. The simulation results confirm the effectiveness of the proposed scheme.

1. Introduction

Stochastic computation, first introduced in 1960s [1], has been widely used in many fields: neural network [2, 3], digital image processing [4–7], channel decoding [8–10], MIMO detection [11, 12], reliability evaluation [13], and so on. The main idea of stochastic computation is to represent continuous values with stochastic bit streams, which makes it possible to perform complex arithmetic computations with simple bitwise operations. One obvious advantage of stochastic computation is that the hardware implementation of the stochastic arithmetic is much simpler than the traditional arithmetic. For example, as shown in Figure 1, multiplication and scaling addition can be performed with an AND gate and a multiplexer, respectively. The simplicity of hardware also brings short critical path and high clock rate. Moreover, all the bits in a stochastic bit stream have the same significance. That means the stochastic arithmetic is unary coding arithmetic which has better fault tolerance than the traditional binary arithmetic. Recently, skew tolerance has also been introduced as another inherent advantage of stochastic computation.

(a)

(b)

(a)
(b)

Figure 1 

Stochastic circuit of arithmetic. (a) Multiplication; (b) scaling addition.

Despite the above advantages, a disadvantage of stochastic computation is that the random fluctuations of stochastic bit streams introduce the random error, which causes the computation result to be a random variable. In [4], it is suggested that, in most cases, the random error will dominate the other errors such as quantization error and approximate error. Thus, the random error should be carefully investigated and addressed in stochastic computation area.

Thus far, many statistical models for random error analysis have been proposed. They are all based on variance evaluations. In the early work, it is assumed that all the input streams of stochastic circuits are BSs. Qian et al. [4] propose the variance evaluation model base on BSs. However, in real application, the random error for stochastic arithmetic based on BSs is often too large. Therefore, people have made effects to reduce random error by using other sequences as input streams of stochastic circuits.

Braendler et al. [3] employ two deterministic bit streams (DBSs) as input streams for the stochastic circuit AND gate. The output stream is also deterministic, which means there are no random fluctuations. Although it eliminates random fluctuations and improves computational accuracy, the use of these DBSs is narrowly restricted to the single AND gate. They can not apply to more complex stochastic circuits.

Alaghi and Hayes [14] propose a general framework for analyzing stochastic circuits with correlated inputs. They demonstrate that using correlated inputs can significantly reduce the random error for some stochastic circuits. However, the use of correlated inputs has its limitation. For example, as illustrated in [15], if the two input streams applied to an AND gate are correlated, say, each has the same bit-pattern of value , then the output will also have value instead of . Hence, the AND gate will not perform multiplication if its inputs are correlated. Similarly, many other commonly used stochastic computational elements can not perform the expected arithmetic with correlated inputs, which largely restricts their applications in stochastic computation systems.

Han et al. [13] use fixed-ones random-permutation sequences (FRSs) as input streams for stochastic circuits. They present the variance evaluation model for the basic stochastic computational elements, including AND gate and invert. Furthermore, they prove that the use of FRSs as input streams reduces random error for stochastic computation compared to the conventional use of BSs. FRSs can be wildly used in stochastic arithmetic. They apply to any combinational stochastic circuit. In [13], FRSs are generated by software and used as input streams of the stochastic circuit for reliability evaluation. However, the low cost hardware generation of FRS is difficult to implement. Thus, FRSs are not suitable for stochastic circuits with input streams generated by hardware.

Najafi et al. [16] use pulse-width modulation (PWM) to time-encode values for more efficient stochastic computing. They represent values by periodic PWM signals. If two PWM signals are not harmonically related, stochastic multiplication of numbers represented by them obtains the best accuracy when running the operation for the least common multiple (LCM) or multiples of the LCM of the period of the inputs. For instance, is represented by a PWM signal with period of 5, namely, , and is represented by a PWM signal with period of 3, namely, . The LCM of the period of and is 15. Thus, when multiplying and with an AND gate, let both of the two bit streams run for 15 clock cycles; that is, , and . Taking the bitwise AND of and , the output stream is , which represents . However, if the operation runs for only 10 clocks, that is, , and , the output stream is , which represents , not the expected value. Similarly, some other stochastic arithmetic, using PWM signals as input streams, can get the expected results only if the operations run for the required time. Hence, we can see that there are length constrains on stochastic computation based on PWM signals. The length of input streams is not flexible but determined by the period of input streams.

From above, we can see that the main issues of stochastic arithmetic based on different input streams are as follows:(i)BSs: large random error;(ii)DBSs or correlated inputs: only applicable for the specific stochastic circuits;(iii)FRSs: difficulties in hardware generation;(iv)PWM signals: length constraints.

To avoid these disadvantages, we need to use other proper sequences as input streams. Considering the autocorrelation between different bits of a sequence, we propose the notion of AS. And then we analyze the random error of stochastic arithmetic on the assumption that input streams are ASs. By definition, BS can be seen as a special AS with equivalent autocorrelation coefficients. Thus, the proposed random error analysis method based on AS is more general than the conventional one based on BS.

According to our analysis results, if proper ASs are used as input streams, the random error can be reduced. Based on that conclusion, we investigate how to reduce random error for any combinational stochastic circuit by employing MCASs and BSs, both of which are ASs, as input streams. Then, We adopt several typical stochastic circuits as case studies and find that the use of MCASs and BSs can significantly improve the calculation performance. MCAS has no difficulties in hardware generation and the hardware cost of the MCAS generator is merely the same as that of the conventional BS generator. Moreover, from the simulation results, the random error reduction scheme based on MCAS and BS is valid for any input stream length and has no length constraints. Hence, by using MCASs and BSs as input streams, we avoid the disadvantages in the previous work and achieve the following goals:(i)Random error reduction: the random error is reduced compared to using only BSs as input streams;(ii)Wide applications: the random error reduction scheme based on MCAS and BS is applicable to any combinational stochastic circuit;(iii)Easy hardware generation: the MCAS and BS generator are easily implemented by hardware;(iv)Flexible length: the MCAS and BS have no length constraints.

Parts of this paper are based on our prior work entitled “Random Error Analysis and Reduction for Stochastic Computation Based on Autocorrelation Sequence,” which has been presented at the IEEE International Symposium on Circuits and Systems (ISCAS), 2014. In the conference paper, we briefly present a general random error analysis method for stochastic computation and illustrate how to reduce random error for a typical stochastic circuit. Compared with the conference version, this journal manuscript has more new contents updated and more insightful discussion, which are stated as follows:(i)Detailed illustration of the general random error analysis method is provided.(ii)The random error reduction scheme for any combination stochastic circuit is proposed.(iii)The complete mathematical proof for the effectiveness of the proposed random error reduction scheme is given.(iv)More typical stochastic circuits are presented as case studies.

The rest of the paper is structured as follows: Section 2 analyzes the random error based on AS. Section 3 illustrates the random error reduction scheme based on MCAS and BS for any combinational stochastic circuit. Section 4 presents several typical case studies. Section 5 gives simulation results. Finally, we give conclusions and future directions of research.

2. Random Error Analysis Method

2.1. Expectation and Variance

In stochastic computation, a stochastic bit stream with length of can be written as The expectation of is defined by and the variance of is defined by

Consider an arithmetic operation with normalized input numbers and a result , which can be expressed by . In the stochastic implementing of , each input number , , is represented by the corresponding stochastic bit stream, denoted by , where . We employ these stochastic bit streams as input streams of stochastic circuit. Then, the stochastic computational result, denoted by , can be obtained from the output stream as follows:

There are two important statistics for stochastic arithmetic. The first one is the expectation of output stream. Since should be an unbiased estimator of the actual result , we have From (2) and (4), (5) can be rewritten as . Thus, the expectation of output stream indicates the arithmetic operation performed by stochastic circuit.

Another important statistic is the variance of output stream. Considering that is an unbiased estimator of the actual result , random error can be calculated in the form of mean square error (MSE) as By (3) and (6), . Therefore, the random error for stochastic computation can be measured by the variance of output stream.

2.2. Autocorrelation Sequence

For a bit stream with length of , the correlation between two different bits can be measured by autocorrelation coefficient, which is defined by , . For example, if is a BS, its autocorrelation coefficients are , ; that is, all the autocorrelation coefficients of BS are equivalent to the square of its expectation.

In the practical applications, input streams may have a variety of autocorrelation coefficients. Considering that, we propose AS, whose autocorrelation coefficients are not necessarily equivalent to each other.

Definition 1. A bit stream , which represents the probability , is called an AS on the following condition:

From Definition 1, AS is more general compared to BS and BS can be seen as special case of AS whose autocorrelation coefficients are equivalent to the square of its expectation.

2.3. Random Error Analysis Based on AS

For clarity, we define the following notations: let BI denote the case that all the input streams are BSs; let AI denote the case that all the input streams are ASs; let denote the expectation of output stream in the case of BI; let denote the expectation of output stream in the case of ; let denote the variance of output stream in the case of ; let denote the variance of output stream in the case of AI.

For any stochastic circuit, we expect that the arithmetic computed in the case of AI is the same as that in the case of BI. As illustrated in Section 2.1, the expectation of output stream indicates the arithmetic operation performed by stochastic circuit. Hence, the expectations of output stream should be equivalent to each other in the two cases and we have Equation (7) is the essential condition for the random error analysis in the case of AI. On the premise of (7), the random error base on AS can be measured by the variance of the output stream as follows:

Example 2. As shown in Figure 1, two stochastic bit streams and are employed as the input streams for an AND gate. When and are BSs, , , . The expectation of output streams can be calculated as Now, consider that the input streams and are ASs with the following properties: or , . Without losing generality, we suppose that . The expectation of output stream can be calculated as By (9) and (10), (7) is satisfied. Thus, the random error can be calculated by (8) as follows: Hence, the variance of output stream in the case of AI is determined by the expectations and autocorrelation coefficients of input streams.

2.4. AI Affection Evaluation

In order to evaluate how much AI affects the random error, we propose differential variance, denoted by , where As suggested in [4], the variance of output stream in the case of BI can be calculated by Then we have the following theorem.

Theorem 3. The range of DV is as follows:

Proof. See our prior work [17].

According to Theorem 3, the upper bound of DV is nonnegative and the lower bound of DV is nonpositive. Thus, DV can be positive, which indicates that the improper use of AS will result in the increase of random error. On the other hand, DV can be negative, which indicates that the proper use of AS will reduce random error. Therefore, we get the following conclusions.(i)Since the random error in the case of AI can be less than that in the case of BI, the random error reduction scheme based on AS is feasible.(ii)Since the random error in the case of AI can be greater than that in the case of BI, it is important to select proper AS to avoid this situation.

3. Random Error Reduction Scheme

In the last section, we find that the proper use of AS is able to reduce the random error for stochastic computation. On the basis of that conclusion, we will discuss how to employ ASs as input streams for the combinational stochastic circuit to reduce the random error.

3.1. Combinational Stochastic Circuit

For any combinational stochastic circuit, the relationship between input streams and output stream can be expressed by where is a constant integer and , . For convenience of illustration, we denote by the subscript . The formulation , where , is called a product term and denoted by . Then, (14) can be rewritten as , where . For the product term , if , is called a factor of . Let denote the set of subscripts of product terms that have as a factor; that is, . Similarly, let denote the set of subscripts of input streams that are factors of ; that is,

Definition 4. An input stream is called a positive input stream (PIS), if it satisfies The set of subscripts of PISs in stochastic circuit is denoted by .

Example 5. Consider the stochastic circuit with Boolean function as follows: , and we have which can be rewritten in the form of (14) as There are three product terms and . is a factor of and , is a factor of and , and is a factor of and . Therefore, , , and . Meanwhile, has and as factors, has as a factor, and has , , and as factors. Hence, , , and .

In order to see whether is a PIS, by (17), we have , , . Thus, (15) is satisfied, and is a PIS. Similarly, and are also PISs. for the stochastic circuit in Example 5 is .

3.2. Mathematical Model Based on MCAS and BS

Definition 6. If a bit stream has the following properties: where is the length of , it is called an MCAS.
For instance, an MCAS with and = 3/8 is as follows: 1, 1, 1, 0, 0, 0, 0, 0.

Consider the combinational stochastic circuit with the input streams and an output stream . In order to indicate the sequence type for each input stream, we introduce the state vector , where There are totally possible states for V. For instance, if , the four possible states are follows: (MCAS, MCAS), (MCAS, BS), (BS, BS), and (BS, MCAS). The expectation and variance of output stream , when state vector is V, are denoted by and , respectively.

As discussed in Section 2.3, in order to guarantee that the arithmetic computed by the combinational stochastic circuit, when state vector is V, is the same as that in the case of BI, V should satisfy On the premise of (21), if the state vector V satisfies then the random error is not greater than that in the case of BI. Thus, the state vector V satisfying (21) and (22) represents a random error reduction scheme for the combinational stochastic circuit.

In order to describe the properties of state vector V, we define the following notations. Let denote the set of subscripts of input streams that are MCASs; that is, . Let denote the set of subscripts of input streams that are MCASs and factors of ; that is, . Let denote the set of subscripts of product terms, of which at least one factor is an MCAS; that is, . Let denote the set of subscripts of product terms whose factors are all BSs; that is, . For instance, in Example 5, assuming that the state vector , then , , , , , and . We also denote by the cardinal of the set . Using the above notations, we have the following lemmas and theorems.

Lemma 7. For any combinational stochastic circuit, if the state vector satisfies then one has that .

Proof. See Appendix A.

Lemma 8. Two input streams and , which are both MCASs, have the following properties:

Proof. See Appendix B.

Theorem 9. For any combinational stochastic circuit, if the state vector satisfies then one has , and .

Proof. See Appendix C. Lemmas 7 and 8 will be used in the proof.

Theorem 10. For any combinational stochastic circuit, if the state vector satisfies then one has , and .

Proof. See Appendix D. Lemmas 7 and 8 will be used in the proof.

Theorems 9 and 10 give two different sufficient conditions for the state vector satisfying (21) and (22), respectively. On the basis of the two theorems, we will propose the random error reduction scheme for any combinational stochastic circuit in the next subsection.

3.3. Random Error Reduction Scheme Based on MCAS and BS

In this subsection, we illustrate how to get the state vector V which represents the random error reduction scheme based on MCAS and BS for any combinational stochastic circuit. Specific steps are stated in Algorithm 1.

Now, we prove that the state vector V obtained from Algorithm 1 satisfies (21) and (22). Consider the following two cases: (I) is a null set; (II) otherwise. In case (I), the valid steps are (), (), (), and (). From criterion (a) in step (), (27) is satisfied. According to Theorem 9, the state vector V obtained from these steps satisfies (21) and (22). In case (II), the valid steps are (), (), (), and (). From criterion (a) and (b) in step (), (28) and (29) are satisfied. According to Theorem 10, the state vector V obtained from these steps also satisfies (21) and (22). Therefore, in both cases, the state vector V obtained from the corresponding steps satisfies (21) and (22).

3.4. Hardware Cost and Power Consumption

In conventional stochastic computation, the input streams for stochastic circuits are all BSs. However, in the proposed random error reduction scheme, some input streams are specified as MCASs. Different input streams are generated by different generators. The hardware structures of the MCAS generator and the BS generator are shown in Figure 2. The MCAS generator consists of an -bit comparator and an -bit upcounter, while the BS generator consists of an -bit comparator and an -bit Liner Feedback Shift Register (LFSR), where is the length of the input stream to be generated. In stochastic computation system, all the MCAS generators can share the same -bit upcounter. By contrast, each BS generator requires a different -bit LFSR. Thus, the total hardware cost of MCAS generators is usually lower than that of BS generators. That is to say, the proposed random error reduction scheme does not require any additional hardware cost.

(a)

(b)

(a)
(b)

Figure 2 

Hardware structure of the input stream generators: (a) MCAS generator; (b) BS generator.

Moreover, due to a lower switching activity, MCAS generator will have less dynamic power consumption than BS generator, which is another inherent advantage of our random error reduction scheme.

4. Case Study

For better understanding of the random error reduction scheme based on MCAS and BS, we take several typical stochastic circuits as case studies. We assume that all the input streams are uncorrelated.

4.1. Case Study 1: AND Gate

In this case study, we illustrate how to get the random error reduction scheme when is not a null set. An AND gate in stochastic logic implements a binary operator multiplication as . We obtain the state vector V satisfying (21) and (22) from the steps given in Algorithm 1.(1)The relationship between input streams and output stream can be written as (2), , and, , . Thus, by Definition 4, and are PISs, and . Since is not a null set, skip step () and go to step ().(3)Since is not a null set, this step is skipped.(4)Since and are PISs, by criterion (a), both of them are likely to be specified as MCASs. Hence, as shown in the first column of Table 1, is chosen from the following candidates: , , and . For each candidate of , we get all the corresponding and see whether criterion (b) is satisfied. If it is, the candidate is valid. For example, if , . Moreover, there is only one product term , and . Then, , , which is equivalent to (29). Criterion (b) is satisfied and the set is a valid candidate. Similarly, the set is also a valid candidate. However, if , , which is opposed to (29). Thus, criterion (b) is not satisfied and the set is not a valid candidate. For each valid candidate, we get the corresponding . As shown in the last column of Table 1, when = or , is maximal, and criterion (c) is satisfied. From above, the set satisfying all the criteria in step () can be written as . Without loss of generality, we set , and is specified as an MCAS. Then, go to step ().(5) is specified as a BS. Thus, the state vector .

4.2. Case Study 2: XOR Gate

In this case study, we illustrate how to get random error reduction scheme when is a null set. An XOR gate in stochastic logic implements the arithmetic as follows: . We obtain the state vector V satisfying (21) and (22) from the steps given in Algorithm 1.(1)The relationship between input streams and output stream can be written in the form of (14) as .(2)If , , and if , . Thus, by Definition 4, and are not PISs, and is a null set. Then, go to step ().(3)By criterion (a), only one of and can be specified as an MCAS. Hence, as shown in the first column of Table 2, is chosen from the following candidates: and . For each candidate, we get the corresponding . As shown in the last column of Table 2, when = or , is maximal, and criterion (b) is satisfied. From above, the set satisfying all the criteria in step () can be written as . Without loss of generality, we set , and is specified as an MCAS. Then, skip step () and go to step .(4)Since is a null set, this step is skipped.(5) is specified as a BS. Thus, the state vector .

4.3. Case Study 3: Stochastic Logic Implementing Bernstein Polynomial

In this case study, we illustrate how to get the random error reduction scheme for the stochastic logic implementing Bernstein polynomial with coefficients in the unit interval. A Bernstein polynomial of degree , denoted by , is defined by , where each is a constant coefficient. In [4], Qian et al. propose the stochastic circuit for the Bernstein polynomial with coefficients in the unit interval, which is shown in Figure 3. For a Bernstein polynomial of degree , there are input streams: , with the following properties: , , and . For simplicity, we will take the Bernstein polynomial of degree 2 as an example, where We obtain the state vector V satisfying (21) and (22) from the steps given in Algorithm 1. (1)The relationship between input streams and output stream can be rewritten in the form of (14) as (2), . Thus, by Definition 4, is a PIS. Similarly, and are also PISs, and . Since is not a null set, skip step () and go to step .(3)Since is not a null set, this step is skipped.(4)Since , , and are PISs, by criterion (a), all of them are likely to be specified as MCASs. Hence, as shown in the first column of Table 3, is chosen from the following candidates: , , , , , , and . For each candidate of , we get all the corresponding and see whether criterion (b) is satisfied. If it is, the candidate is valid. For example, if , then , , , , , , , and . Moreover, there are eight product terms, and , . Hence, ,  , which is equivalent to (29). Criterion (b) is satisfied and the set is a valid candidate. Similarly, the other sets in the first column of Table 3 are also valid candidates. For each valid candidate, we get the corresponding . As shown in the last column of Table 3, when , is maximal, and criterion (c) is satisfied. From above, the set satisfying all the criteria in step () can be written as . Then, go to step ((5) and are specified as BSs. Thus, the state vector .