Abstract
Logarithmic number system (LNS) is an attractive alternative to realize finite-length impulse response filters because of multiplication in the linear domain being only addition in the logarithmic domain. In the literature, linear coefficients are directly replaced by the logarithmic equivalent. In this paper, an approach to directly optimize the finite word length coefficients in the LNS domain is proposed. This branch and bound algorithm is implemented based on LNS integers and several different branching strategies are proposed and evaluated. Optimal coefficients in the minimax sense are obtained and compared with the traditional finite word length representation in the linear domain as well as using rounding. Results show that the proposed method naturally provides smaller approximation error compared to rounding. Furthermore, they provide insights into finite word length properties of FIR filters coefficients in the LNS domain and show that LNS FIR filters typically provide a better approximation error compared to a standard FIR filter.
1. Introduction
Finite-length impulse response (FIR) filters constitute a class of digital filters commonly used for their stability properties and the ability to obtain a linear phase response. The transfer function of an th-order FIR filter is where are the impulse response coefficients.
The filter order and, therefore, the number of multiplications and additions for a straightforward realization grows approximately inversely proportional to the transition bandwidth of the magnitude response [1, 2]. As multiplications traditionally have a larger area complexity and power consumption compared to additions, much work has focused on reducing the number of multiplications in FIR filter realizations by using sparse filters or frequency response masking filters [3–5]. Work has also been done to reduce the complexity of each multiplication, for example, by introducing filter coefficients easily realizable using shifts, additions, and subtractions, sometimes referred to as multiplierless realizations [6–8].
Furthermore, the representation of data and coefficients affects both the switching activity and implementation complexity which, in turn, affects power consumption as well. Commonly, a fixed-point two’s complement number representation is used to represent data in DSP systems, but other number representations have also been investigated as an efficient way of data representation for such systems [9–11]. Among them is the logarithmic number system (LNS) [12], which over the past few decades has been studied as an alternative to fixed-point number systems. The main motivation for doing so is the inherent simplification of basic arithmetic operations as multiplication, division, roots, and powers, due to its properties, which are reduced to addition, subtraction, multiplications, and divisions, respectively. They also have interesting numerical properties as higher dynamic range compared to fixed-point representations for a given number of bits [13] and better round-off noise performance than floating-point arithmetic for a given number of bits [14–16].
With LNS reducing multiplication to an addition, it finds application in low power signal processing systems, including digital filters [13, 17–23]. In [18, 19] LNS has also been proposed to reduce power dissipation in hearing aid devices [24], subband coding [25], and video processing [26]. It has been shown that LNS requires a reduced data word length to achieve the same SNR when compared to fixed-point representations [21, 23], with a reported power saving of nearly in some cases. However, no discussion about the coefficient word length was included nor how to obtain finite word length coefficients.
Most efforts towards utilizing LNS for digital have focused on either implementing the nonlinear conversion to and from LNS, selecting the logarithm basis, or implementing the LNS addition and subtraction efficiently [12, 23, 27–30]. The finite word length filter design has to the best of the authors’ knowledge not been considered but instead relied on rounding the obtained coefficients to the nearest LNS number.
In this work, an integer linear programming (ILP) approach to design optimal finite word length linear-phase FIR filters in the LNS domain is proposed. Here, instead of optimizing filters in the linear domain and converting them into LNS with rounding in the LNS, we optimize the filter directly in the LNS domain with finite word length constraints in the LNS domain. By optimizing the filters directly in the LNS domain it is also possible to compare the required word lengths to obtain further insights regarding the efficiency of LNS.
The rest of the paper is outlined as follows. In the next section LNS numbers are reviewed. In Section 3 ILP is reviewed and the issues involved by performing it in the LNS domain are discussed. This section also includes the formulation and the proposed variable selection and branching direction strategies. Finally, Sections 4 and 5 present the results and conclusions, respectively.
2. The Logarithmic Number System (LNS)
The LNS takes advantage of the fact that multiplications become additions, however, at the cost of increased complexity to implement addition. The LNS representation of a number consists of a triplet as follows [12]: where is a one-bit flag to indicate if is zero, is the sign of , and is the base- logarithm of the absolute value of [21]. Representation capabilities, computational complexity, and conversion to and from LNS numbers greatly depend on the choice of the base [10].
As stated before, the motivation behind using LNS is the simplicity of implementing the multiplication of and which is reduced to the computation of the triplet [21]: where or ; that is, the zero flag of the output, xor ; that is, the sign of the output and the output itself, .
The addition and subtraction are more complex and are given by (4)
In case of FIR filters, the filter coefficients are typically between and . Representing the coefficients in LNS means that all the exponents in the LNS domain will be negative. This means that the larger the magnitude in LNS domain, the smaller it is in the linear domain. As the coefficients become smaller, the LNS number magnitude becomes larger and reaches the maximum number representable using integer and fractional bits, given by . Important to note is that since the LNS numbers will always be negative for (most) FIR filter coefficients, there is no need to use a sign bit to represent the exponent in this case.
Any number smaller than this is not representable. This phenomenon is shown on a linear scale in Figure 1 where two scales are shown for one and two integer bits, respectively. As expected, the logarithmic numbers are unevenly spaced. However, there is a gap between the smallest nonzero number and zero which is much larger than the distance between the smallest and second smallest nonzero numbers. Any number that lies within this space is rounded to either zero or the smallest number. Hence, the number of integer bits will have an impact on the smallest number that can be represented.
(a)
(b)
2.1. Finite Word Length Effects
It is well established that coefficient quantization results in a static deviation of the transfer function, while data quantization results in round-off noise [1, 2]. In the linear domain the quantization happens after the multiplications, as the fractional word length is increased there. However, in the LNS domain, the fractional word length is not increased after the multiplications, as this corresponds to an addition. Instead, the quantization occurs in the addition, because addition, as shown in (4), requires look-up tables and cannot be represented exactly [15]. In the linear domain, the multiplier complexity grows linearly with the coefficient word length, while the adder complexity is independent of the coefficient word length. In the LNS domain, the multiplier complexity is dependent on the smallest of the data and coefficient word lengths, while the adder complexity is dependent on the largest of those.
3. Proposed Integer Linear Programming Design in the LNS Domain
3.1. Integer Linear Programming
Integer linear programming (ILP) is a class of linear programming where some or all the variables are restricted to have integer values [31]. Generally, the standard form to express a linear programming problem is given by where is the number of variables and is the number of constraints.
However, some real world problems require the result to be integers. This requirement is incorporated into the programming model by having an additional constraint . There are two important algorithms for solving ILPs: branch and bound (BB) and cutting plane. (Naturally, for FIR filters the “integers” are fixed-point values with a fractional part. However, to keep the standard nomenclature we select to denote any fixed-point value with integer.)
In this work we use the branch and bound (BB) algorithm which is a general algorithm for finding optimal solutions to integer or combinatorial problems. It consists of enumerating all candidate solutions and discarding candidates by using upper and lower bounds of the quantity being optimized.
ILP problems are nondeterministic polynomial (NP) hard. In order to solve them the integer constraint is relaxed and the variables are allowed to take noninteger values. This, and subsequent, LP-relaxations provide a bound on the best possible solution without the integer constraint. One noninteger variable is selected and branched on by forming two subproblems. One where the variable is restricted to be at least as large as the next higher integer value and one where it is restricted to be at most as large as the next lower integer value. This is illustrated in Figure 2, where the top node represents the LP-relaxation, variable is selected to branch on, and two subproblems are formed by adding the constraint on the edges. This procedure continues until any of the following happens.(1)The subproblem is infeasible, as any possible further branching will not change that.(2)The subproblem results in a solution where all coefficients are integers.(3)The solution of the subproblem is worse than the best obtained integer solution, and, hence, no further subproblems generated from that node can possibly have better values.
In any of these cases the algorithm returns to the previous node and selects another subproblem to solve until all possibilities are exhausted and the optimal integer solution is obtained.
The nontrivial challenge in ILP is to know which variable to branch on and in which order. We will suggest and evaluate a number of branching schemes in later sections.
3.2. Linear Programming Design of FIR Filters
Linear-phase FIR filters can be designed to be linear and are a good candidate for linear programming optimization. The work by Rabiner [32] is one of the earliest papers using this technique to solve digital FIR filter problems. However, this technique has its limitations when finite word length restriction is imposed [33].
In ILP, the FIR optimization problem is the same as in the linear programming case. A general FIR optimization problem can be stated as the following minimization problem [1]: where is the approximation error, is the number of frequency points, and where is the union of the passband and stopband regions and is a dense set of frequency samples taken from the passbands and stopbands, including the band edges. is the desired function to be approximated by . For a standard low-pass filter, is given by [1]
The weighting function specifies the cost of the deviation from the desired function. This basically means that by using this weighting function, one can obtain different deviations in ripples in different frequency bands. The larger the weighting function, the smaller the relative ripple. For a standard low-pass filter, the weighting function can be given as [1]
is the real zero-phase frequency response, which is related to the frequency response as [1]
The magnitude response of is equal to the magnitude of . However, can take on negative values where is a nonnegative function. The resulting filter will be optimized in the Chebyshev or minimax sense. Such filters are also called equiripple filters because all the ripples will be of equal size, subject to the weighting function [1].
The linear relaxation obtained without integer constraints will be the same minimax solution obtained using other FIR filter design techniques, such as the Remez exchange algorithm. Therefore, in later results the linear relaxation is used as a lower bound. Introducing integer constraints will constrain the solution space and increase the approximation error. Furthermore, the resulting filters may no longer be equiripple.
3.3. ILP Design of FIR Filters in the LNS Domain
In the LNS domain, our proposed design method focuses on finding the LNS equivalent of the coefficient values and implementing a branch and bound tree based on LNS integer values. In the branch and bound tree, the coefficients are, when picked as the branching variable, constrained to be LNS integers.
The solution space for an LNS ILP problem is different from that of an ILP problem using linear representation. An example of this is shown in Figure 3, clearly showing the difference between the two spaces.
In the LNS branch and bound tree, after solving a node, some of the coefficients will be LNS integers and some not. The linear relaxation of a noninteger coefficient, if selected, would be rounded up and down.
The first proposed branch variable selection scheme, named as Scheme 1, is to scan all the noninteger coefficients and pick the noninteger variable which has the largest absolute value. This may be efficient since this will introduce the largest quantization error in the linear domain, and, hence, possibly reduce the convergence time. Another scheme, denoted as Scheme 2, instead selects the coefficient, after scanning all noninteger coefficients, which is farthest from being an integer in the LNS domain. This is again based on the idea of introducing large quantization errors early.
For both schemes described above, both combinations of branching direction are evaluated. This means that for both schemes, the difference is calculated between the linear relaxation of the selected variable and its upper and lower bounds. In one implementation, the branching variable is constrained to the bound which gives the least difference first and then upon returning to that node constraining it to the other bound. In the other implementation, this selection is reversed.
A third scheme is proposed, denoted as Scheme 3, where the branching direction was limited to the bound with the least difference. In this scheme, all noninteger coefficients are scanned and the minimum distance to each of its bounds for each variable is calculated. The coefficient having the maximum of these minimum distances is selected as the branching variable.
All the above schemes either picked up the floor or ceiling value of the linear relaxation for the next branch without actually knowing whether that would produce a lower approximation error. The final proposed approach, Scheme 4, uses the same variable selection scheme as Scheme 3 but both subproblems are evaluated before the selection is done on which path to continue along. The decision is based on the most promising path, that is, the one with the smallest approximation error.
All these branch variable selection schemes are for locating the best variable to branch on. Apart from these schemes, the rest of the algorithm is the same for all cases. However, an error threshold is allowed while testing each coefficient for being an LNS integer, which means that the coefficient need not be exactly an LNS integer. Any number within a defined threshold of the corresponding LNS integer is taken as an LNS integer. This is a standard procedure in ILP solving as numerical errors in the computations may lead to noninteger values in the optimal solution, even though they in practice should be considered integers [31].
4. Results
For the results a number of low-pass filter specifications were devised arbitrarily to cover both narrow-band and wide-band filters. The passband and stopband edges are shown in Table 1 and the weighting function is 1 for both passband and stopband. Unless otherwise stated the logarithm base is selected as . Regarding word length, indicates the number of integer bits, and fractional bits, is used for the LNS integer, and denotes fractional bits for the linear integers.
4.1. Comparison of Branching Schemes
A number of optimization runs were carried out with different alternatives. The purpose was to see the efficiency of each scheme and see the impact of word length on different filters.
Table 2 shows the number of nodes visited (subproblems solved) for the first two schemes with different branching directions. “L” denotes that the branching direction is determined on the closest integer, while “H” then denotes the integer with the largest difference from the relaxed value. The number of integer and fractional bits is four and six, respectively, that is, . Table 2 establishes that branching on the direction based on the least difference gives the best result. Also, Scheme 1 proves superior to Scheme 2.
Scheme 1 is further compared with Schemes 3 and 4 and the results are shown in Table 3, again with . This table shows them to be comparable, apart from the case when the filter length is . This shows that Scheme 1 does not converge quickly when filter length is increased. This behavior is further observed in numerous optimization runs. For various integer and fractional bits and filter lengths, Scheme 3 gives, on average, the best convergence time. Scheme 1 appears to perform better when given a narrow transition band and small filter length.
The effect of word length is further studied and results are shown in Figure 4. In Figures 4(a) and 4(b), the number of nodes required to solve spec. , , for three and four integer bits, respectively, while the number of fractional bits varying from to is shown. Similarly, Figures 4(c) and 4(d) show the same for spec. , . When the filter length is , Scheme 1 converges in the least amount of time. Decreasing integer word length to three did not alter the efficiency of this scheme. However, as the filter length was increased while keeping the passband and stopband edges the same, one can see that decreasing the integer word length from four to three had a major impact on the convergence time of Scheme 1. Further optimization runs show the same effect. Table 4 shows, for a few cases, the number of nodes needed by each of these three schemes.
(a)
(b)
(c)
(d)
The figures and tables clearly show that if either (a) the transition bandwidth is increased or (b) the filter length is increased, Scheme 1 converges very slowly. This effect is magnified when the integer word length is decreased. This can be seen in the table in the case of spec. with , where, in relation to spec. , , the filter length is kept the same but the transition bandwidth is increased. The same effect is seen for narrow band filters as well. Based on these observations, one can conclude that either Scheme 3 or 4 is on average the best alternative to solve the given optimization problem.
4.2. Effect of Word Length
Next, the effect of changing the integer and fractional word length on the magnitude response and approximation error was studied. Optimization runs were carried out for a number of specifications given in Table 1. For various optimization runs, either the integer word length was fixed and fractional word length varied or vice-versa. In Figure 5, the magnitude responses for filters with specification 3 are shown. The integer word length has been fixed at four while the fractional word length takes on values of one, six, ten, and fourteen.
To study the effect of changing integer and fractional word length on the approximation error in more detail for specification , the approximation error is plotted in Figure 6. In these figures, there are five curves, one showing the optimal approximation error for continuous coefficients (linear relaxation) and one shows the approximation error for linear integers, while the other three curves are for LNS integer word length three, four, and five, respectively, with total word length changing from four to for each curve. All these figures give a detailed picture of the impact of changing integer and fractional word length (the largest coefficient value, and, hence, the number of most significant bits needed for the coefficients in the linear domain is also a function of filter specification. For a narrow-band low-pass filter, one would require less number of bits than a wide-band filter. This can be realized by considering th band (or Nyquist) filters where a filter with band with rad will have a maximum coefficient value of . However, for simplicity, this aspect is ignored in the comparisons in this paper).
(a)
(b)
(c)
As shown in Figure 6, the LNS filters converge to optimum with both four and five integer bits. However, if the optimal approximation error is small, approximately less than dB, filters with three integer bits also converge. Clearly, the number of fractional bits also has a large impact on the resulting approximation error. It is also shown that the linear domain finite word length filters are always beaten by an LNS filter.
Results on comparing against plainly rounding the coefficients are shown in Figure 7. As expected, results obtained by our proposed optimization method gives a smaller approximation error than obtained by rounding the real valued coefficients for the same specification. This means that it is possible to reduce the coefficient word length using the proposed technique and still meet the specification, hence, reducing the complexity of the arithmetic operators as discussed earlier. Also, by using the proposed method, monotonic results are obtained; that is, increasing the word length always improves the performance.
(a)
(b)
(c)
To further compare the performance of LNS against linear integers, approximation error as a function of filter length is plotted in Figure 8. The figure shows that for the same number of total bits, LNS gives a better approximation error. This improvement increases as the filter length is increased. To strengthen the argument made earlier that filters optimized with word length constraints in the LNS domain give better results than just rounding the linear relaxation, Figure 9 compares the approximation error of optimized filters with that of direct rounding, as a function of filter length. The plot further establishes the advantage of our proposed optimization over results obtained by rounding the real valued coefficients. In this case, the optimization leads to that a lower filter order can be reduced, and, hence, fewer arithmetic operators. Again, the proposed design method leads to monotonically decreasing approximation errors.
(a)
(b)
(c)
(a)
(b)
(c)
Finally, a comparison of the magnitude response of specification three for a filter length of is shown in Figure 10. This further shows that LNS gives better results as compared with linear integers as well as illustrates the benefit of actually optimizing to obtain finite word length coefficients.
4.3. Changing the Base
Earlier works have indicated that the selection of logarithm base has an impact on the round-off noise performance. Hence, several filters are designed with varying bases. In addition, it was earlier shown that the number of integer bits is limiting the obtainable approximation error in some cases and changing the base can be seen to give a similar effect as having a noninteger number of integer bits. For example, using three integer bits and using a base larger than two will reduce the amplitude of the smallest representable number (compare to Figure 1).
In Figure 11 the approximation error for specifications 3 and 6 are shown using three and four integer bits. As seen, the above mentioned effect is clear; as for the three integer bit cases, the approximation error decreases when increasing the base (and increases when decreasing the base). For four integer bits, the effect of the base is not so clear. However, for short word lengths, it can be seen that there is some impact of the base.
(a)
(b)
(c)
(d)
To further illustrate the effect of selecting base the same filters were redesigned with a total word length of seven bits, that is, and , but with a larger range of base values. The results are shown in Figure 12. Here, the same trend is clear and it is also evident that a base of at least two should be selected. However, it should be noted that the word length is rather small and that the same clear effect may be seen for a longer word length.
(a)
(b)
(c)
(d)
5. Conclusion
In this paper, a method for designing finite word length linear-phase FIR filters in the LNS domain was presented. Several branch variable selection and branching direction schemes were suggested and evaluated. The scheme where the branching variable was selected based on finding the largest minimum distance from the closest integer variable and branching in that direction was found to be the best on average among those suggested. The resulting filters are optimal in the minimax sense under finite word length conditions.
It was illustrated by examples that three or four integer bits were the best selection, with three being applicable for larger approximation errors. As opposed to finite word length coefficients in the linear domain, the number of integer bits in the LNS domain determines the smallest nonzero coefficient that can be represented. Using a different and larger logarithm base than two can reduce the amplitude of the smallest representable number and slightly improve the results, especially when using three integer bits.
An interesting topic for further studies is the relation between coefficient word length (determining the magnitude response) and the data word length (determining the round-off noise). While these are completely disconnected in the linear scenario as well [1], the impact is different when LNS numbers are considered. Assuming that the data word length is shorter than the coefficient word length, the least significant bits of each multiplication will be independent of the data and vice-versa for the longer data word length. The impact of this on an architectural level may be interesting to investigate further.
Conflict of Interests
The authors declare that there is no conflict of interests regarding the publication of this paper.