Abstract

This paper considers design of filterbank-based transceivers. A composite error criterion is proposed to capture all the three traditional distortions. Incorporating noise attenuation and filter bandlimiting properties into this error criterion, an optimal design procedure is developed and applied to a transceiver design example, yielding an FIR transceiver that has good frequency-selective properties and is close to perfect reconstruction. As a least-squares solution is given in closed form in each iteration, the algorithm is easy to implement.

1. Introduction

Multirate systems, that is, digital systems with signals of different sampling rates, have wide applications in control [1], signal processing [2], communications, econometrics and numerical mathematics. There are several reasons for this [3]: (1) in large scale multivariable digital systems, often it is unrealistic, or sometimes impossible, to sample all physical signals uniformly at one single rate; in such situations, one is forced to use multirate sampling; (2) multirate systems can often achieve objectives that cannot be achieved by single-rate systems. The study of multirate systems goes back to late 1950s [4]. A renaissance of research in multirate systems has occurred since 1980 in the control, signal processing, and communications communities.

(i)In the control community, two directions of research stand out: first, using multirate control to achieve what single rate control cannot as well as the limitations of doing this, and second, the optimal design of multirate controllers [1].(ii)The driving force for studying multirate systems in signal processing comes from the need of sampling rate conversion, subband coding, and their ability to generate wavelets [2].(iii)In the communications community, multirate sampling is used for blind system identification and equalization.

Intersymbol interference (ISI) is a common problem in telecommunication systems, such as terrestrial television broadcasting, digital data communication systems, and cellular mobile communication systems. Usually the channel distortion results in ISI, which, if left uncompensated, causes high error rates. The solution to the ISI problem is to design a receiver that employs a means for compensating or reducing the ISI in the received signal. The compensator for the ISI is the so-called equalizer. For these problems, the filterbank approach [510] has gained practical interest recently.

The multirate filterbank-based transceiver model was proposed in [7] as a unifying framework able to encompass existing modulations and equalization schemes. Further in [9], the multirate filterbank-based transceiver model, as shown in Figure 1, was studied in detail. The filterbank-based transceiver introduces transmitter redundancy using filterbank precoders and generalizes existing modulations including OFDM, DMT, TDMA, and CDMA schemes encountered with single- and multiuser communications [9]. Recently in [8], based on the filterbank approach, the optimal channel equalization is studied: an optimal receiver filterbank, which is a modified Kalman filter, is obtained.

Motivated by the generality and importance of filterbank-based transceivers, this paper focuses on frequency-domain analysis and optimal design issues for filterbank-based transceivers. The contributions of this work are as follows.

(i)Contrary to a time-domain study [9], our work is developed along the line of frequency-domain analysis. Frequency-domain models are obtained using the blocking technique. Based on such models, an composite error criterion is proposed to quantify the degree to perfect reconstruction. This frequency-domain criterion captures the traditional distortions.(ii)In the transceiver design, we apply and incorporate the model matching methodology [11]: instead of designing for perfect reconstruction, we design for close to perfect reconstruction with effective band separation. This idea was explored in filterbanks [11] and transmultiplexers [12]. Under some mild condition, one can always get arbitrarily close to perfect reconstruction by trading off filter complexity and time delay in reconstruction.(iii)Control of filter stopband energy is incorporated in our design. This is important, since narrowband noise could induce serious impairment due to poor stopbands of filters involved [13]: if the receiving filters have poor stopband attenuation, all the neighboring bands would be affected when there is a strong narrowband noise. The resulting ISI can seriously degrade the system performance [14].

The rest of the paper is organized as follows. In Section 2 the blocked model of the filterbank-based transceiver and the perfect reconstruction are briefly reviewed. In Section 3 the transceiver system is analyzed in several aspects based on the blocked model. Section 4 formulates the optimal design problem as a least squares one and develops an iterative design procedure for the filterbank-based transceiver. The proposed design method is illustrated in detail with an example in Section 5. Finally, Section 6 offers some concluding remarks.

We conclude this section by introducing some notation. The signals are denoted by small letters, for example, 𝑟. 𝑟 (underlining denotes blocking) is the blocked signal. ̂𝑟(𝑧) denotes the 𝑧-transform of 𝑟. The systems are represented as time-domain operators, denoted by capital letters, for example, 𝑇. If a system 𝑇 is linear time invariant (LTI), its transfer function (matrix) is written as 𝑇(𝑧). The notation 2 stands for the 2-norm for transceiver matrices.

2. Blocked Models and Perfect Reconstruction

Figure 1 shows the discrete-time multirate filterbank model for the baseband communication system [9]. It consists of the transmitter filterbank 𝐹𝑚(𝑧) (𝑚=0,,𝑀1), the receiver filterbank 𝐻𝑝(𝑧) (𝑝=0,,𝑃1), and the communication channel modeled by the transfer function 𝐶(𝑧), which is assumed to be causal and stable. The input serial data stream 𝑠(𝑛) and its successively time-advanced versions are downsampled by a factor 𝑀 to get 𝑀 parallel substreams 𝑠0(𝑛),𝑠1(𝑛),,𝑠𝑀1(𝑛) as shown in Figure 1. These substreams are then upsampled by a factor 𝑃 and processed by filters 𝐹𝑚(𝑧); the combined output 𝑢(𝑛) is then transmitted over the channel 𝐶(𝑧), which is corrupted at the output by an additive noise 𝑣(𝑛), assumed to be stationary and white. At the receiver end, the received signal 𝑦(𝑛) and its successively shifted versions are then downsampled by a factor 𝑃, upsampled by a factor 𝑀, and processed by filters 𝐻𝑝(𝑧); the combined output forms the reconstructed signal 𝑟(𝑛).

By blocking the transmitter, channel, and receiver, respectively, we can obtain the blocked model [6, 8] for the multirate system in Figure 1:

̂𝑟𝐻(𝑧)=𝐶(𝑧)𝐹(𝑧)(𝑧)̂𝑠𝐻(𝑧)+̂𝑣(𝑧)(𝑧),(1)where 𝐹(𝑧), 𝐶(𝑧), and 𝐻(𝑧) are the transfer matrices of the blocked transmitter filterbank, the blocked channel, and the receiver filterbank, respectively.

The blocked general multirate system (in the absence of noise) is LTI with 𝑇𝐻(𝑧)=𝐶(𝑧)𝐹(𝑧)(𝑧). The transceiver achieves perfect reconstruction if in Figure 1𝑟 is a delayed version of 𝑠, that is, if there exists nonnegative integer 𝑑 such that 𝑇=𝑇𝑑, where 𝑇𝑑 is the time-delay system with transfer function 𝑇𝑑(𝑧)=𝑧𝑑. Blocking 𝑇𝑑 the same way as we blocked 𝑇, the perfect reconstruction condition is equivalent to [8]𝐻𝐶(𝑧)𝐹(𝑧)(𝑧)=𝑧𝑑0𝑧1𝐼𝑙𝐼𝑀𝑙0,(2) where integers 𝑑 and 𝑙 satisfy 𝑘0 and 0𝑙𝑀1, and 𝐼𝑙 and 𝐼𝑀𝑙 are the 𝑙×𝑙 and (𝑀𝑙)×(𝑀𝑙) identity matrices, respectively. Moreover, if this condition is satisfied, 𝑇(𝑧)=𝑧(𝑘𝑀+𝑙).

3. Analysis

In this section, we study the distortion analysis, the effect of noise, and the frequency selectivity.

3.1. Distortion Analysis

Perfect reconstruction synthesis filterbanks at the transmitter and analysis filterbanks at the receiver allow perfect recovery of communication symbols, but the challenges arise with ISI-inducing channels and noise, either of which destroying the perfect reconstruction property. Many practical transceivers do not achieve perfect reconstruction but get close to perfect reconstruction. In order to measure the degree of closeness to perfect reconstruction, we will introduce three traditional quantities to measure sources of distortions: aliasing distortion, magnitude and phase distortions [2]. These distortion measures are based on the blocked model 𝑇(𝑧).

Lemma 1 (see [15]). An LPTV (linear periodic time variant) system 𝐺 with period 𝑡 can be uniquely decomposed into 𝐺=𝐺𝑡𝑖+𝐺𝑡𝑣(3)satisfying the two properties
(i) 𝐺𝑡𝑖 is the optimal LTI approximation of 𝐺 in the sense that it minimizes 𝐺𝑄(𝑧)(𝑧)2 over the class of LTI 𝑄's (𝐺 denotes the blocked system 𝐿𝑡𝐺𝐿𝑡1, similarly for 𝑄. Here 𝐿𝑡 is the lifting operator, and 𝐿𝑡1 the inverse lifting operator [1]). (ii) 𝐺(𝑧)22𝐺=𝑡𝑡𝑖(𝑧)22𝐺+𝑡𝑣(𝑧)22.

Back to our transceiver problem, the system from 𝑠 to 𝑟 is LPTV with period 𝑀; decompose this into 𝐺ti+𝐺tv, where 𝐺ti is the LTI component and 𝐺tv the time-varying component. Thus we have𝑇𝐺(𝑧)=ti𝐺(𝑧)+tv(𝑧).(4)Aliasing distortion in the system is defined by𝐺AD=tv(𝑧)2(5)

Even if AD is zero, and then the LPTV system reduces to an LTI system 𝐺ti, it may still have errors in magnitude and phase compared with the ideal time delay 𝑧𝑑; define the following quantities:1MD=2𝜋02𝜋|||𝐺tie𝑗𝜔|||12𝑑𝜔1/2,1PD=2𝜋02𝜋sin2𝐺tie𝑗𝜔+𝑑×𝜔𝑑𝜔1/2.(6)Note that MD and PD are defined across all frequencies: (MD)2 is the energy of the magnitude distortion and PD the energy of sine of the phase distortion 𝐺𝜙(𝜔)=ti(e𝑗𝜔)+𝑑𝜔. It is worth noting that there are two reasons why we apply sine to characterize the phase distortion PD: (1) if 𝜙(𝜔) is within ±(𝜋/2), which is usually the case, sin2(𝜙(𝜔)) is a good indicator of the size of 𝜙(𝜔); (2) a connection with the 2-norm-based distortion measure 𝐽, to be introduced, could be conveniently established.

Next we propose a composite distortion measure which captures all the three types of distortions and is relatively easy to use in design. The new distortion measure is the 2-norm of the blocked error transfer matrix:𝑇𝐽=𝑇(𝑧)𝑑(𝑧)2.(7)Such a measure is appropriate because in the next theorem we establish connections between 𝐽 and the three types of distortions discussed earlier.

Lemma 2 (see [16]). Let 𝐺 be a stable LTI system. Comparing 𝐺(𝑧) with the time delay 𝑧𝑑, one has the following inequalities:
|||𝐺𝑒𝑗𝜔𝑒𝑗𝑑𝜔|||||||||𝐺𝑒𝑗𝜔||||||,|||𝐺𝑒1𝑗𝜔𝑒𝑗𝑑𝜔|||2sin2𝐺𝑒𝑗𝜔.+𝑑𝜔(8)

Theorem 1. AD and MD relate to 𝐽 via 𝐴𝐷2+𝑀(𝑀𝐷)2𝐽2,(9)whereas 𝐴𝐷 and 𝑃𝐷 relate to 𝐽 via 𝐴𝐷2+𝑀(𝑃𝐷)2𝐽2.(10)

Proof. From (4) and Lemma 1 we get𝐽2=𝑇𝑇(𝑧)𝑑(𝑧)22=𝐺𝑡𝑖𝐺(𝑧)+𝑡𝑣𝑇(𝑧)𝑑(𝑧)22𝐺=𝑀𝑡𝑖(𝑧)𝑧𝑑22+𝐺𝑡𝑣(𝑧)22.(11)Note (5) to get𝐽2=AD2𝐺+𝑀𝑡𝑖(𝑧)𝑧𝑑22.(12)Now by definition,𝐺𝑡𝑖(𝑧)𝑧𝑑22=12𝜋02𝜋|||𝐺𝑡𝑖e𝑗𝜔e𝑗𝑑𝜔|||2𝑑𝜔.(13)Invoke Lemma 2 to get
|||𝐺𝑡𝑖e𝑗𝜔e𝑗𝑑𝜔|||||||||𝐺𝑡𝑖e𝑗𝜔||||||,|||𝐺1𝑡𝑖e𝑗𝜔e𝑗𝑑𝜔|||2sin2𝐺𝑡𝑖e𝑗𝜔.+𝑑𝜔(14)Combining these two equalities with (13) and noting the definitions of MD and PD in (6), one has
𝐺𝑡𝑖(𝑧)𝑧𝑑2𝐺MD,𝑡𝑖(𝑧)𝑧𝑑2PD.(15)The proof is complete by noting (12) and the above two inequalities.

Then it is clear from Theorem 1 that all distortions (AD, MD, and PD) are bounded above by 𝐽. Therefore, it makes sense to minimize 𝐽 in transceiver design because this suboptimizes the three distortions simultaneously.

3.2. Noise Suppression

In Figure 2, 𝑞(𝑛) represents the noise effect at the receiver end. If we want to minimize the root-mean-square value of 𝑞, it is equivalent to minimize the 2-norm of the blocked transfer matrix from 𝑣(𝑛) (standard white noise) to 𝑞(𝑛). Therefore, the objective function for attenuating the output noise effect can be𝐽𝑁=𝐻𝑧2.(16)

3.3. Frequency Selectivity of Filters

Filters with frequency selectivity are of particular importance in communications systems. In addition, frequency responses of designed filters would be deteriorated in the iterative design procedure, if no constraints are imposed on the filters. In order to obtain better bandlimiting property, we minimize the stopband energy of filters involved. To see this, take the objective function(s) for 𝐻𝑖 in the receiver filterbank as an example; we have𝐽𝐻𝑖=Ω𝐻𝑖|||𝐻𝑖|||(𝑤)2𝑑𝑤,𝑖=0,1,,𝑃1,(17)where Ω𝐻𝑖 defines the stopband frequency interval(s) for 𝐻𝑖. Similarly, 𝐽𝐹𝑖 can be computed.

In this paper, we will consider only FIR filters. The frequency response of a real 𝑁-tap FIR filter 𝐻𝑖 is given by𝐻𝑖(𝜔)=𝑁1𝑛=0𝑖(𝑛)𝑒𝑗𝑛𝜔=𝑇𝑖𝜙𝑖(𝜔),(18)where𝑇𝑖=𝑖(0)𝑖(1)𝑖(2)𝑖,𝜙(𝑁1)𝐻𝑖(𝜔)=1𝑒𝑗𝜔𝑒𝑗2𝜔𝑒𝑗(𝑁1)𝜔.(19)(The superscript 𝐻 indicates the complex conjugate transpose.) The objective function in (17) can then be written as𝐽𝐻𝑖=𝑇𝑖𝑄𝑖𝑖,(20)where the fixed 𝑁×𝑁 matrix 𝑄𝑖 is defined by𝑄𝑖=Ω𝐻𝑖𝜙𝑖(𝑤)𝜙𝐻𝑖(𝑤)𝑑𝑤.(21)The elements 𝑞𝑚𝑛 for 𝑄𝑖 can be calculated easily if Ω𝐻𝑖 is given; for example, assume the filters 𝐻𝑖 have the passband [𝜔𝑝𝑖1,𝜔𝑝𝑖2] over [0,𝜋], then𝑄𝑖=𝜔𝑝𝑖10𝜙𝑖(𝑤)𝜙𝐻𝑖(𝑤)𝑑𝑤+𝜋𝜔𝑝𝑖2𝜙𝑖(𝑤)𝜙𝐻𝑖(𝑤)𝑑𝑤,(22)and thus the elements 𝑞𝑚𝑛 for 𝑄𝑖 are

𝑞𝑚𝑛=𝑤sin𝑝𝑖1𝑤(𝑚𝑛)sin𝑝𝑖2(𝑚𝑛)(𝑚𝑛),𝑚𝑛,𝜋+𝑤𝑝𝑖1𝑤𝑝𝑖2,𝑚=𝑛.(23)

4. Problem Formulation and Design

In view of the new distortion measure 𝐽 discussed in the preceding section, we wish to design transmitter and receiver subsystems to minimize 𝐽. Thus our optimal filterbank-based transceiver design problem using FIR subsystems can be stated as follows: given the FIR channel and desired reconstruction time delay 𝑑, design FIR transmitter and receiver subsystems of some given lengths to minimize 𝐽 subject to some constraint on 𝐽𝑁 in (16) and the stopband energy constraints on 𝐽𝐻𝑖 in (20).

In order to incorporate both the noise attenuation and filter bandlimiting constraints, such an optimal design problem can be recast by including penalties on 𝐽𝑁, 𝐽𝐻𝑖, and 𝐽𝐹𝑖. Because both 𝐹(𝑧) and 𝐻(𝑧) are designable, this optimization problem is in general nonlinear and difficult to solve. Thus we take the following iterative design procedure which turns out to be very effective in the design example to follow.

Step 1. Design transmitter subsystems to satisfy desired frequency limiting properties (without considering reconstruction performance); these are used to initiate the iteration.

Step 2. Fixing the transmitter subsystems, design FIR receiver subsystems by minimizing the following objective function (using the blocked models)𝐻min(𝑧)(𝐻𝐶(𝑧)𝐹(𝑧)𝑇(𝑧)𝑑(𝑧)22+𝛼𝑁𝐻(𝑧)22+𝑃𝑖=1𝛼𝐻𝑖𝐽𝐻𝑖𝐻)=min(𝑧)𝐽1.(24)

Step 3. Fixing the receiver subsystems just designed, now redesign FIR transmitter subsystems by minimizing the following objective function (using the blocked models)𝐹min(𝑧)(𝐻𝐶(𝑧)𝐹(𝑧)𝑇(𝑧)𝑑(𝑧)22+𝑀𝑗=1𝛼𝐹𝑗𝐽𝐹𝑗𝐹)=min(𝑧)𝐽2.(25)

Step 4. Repeat Steps 2 and 3 until the corresponding objective function is sufficiently small.

We note that the idea of iteratively designing transmitter and receiver filters was used effectively in transmultiplexers design in [12]. The advantage of this procedure is evident: by fixing either 𝐹(𝑧) or 𝐻(𝑧) in Steps 2 and 3, the optimization problems become mathematically tractable; in fact, they are finite-dimensional, convex optimization with a quadratic cost function, whose global optimal solution can be always computed. Even analytical solutions can be obtained.

For example, looking at the optimal design problem in Step 2, we define𝐻𝑃(𝑧)=𝐶(𝑧)𝐹(𝑧)𝑇(𝑧)𝑑(𝑧).(26)This system is FIR and hence can be represented by its finitely many coefficient matrices 𝑃𝑖. By Parseval's equality,𝐽21=𝑃𝑧22=[𝑖𝑃trace𝑖𝑃𝑖].(27)Since 𝐹(𝑧) and 𝑇𝑑(𝑧) are given and thus 𝑃(𝑧) depends on 𝐻(𝑧) in an affine manner, it follows that 𝑃𝑖 relates to the coefficients of 𝐻(𝑧) (to be designed) too in an affine manner. It is obvious that both 𝐽𝑁 and 𝐽𝐻𝑖 are of quadratic forms, therefore we can rewrite the quantity in (24) in the following way:𝐽21=𝑀𝐹𝑥𝑏𝑀𝐹𝑥𝑏+𝑥𝑀𝑁𝑥+𝑥𝑀𝐻𝑥.(28)Here 𝑥 is a column vector containing all the parameters in 𝐻(𝑧), to be designed, 𝑏 is a column vector depending on only 𝑇𝑑(𝑧), 𝑀𝐹 is a matrix depending on 𝐹(𝑧) and the way 𝑥 is formed, 𝑀𝑁 depends on 𝛼𝑁 and 𝐻(𝑧), and finally 𝑀𝐻 depends on 𝛼𝐻𝑖 and 𝐻𝑖(𝑧). The matrices 𝑀, 𝑀𝑁, 𝑀𝐻, and 𝑏 can be computed and are independent of the design parameters (𝑥). Now the optimal design problem in Step 2 becomes a least squares problem:min𝑥𝑀𝐹𝑥𝑏𝑀𝐹𝑥𝑏+𝑥𝑀𝑁+𝑀𝐻𝑥.(29)If 𝑀𝐹𝑀𝐹+𝑀𝑁+𝑀𝐻 is invertible, the optimal solution can be obtained to be𝑥opt=𝑀𝐹𝑀𝐹+𝑀𝑁+𝑀𝐻1𝑀𝐹𝑏.(30)From here we can recover the optimal receiver subsystems. The optimal design problem in Step 3 can be solved similarly.

5. Design Example

The filterbank-based transceiver with 𝑀=3 and 𝑃=8 is designed. The channel to be used in this example is𝐶(𝑧)=10.3𝑧1+0.5𝑧20.4𝑧3+0.1𝑧40.02𝑧5+0.3𝑧60.1𝑧7.(31)

The transmitter and receiver filters involved in design are all FIR and causal with a fixed order of 12. The magnitude Bode plots of the initial transmitter filters are given in Figure 3, and the reconstruction time delay is taken as 𝑑=8. The constants 𝛼𝑁, 𝛼𝐻𝑖, and 𝛼𝐹𝑗, reflecting relative weightings among multiple objectives, are tuned in the design process. In our design, these are taken to be 𝛼𝑁=0.02, 𝛼𝐻𝑖=0.03(𝑖=0,1,,7), 𝛼𝐹𝑗=0.01(𝑗=0,1,2).

Next we apply the iterative design procedure to the example. In the first iteration, 𝐽1=5.806, and the resulting 𝛼𝑁𝐻𝐻(𝑧)=0.02(𝑧)22=1.553. After some iterations, the value of the objective function gradually decreases as the number of iterations increases, and finally converges to the value 𝐽1=0.156, 𝐻0.02(𝑧)22=0.0872, and finally 𝐽=0.048. The above comparison on 𝐽1 clearly illustrates the improvement obtained by using the proposed iterative algorithm. The designed transmitter and receiver filters are shown in Figures 4 and 5, respectively.

6. Conclusion

In this paper, we investigated the problem of optimal design of filterbank-based transceivers. We proposed quantities to measure various distortions. We also introduced a composite distortion (𝐽) that captures all distortions. Finally, by incorporating two important practical issues, the noise suppression and filter bandlimiting property, we developed an iterative design procedure based on minimizing the objective function and successfully applied this procedure to design of a filterbank-based transceiver. At each iteration, the least squares solution should be found, thus the final transmitter and receiver filters can be obtained with relative ease.

Acknowledgments

The authors wish to thank the associate editor and anonymous reviewers for providing many constructive suggestions which have improved the presentation of the paper. This research was supported by the Natural Sciences and Engineering Research Council of Canada and the Canada Foundation of Innovation.