Abstract
Commercial User Equipment (UE) testing and certification has become more complex for state-of-the-art mobile communication standards such as 3rd Generation Partnership Project (3GPP) Long-Term Evolution (LTE) due to the extensive use of Multiple Input-Multiple Output (MIMO) transmission techniques. The variety of different MIMO operating modes and the almost unlimited choice of possible multipath channel conditions under which UE performance may be evaluated are not accounted for by established Single Input-Single Output (SISO) Over-The-Air (OTA) performance metrics like Total Isotropic Sensitivity (TIS) and Total Radiated Power (TRP). As pointed out in this contribution, meaningful metrics and cost-effective, low-complexity measurement methods can, nevertheless, be derived by focusing on characterization of the physical attributes of UE and by adopting statistical metrics. Starting from a brief review of the most important MIMO operating modes in the 3GPP LTE standard, the relation between UE properties and UE performance, which is observed in these operating modes, is discussed. Two complementary metrics and corresponding measurement procedures for evaluation of MIMO OTA performance are presented in order to address the diversity of possible propagation scenarios. Measurement results from preliminary implementations of the two proposed measurement procedures, including comparison between different LTE devices, are presented.
1. Introduction
Since the early years of mobile communication development, a method to reliably predict in-the-field performance of a mobile device is highly demanded. OTA test methods were first introduced when Mobile Network Operators (MNOs) experienced that devices with comparable conducted performance exposed significantly different in-the-field performance, including increased rates of dropped connections. Since most small form-factor SISO devices have inherently omni-directional patterns, especially in the low frequency bands, the focus of traditional OTA test is on total antenna efficiency [1].
State-of-the-art mobile communication standards such as 3GPP LTE bear the promise of significantly enhanced cell capacity and per-user throughput. Whether the performance of commercial UE holds up to the promise has become more intricate to predict because MIMO performance depends strongly on UE antenna properties. New OTA measurement methods and performance metrics are necessary. After an initial request by 3GPP RAN4 to address this issue [2], directed to COST 2100 and CTIA, several OTA test method proposals have been made by participants of COST 2100, 3GPP RAN4 and CTIA. The proposals differ significantly in terms of measurement setup and equipment because each approach is trying to solve the issue from a different perspective. Some initial comparison between methods has been conducted in two large-scale Round-Robin Measurement Campaigns [3] without arriving at definite conclusions with respect to advantages and drawbacks of each method [4, 5]. Further measurement campaigns and comparison to simulation [6] and a decision on a suitable test methodology or several methodologies are planned in the near future [7].
2. Goals and Requirements for MIMO OTA Measurements
The goal of a MIMO OTA measurement is to evaluate UE with respect to in-the-field performance for various MIMO transmission modes and an almost infinite choice of multipath channel conditions. An agreed test method should, nevertheless, clearly discriminate between “good’’ and “bad’’ devices without footnotes stating “under certain conditions.” Excellent reproducibility of results is mandatory. Consistent results should be obtained by every test lab.
A further important requirement is low complexity because tests are to be performed for large number of E-UTRA frequency bands and for a variety of different near-field conditions (e.g., with head or hand phantoms), a fact which increases test time substantially. The time required for each measurement should not exceed typical measurement time for established SISO OTA measurement procedures like TIS and TRP. Additional cost for the certification process would have to be transferred to the end-user.
3. MIMO-Operating Modes in LTE
So as to design an OTA test method for MIMO UE, it is mandatory to have a basic understanding of the underlying communication standard. State-of-the-art communication standards employ an aggressive switching strategy in adaptation to the ever-changing multipath channel. LTE is a perfect example to illustrate this issue. The Downlink (DL) of LTE Rel-8 features already 7 basic transmission modes. Further modes are added in Rel-9 and Rel-10. The most relevant MIMO modes are the DL Transmit Diversity (TD) mode and Open-Loop and Closed-Loop Spatial Multiplexing (SM) modes. Apart from the different transmission modes, there are 15 unique combinations of modulation order and coding rate, each referred to in the following context as a Modulation and Coding Scheme (MCS) (note this is different to the definition of parameter in LTE). In reality, transmission mode and MCS are adaptively selected by the evolved Node-B (eNB), taking into account, at its own discretion, feedback from the UE.
Among the different indicators, which are returned by the UE and provide Channel State Information (CSI) feedback to the eNB, the Rank Indication (RI) is fundamental for the selection of the appropriate MIMO transmission mode. SM is possible only as long as the current rank of the channel is larger than one. If the channel is in addition not too rapidly varying over time, the eNB can select Closed-Loop Spatial Multiplexing (CL-SM) mode and use the Precoding Matrix Indicator (PMI) information from the UE to select from a predefined set of precoding matrices the one which optimizes the condition number of the channel matrix. Otherwise, if the channel exposes rapid time variation, not allowing for timely update of PMI, the eNB may switch to Open-Loop Spatial Multiplexing (OL-SM) mode. In this case, random precoding is employed in order to assure an at least statistically, on average, good condition number of the channel matrix. In OL-SM mode, Large-delay Cyclic Delay Diversity (LD-CDD) is in addition introduced as a means to enhance diversity of the channel.
If the rank of the channel or the Signal-to-Noise Ratio (SNR) do not allow for either of the SM modes, DL TD is available as a robust fallback mode, which maximally exploits the diversity of the channel for reliable transmission of single stream. Note that DL TD mode is, nevertheless, a MIMO transmission mode. Space-Frequency Block Coding (SFBC) is used to transmit orthogonal replicas of a single stream from different Basestation (BS) antennas based on the Alamouti scheme [8]. In reality, DL TD is the most important mode as it allows for reliable transmission under poor channel conditions.
A further aspect of adaptivity is MCS selection. Based on Channel Quality Indication (CQI), feedback from the UE the eNB continuously adapts the MCS to current channel conditions. By means of the CQI, the UE indicates allowance for a higher MCS if the current Block Error Rate (BLER) drops below a threshold and requests a lower MCS if the BLER exceeds the threshold.
The above overview is admittedly superficial but considered necessary here to emphasize the role of the interaction between the eNB and the UE in MIMO transmission. Any performance figure like, for example, throughput observed in a real network is a system property, which includes not only UE receive performance and UE feedback reporting quality but also the response of the scheduler in the eNB. It is of uttermost importance to be aware of this fact so as not to arrive at a simplistic and erroneous idea of a realistic test case. To be realistic a time-variant, faded multipath channel is always to be combined with a fully adaptive system comprising not only the UE but also the mode switching and MCS adaptation functionality of the eNB’s scheduler. This functionality is, however, neither supported by presently available test equipment nor is there a standardized scheduler at all. Measurements are presently rather performed under conditions of a Fixed Reference Channel (FRC), which predefines transmission mode and MCS. Exposing UE to a random channel under these fixed settings is all but realistic.
The way out of this difficulty is to characterize the physical UE attributes which determine the UE’s contribution to overall performance. Part of this task requires for OTA measurements, other properties are more easily and reliably characterized by conducted measurements.
4. Relevant Physical Attributes of MIMO UE
4.1. Antenna Properties
A passive -port antenna can be fully characterized by its compound polarimetric pattern, the complex matrix function with , and its scattering matrix . With and denoting the vectors of incoming (TX antenna) and outgoing (antenna RX) wave quantities at the ports, the receive mode is described by and the transmit mode by Radiated power can be written as where is referred to as the radiation matrix. Superscripts , , and denote transposed, conjugate, and Hermitian transposed, respectively.
A closer look at this Hermitian matrix is mandatory because it contains all relevant information about the performance of the antenna system in a statistically isotropic environment. Although the isotropic scattering environment is of rare occurrence, the concept takes its justification from the fact that the orientation of UE in space may be random. We shall see that the radiation matrix is one of the UE attributes to be characterized. To understand its meaning, consider the case that only port is fed, that is, . Available power is then given by and radiated power by with the norm of the th pattern. Obviously the diagonal element is equal to the ratio of radiated over available power from port , that is, to per-port (total) efficiency, Different per-port efficiencies correspond to the different expectation values of received power in a statistically isotropic environment and are, therefore, linked to the concept of antenna imbalance. But we shall see below that a deeper look into this concept is necessary.
The off-diagonal elements of the radiation matrix contain the scalar products between patterns associated with different ports, from which the traditional pattern correlation coefficient is obtained as The correlation corresponds to the expectation value of the observed Complex Envelope Correlation Coefficient (CECC) at the antenna ports in a statistically isotropic environment.
It would not have been worthwhile to introduce the radiation matrix just to come back to familiar parameters like per-port efficiencies and correlation between per-port patterns. It is important to understand that the traditional parameters “antenna imbalance” and “antenna correlation” are actually meaningless for MIMO transmission if considered separately.
A simple thought experiment reveals why. Consider an -port antenna, which receives uncorrelated paths of equal power coming in with different Angle of Arrival (AOA) and isotropic Power Angular Spectrum (PAS). The received signal vector will contain components with different amplitudes and some correlation between the components will be observed. The noise contributions in each component may be considered uncorrelated and of equal power because they are dominated by receiver noise figure. The MIMO detector now basically applies an approximate inverse of the channel matrix to the received signal vector in order to recover the transmitted signal vector, that is, to remove the correlation. The quality of this approximation depends solely on the condition number of the channel matrix. In the present setting, the expectation value of the channel matrix is just the radiation matrix of the antenna. Consequently, the relevant property of the radiation matrix is the attainable rank, which in fact is the number of eigenvalues (assumedly ordered in sequence of decreasing magnitude) for which the ratio of largest to smallest magnitude does not exceed some threshold.
Moreover, in case of CL-SM, the eNB’s precoder and the UE’s MIMO detector make a joint attempt to diagonalize the channel matrix. In either case, unitary transforms are applied to the channel matrix. Hence only those properties of the radiation matrix, which are invariant under unitary transforms, are relevant: the eigenvalues of . The radiation matrix is Hermitian and hence has real eigenvalues, which correspond to the modal efficiencies of the orthogonal radiation modes, which are supported by the antenna system. The discussion can be summarized as follows.(a)The MIMO performance of an -port antenna in a statistically isotropic environment is described by the modal efficiencies , the eigenvalues of the radiation matrix.(b)Total antenna system efficiency is described by the invariant property This figure is equal to the ratio of transmitted over available power in case that the total available power is evenly distributed over uncorrelated feed port signals [9].(c)Modal imbalance is a relevant parameter, whereas conventional per-port “antenna imbalance” is not. The attainable rank of MIMO transmission is restricted by the number of modal efficiencies with value not less than a given (power dependent) fraction of . An imbalance figure can be introduced as the ratio of the arithmetic over the geometric mean of modal efficiencies A perfectly balanced antenna system with will have . An antenna system with higher modal imbalance has larger .(d)“Antenna correlation,” that is, pattern correlation with respect to different port patterns does not convey useful information. We note in passing that antenna correlation is anyway always small if the antenna system is properly matched (Hermitian match with respect to impedance matrix) and if dissipative losses are small [10]. For a lossless antenna system (no dissipation), power balance in fact implies that radiated power is equal to power incident at the ports minus power scattered back from the ports, that is, If a multi-port antenna is correctly matched (at a given frequency), then all elements of the scattering matrix are small in magnitude. The off-diagonal elements of the radiation matrix (corresponding to pattern correlation) are then obviously also small. This statement applies irrespective of the spatial separation between different antenna elements. The latter may have an influence on the bandwidth over which a good multi-port match is attainable but is otherwise not directly related to pattern correlation. Generalization to lossy antennas is straightforward by adding a loss matrix term to the power balance relation.
After having identified the eigenvalues of the radiation matrix, that is, the efficiencies of the orthogonal radiation modes supported by the antenna system as the relevant parameters, a brief look into their physical behaviour is indicated.
Each radiation mode (orthogonal pattern) comes with an individual radiation quality factor defined as the ratio of average reactive energy stored in its near-field over radiated energy per period. As opposed to what the term “quality’’ suggests, a high radiation quality factor is detrimental. It corresponds to narrow matching bandwidth and high dissipation, that is, the modal efficiency of the respective mode is acceptable only over a small bandwidth and may be still poor even there. Reduction of radiation quality factors is the goal of antenna design but unfortunately is subject to fundamental physical limits. When adding further antennas to small form-factor devices, the radiation quality factors of the additional radiation modes typically take increasingly large values. An example for the frequency dependence of modal efficiencies for a simulated 2-port antenna system in a USB dongle is shown in Figure 1. Cyan and blue curves represent efficiencies of the two modes versus frequency. It can be seen that total antenna system efficiency (10) and modal imbalance can be strongly dependent on frequency, which should be considered in test design.
4.2. Reconfigurable and Adaptive Antennas
The above-mentioned physical limitations in antenna design are reasons for introduction of tunable and reconfigurable or even adaptively matched antenna systems (so as to compensate for antenna detuning in a variable near-field environment, e.g., due to user’s head and hand). As a consequence, the antenna system can in principle no longer be treated as a linear system block and, in case of adaptive matching, not even as time invariant. It should be noted in particular that with a tunable antenna system the first nonlinear system block appears in the receive chain before the band filter. New test cases may, therefore, be necessary. Moreover, the benefit (or malfunction) of adaptive matching will not be assessed correctly without a properly chosen set of test cases with phantoms, possibly accounting also for time dependence of the adaptation algorithm. A thorough discussion of these issues is beyond the scope of the present paper. It should be obvious, however, that MIMO OTA test methods, which build on a separate characterization of an assumedly linear and time invariant antenna system in combination with conducted measurements, are not suitable for this situation.
4.3. Self-Interference and Cross-Interference
Self-interference is in reality often the most important cause of poor device performance. Conducted measurements are not capable to detect problems caused by coupling of an interfering signal to the antennas. Cross-interference is important for devices that are connected to a host device such as datacards or USB dongles. Interference from the host computer may differ dramatically between, for example, different laptop models. A standard mock-up is necessary to make comparable measurement for this type of devices.
4.4. Receiver Noise Figure
The noise figure of a receiver is defined as the additional noise power level at a receiver’s output as compared to an ideal receiver. If the effect of nonlinear distortion is approximated as Additive White Gaussian Noise (AWGN), it may be subsumed under the noise figure and the latter be taken as a quality measure for the analog and mixed signal system blocks in the UE’s receive chain. In an “end-to-end’’ measurement, it is not possible to separate receiver noise figure from antenna efficiency. It is, therefore, necessarily included in MIMO OTA measurements.
4.5. Baseband Algorithms
The quality of the algorithms implemented in the digital baseband of UE is crucial for receiver performance and reporting quality (CSI). These aspects, however, can at much lower cost and with superior repeatability be tested in conducted measurements with the full flexibility to apply time-dependent faded channels, different Doppler spectra, and birth-death scenarios with the help of today’s sophisticated fading emulators. In the sequel, it is therefore silently assumed that such conducted tests have been performed separately and the digital baseband is not considered in the context of MIMO OTA measurement.
5. Multipath Channels and UE Properties
5.1. Generic Multipath Scenario, Constellations
In communication theory, the radio channel comprises the propagation channel and the antennas at either side of the link. The topic of this section is the interaction of a multipath scenario with the UE antenna system. Consider a single subcarrier frequency of an OFDM DL supporting parallel data streams. The complex amplitudes of the sinusoids can be described by a vector , which is normalized to unit power according to . In addition, interfering signals may be present and are described in the same fashion by a vector .
Conceptually, we define the Device Under Test (DUT) as the combination of the UE under test and any possible objects in the near-field such as head or hand phantoms. This allows to consider the DUT as being immersed into a superposition of plane waves, incident from AOA , . This superposition is described in terms of the distribution where is a calibration factor (see below) and and are, respectively, and matrices of distributions, referred to as constellations below, which describe the multipath channel for the wanted signals and the interferers, respectively. Both have the same form, namely, with and likewise for . This is a generic description, which covers any real-world scenario as well as any test case. In the context of OFDM, it is sufficient to identify the coefficients in (14) with complex numbers provided that path delays do not exceed the guard period.
The received signal vector at the antenna ports is then, in absence of interferers, given by where The integral over the unit sphere evaluates to an matrix for an -port antenna system. A similar term with exchanged for is to be added to include external interferers.
In order to obtain the channel matrix in real-world units, the factor in (13) must be quantified, that is, the measurement must be calibrated. As usual in anechoic chamber measurements we express in terms of the power , fictitiously received by an ideal isotropic antenna in the center of the UE position without the UE (and phantom) in place when subject to a single co-polarized plane wave. In this setting, (1) reduces to unity and (due to the normalization of the signal vector) from which can be calculated. The operation denotes the Frobenius norm. Since the channel matrix is conventionally referenced to the “inner antenna ports” in the form where is the received signal vector at the input of the detector, it is convenient to scale the received signal vector according to , where denotes receiver noise figure. The correct SNR is thus maintained when we simply substitute an AWGN signal with thermal noise power per component for the noise contribution on the right-hand-side (RHS) of (18). The resulting expression for the channel matrix is
An analogous expression can be written for the external interferers to yield upon combination The additional term accounts for self-interference.
Any MIMO OTA test case obviously amounts to the realization of a sequence of channel matrices (19) and therefore to the selection of a sequence of constellations, each containing incoming (polarized) paths. In an anechoic chamber, each constellation can be realized by means of dual-polarized test antennas. It is understood that the test antennas are either movable over the sphere or selectable from a larger set of fixed test antennas to allow for a sequence of different constellations.
Each choice of amounts to the definition of a sampling operator, which is applied to the compound pattern as described by the projection in (19). The question arises, how to select these operators so as to obtain on the one hand a sufficient characterization of but on the other hand to avoid an overly complex approach. The options range from simply assigning each of the downlink streams to a single test antenna at a time to complex schemes, where pre-faded superpositions of all downlink streams are routed to a large number of test antennas. A large number of antennas allows for emulation of geometrically realistic propagation scenarios as described by Geometry-Based Stochastic Channel Models (GSCMs), for example, the 3GPP Spatial Channel Model Extended (SCME) [11]. Related proposals have been made by several groups [3, 12, 13], and there is a tendency to emulate a GSCM in as much detail as possible, including multiple pre-faded copies of the DL signals, time dependence of the channel, and geometrical detail such as the angular spread of incoming clusters.
The following subsections are devoted to a discussion of these aspects.
5.2. Number of Test Antennas
The number of test antennas has a large impact on cost, complexity, and thereby reproducibility of a MIMO OTA test method. It is, therefore, important to clarify whether a larger number of test antennas will correspond to a better test method or not.
There is an obvious lower bound on the number of test antennas. To investigate UE performance in an MIMO scheme at least test antennas are required to realize a channel of maximum possible rank. Below we assume . The minimum number of test antennas is then . Use of a larger number of test antennas makes sense only if also additional uncorrelated DL signals are generated. This is possible by applying different time-variant delays or phase shifts and distributing these newly generated signals to different antennas. The resulting paths will superimpose in the UE’s antenna system to yield again received signals. We refer to this effect as in situ fading, which (as opposed to pre-fading, see below) is a physical effect and relates to physical attributes of the UE antenna system. Formally, in situ fading is a consequence of the linearity and projection property of the sampling operator showing up in (19). In fact, for any constellation with incoming paths the channel matrix can be represented as a superposition of channel matrices for constellations with only antennas in the form From the perspective of characterizing physical attributes of the UE, the question is, whether we can obtain significant additional information about the compound pattern by applying superpositions of the constellations instead of considering them individually. Although the finally recorded metric (e.g., throughput) is not a linear function of the channel matrix, the answer is negative. It will be shown in Section 6 that a setup with antennas is sufficient to characterize the relevant properties of a UE antenna system, in particular total antenna system efficiency and modal imbalance.
5.3. Pre-Fading
In connection with MIMO OTA test approaches that use more than test antennas, it may be tempting to generate signals for the additional antennas by emulation of a channel model. A common way to do so is to use a fading emulator to apply a multi-tap delay line model to a signal before routing it to a test antenna. We refer to this approach as pre-fading. It corresponds to common functionality of fading emulators. Pre-fading is valuable for conducted measurements where it substitutes for the lack of in situ fading in the UE antenna system. It must be emphasized, however, that the introduction of pre-fading in a MIMO OTA test method, where in situ fading is already present, is unphysical and represents an arbitrary deterioration of the channel, which does not correspond to reality. Consider, for instance, the case of several paths which superimpose in space to create a spatial interference pattern with maxima and minima of available receive power. The diversity performance of a UE antenna system depends on how well it exploits this spatial interference pattern. If the paths are pre-combined before routing them to a test antenna, that is, generated by means of a tapped delay line filter, the spatial interference pattern is lost and replaced by a purely temporal fading characteristic, which is basically sensed in identical form by all UE antennas. Since such proposals, nevertheless, exist [3], it is worthwhile to take a brief look at the main effect of a pre-faded signal when applied under conditions of a fixed MCS.
To this end, it is sufficient to consider a single receiver. Without fading, we shall observe a sudden drop of throughput from nearly to in a small region about the threshold power of the selected MCS as available receive power is reduced below the threshold. This behaviour is approximated by a step function (dashed line) in Figure 2 (the width of the transition region is in the order of to and not relevant for our argument here). The threshold radiated power level at which this transition occurs, assumedly in Figure 2, uniquely describes the receive quality of the UE in this situation. It reflects antenna efficiency, receiver noise figure, and possibly self-interference. In a Rayleigh scenario, the probability that the instantaneous receive power exceeds for a given average available receive power is A fading emulator has to emulate exactly this behaviour. The observed average relative throughput as a function of average power is then basically given by the convolution of the Probability Density Function (PDF) corresponding to (22) with the step function resulting in This dependency is shown in blue in Figure 2. The red curve displays the analogous result for Rician fading with a -factor of 5 dB. We observe that the desired information about the UE receive quality is just blurred by the statistics of the channel. Differences between results obtained this way are characteristic for the measurement setup (channel) but not for the UE.
5.4. GSCM, Adaptivity and Averages
The common starting point of any GSCM is the definition of UE velocity. It is directly translated into Doppler shifts. But in addition it is also used to introduce time dependence into the channel, resulting in time-dependent fading where the UE velocity determines the rate of change. Any MIMO OTA measurement performed under these conditions necessarily amounts to the measurement of a time average, for example, of average throughput. It is tempting to consider such time average to be a particularly “realistic’’ indicator of UE performance. But there are two major issues with this approach if combined with an FRC.(a)The condition number of the channel is varied over time in a poorly predictable way, that is, the rank of the channel may change up to the situation that it is in disagreement with a pre-selected MIMO operating mode.(b)Fading becomes time-dependent and the available receive power is spread out statistically about its average value with the effect that the SNR may be in disagreement with a pre-selected MCS during extended portions of the measurement interval.
Either effect potentially renders the recorded average values meaningless with respect to in-the-field results. In reality, that is, in a fully adaptive system with a real eNB, the UE would have reported the current rank of the channel and the eNB would have switched to a more appropriate transmission mode (such as TD and OL-SM or CL-SM). During a measurement with time-dependent fading, the UE under test may also have desperately reported over parts of the measurement period that it could still attain appreciable throughput if CQI was decremented (i.e., the eNB should switch to a lower order modulation scheme or lower coding rate). But it is not heard. Instead zero throughput is recorded. Similarly, the UE may have reported over parts of the measurement period that the channel is good enough for the next higher order modulation scheme or higher coding rate (CQI increment), but the measurement under conditions of an FRC fails to account for the true performance of the UE.
We conclude that time averages taken with respect to a time-variant channel do not correspond to any real situation if taken under conditions of an FRC. But as opposed to average values, empirical outage probabilities may still convey useful information.
5.5. GSCM and Near-Field Impact
A lot of attention has been given to accurate emulation of geometrical detail of the SCME models such as Angular Spread (AS) in Anechoic Chambers (AC) [14]. In reality, however, such “free-space detail’’ is seldom visible to the UE at all. It is rather masked and strongly modified by the near-field environment of the UE, in case of hand-held devices, for example, by the user’s hand or head. Measurements with appropriate phantoms are well established in existing OTA test plans (e.g., [1]). Since the performance of MIMO UE is potentially more susceptible to the near-field environment than that of a single antenna UE, the quest for realistic test conditions will have to include them.
6. MIMO OTA Metrics and Measurement Methods
6.1. Towards a Direct Device Characterization Method
Based on the discussion and models presented in the previous sections, metrics and methods for MIMO OTA testing are proposed below. The selection of test cases is guided by the following principles.(i)OTA test cases are selected to correspond to those physical UE attributes, which cannot be characterized by conducted measurements but must be characterized by OTA measurements.(ii)The discretionary introduction of features from channel models for which a UE antenna system is transparent, in particular time-dependent fading, is to be avoided. Such features are likely to dominate measurement results and obliterate differences between devices.(iii)In order to assure reproducibility and accurate calibration, each test case should be stripped down in complexity to what is essential for the result.(iv)The influence of the measurement setup on results must be clearly understood and traceable.(v)Test setups that allow for reuse of existing OTA chambers are preferred.
Meaningful and realistic test cases can only be defined in agreement with the fundamental features of a MIMO communication standard, the most important of which is adaptation of the MIMO transmission mode to the channel properties.
The investigation of throughput in SM transmission mode is not meaningful unless it is assured that both the rank of the channel is larger than one and the SNR is sufficiently high. Otherwise, the result will just show that a poor channel is applied, a situation in which TD transmission mode would be invoked in reality. A MIMO-favourable channel is mandatory to evaluate MIMO performance in SM transmission modes.
Conversely, diversity performance cannot be evaluated if the channel is good enough to achieve constant maximum throughput with a single antenna. Diversity performance is best described by sensitivity under poor channel conditions. The MIMO transmission mode which in reality is used in the low SNR regime is TD and hence should also be used for testing in this context.
Two basic test cases are derived from the above consideration, which will be shown to cover the relevant physical attributes of UE:(1)evaluation of noise limited performance in TD mode using a sensitivity-based metric,(2)evaluation of peak performance in SM mode using a throughput-based metric.
They correspond to poor and MIMO-favourable channel conditions, respectively [15–17].
The definition of the metrics must in either case follow the purpose to provide a criterion for in-the-field performance. The UE should perform reasonably well in a virtually unlimited variety of propagation scenarios. Hence performance can only be described statistically. Average quantities are not sufficient to characterize the UE in this respect. Appropriate statistical metrics are outage probabilities with respect to given sensitivity or throughput levels. Statistical metrics allow for a comparison of the UE in terms of the probability that a prescribed minimum performance will be achieved or, conversely, in terms of outage probability of a given transmission mode and MCS [18]. The probabilities are directly related to user experience and network efficiency.
As opposed to averages, observed outage probabilities relative to some performance figure gathered over a set of fixed, identically reproducible propagation scenarios, also remain meaningful under conditions of an FRC.
6.2. Sensitivity in Noise-Limited Scenarios
DL TD mode is meant to provide fair link reliability and throughput under non-optimum channel conditions. The appropriate test case amounts to a characterization of UE sensitivity in a noise-limited scenario, akin to traditional TIS measurements for SISO devices. Differences to traditional TIS measurements arise (i) from the fact that at least orthogonal DL signals are available in DL TD mode which carry the same information and (ii) from the presence of multiple receive antennas and the possible support of Receive Diversity (RD) in the UE. Note that TD and RD are completely independent methods for improvement of link quality. TD is in general based on the Alamouti scheme, in LTE, for instance, in form of SFBC. RD can be realized by Maximum Ratio Combining (MRC), which is the optimum approach and is taken as reference case below. Both schemes work together so as to combine the total available power from all downlink signals and all UE antennas in order to maximize the SNR. The sensitivity is recorded in terms of the received power, which is required for not exceeding a specified BLER threshold. Based on the discussion in Section 5, an explicit model expression for the SNR is readily obtained. Assuming DL TD mode, a power-normalized signal vector (see Section 5.1) and an -antenna UE with MRC (if ), the effective SNR is given by with denotes the Frobenius norm of the matrix (16). We note in passing that the Shannon capacity formula can be rewritten in terms of the above-introduced symbols as where denotes bandwidth.
If a sequence of different constellations is applied, different values of SNR are observed. The distribution of these SNR values and hence the statistics of sensitivity observed over the sequence of constellations characterizes important features of the compound pattern of the UE’s antenna system.
To describe the proposed test procedure in more detail, we consider DL TD mode with (two orthogonal DL signals), and a sequence of constellations whose elements are, apart from irrelevant phase factors, of the form that is, only a single AOA is encountered per measurement [19]. The factor on the RHS of (24), therefore simplifies to in the -th measurement. The setting resembles a conventional sensitivity measurement with the modification that two DL signals are routed to the two orthogonal polarizations of a dual-polarized test antenna.
Let SNRthr denote the threshold SNR level on the Left-Hand-Side (LHS) of (24) below which the specified BLER threshold is exceeded for the selected FRC. The corresponding value on the RHS of (25) gives the sensitivity at AOA , the directional compound sensitivity Note that the RHS of (28) is essentially the compound antenna gain over receiver noise figure at given AOA . The average of (28) over the unit sphere is the Total Compound Isotropic Sensitivity (TCIS) is the radiation matrix (5), denotes total antenna system efficiency after (10), and is the number of UE antenna ports. Note that (29) completely characterizes the most important UE attribute: the ratio of total antenna system efficiency over receiver noise figure. The result can be approximated by choosing AOAs from a proper regular grid in the same way as in conventional SISO OTA test plans. Moreover, this test case can be realized in a conventional OTA test chamber equipped with a single dual-polarized test antenna, for example, a quad-ridged horn antenna. Two output ports of the MIMO Basestation Emulator (BSE), carrying the two orthogonal DL TD signals are routed to the two polarizations of the test antenna simultaneously and with equal power.
It is to the authors’ opinion remarkable that the most relevant MIMO OTA performance figure, TCIS, equivalent to total antenna system efficiency over receiver noise figure, can be obtained with actually less effort than suggested in existing SISO OTA test plans, which require separate measurements for each polarization. But the test procedure, which has been sketched above, is suitable also to obtain a more significant statistical metric at no additional cost. As well known from income statistics, average values do not tell the full story. A UE with the same value of TCIS may still exhibit different outage probabilities. Receive diversity performance will be rated the better the less it depends on the accidental directions of the incoming paths, that is, on the variation of the total power received from both polarizations over all possible AOAs. The Figure of Merit (FOM) to be reported, is therefore, the empirical Cumulative Distribution Function (CDF) of sensitivity, which is readily obtained from recorded directional sensitivity values (28). The proposed metric is formally introduced as a function of applied power by and of course depends on choice of FRC, threshold BLER, and applied sequence of constellations . The CDF (30) allows for comparison of devices in terms of outage probability of a given FRC at given power level for arbitrary AOA, a quantity which is expected to bear a close relation to the probability of dropped service in an arbitrary propagation scenario.
For illustration, Figure 3 displays sensitivity patterns of a High-Speed Packet Access (HSPA) dongle plugged into a laptop. Figures 3(a) and 3(b) display the conventional separate sensitivity patterns for a -polarized wave and -polarized wave, respectively. The result of Figure 3(c) corresponds to the above-described procedure with uncorrelated signals of half the total power applied to the two polarizations simultaneously. It can be observed that the deep troughs in Figures 3(a) and 3(b) do not show up in Figure 3(c), which approaches a more spherical shape indicating an effective receive diversity.
(a)
(b)
(c)
The CDFs (30) of three different LTE devices of different form-factor (and operating in different bands) are presented in Figure 4. A measurement resolution of is used. The CDFs have been generated with a weight correction for each individual sensitivity measurement corresponding to the relative fraction of the surface of the unit sphere, which is represented by the corresponding AOA. As illustrated by the shaded regions in Figure 4, an MNO may decide to define a rejection region in the CDF plot. A device might for instance be rated acceptable if it exposes a compound sensitivity better than (RS EPRE) with a probability of at least and still better than with a probability of at least . Note that the numerical values here are just for illustration and arbitrarily selected. Actual values are likely to be obtained only by comparison of (30) against in-the-field results.
6.3. Peak Performance in MIMO-Favourable Scenarios
The capability of a device to benefit from multipath propagation by providing high throughput can only be tested if the propagation environment is suitable for such transmission. MIMO peak performance is to be measured for SM modes under conditions of a full rank propagation channel and high SNR. A throughput-based metric is appropriate to evaluate whether the UE antenna system meets the performance target. The metric and accompanying test procedure, which is proposed below, covers the physical UE attributes modal efficiencies and modal imbalance and is therefore complementary to what was obtained by sensitivity measurement in a noise-limited scenario in the previous section.
Figure 5 depicts the concept of a test setup with two dual-polarized horns, which can independently be set to AOA and AOA , respectively. For each pair of AOAs, all possible combinations of polarizations can be realized. The setup, therefore, allows for full characterization of the UE with antennas. In this context, it can accommodate for a MIMO scheme. For evaluation of the UE with more antennas, additional test antennas may be added to the setup. For the presently proposed approach, we consider the simple case of a MIMO transmission in SM mode. Among all possible constellations that can be realized with the setup after Figure 5, we restrict ourselves to those which are of the simple form with exactly one element in either matrix on the RHS of (31) equal to one and the other zero. Each of the two orthogonal, equal power downlink signals is then routed to exactly one out of the 4 available ports at a time. The measurement procedure involves a sequence of such constellations with and taken from a predefined grid on the unit sphere for all possible combinations of polarization, that is, , , , and . Throughput is recorded for each measurement.
Before going into further detail of the procedure, let us briefly consider which UE antenna attributes can in principle be characterized with this approach. LTE offers a CL-SM and an OL-SM DL transmission mode. Provided that “good” channel information is available at the receiver, a situation which would be assumed as a prerequisite for CL-SM mode, the Shannon capacity formula can be approximated by for a given constellation. Assume now that we apply a sequence of constellations of the form (31), which corresponds to a (doubly) isotropic PAS for both polarizations and both downlink signals. Such sequence can be obtained by choosing and from a proper regular grid on the unit sphere. Obviously then, as angular resolution is increased, the average of the term approaches the radiation matrix . In the limit, we would obtain the isotropic average of (32) in the form Since we intend to test in the high SNR regime, we may assume that the SNR is high enough to neglect the identity matrix in (33) as proposed in [20]. The isotropic average (33) can then be approximated by The first term on the RHS of (34) corresponds to the isotropic average of capacity in the optimum case of a balanced -port MIMO antenna system with total antenna system efficiency . The second term on the RHS (34) describes the capacity degradation due to modal imbalance. It is always nonpositive and equal to zero if and only if the antenna system is perfectly balanced with equal modal efficiencies . The approximation (34) can be rewritten in the form where superscript indicates a quantity given in decibels and is the imbalance figure of the UE antenna system as introduced in (11). Note that both and are invariant properties of the radiation matrix as discussed in Section 4.1. We conclude that with diligent choice of SNR0, such that would allow for almost the maximum throughput supported by a given FRC, an observed degradation of the isotropic average of throughput can be attributed to modal imbalance, that is, the latter attribute can be characterized by the proposed measurement approach.
The above derivation has been made only for the purpose of a qualitative interpretation of the significance of throughput measurements in the high SNR regime. It is neither necessary nor feasible to choose the AOAs from a grid on the unit sphere with resolution fine enough to accurately approximate in (33). Even for a setup with 2 test antennas only, there are 4 degrees of freedom . Hence, at an angular resolution comparable to traditional TIS/TRP measurement, instead of say samples (2 polarizations times grid points on the unit sphere), samples would be required. It must be emphasized that this observation holds equally for measurements in an anechoic chamber and in a reverberation chamber. The number of samples, that is, different constellations required for sampling with a prescribed resolution, can also not be reduced by employing more antennas. It should be clearly understood that with a larger number of test antennas we can only generate linear combinations of those constellations, which were also realizable with the smaller number of test antennas. Only when taking a larger number of samples, that is, resorting to a longer sequence of different constellations, further independent samples can be obtained. It is, therefore, inevitable, but as to presently available experimental evidence also sufficient, to resort to a sequence of constellations, which is just large enough to deliver trustworthy statistics of throughput. There are some minimum requirements on the selection of such sequence of constellations, which are experimentally confirmed and will be illustrated with examples below.(a)It is mandatory to consider both polarizations in all cases, that is, for each pair of AOA all 4 possible combinations of polarizations must be accounted for.(b)It is mandatory to achieve a fair representation of the assumed PAS. An “economization” of MIMO OTA measurement by, for example, restriction to AOAs, which are exclusively in the azimuthal plane may yield grossly misleading results.
Before turning to the proposed statistical metric for the present test case, some considerations with respect to the MIMO transmission modes, which may be selected for the measurement, that is, CL-SM or OL-SM, are necessary. In view of the above discussion, CL-SM appears to be well suited for the intended characterization. It comes with the additional advantage to minimize possible bias introduced through the test scenario by virtue of adaptive precoding. Investigation of CL-SM performance, however, makes sense only if the eNB emulator supports adaptive precoding based on the UE’s PMI report and adaptive per-stream CQI-based MCS selection. It must further allow to have the UE’s RI report ignored. These features are not yet in general supported by test equipment. OL-SM comes with an inherent randomization of the channel via LD-CDD and random precoding. As a consequence, throughput observed in each particular constellation represents by itself a statistical average of different channel realizations. Furthermore, when using OL-SM mode meaningful statistics can be gathered under conditions of an FRC provided that the power level is selected properly. In view of the limitations of presently available test equipment, OL-SM mode is therefore assumed below.
The proposed metric is the empirical Complementary Cumulative Distribution Function (CCDF) of observed relative throughput written as function of relative throughput in the form where denotes the set of applied constellations and is the applied power level.
Figure 7 shows an example for this metric. It relates to a commercially available LTE device, a USB dongle for E-UTRA band class 7. The device has connectors for two external antennas (Figure 6). For this proof-of-concept example, it was decided to use external antennas to facilitate interpretation of results. Two dipoles were used in different arrangements:(a)collinear placement with half-wavelength separation (“canonical dual dipole arrangement”) as shown in Figure 6(a),(b)orthogonal placement with half-wavelength separation (“crossed dipoles’’) as shown in Figure 6(b),(c)similar to (a) and (b) but with 45° inclination between the axes of the two dipoles.
(a)
(b)
The theoretical patterns of these antenna arrangements are of course disturbed by the presence of the laptop and connecting cables.
The CCDF of throughput for the 3 antenna arrangements were recorded for an FRC with 16QAM and a 10 MHz bandwidth (DL 2655 MHz, UL 2535 MHz) at a power level corresponding to kHz. A sequence of only 488 constellations (including different combinations of polarization) was applied in this example with a resolution of in and but fixed and only the two cases and . The choice of and was restricted by the chamber used for this measurement, which did not allow moving the second antenna automatically. The throughput observed for each constellation is weighted in the CCDF with the relative fraction of the surface of the unit sphere it represents, that is, the data reflect a two-path uniform PAS. Despite the fact that the sequence of applied constellations appears insufficient for a full characterization, the results in Figure 7 confirm that we can already clearly distinguish the three arrangements (a), (b), and (c). The arrangement (a) (collinear dipoles) performs worst and one may assume that this observation carries over to in-the-field performance where cross-polarized antennas are employed at the basestation.
One way to make use of the FOM in Figure 7 is to introduce again a “rejection region” as performance criterion. The shaded box in Figure 7 is given for illustration. It corresponds to the criterion that “UE should provide at least throughput in of all test cases.” The MNO can specifically tailor this requirement based on network quality requirements and user expectation.
Suitable chamber setups to be used in automated commercial test systems are presented in Figure 8. These two setups have extensions made to distributed axis chambers with an additional boom mounted at the opposite wall or the ceiling. The example presented in Figure 7 corresponds to a rudimentary implementation after Figure 8(b).
(a)
(b)
Further measurement results that were obtained in a chamber setup after Figure 8(a) are given in Figure 9. The figure displays the CCDF of throughput for 4 LTE devices of different form-factors (and operating in different bands). Results were recorded for an FRC with 16QAM and a 10 MHz bandwidth at a power level corresponding to kHz. A sequence of constellations (including different combinations of polarization), which have their AOA components limited to the azimuthal plane of the DUT, was applied in this example. The result permits to discriminate the devices although constellations with elevated AOAs are desirable in further investigations.
The question remains, how many constellations are actually required to sufficiently characterize a device. The question has been addressed by simulation, and results are presented in Figure 11. The simulation is based on a realistic compound pattern of an LTE datacard in a laptop with two antennas, operating in OL-SM mode in a MIMO configuration. The patterns are indicated in Figure 10 (colored and gray pattern, resp.). The simulation tool [6] is based on the LTE simulator developed by the Institute of Communications and Radio-Frequency Engineering of TU Vienna [21]. The MCS corresponds to , that is, QPSK with coding rate of . The maximum number of HARQ transmissions was set to 4. The DL channel bandwidth was chosen to be 5 MHz (25 Resources Blocks (RBs)). To save simulation time, received signals were captured after the Zero Forcing (ZF) receiver to estimate throughput for the selected MCS based on Error Vector Magnitude (EVM).
CCDFs of estimated throughput for dB for different numbers of randomly selected constellations are given in Figure 11. The throughput observed for each constellation is weighted in the CCDF with the relative fraction of the surface of the unit sphere it represents, that is, the data reflect a two-path uniform PAS. It is observed that a level of hundreds of constellations is already sufficient for convergence of the CCDF even for a complicated pattern as in Figure 10.
7. Conclusion
In this paper, a straightforward test approach is presented, which focuses on the characterization of the physical attributes of UEs. Statistical metrics are adopted to adequately represent the variety of possible propagation scenarios. The approach is of low complexity and allows for reuse of existing test sites and equipment. The relevant device properties are covered by two test cases.
The first test case covers total antenna system efficiency, receiver noise figure, and self-interference. It bears similarity to traditional TIS measurement for SISO devices. The statistical metric that is reported maps directly to in-the-field outage probability. The second test case covers modal imbalance, pattern and polarization selectivity. The measurement setup in this case realizes a maximum rank channel. The statistical metric, which is reported here, describes the probability of attaining theoretical peak performance in SM mode.
The examples that have been presented support the conclusion that the approach is suitable to distinguish very well between good and bad devices. Final validation of the predictive power of MIMO OTA test approaches, however, requires a thorough comparison between lab results and in-the-field observations, which has to be the program of future work.
Abbreviations
| 3GPP: | 3rd Generation Partnership Project |
| AC: | Anechoic Chamber |
| AOA: | Angle of Arrival |
| AS: | Angular Spread |
| AWGN: | Additive White Gaussian Noise |
| BLER: | Block Error Rate |
| BS: | Basestation |
| BSE: | Basestation Emulator |
| CCDF: | Complementary Cumulative Distribution Function |
| CDF: | Cumulative Distribution Function |
| CECC: | Complex Envelope Correlation Coefficient |
| CL-SM: | Closed-Loop Spatial Multiplexing |
| CQI: | Channel Quality Indication |
| CSI: | Channel State Information |
| DL: | Downlink |
| DUT: | Device Under Test |
| eNB: | evolved Node-B |
| E-UTRA: | Evolved Universal Terrestrial Radio Access |
| EVM: | Error Vector Magnitude |
| FOM: | Figure of Merit |
| FRC: | Fixed Reference Channel |
| GSCM: | Geometry-Based Stochastic Channel Model |
| HARQ: | Hybrid Automatic Repeat Request |
| HSDPA: | High-Speed Downlink Packet Access |
| HSPA: | High-Speed Packet Access |
| LD-CDD: | Large-Delay Cyclic Delay Diversity |
| LHS: | Left-Hand-Side |
| LTE: | Long-Term Evolution |
| MCS: | Modulation and Coding Scheme |
| MIMO: | Multiple Input-Multiple Output |
| MNO: | Mobile Network |
| MRC: | Maximum Ratio Combining |
| OFDM: | Orthogonal Frequency Division Multiplex |
| OL-SM: | Open-Loop Spatial Multiplexing |
| OTA: | Over-The-Air |
| PAS: | Power Angular Spectrum |
| PDF: | Probability Density Function |
| PMI: | Precoding Matrix Indicator |
| QAM: | Quadrature Amplitude Modulation |
| QPSK: | Quaternary Phase-Shift Keying |
| RB: | Resource Block |
| RD: | Receive Diversity |
| RHS: | Right-Hand-Side |
| RI: | Rank Indication |
| SCME: | 3GPP Spatial Channel Model Extended |
| SFBC: | Space-Frequency Block Coding |
| SISO: | Single Input-Single Output |
| SM: | Spatial Multiplexing |
| SNR: | Signal-to-Noise Ratio |
| TCIS: | Total Compound Isotropic Sensitivity |
| TD: | Transmit Diversity |
| TIS: | Total Isotropic Sensitivity |
| TRP: | Total Radiated Power |
| UE: | User Equipment |
| UL: | Uplink |
| ZF: | Zero Forcing. |
Acknowledgments
The authors would like to thank Mr. Aleksander Krewski for kindly supplying the simulation results in Figures 1 and 10. Part of the measurements presented in Figure 7 was conducted in the test lab of CETECOM GmbH, Essen, Germany. The authors gratefully acknowledge support by Mr. Rachid Acharkaoui. The research project on which this paper is based was funded by the German Ministry for Education and Research (BMBF) under project no. 17N1408. Responsibility for the contents is with the authors.