Abstract
A decorrelation and analog-to-digital conversion scheme aiming to reduce the power consumption of infrared image sensors is presented in this paper. To exploit both intraframe redundancy and inherent photon shot noise characteristics, a column based 1D Haar analog wavelet transform combined with variable gain amplification prior to A/D conversion is used. This allows to use only an 11-bit ADC, instead of a 13-bit one, and to save 15% of data transfer. An pixels test circuit demonstrates this functionality.
1. Introduction
Modern high-performance infrared sensors, like CdHgTe-based ones, require low-power consumption and digital output to reduce their cost and increase their ease of use, by avoiding the need for analog components on proximity board. However, when developing large format sensors (e.g., ) the bottlenecks of analog-to-digital conversion and data transfer for low-power compliance worsen. Thus, the first two main contributors to power consumption, to consider for minimizing it, are the analog-to-digital converters (ADC-) and the drivers for data transfer off the chip.
Several digital read-out circuits have been demonstrated, relying on pixel-level [1], column-level [2], or array-level A/D conversion. In such sensors, power optimization is focused on the ADC itself, and each pixel signal is treated as completely independent, in time and space, from the others. Thus no specific transfer rate optimization is implemented. Moreover, the ADC noise figure is defined with regards to the lowest-input signal noise, without considering the signal and noise dependency in the case of photons; this leads to over-conservative conversion for large input fluxes.
To target the data transfer power, compression is a well-known technique used in image processing to reduce the bit rate. Compression algorithms are composed of two steps: firstly, data are decorrelated using either a predictor or a transformation, then entropy coding is applied to reduce the bit rate. Implementations of compression are mostly digital; however, decorrelation schemes can also be implemented in the analog domain [3–5].
This paper, by exploiting the input signal characteristics as well as the inherent spatial redundancy, targets a decrease of both the ADC resolution and the amount of data transfer. It presents a decorrelation scheme based on a modified first-level Haar decorrelation combined with a variable gain applied to its coefficients accordingly to their probability density function (PDF).
This paper is organised as follows. Section 2 discusses the main noise contributions in an infrared image sensor. In Section 3, considering the pixels spatial correlations, the Haar wavelet decorrelation is introduced and its output on a real image discussed. Section 4 presents the proposed mixed wavelet transform and adaptive gain principle. Section 5 details its integrated electronic implementation, while the associated test results are discussed in Section 6.
2. Signal-to-Noise Ratio in Image Sensors
The classical read-out chain of a digital image sensor is composed of a photodiode, a current integrator, a pixel follower, and an out-of-focal-plane ADC. Photons whose energy falls into the photodiode material bandgap are converted to electrons. These electrons generate a photocurrent in the reverse biased photodiode. After integration, the resulting charge creates a voltage difference which is then transferred and A-to-D converted. The main noise sources [6] all along the chain are as follows: the shot noise which comes from photodiode current and has a noise power equal to the number of electrons generated during the photo conversion; the kTC or reset noise which appears at the reset of the integration capacitor; the read-out circuit noise which depends on the circuit architecture and technology used; the quantization noise which is usually considered to be a white noise inversely proportional to the ADC effective number of codes.
The calculation results on Figure 1 show how these noise contributions change with the input signal, in the case of a typical infrared imager characterized by a 500 fF integrating capacitor, a 1.6 V full scale at the ADC input and a 130 μVrms read-out noise (ROIC). The latter is a realistic value for a commercial product with resolution and frame rate higher than pixels at 50 Hz. The highest part of the dynamic range is dominated by the photocurrent shot noise whereas the quantization noise limits the SNR for low photon flux (Figure 1). The goal of infrared detection is to keep the overall circuit noise contribution below photon shot noise over the dynamic range of interest, to ensure background limited infrared photodetection (BLIP).
3. Pixel-to-Pixel Correlation
Real life images exhibit significant spatial redundancies. This is already widely used in image standards like JPEG to minimize data storage. To efficiently take benefit from those redundancies, image transformation is needed. transformation changes the domain of analysis from spatial domain to frequency domain in the case of discrete cosine transform DCT [5] or to wavelet domain in the case of discrete wavelet transform DWT [4, 7]. Most of the information is contained in the low-frequency coefficients for DCT or the high-level ones for DWT.
The case of the two-dimensional integer Haar wavelet decorrelation is considered in this study for its low hardware requirement: it is only based on adders and substractors. A 1st order Haar wavelet transform converts the raw output data of a group of pixels in one binning coefficient and 3 detail coefficients , according to the following equations [4]: corresponds to the mean value of the four pixels, while , , and depend on the gradients within the group.
Figure 2 presents a raw infrared image and its associated pixel value PDF (probability density function). The corresponding coefficients PDFs are shown Figure 3. It clearly appears that the detail coefficients exhibit a small deviation around 0, as neighbouring pixels have correlated values.
4. Mixed Wavelet Transform and Adaptive Gain Principle
In this work, pixel-to-pixel correlation is exploited thanks to a decorrelation scheme based on a 1st order Haar wavelet transform. Contrary to the case of visible images, in the infrared domain reference video streams are not available for benchmarking of image processing solutions. Thus we built our own set of infrared video data by acquiring images in various conditions including indoor and outdoor, night and day, static and moving scenes. A statistical analysis of these infrared images shows that more than 99% of the detail coefficients lie in the [−FS/8, FS/8] interval, FS being the full scale, which confirms their small deviation around 0. Hence they can be amplified by 4 prior to digitization without inducing saturation.
In addition, compared to the signal power, the shot noise power follows a square-root law. Consequently the ADC quantization noise has more impact at low flux. This impact can be reduced by preamplifying the signal or the binning coefficient which is its spatial mean, with high gain at low flux and low gain at high flux. As binning coefficients are spatially correlated, a predictive scheme can be used to anticipate their values. The predictor can be a combination of spatiotemporal neighboring binning coefficients previously determined. However, taking the last binning coefficient calculated as a prediction presents the advantage of an easy implementation and offers good approximation performance. In this proposed solution, we chose to implement such an adaptive gain prior to the AD Conversion. We selected gain values of 4, 2, and 1 for the ranges defined by the intervals [0, FS/4], [FS/4, FS/2], and [FS/2, FS], respectively. This allows to decrease the ADC effective number of bits (ENOB) by 2, provided the pre-amplifier’s noise is low enough.
Combining these two ideas led us to the following new concept depicted on Figure 4: one binning and three detail coefficients are generated per group of pixels by the discrete wavelet transform (DWT). For the detail coefficients the gain is set to 4 and for the binning coefficient it is set to 1, 2, or 4 depending on an estimation from the previous group of pixels in the column, thus further exploiting the spatial correlation within the image.
5. Silicon Demonstrator
This concept has been implemented in a test chip, on a 3.3 V, 0.35 μm CMOS process (Figure 5). The matrix format is 16 rows by 8 columns made of typical infrared 25 μm by 25 μm pixels. For testability purpose each pixel voltage can be set to an arbitrary value within the dynamic range (“electrical image”). The variable gain algorithm is simple (Figure 6): within the fixed set the gain used for the next block is twice, equal to or half the one used for the current block when the current value falls, respectively in the low, mid, or high part of the dynamic range. These parts are defined by two tunable digital thresholds. At each new frame, the gain is reset to its minimum value. The analog DWT/gain block is based on a standard switched capacitor schematic (Figure 7). Sample capacitors are first precharged to the voltage (, then ). Then the charges are transferred to the amplifier’s feedback capacitors with appropriate switch conditions to ensure the desired polarity and gain. The resulting voltage is then sampled hold, and converted by a 11-bit pipeline ADC which is shared by 8 columns.
The building blocks of this small format image sensor are designed to be compatible with a standard-size component (e.g., ): the number of lines can be increased by simply adding pixel blocks and the number of columns can be increased by abutting blocks composed of 1 ADC and 8 columns.
Thanks to a configuration register the chip can be set up in several test modes allowing to independently characterize the “electric image” writing process, the image analog read-out, the analog wavelet transform, and the AD conversion.
6. Test Results
The chip (Figure 8) has been tested at typical infrared imager operating temperature. Binning and detail coefficients measured results are presented Table 1. Slopes and offsets are derived by linear regression on the measured “coefficient versus input voltage” curves. The measurement was performed by applying the same value to the four inputs () and sweeping this value over the whole dynamic range. The theoretical results for such conditions are , with , and . The measured values exhibit some deviations which are due to nonsymmetrical parasitic capacitance effects (e.g., capacitance between substrate and bottom plate of a poly-poly capacitor) in the switched capacitor block. Like in standard image sensors, off chip offset and gain corrections can partly compensate for this; this has not been used in this study.
An example of measurements on an “electrical image” with a specific pattern is given Figure 9. The reconstructed image is obtained from the measured DWT coefficients by applying (off-chip) the inverse-DWT without any digital calibration.
(a) Test pattern
(b) Reconstructed image
Figure 9 shows that the reconstructed image matches the original one, but with an attenuated contrast in the transition region between the darkest and the brightest zones. This effect is due partly to the above-mentioned deviations and mostly to the read-out concept. Indeed, in high-contrast regions, the high-gain-to-low-gain transition might be spatially too slow. Also, some detail coefficients might lie outside the [−FS/8, FS/8] interval. Both cases will lead to saturation. Figure 9 input image was built to illustrate those effects. On real images, this quite improbable case will mostly occur around the “dead” pixels and will not excessively affect the image quality.
Also due to the parasitic capacitance effects, each block in the contrasted regions of the reconstructed image exhibits a column-oriented fixed pattern. This effect can be significantly reduced by both design optimization and off-chip calibration.
The analog blocks were designed to be compatible with a image sensor working at 60 frames per s. The measured power consumption of the DWT/gain block is 100 μW. The ADC exhibits an 11-bit ENOB at 246 kHz for 600 μW. This 700 μW proposed solution is intended to replace a 13-bit ENOB ADC which consumes 2.4 mW. Thus, we can expect a significant (>70%) power saving on the ADC contribution. In addition, switching from 13 bits to 11 bits, the dynamic power consumption due to data transfer will decrease by 15%; since the 2-bit gain information does not need to be transferred, it can be computed offchip by running the same gain setting algorithm on the digital data.
In the worst case, that is, gain of 4 (low flux), the output referred noise due to the DWT/gain block circuitry (amplifiers, charge injection, kT/C noise) was measured to be 200 μVrms. Quadratically added to the 310 μVrms quantization noise of the 11-bit ADC (on a 2.2 V full scale), this leads to a total 370 μVrms noise voltage added to the signal previously amplified by 4. So, when scaled to the original signal, the equivalent noise voltage is 92 μVrms, which corresponds to the quantization noise of a 12.8-bit ENOB on a 2.2 V full scale. In other words, compared to a 13-bit solution (77.5 μVrms quantization noise), the SNR loss, that is, the increase of the low end of the BLIP flux range, is 19%.
7. Conclusion
This paper presents a column-wise decorrelation scheme for high-resolution IR imagers and its silicon implementation and tests. This architecture exploits both intraframe redundancy and inherent photon shot noise characteristics to achieve a reduced power consumption while preserving the background limited infrared photodetection objective. It is based on a first-level Haar wavelet transform with predictive estimation of binning coefficients dynamic range. Combining into an analog implementation, an adaptive gain for the binning coefficients and a fixed gain for the detail coefficients allow to use a lower resolution ADC thus reducing the power consumption. Further power reduction is achieved thanks to the reduced amount of data to be transmitted. Overall, an image quality equivalent to a 12.8 bit direct quantization scheme is achievable. Those results prove the worthiness of the adaptive gain and analog wavelet transform approach to decrease the power consumption of infrared image sensors by allowing to relax the ADC resolution and, in addition, by, accordingly, decreasing the output switching activity.