Review Article  Open Access
CMOS Image Sensor with OnChip Image Compression: A Review and Performance Analysis
Abstract
Demand for highresolution, lowpower sensing devices with integrated image processing capabilities, especially compression capability, is increasing. CMOS technology enables the integration of image sensing and image processing, making it possible to improve the overall system performance. This paper reviews the current state of the art in CMOS image sensors featuring onchip image compression. Firstly, typical sensing systems consisting of separate imagecapturing unit and imagecompression processing unit are reviewed, followed by systems that integrate focalplane compression. The paper also provides a thorough review of a new design paradigm, in which image compression is performed during the imagecapture phase prior to storage, referred to as compressive acquisition. Highperformance sensor systems reported in recent years are also introduced. Performance analysis and comparison of the reported designs using different design paradigm are presented at the end.
1. Introduction
Image sensors are found in a wide variety of applications, such as biomedical microsystems, mobile devices, personal computers, and video cameras [1]. Since the image resolution and frame rate keep on increasing, imageprocessing capabilities are becoming an important consideration for both still image and video devices. Compression is one of the most demanding of the processing steps. Still, image compression is achieved by removing spatial redundancy, while in video devices, temporal redundancy can be used to further improve the compression performance. Various image compression algorithms and coding schemes have been proposed, such as predictive coding, Discrete Cosine Transform (DCT) based compression algorithms [2–4], and waveletbased compression algorithms [5–8]. International committees publish standard coding schemes for both still image and video stream, such as the Joint Photographic Experts Group (JPEG) standard series [9, 10], the H.26x [11] standard series published by the International Telecommunications Union (ITU), and Moving Picture Experts Group (MPEG) standards [12] published by the International Organization for Standardization (ISO) and the International Electrotechnical Commission (IEC).
VLSI circuit implemented image/video compression standards have been reported in the early literature [13–15]. Improvement in CMOS technology results in CMOS image sensor competitive to CCD image sensor in many applications [16, 17]. Correspondingly, the market share of CMOS image sensor has rapidly increased from the late 1990s. CMOS technology enables the integration of image sensing and image processing, making the CMOS image sensor the optimum solution to improve the performance of the overall system. In the last several decades, image sensors integrating different onchip compression algorithms, such as predictive coding [18], waveletbased image processing [19], DCTbased image processing [20], conditional replenishment [21], SPIHT algorithm [22], and FBAR and QTD processing [23] have been developed. A simplest way to design such a system is to implement different functions using sperate circuit units, such as imagesensing unit and imagecompression processing unit. Another option is to implement image compression into the sensor array, which is referred to as focalplane compression. If imagesensor system integrates focalplane compression, imageprocessing is performed near the focal plane once the pixel value is captured. Higher image quality and higher processing rate can be expected in such a system. Recently, a new design paradigm referred to as compressive acquisition [24] proposed to carry out image compression during the image capture phase, moving the imageprocessing phase prior to the storage phase. It combines image capture with compression, enabling lower onchip storage requirement.
Some widely used imagecompression algorithms are firstly reviewed in Section 2. Section 3 compares the different paradigms that are used in the development of CMOS imagesensors that integrate onchip imagecompression algorithms. These design paradigms include imagesensor array integrated offarray image compression processor, imagesensor array integrated focalplane compression, and compressiveacquisition imagesensor array performing compression processing during the capture phase. At the end of Section 3, some highperformance system reported in recent years is also introduced, followed by performance analysis and comparison between the reviewed designs. Section 4 concludes the overall paper.
2. Standard Image/Video Compression
2.1. Still ImageCompression Algorithms
Image compression reduces the amount of data required to represent a digital image by removing data redundancy. Imagecompression algorithms can be classified into lossy and lossless.
In lossy imagecompression, there is a tradeoff between the compression ratio and the reconstructed image quality. If the distortion caused by compression can be tolerated, the increase in compression ratio becomes very significant. Lossy imagecompression algorithms can be performed in either spatial domain or transform domains (such as frequency domain). A classical lossy imagecompression scheme is lossy predictive coding. The predictive value is quantized using limited bits. There are different effective predictors, such as the gradientadjusted predictor (GAP) and the median adaptive predictor (MED). Another way to compress the image is to firstly map the image into a set of transform coefficients using a linear, reversible transform, such as Fourier transform, discrete cosine transform (DCT) transform [2–4], or wavelet [5–8] transform. The newly obtained set of transform coefficients are then quantized and encoded.
Lossless imagecompression algorithms are errorfree compression, which are widely used in medical applications, satellite imagery, business documentation compression, and radiography area because any information loss is undesirable or prohibited. Typically, compressing an image consists of removing either coding redundancy or interpixel redundancy, or both. The simplest way to compress an image is to reduce the coding redundancy using variable lengthcoding schemes. A variable lengthcoding scheme maps source symbols to a variable number of bits. Huffman coding [25, 26] and arithmetic coding [27, 28] are all wellknown variable lengthcoding strategies. An effective way to reduce interpixel redundancy is to use bitplane coding schemes. In a bitplane coding scheme, an image is firstly decomposed into a series of binary images, and then those images are compressed by a binary compression algorithm, such as runlength coding scheme and contourtracing and coding scheme. Another way to remove interpixel redundancy is to use LempelZivWelch (LZW) coding scheme [29–31], which replaces character strings with single codes without the requirement of any prior knowledge of the probability of occurrence of the symbols.
In order to make sure that compressed images can be decoded correctly, standardization of image compression algorithms becomes very significant. The Joint Photographic Experts Group committee proposed the DCTbased JPEG standard [9, 10] and the waveletbased image compression standard JPEG 2000 [32–36], in 1992 and 2000, respectively.
2.2. VideoCoding Schemes
Research in videocoding compression can be traced back to early 1980s. From that time on, the Consultative Committee for International Telegraph and Telephone (CCITT) and the International Organization for Standardization (ISO) began to standardize the various videocoding schemes. Later, H.26x [37–40] and MPEG1 [41]/MPEG2 [42]/MPEG4 [43–45] are published and used in various video devices and applications. The newly reported video coding scheme, the socalled distributed video coding (DVC) [46–48], is based on the SlepianWolf [49] and the WynerZiv [50] theorems in information theory published in 1970s. As DVC provides better compression efficiency than the widely used standards, there is much research focusing on this area in recent years.
3. OnChip Image Compression for CMOS Image Sensors
3.1. Image Sensors with OffArray Image Compression
In the 1990s, taking advantage of the development of CMOS technology, CMOS image sensor began attracting the attention of designers. The improvement of manufacturing processing and the use of noisereduction techniques decreased the noise levels in CMOS image sensors, making them comparable to the conventional CCD image sensors. Using CMOS image sensor, peripheral imageprocessing circuits can be easily integrated with the sensor array.
3.1.1. Typical Design Reviews
A typical smart imagesensor system implements the imagecapturing device and the image processor into separate functional units: an array of pixel sensors and an offarray processing unit as illustrated in Figure 1. Active Pixel Sensor (APS) array is widely used to convert light intensity into analog signals. A standard APS architecture includes a photodiode, a transfer gate, a reset gate, a selection gate, and a source follower readout transistor. The reset gate resets the photodiode at the beginning of each capture phase. The source follower isolates the photodiode from the data bus. The analog signals from the sensor array take raw pixel values for further image processing. Different designs have been reported in the literature implementing different compression algorithms in the offarray processor.
In [18, 51–54], predictive coding schemes are implemented in the offarray columnlevel processor. Predictive coding is a very useful imageprocessing scheme with low computational complexity. It removes correlation between neighboring pixels. The simplest lossless predictive coding scheme, the gradientadjusted prediction (GAP) integrated with imagesensor array is first proposed in [51]. The mean value of the upper and the left neighbors are used as the predictive pixel value . Pixels in the first row and first column are not predicted. An analogpredictive circuit is used to calculate both the predictive pixel value and the prediction error between the predictive value and the raw value. A more complexity version of GAP using the sum of the weighted upper, left, top left, and top right neighboring pixels is implemented in [52]. In [18, 53], the median adaptive predictor (MED predictor) is implemented in the offarray logic, predicting pixel value using the upper pixel value (denoted as ), the left pixel value (denoted as ), and the top left pixel value (denoted as ) as expressed in Architecture of the overall system is illustrated in Figure 2. Pixellevel capacitors are used to buffer the raw captured pixel value before further processing. After readingout one pixel , a differential amplifier, which is integrated in the analogpredictor circuit, subtracts the photogenerated voltage from the reset voltage so as to reduce the pixel offset fixedpatten noise. Both the analog raw pixel values and the analog predictive values are converted into digital values in columnlevel singleslope A/D converters. The digital counters of the singleslope A/D converts are also used to generate GolombRice codes by counting the total times that the counter overflows the counting range (0 to ). Testing results show a compression ratio of around 1.5 can be achieved, which is comparable to the lossless compression standards.
In [55], the authors proposed an integration of image sensing and waveletbased image processing. The overall sensor array consists of APSs, each of which integrates a photodiode, a reset transistor, and a source follower. Correlated double sampling (CDS) circuits are used in every column to reduce the fixed pattern noise (FPN), pixel KTC noise, and 1/f noise. iterations are required by the Haar transform, as the overall raw image has been divided into blocks. Only adder and subtractor processing elements are required in each column parallel basic Haar transform unit circuitry to complete the Haar transform. Row pixel values readout from the column CDS circuits and a copy of those values are buffered in two parallel capacitors. The calculation of Haar transform is performed by switches and capacitors. By integrating the image processing, the amount of data required to be transformed during communication is reduced. This improvement of performance is significant for highresolution and highframe rate image sensing, as well as for power constrained and bandwidth constrained devices.
Highthroughput image sensors integrated with offarray image compression have been reported in [56, 57]. In [56], a CMOS image sensor integrated together with a 2D Haar wavelet imagecompression algorithm has been proposed. 1.4 GMACS (Giga Multiply Accumulate Operations Per Second) throughput with SVGA imager resolution is achieved, while in [57], 4 GMACS throughput with HDTV 1080i resolution is achieved using an image sensor integrating an onchip DCT processing. Both of the two proposed designs were mixedsignal CMOS image sensors, which took the advantage of both analog and digital design techniques as illustrated in Figure 3. The combination of weighted spatial average and oversampling quantization carried out in a single conversion cycle of the modulated ADC enables realtime focalplane processing. While the implementation of digital logic enables digital output with high accuracy.
3.1.2. Efficiency of the ImageSensor Array Readout
At the beginning of CMOS sensors use, designers were apt to treat the image capture device and the image processing unit as two independent units only connected by the readout interface, which enables row by row scanning of the analog raw pixel values from the sensor array. But later, embedded pixellevel control logic and corresponding offarray readout circuit becomes a better option in such a system design, as it can greatly improve the efficiency of further image compression processing.
In [58], a CMOS block MAtrix Transform Image architecture (MATIA) is designed as a front end for JPEG compression as illustrated in Figure 4. A highswing cascode operational amplifier is used for pixel readout, as well as for current measurement for programming floating gates in the following processing units. The matrix coefficients are stored using an array of floatinggate circuits, each cell of which is a polysilicon gate surrounded by silicon dioxide. The charge on the floating gate can be permanent as the surrounded silicon dioxide provides highquality insulator. A column of current to voltage () converters are connected to the floatinggate array to convert the respective current to bias voltage for the matrixvector multiplication. Current mode differential vector matrix multiplier (VMM) is used to perform matrixvector multiplication, instead of the traditional voltage implementation, because of the VMM's high processing speed, low power consumption, and high linearity. Taking advantage of the programmability of the proposed architecture, twodimensional (2D) transforms or filter operations on an entire image, or block matrix operations on subimages can be performed. Various block transforms can be implemented, such as the DCT, the discrete sine transform (DST), and the Haar transform. It can be extended to different applications, including motion estimation, depth computation from stereo, and spatial or temporal compression and filtering. In [58], the 2D DCT and Haar transform are carried out in MATIA as examples, in both of the two examples, elemental block size is used.
In one of the most recent published work which integrated wavelet in the offarray processor [19, 59], two capacitors are implemented at the pixellevel to store both the reset voltage and the integrated photocurrent for subsequent multiple sampling processing as illustrated in Figure 5. As nondestructive readout is used in the readout interface, one very interesting feature of the pixel architecture is that the spatial image processing is available as well as temporal image processing based on frame difference calculation. The proposed work consists of APS array. Columnbased processing elements are used to perform the blockmatrix transform on the readout pixel values as expressed in which requires pixelwise signed multiplication and crosspixel addition processing that will be carried out in the mixed signal VLSI domain columnbased logic circuit. The column based circuit consists of a sign unit, a binary analog multiplier, an accumulator, and multiplying analog to digital converters (MADCs). A switch matrix takes the blockmatrix coefficients value from the binary analog multiplier and the corresponding sign bits from the sign unit and sends these signal to the binary analog multiplier. The MADCs are used to multiply the readout pixel values with respective digital coefficients for convolutional transform. The proposed computational functionality of the sensor array is validated onchip Haar discrete wavelet transform (DWT) based image compression. The transform results are compared to a threshold before data transmission. Transform results smaller than the threshold are filtered out.
Reference [60] reported a highspeed (>1000 fps) CMOS image sensor with a resolution of integrated together with a DCT processor. In order to achieve a highprocessing speed, global electronic shutters are implemented with pixellevel samplehold function unit as illustrated in Figure 6. Raw pixel values are read out and digitized into a 10bit digital signal in parallel ADC row by row. The 10bit digital pixel values are buffered and rearranged in an input buffer memory before being sent to the array of image compression processing elements (ICPE). 2D DCT, quantization, zigzag scanning, and Huffman coding are carried out in the ICPEs. In the DCT processing, elemental block size is used instead of the more commonly used DCT elemental block size, so as to reduce the amount of inner product computations. Experimental results show that for a image sensor array, 3000 fps can be achieved under an operation frequency of 16.8 MHz. The frame rate can be as high as 8500 fps if 47.6 MHz operation frequency is used. Thus, with the proposed architecture, 1 Mpixel, 3000 fps digitalimage sensor can be realized under an operating frequency of 53 MHz.
Compared to one by one readout, blockbased readout is a more efficient way to prepare raw pixel data for further blockbased transforms. In [61], The sensor array is divided into blocks which is the elemental block size of the cosine coefficient matrix used in the following analog 2D DCT processing unit and a subsequent analogtodigital converter/quantizer (ADC/Q). Raw pixel data values are read out from the sensor array block by block during one readout circle. There are two stages in the readout phase. At the first stage, a frontend amplifier and a successive fully differential amplifier convert the signal charge into voltage using a 100 fF capacitor and shift the voltage range to be suitable for the subsequent signal processing. Both amplifiers are designed based on switched capacitor technique. At the second stage, CDS scheme is carried out to reduce the 1/f noise and offset voltage deviation. 2D DCTbased compression algorithms are implemented in the offarray processor as illustrated in Figure 7 to compress the raw captured pixel values [20, 62, 63]. The 2D DCT is performed using analog 1D DCT processor. The 1D DCT processor consists of 32 coefficient multiplications and 32 additions and switches control logic for weighted summation. 8 rows are read out and calculated at one time. The intermediate results are stored in an analog memory, each of which consists of 4 switches and 2 capacitors. 1D DCT is performed by 2 clock cycles. Thus, it takes 32 clock cycles to finalize an 2D DCT computation. 9bit ADC with differential nonlinearity (DNL) of less than 0.5 least significant bit (LSB) is used to digitize and quantize the analog 2D DCT results in order to keep a high PSNR (over 40 dB). Variable length coding is utilized to further remove data redundancy before transmission. The overall sensor array consists of active pixel sensors (APSs). Taking the advantage of parallel processing, the operation frequency is only 62 kHz at 30 fps.
In [64], a sensor array integrated enables three different processing modes, integrated intensity mode (Imode), spatial contrast mode (Cmode) and temporal difference mode (Tmode), during the readout phase using pixellevel control units. Only 11 transistors are required for temporal redundancy removing. Pixellevel capacitors are used to buffer raw capture pixel values or value from a selected neighbor. Winnertakesall (WTA) and losertakesall (LTA) circuits are used to find out a brightest and a darkest pixel value among some selected candidates. Under Imode, raw pixel values are captured, which maximum and minimum differential values between 4 neighboring pixels can be calculated online in the offarray column based circuit under Cmode. Under Tmode, differential value between pixel values within neighboring frames are calculated in real time during the readout phase. All the three modes are implemented in pixellevel processor controlled by pixellevel switches.
3.1.3. HighPerformance CompressionProcessor Design
In fact, the performance of the offarray processor is one of the key factors that affect the performance of the overall system. Highperformance offarray processor designs have been reported in the literature.
In [65], an digital DCT processor is proposed, in which a variable threshold voltage is used to reduce the active power consumption with negligible overhead in speed, standby power consumption, and chip area of a 2D DCT processing core for portable equipment with HDTAresolution video compression and decompression. Comparing the design reported in [61] with that of [65], the power consumption reported in [61] is only about half of the power consumption reported in [65].
In [66], a lowpower, realtime JPEG encoder is presented with a resolution of up to . Eight different optional output resolutions are available in the proposed system. A buffer is used as an interface between the CMOS image sensor and the encoder, which read out and rearrange raw captured pixel values from the sensor array. The reported JPEG encoder is fully compliant with the JPEG standard, including DCT, quantizer, runlength coding, Huffman coding, and packer unit. The DCT processing element in the proposed work consists of 3level pipeline processing units. In the first level, raw pixel values are read out, while arithmetic distribution and DCT coefficient generation are carried out in the second and the third level unit, respectively. Experimental results showed that a 15 fps is achieved at a highest output resolution (), while 30 fps is achieved at a lower resolution ().
In 2000, Philips National Research Laboratory developed a Transpose Switch Matrix Memory (TSMM) to reduce power consumption of onchip image compression processing that required block level communication. This was done by enhancing the dataaccess efficiency and the memory utilization. The proposed TSMM is used in a highly parallel singlechip CMOS sensor processor, Xetal [67], to perform JPEG compression at video rate (30 fps) with a resolution of [68]. The Xetal is a singleinstruction, multipledata (SIMD) linearprocessing array for pixellevel image processing. 640 ADCs are used to digitize the analog values. Recursive block DCT is carried out in 320 offarray processing elements (PEs), 80 TSMM units and an embedded 40line memory as illustrated in Figure 8. The TSMM is designed to facilitate the blocklevel interprocessor communication in Xetal, so as to perform DCT or JPEG on it. It consists of a matrix of registers, using switches to enable access to horizontal and vertical buses. The flexibility databus controlling of TSMM simplifies the implementations of the transposition of DCT coefficients and the Zigzag output scanning.
In [69], a VLSI circuit for wavelet image compression was reported. In the proposed architecture, there are four major processing elements: (i) a data format conversion processing element, which converts the raw pixel value from BAYERRGB format into BAYERYY format; (ii) a wavelet transformunit, which performs 1D wavelet transforms firstly on rows and then on columns; (iii) a binary adaptive quantizer, which quantizes the wavelet transform coefficients; (iv) a significant coefficient pyramid coder, which further reduces the coding redundancy. Testing results show that under an operating frequency of 25 MHz, a processing rate of 1.5 M pixels/s is achieved. Color image with a resolution of can be compressed within 1 second if faster onchip memory is available.
3.2. Image Sensors with FocalPlane Image Compression
However, the computational complexity of most standard compression algorithms is still high, such as those mentioned in the previous subsection. While the image resolution and frame rate keep increasing, the processing time becomes a bottleneck in highspeed sensor design. In order to increase the processing rate, pixellevel processor are used to implement inarray parallel processing.
3.2.1. APS Array with FocalPlane Compression Processor
In fact, as early as in 1997, imagers integrated with parallel inarray processing circuit were being reported in [21, 70]. In the proposed work, a computational image sensor explores the parallel nature of image signal by integrating conditional replenishment algorithm, a pixellevel video compression schemes, which removes temporal redundancy of pixels between consecutive frames. As illustrated in Figure 9, in each pixel, raw capture pixel value is firstly stored in a capacitor. A pixellevel differential amplifier is used to compare the newly captured value with the pixel value of the previous frame that is buffered in another capacitor. Only activated pixels are readout, in which the difference between the newly captured pixel value and the previously captured pixel value is larger than a threshold. Analysis in [21] shows that the higher the frame rate is, the lower the percentage of active pixels in the sensor array. Differences caused by the motion between the frames are much less while a higher frame rate is used. The threshold and the frame rate are adjusted in real time so that the total number of activated pixels read out for one frame is almost a constant. A compression ratio within 5 : 1 to 10 : 1 can be achieved without significant degradation of the video quality with low motion activity. show video stream Different methods to encode the activated pixel address are compared in [21]. In [71], based on the onsensor compression architecture, the authors further implemented offarray coding scheme [70, 71].
Image sensors integrated similar pixellevel processing element to remove temporal redundancy have been widely reported in the literature. In [72, 73], a visionsensor integrated pixellevel processor which can asynchronously respond to changing events between frames. Architecture of the pixel combines an active continuoustime logarithmic photosensor and a wellmatched selftimed switched capacitor amplifier. Once the changing of the illumination intensity exceeds a threshold, a request will be sent out. Offarray addressevent representation (AER) processor will respond to those request asynchronously. In [74], pixellevel processor is embedded into an APS array. Two capacitors are used to buffer the pixel value of the newly capture frame and the previous frame, respectively. The differential value between the two buffered pixel values are compared with a threshold during the readout phase. In [75], a sensor array with the QVGA resolution integrated lossless temporal compression has been proposed. Only pixels with a new capture value different from the previous one will be sampled, resulting in reduction of power consumption, bandwidth and memory requirement. A change detector is implemented in pixellevel circuit to perform the proposed processing.
References [22, 76–80] proposed an APS array integrated pixellevel parallel prediction image decomposition algorithm and Set Partitioning In Hierarchical Trees (SPIHT) coding scheme. The proposed prediction scheme enables image decomposition with lower computational complexity, while compared with standard waveletbased algorithms as detailed in [77]. Within each block, except for the topleft corner, all the pixel values are predictive values calculated from summation of weighted neighboring pixel values. The proposed predictiondecomposition algorithm is performed parallel in pixellevel circuit, which enables a higher frame rate as claimed in [80]. The pixellevel chargebased computational circuit consists of comparator, capacitors, and control logic circuit (totally thirteen transistors and four capacitors in each pixel) as illustrated in Figure 10. Four different pixels are required for onelevel prediction. Capacitors are used to buffer current pixel values for residual computation and to buffer nearby pixel values for the prediction decomposition processing. For an level image composition, only raw pixel values are required to be recorded. More details of a chargedbased computational circuit can be found in [76, 78]. Computational error caused by parasitic capacitances were analyzed in [22]. Estimated coefficients for nine subbands obtained from both the gray gradient test image and the test pixels built outside the sensor array show that the coefficients are consistent for different subbands and close to the approximate theoretical values. Another main source of computational error is the error caused by charge injection. The latter can be minimized by improving the prediction accuracy. In order to further improve the performance of the sensor array, CDS is used to reduce the FPN inherent in the mismatch of threshold voltage. The linearity of the proposed prediction decomposition scheme enables the integration of the residual computation circuits in the signal path of the CDS function.
3.2.2. DPS Array Integrated Compression Processor
Among different architecture of CMOS image sensors, the digital pixel sensor (DPS) is the most recently proposed architecture [81–85]. It integrates massively parallel conversion and provides digital readout circuit, enabling easy implementation of parallel processing. Higher processing rate can be expected while using DPS to acquire raw pixel values for onchip imagecompression processing.
References [23, 86] proposed a DPS array sensor integrated together with an onchip adaptive quantization scheme based on Fast Boundary Adaptation Rule (FBAR) and Differential Pulse CodeModulation (DPCM) procedure followed by an online Quadrant TreeDecomposition (QTD) processing. The overall sensor array is operated in two separate phases: integration phase and readout phase. As illustrated in Figure 11, the integration phase begins from a reset operation, during which voltage of the photodiode is pulled up to , and a global counter is reset to all one at the same time. After resetting, the voltage of the photodiode is discharged proportionally to the illumination intensity, meanwhile, the global counter began down counting. Once the voltage of the photodiode achieves a reference voltage value, the counting result of the global counter will be written into a pixellevel memory as the digitized raw captured pixel value. During the readout phase, the sensor array is used as a matrix of 8bit memories. In the offarray logic, QTD compression is performed by building multihierarchical layers of tree corresponding to a quadrant using Hilbert scanning. By using the Hilbert scanning scheme, the storage requirement for adaptive quantizer is reduced, as Hilbert scanning scheme keeps spatial continuity, while scanning the image quadrants. The raw pixel values are quantized by an adaptive quantizer designed based on FBAR. FABR minimizes the th power law distortion, which is the most commonly used distortion measure scheme. One interesting feature of the proposed work is that the FBAR algorithm is performed on the predictive error rather than the pixel itself using DPCM, which results in compressed dynamic range.
3.3. Compressive Acquisition Image Sensor
In the mentioned applications, the image is first acquired, and then compressed. Raw pixel data values are buffered during readout phase in offarray storage or directly buffered into inarray storage before being compressed, resulting in a very high storage requirement. In [24], a new concept of compressive acquisition has recently been proposed to perform compression online during the image capture phase, prior to the storage phase. This approach shifts the design paradigm from the conventional: Capture Store Compress to the newly proposed: Capture Compress Store as illustrated in Figure 12. The paper illustrates the potential advantages of such a new design paradigm namely: (i) reduced silicon area required for the DPS, (ii) reduce onchip storage requirement, and (iii) compression processing integrated within the pixel array, enabling the concept of parallel processing. The basic idea behind a compressive acquisition CMOS image sensor is to combine image compression processing with image capture, so as to reduce the onchip storage requirement. There is obviously a tradeoff between reduction of storage requirement and the complexity of pixellevel circuit design. The key question affecting the success of this approach is whether it is possible to simplify the compression processing to enable a powerful and yet simple pixel design.
(a)
(b)
Different compression algorithms that are suitable to be integrated with a compressive acquisition CMOS image sensor have been proposed. In [24], a blockbased compression algorithm, which can be implemented at the pixellevel with very limited hardware complexity, and hence limited overhead in terms of silicon area was proposed. The overall sensor array of digital pixel sensors is divided into blocks. Within each block, the brightest raw pixel value is recorded in blocklevel memory. The differential value between pixel value and the brightest pixel value within a same block is calculated and quantized during the imagecapture phase. Only the quantized differential values are recorded instead of 8bit raw pixel values.
In [87], an online 1bit predictive coding algorithm using Hilbert scanning scheme that can be implemented in pixellevel circuit is proposed. Compared to conventional DPS that integrated 8bit pixellevel memory, the proposed architecture reduces more than half the silicon area by sampling and storing the differential values between the pixel and its prediction, featuring compressed dynamic range and hence requiring limited precision. In order to further improve the reconstructed image quality, Hilbert scanning is used to read out the pixels values instead of conventional raster scanning. The Hilbert scanning path is all carried out by hardware wire connection without increasing the circuit complexity of the sensor array. Reset pixels are inserted into the scanning path to overcome the error accumulation problem inherent in the predictive coding.
In [88], an online compression algorithm is proposed for compressive acquisition CMOS image sensor which removes spatial redundancy by sampling selected pixel values instead of all the pixel values within the overall sensor array. Blockbased division is performed on the overall sensor array. In each block, the raw pixel values are reordering after captured. Only the brightest and darkest pixels are sampled and stored in blocklevel memories. In imagereconstructing phase, different block models are built based on analysis of the relationship between neighboring blocks. Taking advantage of the feature of DPS, the online sorting process can be implemented in pixellevel circuit with very low overhead.
In [89], the concept of compressive acquisition CMOS image sensor is extended to temporal domain. An efficient intra and interframe compressiveacquisition scheme which removes spatial and temporal redundancy prior to storage is proposed in this paper. The overall sensor array is divided into quadrants, whereby a brightest pixel value within each quadrant is selected as a quadrant value to represent the quadrant. A reference array is built to track the quadrant value changes between different frames. The reference array is updated adaptively by an offarray judge logic after capturing one frame. The background versus nonbackground classification results referring to the comparison result between the reference array and the matrix of all the quadrant values are used to remove the data redundancy before transmission.
3.4. Performance Analysis and Comparison
Image sensors with integrated onchip image compression featuring highresolution, highthroughput, highframe rate, or low power consumption, have been proposed in recent years. The chip reported in [58] integrating JPEG processing is operated in subthreshold domain enabling very lower power consumption (80 μW/frame). 1.4 GMACS (Giga Multiply Accumulate Operations Per Second) throughput with SVGA imager resolution is achieved in [56], while in [57], 4 GMACS throughput with HDTV 1080i resolution is achieved using an image sensor integrating an onchip DCT processing. In [22], a frame rate of 3000 fps can be realized with a power consumption of 0.25 mW.
Table 1 compares the features of the abovecited designs. The traditional design paradigm implements the image pixel sensor and the imagecompression processor as two separate functional units connected by an offarray scanbased readout interface. Typically, more complexity standard compression algorithms can be implemented in an offarray processor, resulting in a higher quality of the reconstructed image. A PSNR value higher than 30 dB can be easily achieved as shown in the table. Different architectures of A/D convertor are used in those designs as compared in Table 2.
 
APS stands for Active Pixel Sensor. The image pixel sensor and the image compression processor are two separate functional units connected by offarray readout interface. Simple pixellevel logic is used in order to better apt the processing procedure of the compression algorithm used in the offarray processor. Typically, customized readout logic is required for such a image sensor array. PSNR stands for Peak SignaltoNoise Ratio. The compression processing is performed in pixellevel focalplane processor. 
However, traditional scanbased readout circuits have limitations on the volume of data that can be readout. While with each new generation of image sensors, the image resolution and frame rate keep increasing, as illustrated in Table 3. CMOS image sensor integrating pixellevel processor have been studied to solve this issue [22–24, 70, 73]. Focalplane compression processing can be implemented in either the analog domain [22, 70, 72], or digital domain [23, 24]. A higher processing rate can be expected by using focalplane processor. In [22], pixellevel chargebased computational circuit are used to perform the predictiondecomposition algorithm. Processing rate of the proposed system can be as high as 3000 fps. However, due to the limitation of the processing capability in pixellevel processor, the reconstructed image quality is reduced by performing processing in pixellevel processor. Imagesensor array integrated pixellevel parallel processor enables a processing rate up to 10000 fps has also been reported as in [90].

Using digitaldomain processor, a smaller pixel size can typically be expected, as no capacitor is required in pixellevel circuit. However, the noise introduced by using pixellevel circuit for compression processing is always a big disadvantage of focalplane processor. In general, dark current is the leakage that flows through the photosensitive device even when no photons are entering the device, and can result in both FPN and temporal noise. The switching activities, due to the embedded pixel level processor, can result in higher substrate noise near the photosensitive device. As a consequence, an increase in temporal noise correlated with the switching activities may result. This can, however, be minimized if switching is prohibited during the integration cycle.
4. Conclusion
Compression is one of the most demanding image processing tasks. CMOS technology enables the integration of sensors and image processing, making it a very interesting and promising technology for future image systems. A review of the state of the art onchip image compression integrated with CMOS image sensor has been provided. The paper begins from the traditional architecture in which image capture device and image compression processor are implemented in separate units connected by the readout interface. Architecture of imagesensor array integrated inarray pixellevel processor is used to improve the performance of the overall system. However, the image is always first acquired and then quantized and compressed. A new design paradigm referred to as compressive acquisition is also reviewed in this paper showing an interesting concept which performs compression processing during the capture phase prior to the storage phase, thus a reduction of inarray storage requirement can be expected. Recently proposed image sensors integrated together with onchip image compression processing featuring high performance, such as high resolution, high throughput, high frame rate, or low power consumption, have been reviewed in this paper.
The development of CMOS image sensor featuring onchip image compression has been very fast in the last decade. As the image resolution keeps increasing, the amount of data for each frame enhances, resulting in requirement of efficient compression processing capability or higher throughput capability. In addition, image sensors are widely used in consumer electronic applications, most of which are battery powered, lower power consumption capability is very important in nowadays imagesensor design. We believe that in the near future, the performance of CMOS image sensor integrated with onchip image compression will enable to meet the requirement of modern imager system.
Acknowledgments
The authors would like to thank Mr. Berin Martini and Mr. Matthew Law for their helpful suggestions and discussion. This work was supported by a grant from the Research Grant Council (RGC) of Hong Kong, CERG Grant reference no. 610509.
References
 A. El Gamal and H. Eltoukhy, “CMOS image sensors,” IEEE Circuits and Devices Magazine, vol. 21, no. 3, pp. 6–20, 2005. View at: Publisher Site  Google Scholar
 N. Ahmed, T. Natarajan, and K. R. Rao, “Discrete cosine transfom,” IEEE Transactions on Computers, vol. 23, no. 1, pp. 90–93, 1974. View at: Google Scholar
 R. C. Reininger and J. D. Gibson, “Distribution of the twodimensional DCT coefficients for images,” IEEE Transactions on Communications, vol. 31, no. 6, pp. 835–839, 1983. View at: Google Scholar
 K. Rose, A. Heiman, and I. Dinstein, “DCT/DST alternatetransform image coding,” IEEE Transactions on Communications, vol. 38, no. 1, pp. 94–101, 1990. View at: Google Scholar
 R. Wilson, “Wavelets?” in IEE Colloquium on Applications of Wavelet Transforms in Image Processing, pp. 1/1–1/6, London, UK, 1993. View at: Google Scholar
 E. Cetin and O. N. Gerek, “Block wavelet transform for image coding,” IEEE Transactions on Circuits and Systems for Video Technology, vol. 3, no. 6, pp. 433–435, 1993. View at: Publisher Site  Google Scholar
 D. Park and M. HO. Lee, “Image compression based on best wavelet packet bases,” IEEE Transactions on Consumer Electronics, vol. 40, no. 3, pp. 527–537, 1994. View at: Google Scholar
 L. H. Croft and J. A. Robinson, “Subband image coding using watershed and watercourse lines of the wavelet transforms,” IEEE Transactions on Image Processing, vol. 3, no. 6, pp. 759–772, 1994. View at: Publisher Site  Google Scholar
 W. Pennebaker and J. Mitchell, JPEG, Still Image Data Compression Standard, Van Nostrand, New York, NY, USA, 1993.
 ISO/IEC 144951:1999, “Information TechnologyLossless and NearLossless Compression of ContinuousTone Still Images:Baseline[S],” December 1999. View at: Google Scholar
 T. Wiegand, G. J. Sullivan, G. Bjøntegaard, and A. Luthra, “Overview of the H.264/AVC video coding standard,” IEEE Transactions on Circuits and Systems for Video Technology, vol. 13, no. 7, pp. 560–576, 2003. View at: Publisher Site  Google Scholar
 D. J. Le Gall, “MPEG (moving pictures expert group) video compression algorithm: a review,” in Image Processing Algorithms and Techniques II, vol. 1452 of Proceedings of SPIE, pp. 444–457, San Jose, Calif, USA, February 1991. View at: Publisher Site  Google Scholar
 B.F. Wu and C.F. Lin, “Memoryefficient architecture for JPEG 2000 coprocessor with large tile image,” IEEE Transactions on Circuits and Systems II, vol. 53, no. 4, pp. 304–308, 2006. View at: Publisher Site  Google Scholar
 Y. Li and M. Bayoumi, “VLSI architectures for JPEG 2000 EBCOT,” in Proceedings of the 40th Asilomar Conference on Signals, Systems, and Computers (ACSSC '06), pp. 907–911, OctoberNovember 2006. View at: Publisher Site  Google Scholar
 L. Liu, N. Chen, H. Meng, L. Zhang, Z. Wang, and H. Chen, “A VLSI architecture of JPEG2000 encoder,” IEEE Journal of SolidState Circuits, vol. 39, no. 11, pp. 2032–2040, 2004. View at: Publisher Site  Google Scholar
 D. Litwiller, “CCD vs. CMOS: facts and fiction,” Photonics Spectra, vol. 35, no. 1, pp. 154–158, 2001. View at: Google Scholar
 B. S. Carlson, “Comparison of modern CCD and CMOS image sensor technologies and systems for low resolution imaging,” in Proceedings of the 1st IEEE International Conference on Sensors (Sensors '02), vol. 1, pp. 171–176, June 2002. View at: Google Scholar
 W. D. LeónSalas, S. Balkir, K. Sayood, N. Schemm, and M. W. Hoffman, “A CMOS imager with focal plane compression using predictive coding,” IEEE Journal of SolidState Circuits, vol. 42, no. 11, pp. 2555–2572, 2007. View at: Publisher Site  Google Scholar
 A. Nilchi, J. Aziz, and R. Genov, “Focalplane algorithmicallymultiplying CMOS computational image sensor,” IEEE Journal of SolidState Circuits, vol. 44, no. 6, pp. 1829–1839, 2009. View at: Publisher Site  Google Scholar
 S. Kawahito, M. Yoshida, M. Sasaki et al., “A CMOS image sensor with analog twodimensional DCTbased compression circuits for onechip cameras,” IEEE Journal of SolidState Circuits, vol. 32, no. 12, pp. 2030–2040, 1997. View at: Google Scholar
 K. Aizawa, H. Ohno, Y. Egi et al., “On sensor image compression,” IEEE Transactions on Circuits and Systems for Video Technology, vol. 7, no. 3, pp. 543–548, 1997. View at: Google Scholar
 Z. Lin, M. W. Hoffman, N. Schemm, W. D. LeonSalas, and S. Balkir, “A CMOS image sensor for multilevel focal plane image decomposition,” IEEE Transactions on Circuits and Systems I, vol. 55, no. 9, pp. 2561–2572, 2008. View at: Publisher Site  Google Scholar
 S. Chen, A. Bermak, and Y. Wang, “A CMOS image sensor with onchip image compression based on predictive boundary adaptation and memoryless QTD algorithm,” IEEE Transactions on Very Large Scale Integration (VLSI) Systems, no. 99, pp. 1–10, 2010. View at: Publisher Site  Google Scholar
 M. Zhang and A. Bermak, “Compressive acquisition CMOS image sensor: from the algorithm to hardware implementation,” IEEE Transactions on Very Large Scale Integration (VLSI) Systems, vol. 18, no. 3, pp. 490–500, 2009. View at: Publisher Site  Google Scholar
 D. A. Huffman, “A method for the construction of minimumredundancy codes,” Proceedings of the IRE, vol. 40, no. 9, pp. 1098–1101, 1952. View at: Google Scholar
 J. Abrahams, “Code and parse trees for lossless source encoding,” in Proceedings of the International Conference on Compression and Complexity of Sequences, pp. 145–171, June 1997. View at: Google Scholar
 P. G. Howard and J. S. Vitter, “Arithmetic coding for data compression,” Proceedings of the IEEE, vol. 82, no. 6, pp. 857–865, 1994. View at: Publisher Site  Google Scholar
 A. Moffat, R. Neal, and I. H. Witten, “Arithmetic coding revisited,” in Proceedings of the 5th Data Compression Conference (DCC '95), pp. 202–211, March 1995. View at: Google Scholar
 A. Lempel and J. Ziv, “On the complexity of finite sequences,” IEEE Transactions on Information Theory, vol. 22, no. 1, pp. 75–81, 1976. View at: Google Scholar
 J. Ziv and A. Lempel, “A universal algorithm for sequential data compression,” IEEE Transactions on Information Theory, vol. 23, no. 3, pp. 337–343, 1977. View at: Google Scholar
 J. Ziv and A. Lempel, “Compression of individual sequences via variablerate coding,” IEEE Transactions on Information Theory, vol. 24, no. 5, pp. 530–536, 1978. View at: Google Scholar
 ISO/IEC JTCl/SC29/WG1 N505, “Call for contributions for JPEG 2000 (JTC 1.29.14, 15444): Image Coding System,” March 1997. View at: Google Scholar
 ISO/IEC JTCl/SC29/WG11 N390R, “New work item: JPEG 2000 image coding system,” March 1997. View at: Google Scholar
 C. Christopoulos, Ed., “JPEG2000 Verification Model 8.0 (technical description),” ISO/IEC JTC1/SC29/WGI N1822, July 2000. View at: Google Scholar
 M. Boliek, C. Christopoulos, and E. Majani , “JPEG2000 Part I Final Drafi International Standard,” (ISO/IEC FDIS 154441), ISO/IEC JTC1/SC29/WG1 N1855, August 2000. View at: Google Scholar
 C. Christopoulos, “The jpeg2000 still image coding system: an overview,” IEEE Transactions on Consumer Electronics, vol. 46, no. 4, pp. 1103–1127, 2000. View at: Google Scholar
 “Video Codec for Audiovisual Services at $p\times 64$ kbit=s ITUT Recommendation H.261, Version 1,” ITUT, ITUT Recommendation H.261 Version 1, 1990. View at: Google Scholar
 “Generic Coding of Moving Pictures and Associated Audio InformationPart 2: Video,” ITUT and ISO/IEC JTC 1, ITUT Recommendation H.262 and ISO/IEC 13 8182 (MPEG2), 1994. View at: Google Scholar
 “Video Coding for Low Bit Rate Communication,” ITUT, ITUT Recommendation H.263 version 1, 1995. View at: Google Scholar
 “Draft ITUT recommendation and final draft international standard of joint video specification (ITUT Rec. H.264/ISO/IEC 14 49610 AVC,” in Joint Video Team (JVT) of ISO/IEC MPEG and ITUT VCEG, JVTG050, 2003. View at: Google Scholar
 ISO/IEC 111722, “Information TechnologyCoding of Moving Pictures and Associated Audio for Digital Storage Media at up to about 1.5 Mbit/s vodep,” Geneva, Switzerland, 1993. View at: Google Scholar
 ISO/IEC 138181, “Information technology—generic coding of moving pictures and associated audio information,” 2004. View at: Google Scholar
 T. Sikora, “The MPEG4 video standard verification model,” IEEE Transactions on Circuits and Systems for Video Technology, vol. 7, no. 1, pp. 19–31, 1997. View at: Google Scholar
 T. Sikora, “MPEG4 very low bit rate video,” in Proceedings of the IEEE International Symposium on Circuits and Systems (ISCAS '97), pp. 1440–1443, June 1997. View at: Google Scholar
 T. Sikora and L. Chiariglione, “MPEG4 video and its potential for future multimedia services,” in Proceedings of the IEEE International Symposium on Circuits and Systems (ISCAS '97), pp. 1468–1471, June 1997. View at: Google Scholar
 A. Aaron, R. Zhang, and B. Girod, “WynerZiv coding of motion video,” in Proceedings of the 36th Asilomar Conference on Signals Systems and Computers, pp. 240–244, November 2002. View at: Google Scholar
 B. Girod, A. M. Aaron, S. Rane, and D. RebolloMonedero, “Distributed video coding,” Proceedings of the IEEE, vol. 93, no. 1, pp. 71–83, 2005. View at: Publisher Site  Google Scholar
 R. Puri, A. Majumdar, and K. Ramchandran, “PRISM: a video coding paradigm with motion estimation at the decoder,” IEEE Transactions on Image Processing, vol. 16, no. 10, pp. 2436–2448, 2007. View at: Publisher Site  Google Scholar
 D. Slepian and J. K. Wolf, “Noiseless coding of correlated information sources,” IEEE Transactions on Information Theory, vol. 19, no. 4, pp. 471–480, 1973. View at: Google Scholar
 A. D. Wyner and J. Ziv, “Ratedistortion function for source coding with side information at the decoder,” IEEE Transactions on Information Theory, vol. 22, no. 1, pp. 1–10, 1976. View at: Google Scholar
 D. León, S. Balkir, K. Sayood, and M. W. Hoffman, “A CMOS imager with pixel prediction for image compression,” in Proceedings of the IEEE International Symposium on Circuits and Systems, vol. 1, pp. 776–779, May 2003. View at: Google Scholar
 W. D. Leon, S. Balkir, K. Sayood, and M. W. Hoffman, “Chargebased prediction circuits for focal plane image compression,” in Proceedings of the IEEE International Symposium on Cirquits and Systems (ISCAS '04), pp. 936–939, May 2004. View at: Google Scholar
 W. D. LeonSalas, S. Balkir, K. Sayood, M. W. Hoffman, and N. Schemm, “A CMOS imager with focal plane compression,” in Proceedings of the IEEE International Symposium on Circuits and Systems (ISCAS '06), pp. 3570–3573, May 2006. View at: Google Scholar
 W. D. LeonSalas, S. Balkir, N. Schemm, M. W. Hoffman, and K. Sayood, “Predictive coding onsensor compression,” in Proceedings of the IEEE International Symposium on Circuits and Systems (ISCAS '08), pp. 1636–1639, May 2008. View at: Publisher Site  Google Scholar
 Q. Luo and J. G. Harris, “A novel integration of onsensor wavelet compression for a CMOS imager,” in Proceedings of the IEEE International Symposium on Circuits and Systems (ISCAS '02), pp. 325–328, May 2002. View at: Google Scholar
 A. Olyaei and R. Genov, “CMOS wavelet compression imager architecture,” in Proceedings of the 7th IEEE Emerging Technologies Workshop: Circuits and Systems for 4G Mobile Communications (ETW '05), pp. 104–107, June 2005. View at: Google Scholar
 A. Olyaei and R. Genov, “Focalplane spatially oversampling CMOS image compression sensor,” IEEE Transactions on Circuits and Systems I, vol. 54, no. 1, pp. 26–34, 2007. View at: Publisher Site  Google Scholar
 A. Bandyopadhyay, J. Lee, R. W. Robucci, and P. Hasler, “MATIA: a programmable 80 μW/frame CMOS block matrix transform imager architecture,” IEEE Journal of SolidState Circuits, vol. 41, no. 3, pp. 663–671, 2006. View at: Publisher Site  Google Scholar
 A. Nilchi, J. Aziz, and R. Genov, “CMOS image compression sensor with algorithmicallymultiplying ADCs,” in Proceedings of the IEEE International Symposium on Circuits and Systems (ISCAS '09), pp. 1497–1500, May 2009. View at: Publisher Site  Google Scholar
 Y. Nishikawa, S. Kawahito, M. Furuta, and T. Tamura, “A highspeed CMOS image sensor with onchip parallel image compression circuits,” in Proceedings of the IEEE Custom Integrated Circuits Conference (CICC '07), pp. 833–836, September 2007. View at: Publisher Site  Google Scholar
 S. Kawahito, M. Yoshida, M. Sasaki et al., “Compressed digital output CMOS image sensor with analog 2D DCT processors and ADC/quantizer,” in Proceedings of the 43rd IEEE International SolidState Circuits Conference. Digest of Technical Papers (ISSCC '97), pp. 184–185, San Francisco, Calif, USA, February 1997. View at: Publisher Site  Google Scholar
 S. Kawahito, Y. Tadokoro, and A. Matsuzawa, “CMOS image sensors with video compression,” in Proceedings of the 3rd Conference of the Asia and South Pacific Design Automation (ASPDAC '98), pp. 595–600, February 1998. View at: Google Scholar
 S. Kawahito, M. Yoshida, M. Sasaki et al., “A CMOS smart image sensor LSI for focalplane compression,” in Proceedings of the 3rd Conference of the Asia and South Pacific Design Automation (ASPDAC '98), pp. 339–340, February 1998. View at: Google Scholar
 D. Kim and E. Culurciello, “A compactpixel trimode vision sensor,” in Proceedings of the IEEE International Symposium on Circuits and Systems (ISCAS '10), pp. 2434–2437, Paris, France, May 2010. View at: Publisher Site  Google Scholar
 T. Kuroda, T. Fujita, S. Mita et al., “A 0.9V, 150MHz, 10mW, 4 ${\text{mm}}^{2}$, 2D discrete cosine transform core processor with variable thresholdvoltage (VT) scheme,” IEEE Journal of SolidState Circuits, vol. 31, no. 11, pp. 1770–1777, 1996. View at: Publisher Site  Google Scholar
 K.Y. Min and J.W. Chong, “A realtime JPEG encoder for 1.3 mega pixel CMOS image sensor SoC,” in Proceedings of the 30th Annual Conference of IEEE Industrial Electronics Society (IECON '04), pp. 2633–2636, November 2004. View at: Google Scholar
 R. P. Kleihorst, A. van der Avoird, A. A. Abbo, M. J. R. Op de Beeck, H. J. L. van Herten, and L. H. M. Sevat, “A lowpower highperformance singlechip digital camera,” Philips Research Laboratories, Eindhoven, The Netherlands. View at: Google Scholar
 J. Y. F. Hsieh, A. van der Avoird, R. P. Kleihorst, and T. H. Y. Meng, “Transpose memory for video rate JPEG compression on highly parallel singlechip digital CMOS imager,” in International Conference on Image Processing (ICIP 2000), pp. 102–105, September 2000. View at: Google Scholar
 M. Schwarzenberg, M. Traeber, M. Scholles, and R. Schueffny, “VLSI chip for wavelet image compression,” in Proceedings of the IEEE International Symposium on Circuits and Systems (ISCAS '99), vol. 4, pp. 271–274, Orlando, Fla, USA, May 1999. View at: Google Scholar
 K. Aizawa, Y. Egi, T. Hamamoto et al., “Computational image sensor for on sensor compression,” IEEE Transactions on Electron Devices, vol. 44, no. 10, pp. 1724–1730, 1997. View at: Google Scholar
 S. Narisawa, K. Masuda, and T. Hamamoto, “High speed digital smart image sensor with image compression function,” in Proceedings of 2004 IEEE AsiaPacific Conference on Advanced System Integrated Circuits, pp. 128–131, August 2004. View at: Google Scholar
 P. Lichtsteiner, C. Posch, and T. Delbruck, “A 128 × 128 120dB 30mW asynchronous vision sensor that responds to relative intensity change,” in Proceedings of the IEEE International SolidState Circuits Conference (ISSCC '06), pp. 491–508, February 2006. View at: Google Scholar
 P. Lichtsteiner, C. Posch, and T. Delbruck, “A 128 × 128 120 dB 15 μs latency asynchronous temporal contrast vision sensor,” IEEE Journal of SolidState Circuits, vol. 43, no. 2, pp. 566–576, 2008. View at: Publisher Site  Google Scholar
 Yu. M. Chi, U. Mallik, M. A. Clapp, E. Choi, G. Cauwenberghs, and R. EtienneCummings, “CMOS camera with inpixel temporal change detection and ADC,” IEEE Journal of SolidState Circuits, vol. 42, no. 10, pp. 2187–2196, 2007. View at: Publisher Site  Google Scholar
 C. Posch, D. Matolin, and R. Wohlgenannt, “A QVGA 143dB dynamic range asynchronous addressevent PWM dynamic image sensor with lossless pixellevel video compression,” in IEEE International SolidState Circuits Conference (ISSCC '10), pp. 400–401, San Francisco, Calif, USA, February 2010. View at: Publisher Site  Google Scholar
 W. D. Leon, S. Balkir, K. Sayood, and M. W. Hoffman, “Chargebased prediction circuits for focal plane image compression,” in Proceedings of the IEEE International Symposium on Cirquits and Systems (ISCAS '04), vol. 4, pp. 936–939, May 2004. View at: Google Scholar
 Z. Lin, M. W. Hoffman, W. D. LeonSalas, N. Schemm, and S. Balkir, “A CMOS image sensor for focal plane decomposition,” in Proceedings of the IEEE International Symposium on Circuits and Systems (ISCAS '05), pp. 5322–5325, May 2005. View at: Publisher Site  Google Scholar
 Z. Lin, M. W. Hoffman, W. D. Leon, N. Schemm, and S. Balkir, “Effects of chargebased computation nonidealities on CMOS image compression sensors,” in Proceedings of the IEEE International Symposium on Circuits and Systems (ISCAS '06), pp. 5187–5190, May 2006. View at: Google Scholar
 Z. Lin, M. W. Hoffman, W. D. Leon, N. Schemm, and S. Balkir, “A CMOS frontend for a lossy image compression sensor,” in Proceedings of the IEEE International Symposium on Circuits and Systems (ISCAS '07), pp. 2838–2841, May 2007. View at: Google Scholar
 Z. Lin, M. W. Hoffman, W. D. Leon, N. Schemm, and S. Balkir, “A CMOS image sensor with focal plane SPIHT image compression,” in Proceedings of the IEEE International Symposium on Circuits and Systems (ISCAS '08), pp. 2134–2137, May 2008. View at: Publisher Site  Google Scholar
 B. Fowler, A. El Gamal, and D. X. D. Yand, “CMOS area image sensor with pixellevel A/D conversion,” in Proceedings of the IEEE International SolidState Circuits Conference (ISSCC '94), pp. 226–227, February 1994. View at: Google Scholar
 S. Kleinfelder, S. Lim, X. Liu, and A. El Gamal, “A 10000 frames/s CMOS digital pixel sensor,” IEEE Journal of SolidState Circuits, vol. 36, no. 12, pp. 2049–2059, 2001. View at: Publisher Site  Google Scholar
 A. Kitchen, A. Bermak, and A. Bouzerdoum, “PWM digital pixel sensor based on asynchronous selfresetting scheme,” IEEE Electron Device Letters, vol. 25, no. 7, pp. 471–473, 2004. View at: Publisher Site  Google Scholar
 A. Kitchen, A. Bermak, and A. Bouzerdoum, “A digital pixel sensor array with programmable dynamic range,” IEEE Transactions on Electron Devices, vol. 52, no. 12, pp. 2591–2601, 2005. View at: Publisher Site  Google Scholar
 J. M. Margarit, J. Sabadell, L. Terés, and F. SerraGraells, “A novel DPS integrator for fast CMOS imagers,” in Proceedings of the IEEE International Symposium on Circuits and Systems (ISCAS '08), pp. 1632–1635, May 2008. View at: Publisher Site  Google Scholar
 S. Chen, A. Bermak, W. Yan, and D. Martinez, “A CMOS image sensor with on chip image compression based on predictive boundary adaptation and QTD algorithm,” in Proceedings of the 6th IEEE Conference on SENSORS (SENSORS '07), pp. 531–534, October 2007. View at: Publisher Site  Google Scholar
 M. Zhang and A. Bermak, “Architecture of a digital pixel sensor array using 1bit hilbert predictive coding,” in Proceedings of the IEEE International Symposium on Circuits and Systems (ISCAS '09), pp. 1501–1504, May 2009. View at: Publisher Site  Google Scholar
 M. Zhang and A. Bermak, “Compressive acquisition CMOS image sensor using online sorting scheme,” in Proceedings of the International SoC Design Conference (ISOCC '09), pp. 169–172, Busan, Korea, November 2009. View at: Publisher Site  Google Scholar
 M. Zhang and A. Bermak, “Digital pixel sensor with online spatial and temporal compression scheme,” in Proceedings of the 12th International Symposium on Integrated Circuits (ISIC '09), pp. 155–158, Singapore, 2009. View at: Google Scholar
 J. Dubois, D. Ginhac, M. Paindavoine, and B. Heyrman, “A 10 000 fps CMOS sensor with massively parallel image processing,” IEEE Journal of SolidState Circuits, vol. 43, no. 3, pp. 706–717, 2008. View at: Publisher Site  Google Scholar
Copyright
Copyright © 2010 Milin Zhang and Amine Bermak. This is an open access article distributed under the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.