Abstract
We propose a novel super-resolution multisource images fusion scheme via compressive sensing and dictionary learning theory. Under the sparsity prior of images patches and the framework of the compressive sensing theory, the multisource images fusion is reduced to a signal recovery problem from the compressive measurements. Then, a set of multiscale dictionaries are learned from several groups of high-resolution sample image’s patches via a nonlinear optimization algorithm. Moreover, a new linear weights fusion rule is proposed to obtain the high-resolution image. Some experiments are taken to investigate the performance of our proposed method, and the results prove its superiority to its counterparts.
1. Introduction
Fusion of multisource images that came from different modalities is very useful for obtaining a better understanding of the environmental conditions, for example, the fusion of multifocus images, the infrared (IR) images and visible images, the medical CT images and MRI images, and the multispectrum images and panchromatic images. Nowadays multiresolution based fusion approaches have been one of the popular techniques that is investigated by many researches and proves to present state-of-the-art result [1–4], including pyramid-based methods and discrete wavelet transform- (DWT-) based methods. In recent years, a new developed compressive sensing (CS) [5–8] theory is introduced into image fusion. It is well known that the compressive sensing theory provides a possible way of recovering sparse signals from their projection onto a small number of random vectors, so compressive sensing indicated a possible way of recovering high-resolution signals from their low-resolution version.
Assume that a signal is compressible under a dictionary , where is the number of nonzero components of . The main idea of CS is to recover the original signal from its compressive measurements , where . Under the condition that the matrix satisfies the restricted isometry property (RIP), the signal can be accurately recovered from only measurements [5], by solving such an optimization problem, Therefore, there are many advantages of combining the CS technique and image fusion application [9–14].
Nowadays the applications of compressive sensing technology into image processing can be classified into three categories: compressive sensing based imaging [15–21], compressive sensing based image processing [22–26], and “compressive sensing” form applications [27–29]. Imaging is one of the most successful applications of compressive sensing theory, where a few sensors or low-resolution sensors are employed to achieve high-resolution imaging, such as optical imaging [16, 17], medical imaging [18, 19], and hyperspectral imaging [20, 21]. Compressive sensing is also used to transform images to other spaces to obtain more efficient analysis, such as the texture classification [22] and super-resolution image construction [23]. Numerous works are of the “compressive sensing” form applications; that is, if the task can be reduced to the optimization problem shown in (1), these works are also called compressive sensing based applications.
In image fusion, most of the available compressive sensing based fusion schemes are of “compressive sensing” form [9–14, 27, 28]; that is, they did not consider the simultaneous fusion and super-resolution of multisource images. In this paper, we indicate another solution for simultaneous fusion and super-resolution of multisource images via the recent developed compressive sampling theory. Under the sparsity prior of images patches and the framework of the compressive sensing theory, the multisource images fusion is reduced to a signal recovery problem from the compressive measurements. A set of multiscale dictionaries are learned from some groups of high-resolution sample image’s patches via a nonlinear optimization algorithm. Moreover, a new linear weights fusion rule is proposed. Some experiments are taken to investigate the performance of our proposed method, and the results prove its superiority to its counterparts.
The rest of this paper is organized as follows. Our proposed simultaneous fusion and super-resolution scheme of multisource images is expounded in Section 2. In Section 3, some experiments are made to compare the proposed method with other related segmentation approaches. The conclusions are finally summarized in Section 4.
2. Simultaneous Fusion and Super-Resolution Scheme of Multisource Images
In this section, the foundations of our proposed method are illustrated, including the super-resolution multisource images fusion, the super-resolution multisource images fusion via compressive sensing, and the dictionary learning algorithm used in our approach.
2.1. Super-Resolution Multisource Images Fusion
Assume that the multisource images to be fused are low-resolution images; that is, the th source images are a low-resolution version of : where is the number of source images, is the down-resolution operator, and is the measurement noise of the th source image. We aim to recover a high-resolution image from the multisource low-resolution images .
The patches based fusion is adopted in our method; that is, is processed in raster-scan order, from left to right and top to bottom, and then sequentially recovered. Let denote the th local patch vector extracted from a high-resolution fusion image at the spatial location , where denotes a rectangular windowing operator and the overlapping is allowed. Given a set of () LR patches taken from (), , we have A simple example of the matrix is as follows: Our aim is to reconstruct the fusion image from the high-resolution patches () from a set of the corresponding () HR patches (). This is a simultaneous fusion and super-resolution problem of multisource images.
2.2. Super-Resolution Multisource Images Fusion via Compressive Sensing
According to the recent developed compressive sampling theory [5, 6], it is capable of recovering (; ) from under the sparsity prior of ; that is, can be represented as a sparse linear combination by an overcomplete dictionary that is not coherent with the measurement (or sampling) matrix ; that is, Here the “sparsity” of the decomposition coefficient means , and is the number of elements (or atoms) in the dictionary . Under this sparsity assumption, can thus be reconstructed by taking only measurements. As soon as the sparse coefficient is determined by estimation of can be obtained using (5).
In our method, a linear fusion rule is performed on the with each . Considering the patch by patch processing pattern, we write the low-resolution fusion patch as Therefore, its high-resolution version can be written as In our proposed method, we determine the weights according to the following formula: Because patches are highly redundant and the recovery of from becomes an overdetermined system, it is straightforward to obtain the following least-square solution in the patch aggregation:
2.3. Dictionary Learning Algorithm
The compressibility of patches shown in (5) is the sparsity prior used in our method. In order to generate several overcomplete dictionaries () that can represent well the underlying HR patches, we propose an algorithm to adaptively tune the dictionary from a set of High-Resolution multisource sample image’s patches. In this section, we will reduce the learning of dictionary () as another sparsity-oriented optimization problem. Recent research on image statistics suggests that image patches can be well represented as a sparse linear combination of elements from an appropriately chosen overcomplete dictionary. Under this assumption, the HR image patches set () sampled from some training HR images can be represented as a sparse linear combination in a dictionary () (); that is, with the sparse coefficient vectors and . The objective of designing is to make the reconstruction error over () be minimal under the sparsity assumption; that is, We reformulate (12) as follows: where is the coefficients matrix. In order to solve it, the KSVD dictionary learning algorithm is used to train the dictionaries () from () [30, 31].
3. Experiment Results
For evaluating the performance of the proposed fusion algorithm, in this section we have implemented them on some multisource images, including the multifocus images, infrared (IR) images, and visual images, as shown in Figure 1. The size of all images used in the test is 256 lines × 256 columns and we aim to recover the 512 lines × 512 columns images. We compare our method with the following two related methods.
(a)
(b)
(c)
(d)
(e)
(f)
(g)
(h)
Method (1). Consider the multimodal image fusion with joint sparsity model [32].
Method (2). Consider the image features extraction and fusion based on joint sparse representation [33].
For evaluating the performance of the proposed algorithm, the computed results are compared by visual quality subjectively and by some guidelines in fusion. The simulations are conducted in MATLAB R2009 on PC with Intel Core 2/1.8 G/1 G.
The fusion results of three methods are shown in Figures 2, 3, 4, and 5, and from left to right are Method 1, our method, and Method 2. From them we can see that the fusion images of our method can get higher-resolution images at the same time of fusing the multisource images. Compared with the rules in [32, 33], our proposed method has better preservation of directional information. The numerical guidelines are shown in Table 1, where some measures including the entropy , mutual information (MI), average gradient (AG), standard deviation (SD), cross entropy , universal image quality index (UIQI) [34], and correlation coefficient (CC) are calculated from the fusion images derived by different methods.
Here UIQI is used to estimate the subjective vision effect, which combined the spatial correlation, wrap of mean, and variance together, and it can embody the comparability between the fused image and original images. It is defined as where is the covariance of the fused source images and and and are the standard variance and the mean of the image , respectively. From it we can see that the numerical result accords with the subjective result.
4. Conclusions
In this paper we propose a novel super-resolution multisource images fusion scheme based on compressive sensing and dictionary learning. Under the sparsity prior of images patches and the framework of the compressive sensing theory, the multisource images fusion is reduced to a signal recovery problem from the compressive measurements. A new linear weights fusion rule is proposed. A set of multiscale dictionaries are learned from several groups of high-resolution (HR) sample image’s patches, and a higher resolution fusion image can be obtained from multisource images. Some experiments are taken and the results prove its superiority to its counterparts.
Conflict of Interests
The authors declare that there is no conflict of interests regarding the publication of this paper.
Acknowledgment
This project is supported by the Natural Science Foundation of Jiangsu Province of China, under Grant no. BK20130769.