A STUDY OF IMAGE FUSION TECHNIQUES FOR MULTISENSOR SATELLITE DATASET

Abstract

This study intends to provide an exhaustive treatise on various image fusion techniques; analysis and testing using different satellite remote sensing imagery. In remote sensing applications, the increasing availability of space borne sensors gives a motivation for the development of different image fusion techniques. Several situations in image processing require high spatial and spectral resolution in a single image and most of the available equipment is not capable of providing such data convincingly. Image fusion techniques allow for the integration of different information sources such that the fused image can have complementary spatial and spectral resolution characteristics. The objective of image fusion in remote sensing is to combine the geometric detail of the panchromatic image and the color information of the multi-spectral image to produce a high-resolution multi-spectral image suitable for mapping, information extraction, classification, feature enhancement, land–cover classifications, land resources, urban scenario and environment monitoring. One of the most challenging applications in satellite image fusion is to fuse PAN and MS images acquired from different satellite sensor without introducing inconsistencies or artifacts, which may tamper the attribute of the fused image. Based on the advantages and limitations of the existing fusion techniques, studies have been carried out made either to enhance the performances of the existing techniques or to provide new solutions to the image fusion problem i.e. spectral distortion. In order to resolve the colour distortion problem, techniques based on spatial-frequency domain have been introduced and have demonstrated superior performance in terms of producing high spectral fidelity pan-sharpened images. In other words, better approaches of pan-sharpening based on spatial-frequency domain have been proposed in order to reduce the spectral distortion compared to the distortion produced by existing fusion techniques based on colour based, statistical based and multi-scale transform based domain. Further, the goal of this study is to produce pan-sharpened images with high spectral fidelity possible as there is importance of such images in various applications, ranging from land use/land cover classification to road extraction. ii In this study, three new approaches have been proposed for the fusion of PAN and MS images, in order to reduce the problem of spectral distortion comparatively, which is prominent in existing fusion techniques. The first proposed approach is to use, Pseudo Wigner Distribution (PWD) to fuse PAN and MS image and compare the same with other fusion techniques belonging to colour based, statistical based and multi-scale transform based methods. The second proposed approach is used to introduce a HSI and Pseudo Wigner Distribution (HSI-PWD) integrated fusion approach in order to reduce the spectral distortion problem associated with HSI fusion method and compare with the existing HSI based hybrid fusion techniques, such as, HSIDWT, HSI-SWT and HSI-NSCT. The third proposed approach is to use Principal Component Analysis (PCA) along with Pseudo Wigner Distribution (PCA-PWD) to reduce the spectral distortion problem, which is prominent in PCA fusion technique and compare the same with the existing PCA based hybrid fusion techniques, such as, PCA-DWT, PCA-SWT and PCA-NSCT. Furthermore, this study also proposes the utility of new quantitative metric in image fusion i.e. Tamper Assessment Factor (TAF), to measure the quality of the fused image. The accuracy of pan-sharpening techniques has been evaluated, both qualitatively and quantitatively. In qualitative evaluation, the fused images are compared to the original PAN and MS images visually. The amount of color distortion and enhancement of spatial detail in the fused images is examined by the users. This is done by evaluating the fused images in terms of some visual metrics such as sharpness, existence of noisy areas, missing spatial detail, and distortions in the geometry of the objects, such as buildings, surfaces, roads, etc. In the quantitative evaluation, the spectral and spatial distortions occurred during the fusion are determined using quantitative evaluation metrics. In this study, nine quality metrics, Root Mean Square Error (RMSE), Peak Signal-to-Noise Ratio (PSNR), Correlation Coefficient (CC), Relative Dimensionless Global Synthesis Error (ERGAS), Relative Average Spectral Error (RASE), Universal Image Quality Index (UIQI), Mean Structural Similarity Index (MSSIM), Spatial Correlation Coefficient (SCC) and Tamper Assessment Factor are used. The quality metrics such as, RMSE, PSNR, CC, UIQI, MSSIM, SCC, are more suitable for band-wise analysis, while, ERGAS, RASE and TAF are more suitable for estimating the global spectral quality of the fused images. The objective metrics belonging to spectral, structural similarity and iii spatial category has been selected in order to provide spatial, spectral, and structural similarity information of the fused image. In order to meet the requirement of objectives defined, as well as, to overcome the limitation of existing available readymade software packages, a software package has been developed named as ‘Image Fusion and Accuracy Assessment (IFAS)’ software package. IFAS has been developed with a Graphical User Interface developed in MATLAB environment. The competence, strength and potential of IFAS is that it takes into account both existing and latest developed pan-sharpening techniques under one platform. Further, the pan-sharpening techniques which are incorporated in the software package are Colour based, Statistical based, Multi-scale Transform based, Spatial-frequency based and Hybrid based pan-sharpening techniques, for the fusion of PAN and MS satellite images, along with its assessment of accuracy. Further, the proposed fusion techniques has been compared with the existing fusion techniques in terms of different fusion rules, computational time, qualitative and quantitative parameters, using World View-2 (PAN and MS) and IKONOS (PAN and MS), CARTOSAT-1 (PAN) and LANDSAT-8 (MS) data. The analysis in this research work also highlight the importance of shift-invariant property in image fusion i.e. to investigate the performance of shift-variant and shift-invariant fusion techniques in context of satellite image fusion. The analysis of result shows that the irrespective of the datasets used, the proposed fusion technique i.e. PWD emerges as one of the most effective fusion technique by minimizing the trade-off between spatial and spectral fidelity, along with the balanced computational time i.e. preservation of edge (spatial details) and preservation of spectral information (color details) are achievable at the cost of slight increment in the computational time. Further, the statistical analysis shows that the proposed fusion techniques using Average and Maximum Fusion Rule provides better fusion results in terms of well-known global evaluation indices and is best in preserving the spectral information of the MS image, when compared to the existing fusion techniques using the same fusion rules such as, DWT, SWT and NSCT. Amongst the different fusion rules, Average Fusion Rule produces satisfactory result in terms of spectral quality, while, iv the Maximum Fusion Rule produces satisfactory results in terms of spatial quality. Further, the fusion techniques using Average Fusion Rule outperforms Maximum Fusion Rule in terms of computational time i.e. techniques using Average Fusion Rule takes less time to perform fusion. The proposed hybrid techniques i.e. HSI-PWD and PCA-PWD based on spatialfrequency domain using Average and Maximum Fusion Rule emerged as one of the most effective fusion technique to overcome the problem of spectral distortion associated with HSI and PCA techniques. It produces better results in terms of well known metrics, when compared to HSI or PCA hybrid techniques based on MST technique i.e. HSI-DWT, HSI-SWT, HSINSCT, PCA-DWT, PCA-SWT and PCA-NSCT. Combining the results of qualitative and quantitative evaluation performances, HSI-PWD and PCA-PWD based hybrid fusion techniques using different set of fusion rule yields the highest performance amongst all the existing hybrid techniques. The shift-invariant techniques such as, SWT, NSCT and PWD, using different set of fusion rules (Avg and Max) yields better results in term of well known indices when compared to shift-variant technique, such as, DWT technique using the same fusion rules. This may be due to the involvement of sub-sampling process in DWT method, which cause artifacts in the resulting fused image. Therefore, it can be inferred that the performance of image fusion is greatly affected by the shift-variance property. Thus, it can be concluded from this study is that the proposed approaches based on spatial-frequency domain i.e. PWD, HSI-PWD and PCA-PWD may be considered a general approach to enhance the spatial resolution of MS images, without sacrificing the spectral content of MS images. In other words, analysis of non-stationary image can be analyzed efficiently by using PWD, HSI-PWD and PCA-PWD fusion techniques for the fusion of World View-2, IKONOS, LANDSAT-8 and CARTOSAT-1 satellite images, when compared to other techniques based on colour, statistical and multi-scale transform domain