MULTIMODAL HYPERSPECTRAL IMAGE UNMIXING USING CROSSED ATTENTION TRANSFORMER

Abstract

The ability of deep learning (DL) to represent characteristics has brought hyperspectral unmixing (HU) to the forefront of technological advancement. In comparison to model-driven linearized approaches, deep learning auto-encoders (AEs) are able to capture nonlinear hyperspectral image components without the need for supervision. Hyperspectral imaging (HSI) is a method that utilizes light to gather and analyze data over the whole electromagnetic (EM) spectrum. It disperses light into many small bands, often spanning from visible to infrared wavelengths, and quantifies the light’s uninterrupted spectrum for every pixel in a given image. HSI is capable of identifying each feature on Earth’s surface by generating a distinct spectral fingerprint for each one. When encountering challenging circumstances, such as when working with end members of materials that are similar to one another, hyperspectral images may prove to be more successful for unmixing. Unmixing multimodal hyperspectral images is accomplished by the utilization of a crossed-attention transformer. The integration of HSI and LiDAR patch projections is accomplished by the utilization of a crossed-attention transformer module. It is simple to transmit information that can be relied upon between modes. When it comes to queries (Q), the technique makes use of LiDAR patch tokens, whereas HS patch tokens are used for keys (K) and values (V). The results obtained from actual datasets demonstrate that the technique that was presented is effective.