INFRARED SIGNAL PROCESSING ALGORITHMS AND ARCHITECTURES FOR THERMAL IMAGING SYSTEMS

Abstract

Thermal imaging systems are used for various applications extending from surveillance and reconnaissance to long range target acquisition, engagement and missile guidance and are truly passive devices. These systems are true force multiplier as they allow weapons and equipments to be used with efficacy during day and night, in fair and bad weather and belong to one of the class of night vision sensors. Two types of systems based upon image intensification and thermal imaging systems are currently in service. Image intensification systems amplify the available residual light from moon and stars, which is reflected by the scene being viewed. However, these devices cannot offer ranges exceeding 1 Km and their performance is degraded in complete darkness and poor atmospheric conditions. Thermal imaging systems make use of the temperature and emissivity differences of the targets with respect to the background for the generation of real time picture. These systems provide night vision capability in completes darkness and are partially affected by smoke dust or haze. These systems can detect objects even through camouflage nets and are not blinded by searchlights, flares and fires. It is generally agreed that most of the next generation infrared systems require large format infrared focal plane arrays (IRFPA) operating in 3-5um, 8-12um and in dual spectral bands covering both 3-5 \xm and 8-12 urn wavelength regions. The various reasons for such developments are higher performance, suppression of clutter and increase of the target contrast, which lead to longer detection and recognition ranges, improved resistance to countermeasures such as decoys and camouflage, and improved target acquisition. The signal and image processing, in or behind the focal plane is as essential to high performance systems as the optics and the focal plane array sensor. State of the art optics and focal plane arrays (FPAs) will not give us the next generation infrared sensor systems unless these technologies are coupled with near noise less signal conditioners and readout circuits followed by a sophisticated signal processor's target acquisition capabilities. Thus the development of new and high performance sensors also u call for the development of newer signal processing algorithms and architectures for improved target detection and engagement capabilities and the performance of the system is often dictated by the performance of the signal processing. The IRFPA technology has advanced immensely in recent years, resulting in design and development of IRFPA having large number of pixels at the focal plane with smaller pitch and better noise equivalent temperature difference (NETD), thus improving the performance of infrared imaging system significantly. However, it results in several problems and most serious of them is sensor non-uniformities. Non-uniformities are mainly attributed to the difference in the photo-response of each detector in the FPA. Other degrading factors such as FPA temperature, finite lens aperture, finite photosensitivity of the detector and under sampling add to these non-uniformities. These spatial and temporal non-uniformities result in degrading the temperature resolving capabilities of infrared system considerably. In this thesis we have described different algorithms for correcting sensor non-uniformities. A new approach of correcting the sensor non-uniformities based on variation of integration time of the FPA has been described, analyzed and implemented. The performance of the algorithm is evaluated by defining performance parameter residual non-uniformity (RNU). Results have demonstrated that this method reduces the spatial noise to less than 0.6%, which is significantly lower than the temporal noise. Hardware architecture for implementing the proposed algorithm in real time is described. Three scene based non-uniformity correction algorithms are also described in the thesis. The first scene based algorithm uses the image statistics for calibration. It is based upon the assumption that scene parameters such as mean and variance of the pixels do not vary from one frame to another. This algorithm corrects both, additive and multiplicative types of fixed patter noise. The second scene-based technique uses Kalman filter for non-uniformity correction. Gain and offset non-uniformities associated with IRFPA are treated as random variables and drift in gain and offset associated with the detector pixel element is assumed to be negligible from one frame to another. A method to initialize the Kalman filter based using scene information is described. Third technique 111 is based upon adaptive filter for correcting sensor non-uniformities. This algorithm uses local parameters such as local mean and local variance of a pixel to compute the expected value of the scene at each pixel location. Scene-based algorithms are applied on infrared images with simulated non-uniformities and real infrared images with non-uniformities. The performances of these algorithms are evaluated by defining the performance evaluation parameter root mean square error. Results obtained have demonstrated that proposed methods could reduce the fixed pattern noise to a level that is almost free from the severity of the non-uniformity. Hardware architectures for implementing these algorithms in FPGA are described and tested with 320x240 element InSb based infrared imaging system. Thesis also deals with the design, analysis and implementation of infrared image processing algorithms for image enhancement and target detection. Seven different algorithms are described for image enhancement and an algorithm based upon morphological operations for target detection described. These algorithms are optimized to give enhanced output for maximum types of the infrared images and ensure real time implementation on a FPGA based embedded system. Three Three-performance parameters namely image contrast, image quality index (Q) and peak signal to noise ratio (PSNR) are described to evaluate the performance of these algorithms. Hardware designs for implementing these image-processing algorithms have been carried out based upon FPGA based embedded system. Architectures of theses algorithms are optimized to meet the real time constraints as well as obtain the desired performance.