TARGET LOCALIZATION WITH VIDEO STABILIZATION USING VISUAL FEATURE TRACKING

Abstract

Visual surveillance is one very important requirement for present and future computer vision technologies. Video may be recorded using a digital camera which may be handheld or mounted on some moving platform of a vehicle. Consequently, the recorded video is generally unstabilized on account of unwanted and unintentional jerks and motion of the camera. This unwanted motion may be due to shaking of hand, vehicle running on irregular terrain or due to vehicle engine vibrations. Such unstabilized and shaky videos are of poor quality, noisy, motion blurred and neither suitable for viewing nor for post processing like target detection and tracking. Target detection and tracking is one prime research problem in computer vision area due to its increasing demand in many visual applications ranging from dynamic motion based detection, face detection, threat detection, automatic visual surveillance, indexing of video, crowd management, behavioral analysis, automatic vehicle drive, military search and track of potential threats for integrated fire control solutions. This problem of target tracking poses challenge due to scene illumination variations, non-rigid target body, frequent pose variation of target, occlusion, information loss due to conversion of scene from 3D to 2D, image noise, complex target motion, different shapes of target and so on. In order to enable the use of unstabilized and shaky videos for surveillance and other purposes, this thesis addresses two major issues: (1) Video stabilization, and (2) localization of targets of interests in the video scene. Video stabilization is a necessary requirement for creating stable video footage that can be used reliably for visual communication, smooth high quality visuals and for any post processing of video. There are three types of video stabilization techniques available in the literature, namely, electronic image stabilization (uses sensors and actuators hardware), optical image stabilization (uses moving optical components) and digital image stabilization (uses image processing techniques for detection and compensation of global frame motion vector). In this thesis, we propose to use feature tracking based image/video processing technique to meet this requirement. Feature tracking is used to estimate the frame-to-frame motion in the video. The unwanted motion is then estimated and removed from the video to display the video footage as if the unwanted motion had not occurred. The resulting stabilized video enables the user to easily detect, track and locate targets in the video scene. Target localization involves estimating the world position of an identified target while visual tracking may be defined as estimating the frame-to-frame trajectory of the object of interest in a video scene. There are two main components in visual tracking, viz. detection of target (by extracting target features) and tracking of target (by estimating local motion vector of target). Visual tracking has always been a challenging research problem i among the researchers in the field of computer vision. Development of algorithm for visual tracking involves target representation (point, geometrical shapes, silhouette, articulated, contour or skeletal model), target appearance (probability density, templates, active or multi-view appearance models), selection of features for visual tracking (color, edge, optical flow or texture), target detection (point detector, subtraction of background, segmentation, mean-shift clustering, graph cut method, active contours, supervised learning (adaptive boosting, support vector machines) and finally visual tracking (point, kernel or silhouette). In this thesis, several different visual features, such as Gabor wavelet features, Speeded Up Robust Feature (SURF), etc. are used for precise detection and localization of targets while Kalman filtering for tracking these targets. Real time operation of visual object trackers is indispensable for almost all of its applications. Since object detection and tracking algorithms are generally computationintensive, this thesis also proposes hardware realization of the proposed techniques for video stabilization and target localization. Further, option for incorporating hot swappable redundancy in the system is also considered in this thesis so as to improve the reliability of the system. This thesis begins with an introduction to the problem of video stabilization and target localization in Chapter 1 followed by a discussion on the state-of-the-art in the development of respective technology in video stabilization and target localization in Chapter 2. Our proposed methods for digital video stabilization are presented in Chapter 3. Chapter 4 discusses our proposed target detection and tracking algorithms. Methods for target localization in unstabilized videos, with simultaneous video stabilization are developed in Chapter 5. Hardware implementation of the above proposed methods for video stabilization and target localization is proposed in Chapter 6. The conclusions drawn from this work and a discussion on possible future work are finally presented in Chapter 7. In summary, this thesis contributes in the development of algorithms for digital video stabilization and target localization. Hardware development for electronics image stabilization is also proposed. The thesis is well supported with promising results and comparative analysis with benchmarked algorithms proving the effectiveness of the algorithms.