NEW FEATURE DESCRIPTORS FOR IMAGE RETRIEVAL, OBJECT TRACKING AND SHOT DETECTION

Abstract

Image retrieval has been a popular research area due to extensive online and o ine image database. Content based image retrieval (CBIR) has served well in the areas of education, multimedia, medical diagnosis, art collections, scienti c databases, etc. Feature extraction and similarity detection are measure aspects of a CBIR system. Similarly, object tracking and shot boundary detection are the standard computer vision applications which required pro cient feature extraction methods. This research work develops and integrates feature extraction methods for CBIR, object tracking and shot boundary detection applications. Application of chapter 2 to 6 is content based image retrieval systems for di erent databases, chapter 7 targets an object tracking problem and nally a shot boundary detection problem is solved in chapter 8. Chapter 2, proposes two techniques using discrete wavelet transform and local feature descriptors. Local patterns utilize the neighboring pixels to get the local information of the image. Discrete wavelet transform (DWT) is rst applied to acquire the subband images and then direction based local patterns, local extrema pattern (LEP) and directional local extrema pattern (DLEP) are used to extract local directional information of DWT subband images. Both the patterns work in four speci c directions. In rst method, LEP is uniformly applied to all the subband images. Moreover, in second method, based on the direction information of the wavelet coe cient, corresponding DLEP is applied. Wavelet has proved its directional information signi cance and hence it helps LEP and DLEP to create more orientated features. i In Chapter 3 and 4, local information is extracted using local patterns and that information further organized in a feature vector using co-occurrence of pixel pairs in pattern map. Most of the local pattern that have been proposed by researchers, used only occurrence of each pattern value in the pattern map. Besides, in this work, pixels are analyzed in occurrence of pattern value pairs and on the basis of occurrence values corresponding feature vectors are formed. In Chapter 2, HSV color space is used for extracting color information using histograms of hue and saturation components and LEP is extracted from value component. Further, to extract co-occurrence information, gray level co-occurrence matrix (GLCM) is derived from LEP map. In Chapter 4, co-occurrence matrix is utilized in di erent directions and distances to obtain more local directional information. In this chapter, center symmetric local binary pattern (CSLBP) are employed to acquire the local information and GLCM of 0 , 45 , 90 and 135 orientation and one and two distances are applied to CSLBP map. Di erent combinations are analyzed for performance in CBIR application and results are projected accordingly. Two novel local patterns are proposed based on pixel directions and mutual relationship of neighboring pixels in chapter 5 and 6. Local tri-directional pattern (LTriDP) for texture features is proposed in chapter 5. It extracts information of each neighboring pixel related to a center pixel in three speci c directions. On the basis of thresholding of neighboring pixel with other three neighboring pixel, a ternary pattern (0, 1 or 2) is assigned to corresponding pixel. Also, one magnitude pattern is extracted using same pixel. Both patterns are combined and called local tri-directional pattern and used as a feature descriptor of CBIR system. In chapter 6, local neighboring di erence pattern (LNDP) is proposed which deals with mutual relationship of neighboring pixels. Relationship of each neighboring pixel is calculated with two other adjacent neighboring pixels and pattern map is created. In feature extraction, LNDP is combined with LBP as they are compliment with each other since LBP extracts the information regarding center and neighboring pixel relationship and LNDP extracts mutual relationship of neighboring pixels. Combined feature is applied to textural and natural image database for image retrieval. Chapter 7 and 8 are based on video problems of object tracking and shot boundary detection. A new texture feature is proposed called local rhombus pattern and ii combined with HSV color histograms in chapter 7. Local rhombus pattern creates a local patterns using four neighboring pixels of each center pixel in image. Feature extraction is performed using color and texture information of objects in the video and mean shift tracking algorithm is used for tracking the object. In chpater 8, a hierarchical approach is applied to extract shot boundaries. Two step approach is implemented using RGB color histogram and local binary pattern (LBP). Hierarchical method using global and local features helped in reducing the extra number of keyframes from repeated shots in video sequence.