PREDICTION BASED SEAM CARVING FOR VIDEO RETARGETING

Abstract

Image Retargeting is content aware image resizing which takes into account the resizing of image according to the resolution of display screen but without distorting salient content of image and without the loss of important information. This is challenging task as important areas of image must be preserved to maintain its aesthetics while resizing it.Image resizing techniques like scaling and cropping does not produce adequate results as scaling scales the image equally from all directions distorting the fineness of objects and cropping may remove important information from image. Similar to image retargeting is the video retargeting where video is transformed to fit in an arbitrary display while considering the saliency of frames in video. As the video is sequence of frames inducing motion and human eye is more sensitive to movements. So in addition to saliency, retargeting should consider the temporal coherency as well where pixels are removed from almost similar position in subsequent frames so that similar kind of data is removed from each frame to maintain smoothness in videos. Temporal coherency is essential in video so that frames are aligned in similar way as if they were in original video. Hence video retargeting aims to have utmost balance between spatial coherency and temporal coherency. Various algorithms are proposed on warping based techniques, patch based techniques, seam based technique to retarget videos. Each has its own advantages and disadvantages. We presenta prediction based spatio-temporal seam carving scheme for video retargeting. Itresizes the video maintaining appropriate balance between spatial and temporal coherence. In a video frame, the proposed approach finds a‘temporal’ seam by using Kalman filter estimation and then modifies it with the help of ‘spatial’ seam considering both spatial and I vi temporal coherency. Unlike image retargeting, it is of utmost importance in retargeting a video frame to consider temporal coherency along with spatial coherency to remove or replicate unimportant background portion. This will ensure that insignificant amount of motion artifacts are introduced during resizing. The proposed Kalman filter based approach not only predicts a spatio-temporal seam to mark a portion of the frame where there is more possibility of having spatially and temporally coherent seam, but also has low time complexity. The proposed approach outperforms other state-of-the-art video retargeting methods which are illustrated by experimental results.