USING MORPHABLE FACE MODEL TO IMPROVE STEREO RECONSTRUCTION AND VISUALISING THE MODEL ON A SMARTPHONE

Abstract

Human faces are similar in global properties, including location of main features, size, aspect ratio, but can vary considerably in details across individuals gender, race or facial expression. Due to the loss of one dimension in the image acquisition process, the retrieval of the true 3D geometry is difficult and a so called ill-posed problem, 3D reconstruction from images. Stereo Reconstruction from image pair is a standard method for 3D acquisition of human faces. Depending on available imagery and accuracy requirements the resulting 3D stereo reconstructions may have deficits. The stereo surface reconstruction has lots of holes, and limited texture information. In this work we remedy such deficits combining the 3D stereo reconstruction with a generic Morphable Face Model. The 3D morphable face model has smooth shape information which can be modified specific to a face image. For the improved reconstruction, prior shape information can be obtained by already developed methods, which uses landmarks to fit a morphable model to a single image. A Major part of the thesis is devoted to improvement in stereo face reconstruction pipeline by allowing to prefer information from the single image reconstruction whenever the stereo reconstruction shows untypical deviations from the expected 3D features of a human face. From a pair of stereo images, one is used for single image reconstruction and the combination gives the stereo model. The two reconstruction are then combined to result in the deformed face model. The fusion of models is conducted in a global and a local transformation stages. To include high frequency color information in the model, texture is extracted from the face image. A comparison of the output with high quality face scan is also presented. The fusion outcome results in more accurate face reconstruction than the either of the two. Finally, the resultant deformed face model is visually presented on a smartphone using cardboard, which addresses the modern trend of low cost devices in virtual 3D visualization