Abstract:
The aim is to address the problem of exploiting multiple sources of
information for object classi cation tasks when additional modali-
ties that are present in the labeled training set are not available for
inference. Considering the practicality of RGB-D object classi er, a
modality hallucination architecture using multi-modal ConvNets has
been proposed to incorporate depth information at training time.
The modality hallucination network is trained to mimic mid-level
features of depth images and learns a new RGB image representa-
tion . The single modality RGB test image is jointly processed using
hallucination and RGB network and it outperforms the RGB model.
As the deep networks based object classi ers require prohibitive run-
times to process images for real world applications, knowledge distil-
lation framework has been proposed for the modality hallucination
architecture with improved accuracy
