IMAGE CAPTIONING USING NEURAL NETWORKS

Abstract

Image captioning in the general sense, is the process of generating a concise and clear description of an input image. For automatic image captioning there are some challenges in how to extract visual information from the provided image and how to transform this information into a proper meaningful text in a natural language. We can summarise the task of Automatic Image Caption generation as follows: Given an Image automatically generate a caption of the image describing the objects in the image and their relationships in Natural Language ❖ Detect Objects in the scene. ❖ Detect Relationship between the objects. ❖ Describe the information using Natural Language. All these phases are carried out sequentially to form meaningful captions from the image. First we need to detect objects in the scene so that their relationship can be understood, the number of objects and the depth of the relationship captured both create an impact on the end result i.e. the generated captions and depend on various factors such as the model chosen, type of attention etc. Although being able to automatically describe the contents of an image in a natural language can be a very challenging task, it also has a high pay-off as we will see ahead.