Multi-modal Sentiment Analysis

Abstract

Sentiment analysis is a rising field in Natural Language Processing. Finding people’s ideas has a significant advantage over economics as well as business. Mainly, it helps to improve a product or service quality provided to the people. In general, the words uttered by people help to predict sentiment, but in few cases, words alone can’t be much helpful, e.g., when you are making sarcasm. So to overcome this problem, we need more information about the speakers’ talking style and facial expressions while uttering the words. Knowing these things helps to learn about people’s opinions on some specific topic. Using all the information together to predict sentiment makes multi-modal sentiment analysis an exciting field. Unlike unimodal, when modalities combined in multi-modal give more details about the context and speaker’s emotional state. Connecting all the modalities is a pretty challenging task. The system perfor mance depends on how and when the modalities get combined. Joining the modalities plays a vital role because combining very early may increase the system’s noise by degrading the performance, and connecting at the very end, cause them to lose some semantics. So connecting should happen in a way it increases overall system perfor mance. The proposed model uses popular sentiment analysis benchmarks, Youtube, MOSI, and MOSEI, demonstrating significant gains over state-of-the-art models.