OPENSOURCEINTELLIGENCEUSINGNATURAL LANGUAGEPROCESSINGANDMACHINE LEARNING

Abstract

Open Source Intelligence (OSInt) is the collection, collation, corroboration, and analysis of publicly available data sources over the internet to generate actionable intel ligence. The entire landscape of OS Int comprises data collection from various specific areas of interest using dedicated tools and technological solutions. Intelligence searches are generally very sensitive in nature from a security point of view, so we decided to use a neutral but equally effective topic of #BoycottBollywood for this project. In this project, we have conducted an intelligence-based study and investigation using Open Source data(Twitter and web search) and analysed it using Natural Language Process ing (NLP) and Machine Learning(ML). The thorough analysis of # Boycott Bollywood trend over the internet in the year 2022-23 was conducted successfully by carrying out its clustering, sentiment analysis, active user profiling, and predictive analysis. The re search results highlight the effects of this trend on the Bollywood industry. Information overload is a genuine concern in this scenario, wherein an enormous amount of data is available over the internet. Overall, generating credible OS int, which bears some results, requires considerable resources and efforts in terms of technical solutions and human resources to operate those technical assets, carry out analysis, and generate some actionable intelligence. Automating this process using emerging technologies like Nat ural Language Processing (NLP) and Machine Learning(ML) to mitigate the enormous efforts and time lag due to manual processing will pave the way for advancement in credible intelligence generation. This work intends to create a digital platform of OSInt generation using NLP and ML, whichto date, was done manually. The use of this estab lished platform by various intelligence agencies can be explored. It will help an analyst filter out noise from relevant data, track similar data from multiple sources (cluster ing) for corroboration, annotate and time-stamp information collected, and tag/classify collected data for further dissemination.