Topic Modelling and Classification of Farmers Queries

Abstract

In today’s world, it is crucial to find new ways to obtain accurate information about agricul tural issues. Agricultural policymakers heavily depend on expert systems to identify issues and seek out potential solutions. Nevertheless, there is currently a lack of a comprehensive system capable of gathering and analyzing vast amounts of data pertaining to the challenges encountered by farmers in developing nations. Therefore, there is an urgent requirement for efficient tools that can categorize and promptly address farmers’ queries. This thesis report investigates the use of BERTopic modeling for topic modeling of agri cultural datasets containing farmers queries. The growing amount of data generated in the agriculture sector has created a need for effective tools to analyze the data and extract mean ingful insights. Topic modeling is a widely used technique for identifying topics and their underlying themes in large datasets. This study proposes the use of BERTopic, a state-of-the art topic modelling algorithm based on the transformer architecture, to analyze an agricul tural dataset from multiple regions of India, containing queries related to plant protection, weather, fertilizer use and availability etc. The dataset is preprocessed to remove stop words and other irrelevant information, and BERTopic is applied to identify the most important topics and their associated keywords. The results show that BERTopic has the ability to identify relevant topics. Several queries related to plant protection, weather, fertilizer use and availability etc. are identified. These insights can aid policymakers and farmers in making informed decisions related to crop pro duction and management. We then experimented with several machine learning algorithms, including Support ML Algorithms such as LinearSVCn(Support Vector Classification), Random Forests (RF), and Multinomial Naive Bayes (MNB), to classify the queries into their respective classes. We evaluated the performance of these algorithms using standard metrics such as recall, preci sion, and accuracy.