ENHANCING FAKE-NEWS DETECTION WITH ADAPTIVE RATIONAL GUIDANCE USING LLM(GPT-4O, GPT-3.5, CLAUDE-3.5) GENERATED FEATURES

Abstract

Fake-news poses significant threats to society, affecting public opinion, political stability, and public health. Effective detection of fake-news is crucial, yet challenging, due to its complex and evolving nature. This thesis explores the enhancement of fake-news detection using an Adaptive Rational Guidance Network (ARGN), which integrates rationales generated by advanced Large Language Models (LLMs) such as Claude 3.5 Sonnet , GPT-4o and GPT-3.5. Initially, we develop a baseline of ARGN model, utilizing content-only data for prediction. Subsequently, rationales were generated for the same data through textual and commonsense analyses conducted by different LLMs. These rationales were then incorporated into the ARGN model, allowing for a comparative analysis of models using content-only data versus those using both content and LLM-generated rationales. The results demonstrated a notable enhancement in fake-news detection accuracy when rationales were included. The thesis presents a comprehensive evaluation of the ARGN model's performance across various metrics, including F1-score, Accuracy, SPAUC, Precision & Recall. Results indicate that incorporating LLM-generated rationales enhances model performance, giving deeper insight into and ability to recognize false information. Claude-3.5 emerged as the most effective model, consistently outperforming GPT-4o and GPT-3.5 across multiple metrics. This research underscores the potential of integrating advanced LLMs with adaptive models to improve fake-news detection. The proposed ARGN model not only enhances detection accuracy but also offers a robust framework for leveraging multi-perspective analyses in natural language processing tasks. Future work could explore further optimization and application of this model in diverse domains requiring high-stakes decision-making. Keywords: Fake news detection, Adaptive Rational Guidance Network, Large Language Models, BERT, Co-Attention Networks, Accuracy, Precision, Recall, AUC, Machine Learning, Online Misinformation, Text Classification, Natural Language Processing, ARG Model, Adaptive Attention Mechanisms, GPT-4o, GPT-3.5, Claude-3.5, rationales, textual analysis, commonsense analysis, model performance, F1 score.