INTEGRATING DIFFUSION MODEL FOR VIRTUAL EXEMPLAR GENERATION AND LLM TO PROMPT-BASED INCREMENTAL LEARNING TECHNIQUES

Abstract

Most traditional machine learning and deep learning algorithms assume the entire dataset to be available before beginning the training process. But this is not the case with most realworld scenarios. In real-world scenarios, the data keeps changing with time. Retraining the entire dataset from the start can be computationally very expensive. Simply updating the models with the incoming data will lead to the problem of catastrophic forgetting. Incremental learning also known as continual learning or lifelong learning techniques aims to develop algorithms that enable existing models to learn new knowledge from sequentially incoming data while retaining the existing knowledge, mitigating the catastrophic forgetting problem and also without extensive retraining of the previous data. In this work, a novel technique to enhance the performance of the existing incremental learning algorithms has been proposed. These include a novel exemplar generation technique like use of generative models for using stable diffusion model, and also integration of LLMs to the existing prompt-based incremental learning algorithms. This work mainly focuses on class incremental learning settings for the image classification task and were experimented on popular benchmark datasets like Split CIFAR100 and Split ImageNet-R. Our proposed novel methods have outperformed existing methods in all benchmark settings.