COLLABORATIVE FILTERING BASED RECOMMENDATION SYSTEM TO MITIGATE THE DATA SPARSITY AND COLD START PROBLEM

Abstract

A recommendation system (RS) serves as an information filtering tool that generates relevant recommendations for goods or products that may be of interest to a group of users. Thus, it presents an alternate way to information filtering based on human preferences. In real life as the amount of information increases, it becomes very challenging for the Internet users to sort valuable information in a short period of time. In dealing with this, a Recommendation System (RS) plays an important role. RS tries to automate the recommendations received from family, friends, and co-workers on a regular basis. It gathers a large amount of information on the activities, preferences, interest or taste of its users for a set of items i.e., movies, hardware items, garments and so forth and makes use of this gathered information to provide suggestions to different users. The Collaborative Filtering (CF) technique of RS is one of the most successful, popular and extensively used approaches. It is based on the idea that if a person has similar preferences for a collection of items in the past, he or she will have similar preferences in the future. A CF-based recommendation system first determines the similarity among users or items to find out the neighbours, and then makes recommendations to individual users. To compute the similarity, both traditional and advanced similarity measures are used. The traditional measures include Cosine, Jaccard, Pearson Correlation Coefficient and some popular advanced measures are Proximity Impact Popularity (PIP), Jaccard Mean Squared Difference (JMSD), Triangle Multiplying Jaccard (TMJ), etc. These measures use some basic information such as user id, item id and ratings to compute the similarity. Most of these similarity measures either suffer from data sparsity and/or cold start problems. The data sparsity problem occurs when the ratio of estimated ratings to available ratings is exceedingly high. With an increase in the number of users and items, the problem worsens and the provided recommendations are of relatively low accuracy. On the other hand, when a user does not have enough ratings to make recommendations, it is known as the cold start problem. Because of these limitations, the resultant recommendations have a low degree of accuracy. As a result, a method that can overcome these drawbacks and compute effective similarity is required. Even if the system succeeds in computing effective similarity, there might be possibility that computed similarity isn't accurate. The reason behind this is, most of the CF approaches only focus on the historical preference of the users, but these preferences can change over time. Therefore, it is important to include the time function in the similarity computation process. There are various time decay functions i.e., exponential, concave, convex etc that can be applied on the various levels of the recommendation process. This includes, i.e., similarity computation level, prediction level and rating matrix level. The concept of time is also integrated with the deep learning technique to give weightage to the most recent ratings. It overcomes the issue of selecting an appropriate similarity measure and increases the system efficiency by correcting the noisy ratings. Thus, the research work focuses on the development of an efficient Collaborative filtering approach to handle the data sparsity and cold start problem. Also, a time-based recommendation system is developed to give weightage to the most recent ratings. In addition, Time decay based Deep Neural Network for RS is also proposed to overcome the limit of data sparsity of similarity measures in CF technique. Chapter 1 forms the motivation for the research, defines the notion of context and lists the contributions of this thesis. Chapter 2 discusses the related work in the area of Recommendation System, Collaborative Filtering technique, Time decay functions and Deep learning technique in Recommendation System. Chapter 3 proposes a new similarity measure, using the traditional approach. In this approach, an Efficient Gowers- Jaccard Sigmoid Measure (EGJSM) is developed which uses user, item and rating information to compute the similarity among users/items. This measure overcomes the limitations of the existing similarity measures by taking into account not only the local context of user ratings, but also the users’ overall preferences. Experiments were carried out on five real world datasets, and the results were compared against a variety of state-of-the-art similarity metrics. Chapter 4 proposes a new similarity measure CgS that utilizes cognitive information such as time, gender etc. along with the user, item and rating information to compute similarity. The CgS method produced better results than the proposed EGJSM method. Chapter 5 includes the time context in the recommendation process because traditional and previously developed similarity measures are not able to adequately reveal the change in users' interests. Therefore, an efficient measure that considers time context is proposed in this chapter. These time decay functions, i.e., exponential, convex, linear, power, etc., are utilized at various levels of the recommendation process, i.e., similarity computation, rating matrix, and prediction level to compute similarity. Experimental results over three real datasets suggest that the power decay function outperforms other existing techniques when applied at the rating matrix level. Chapter 6 develops a Time based Deep Neural Network (TD-DNN) for the Recommendation System. The main feature of TD-DNN is its ability to handle the noisy data while recommending the items. Also, it incorporates users' tastes and interests that change over time that traditional techniques fail to do. In this, initially noisy ratings from the dataset are detected and corrected using the matrix factorization approach and then, a power decay function is applied at the non-noisy rating matrix to provide more weightage to the recent ratings. This non-noise weighted matrix is fed into the deep learning model to generate the predicted ratings. Chapter 7 summarizes the work, presents important conclusions and discusses future research directions.