Adaptive Hybrid Collaborative Filtering Recommendation System (AHCF)

Abstract

Recommendation systems play a vital role in boosting the organization’s profit, especially for e-commerce platforms such as Amazon. These systems focus on targeting specific products to users and predicting user preferences and interests. However, recommendation systems are plagued with many challenges, such as adapting them to changes in user preferences and taste, and the effectiveness of recommendations made also determines the ability to retain and engage new users, as new user conversion to clients. This thesis proposes to use an adaptive hybrid collaborative approach to making recommendations to users. Four algorithms are combined: the Alternate Least Squares (ALS), KMeans clustering, Latent Dirichlet Allocation (LDA) and KMeans streaming. The recommender engine developed is in itself a multi-hybrid system as it not only combines four (4) algorithms but also combines the collaborative technique and content-based techniques of making a recommendation. Thus, the approach adopted can be used on datasets that contain rating information, textual descriptions or both. Three servers are leveraged in the implementation, consisting of the Scala server, PHP and Angular JS server and the MySQL database server for the storage of the results from the recommender engine. Various industry-standard metrics are adopted for the individual algorithms in addition to their computational times. These metrics include Root Mean Square Error(RMSE) for the ALS, Within Cluster Sum of Squares(WCSS) for KMeans, Log Perplexity and Log-Likelihood in the LDA. The memory estimates footprints and computational time on retraining the model are recorded for the KMeans streaming. The recommender engine is tested primarily on the 100K and 1M movieLens datasets and some portions of the 20M dataset are used. The implementation is compared with benchmark recommender algorithms via GitHub and existing offline implementations. In terms of retraining, the Adaptive Hybrid Collaborative Filtering Recommendation System(AHCF) developed improves a recommendation’s computational time concerning the offline model by 50%. The AHCF has an accuracy measure of 0.88-3.0 on RMSE values for the chosen datasets on increasing rank but less than 8 for 5 other datasets adopted. The other datasets range from restaurant datasets, anime, dating datasets, books and e-commerce. These results are taken for the 1M and 100K datasets. The unique contributions made in this research include combining multiple algorithms into one recommender engine that leverages textual and rating information at the same time. Improvements in computational efficiency as against offline models that are designed for a real-time update of recommendations by half on retraining. The generic nature of the algorithm also makes it useful to be used in many domains that leverage informative text and rating information. The model is also open source and available to all users. In a nutshell, the research embraces the efficiency of updating user preferences in real-time and making personalized recommendations by adapting to user preferences over short time intervals.