Efficient continuous KNN join processing for real-time recommendation

Abstract

Along with the sustainable and rapid accumulation of user-generated contents in social networking websites, how to push a certain content to the corresponding interested users, named recommendation for short, has successfully received wide attention. Considering the continuous updated contents and the constant changing of users’ interests, recommendation is expected to be completed immediately to send the most fresh content to appropriate users after the corresponding new contents have become available. In other words, recommendation needs to meet real-time requirements to fit the content-consumption behavior of users. In the traditional recommendation system field, the corresponding attributes of users and contents could be characterized by feature vectors in a certain high-dimensional space, subsequently the recommendation problem could also be converted into how to obtain the K appropriate contents for each user, which could be called kNN join. Due to the massive, high-dimensional, and continuously updated contents, the corresponding recommendation based on traditional kNN join (continuously updating the kNN join results) will be undoubtedly faced with unacceptable computational costs. In this paper, we propose a locality-sensitive hashing (LSH)–based index called LSHI, which is built on user set to find the specific users who might be affected by the updated contents efficiently. With the help of LSHI, the recommendation lists of the affected users could be adjusted accordingly and the holistic effectiveness of the recommendation (for all users) could be guaranteed simultaneously. Finally, extensive experiments have been conducted to demonstrate the superiorities of our proposed method in this paper.