Offline and Online Evaluation of Recommendations

Alejandro Bellogin, Alan Said. 2018, "Offline and Online Evaluation of Recommendations". Collaborative Recommendations.


Recommender Systems have been a popular research topic within personalized systems and information retrieval since the mid nineties. Throughout this time, various models of recommendation have been developed, e.g., approaches using collaborative filtering for purposes such as retrieval of ranked lists of items for consumption, or for the rating prediction task (which was made very popular through the Netflix prize). Today, the use of recommender systems has spread to a very wide area of topics, including personalized healthcare, online news portals, food, social networks, exercise, jobs, investment, transportation, shopping, etc. Given the various situations recommendations can be applied to, it follows that evaluation of these systems needs to be tailored to the specific setting, domain, user-base, context, etc. This chapter aims to give an overview of some of the more commonly used evaluation methods and metrics used for various types of recommendation techniques. We also provide a summary of the available resources in this topic, in addition to some practical considerations, experimental results, and future directions about evaluation in recommendation.

Collaborative Recommendations