Workshop on reproducibility and replication in recommender systems evaluation

Alejandro Bellogin, Pablo Castells, Alan Said, Domonkos Tikk. 2013, "Workshop on reproducibility and replication in recommender systems evaluation". Proceedings of the 7th ACM conference on Recommender systems - RecSys ‘13.

Abstract

Experiment replication and reproduction are key requirements for empirical research methodology, and an important open issue in the field of Recommender Systems. When an experiment is repeated by a different researcher and exactly the same result is obtained, we can say the experiment has been replicated. When the results are not exactly the same but the conclusions are compatible with the prior ones, we have a reproduction of the experiment. Reproducibility and replication involve recommendation algorithm implementations, experimental protocols, and evaluation metrics. While the problem of reproducibility and replication has been recognized in the Recommender Systems community, the need for a clear solution remains largely unmet, which motivates the present workshop.

Publication
Proceedings of the 7th ACM conference on Recommender systems - RecSys ‘13