User-centric recommender system evaluation

Whereas originally the field of recommender systems heavily focused on offline evaluation, recently awareness has grown that the usability and user experience of recommender systems should be tested in online evaluations with real users. User-centric evaluation methods can be broadly categorized into qualitative user-studies and quantitative user experiments or field trials. User studies typically combine objective evaluation measures with subjective evaluation measures, typically in the form of design critiques, interviews, and questionnaires.