Category:User-centric evaluation

From RecSysWiki
Jump to navigation Jump to search

Whereas originally the field of recommender systems heavily focused on offline evaluation, recently awareness has grown that the usability and user experience of recommender systems should be tested in online evaluations with real users. User-centric evaluation methods can be broadly categorized into qualitative user-studies and quantitative user experiments or field trials. User studies typically combine objective evaluation measures with subjective evaluation measures, typically in the form of design critiques, interviews, and questionnaires.

User-centric evaluation has had difficulties gaining popularity as an evaluation method, because it is often difficult to test new algorithms or systems with real users. Early examples in the recommender systems literature have been of questionable quality. There have been a few suggestions for standardization and simplification of the user-centric evaluation process.