Difference between revisions of "Objective evaluation measures"
Zeno Gantner (talk | contribs) |
|||
| Line 1: | Line 1: | ||
| − | Aside from the typical | + | Aside from the typical recommender system [[measures]] such as [[accuracy]] and [[precision]], in live experiments several other '''objective evaluation measures''' can be taken. These typically concern the users' behavior with the system: number of log-ins, session length, [[clicks]], item views, and [[purchases]]. Objective evaluations provide a ground truth for the effect of the system on its users. However, it is sometimes hard to interpret differences in user behavior. For example, if users of a video recommender system click on more clips to watch, does this mean that the user experience is better (more consumption) or worse (more browsing)? In this case, the number of clips watched from beginning to end would be a better measure. Better yet, one can combine objective and [[subjective evaluation measures]]. In [[quantitative user experiments or field trials]], this entails correlating the objective and subjective measures. In [[qualitative user-studies]] (specifically observational studies), this entails asking the user to think aloud while performing tasks/work. |
[[Category:Evaluation]] | [[Category:Evaluation]] | ||
[[Category:Evaluation measure]] | [[Category:Evaluation measure]] | ||
[[Category:User-centric evaluation]] | [[Category:User-centric evaluation]] | ||
Latest revision as of 09:46, 6 June 2011
Aside from the typical recommender system measures such as accuracy and precision, in live experiments several other objective evaluation measures can be taken. These typically concern the users' behavior with the system: number of log-ins, session length, clicks, item views, and purchases. Objective evaluations provide a ground truth for the effect of the system on its users. However, it is sometimes hard to interpret differences in user behavior. For example, if users of a video recommender system click on more clips to watch, does this mean that the user experience is better (more consumption) or worse (more browsing)? In this case, the number of clips watched from beginning to end would be a better measure. Better yet, one can combine objective and subjective evaluation measures. In quantitative user experiments or field trials, this entails correlating the objective and subjective measures. In qualitative user-studies (specifically observational studies), this entails asking the user to think aloud while performing tasks/work.
