Objective evaluation measures

Aside from the typical recsys measures such as accuracy and precision, in live experiments several other objective evaluation measures can be taken. These typically concern the users' behavior with the system: number of log-ins, session length, clicks, item views, and purchases. Objective evaluations provide a ground truth for the effect of the system on its users. However, it is sometimes hard to interpret differences in user behavior. For example, if users of a video recommender system click on more clips to watch, does this mean that the user experience is better (more consumption) or worse (more browsing)? In this case, the number of clips watched from beginning to end would be a better measure. Better yet, one can triangulate the objective measure with subjective evaluation measures.