Objective evaluation measures - Revision history

Zeno Gantner at 13:46, 6 June 2011

2011-06-06T13:46:42Z

Usabart at 02:46, 1 March 2011

2011-03-01T02:46:00Z

92.230.103.21 at 21:25, 21 February 2011

2011-02-21T21:25:40Z

Usabart at 21:23, 21 February 2011

2011-02-21T21:23:47Z

Alan at 21:16, 21 February 2011

2011-02-21T21:16:27Z

Usabart: Created page with "Aside from the typical recsys measures such as accuracy and precision, in live experiments several other objective evaluation measures can be taken. These typically concern the u..."

2011-02-21T21:13:12Z

Created page with "Aside from the typical recsys measures such as accuracy and precision, in live experiments several other objective evaluation measures can be taken. These typically concern the u..."

New page

Aside from the typical recsys measures such as accuracy and precision, in live experiments several other objective evaluation measures can be taken. These typically concern the users' behavior with the system: number of log-ins, session length, clicks, item views, and purchases. Objective evaluations provide a ground truth for the effect of the system on its users. However, it is sometimes hard to interpret differences in user behavior. For example, if users of a video recommender system click on more clips to watch, does this mean that the user experience is better (more consumption) or worse (more browsing)? In this case, the number of clips watched from beginning to end would be a better measure. Better yet, one can triangulate the objective measure with [[subjective evaluation measures]].

@@ Line 1: / Line 1: @@
-Aside from the typical recsys measures such as accuracy and precision, in live experiments several other '''objective evaluation measures''' can be taken. These typically concern the users' behavior with the system: number of log-ins, session length, clicks, item views, and purchases. Objective evaluations provide a ground truth for the effect of the system on its users. However, it is sometimes hard to interpret differences in user behavior. For example, if users of a video recommender system click on more clips to watch, does this mean that the user experience is better (more consumption) or worse (more browsing)? In this case, the number of clips watched from beginning to end would be a better measure. Better yet, one can combine objective and [[subjective evaluation measures]]. In [[quantitative user experiments or field trials]], this entails correlating the objective and subjective measures. In [[qualitative user-studies]] (specifically observational studies), this entails asking the user to think aloud while performing tasks/work.
+Aside from the typical recommender system [[measures]] such as [[accuracy]] and [[precision]], in live experiments several other '''objective evaluation measures''' can be taken. These typically concern the users' behavior with the system: number of log-ins, session length, [[clicks]], item views, and [[purchases]]. Objective evaluations provide a ground truth for the effect of the system on its users. However, it is sometimes hard to interpret differences in user behavior. For example, if users of a video recommender system click on more clips to watch, does this mean that the user experience is better (more consumption) or worse (more browsing)? In this case, the number of clips watched from beginning to end would be a better measure. Better yet, one can combine objective and [[subjective evaluation measures]]. In [[quantitative user experiments or field trials]], this entails correlating the objective and subjective measures. In [[qualitative user-studies]] (specifically observational studies), this entails asking the user to think aloud while performing tasks/work.
 [[Category:Evaluation]]
 [[Category:Evaluation measure]]
 [[Category:User-centric evaluation]]

← Older revision		Revision as of 02:46, 1 March 2011
Line 3:		Line 3:
	[[Category:Evaluation]]		[[Category:Evaluation]]
	[[Category:Evaluation measure]]		[[Category:Evaluation measure]]
		+	[[Category:User-centric evaluation]]

← Older revision		Revision as of 21:25, 21 February 2011
Line 1:		Line 1:
	Aside from the typical recsys measures such as accuracy and precision, in live experiments several other '''objective evaluation measures''' can be taken. These typically concern the users' behavior with the system: number of log-ins, session length, clicks, item views, and purchases. Objective evaluations provide a ground truth for the effect of the system on its users. However, it is sometimes hard to interpret differences in user behavior. For example, if users of a video recommender system click on more clips to watch, does this mean that the user experience is better (more consumption) or worse (more browsing)? In this case, the number of clips watched from beginning to end would be a better measure. Better yet, one can combine objective and [[subjective evaluation measures]]. In [[quantitative user experiments or field trials]], this entails correlating the objective and subjective measures. In [[qualitative user-studies]] (specifically observational studies), this entails asking the user to think aloud while performing tasks/work.		Aside from the typical recsys measures such as accuracy and precision, in live experiments several other '''objective evaluation measures''' can be taken. These typically concern the users' behavior with the system: number of log-ins, session length, clicks, item views, and purchases. Objective evaluations provide a ground truth for the effect of the system on its users. However, it is sometimes hard to interpret differences in user behavior. For example, if users of a video recommender system click on more clips to watch, does this mean that the user experience is better (more consumption) or worse (more browsing)? In this case, the number of clips watched from beginning to end would be a better measure. Better yet, one can combine objective and [[subjective evaluation measures]]. In [[quantitative user experiments or field trials]], this entails correlating the objective and subjective measures. In [[qualitative user-studies]] (specifically observational studies), this entails asking the user to think aloud while performing tasks/work.
		+
	[[Category:Evaluation]]		[[Category:Evaluation]]
		+	[[Category:Evaluation measure]]

← Older revision		Revision as of 21:23, 21 February 2011
Line 1:		Line 1:
−	Aside from the typical recsys measures such as accuracy and precision, in live experiments several other '''objective evaluation measures''' can be taken. These typically concern the users' behavior with the system: number of log-ins, session length, clicks, item views, and purchases. Objective evaluations provide a ground truth for the effect of the system on its users. However, it is sometimes hard to interpret differences in user behavior. For example, if users of a video recommender system click on more clips to watch, does this mean that the user experience is better (more consumption) or worse (more browsing)? In this case, the number of clips watched from beginning to end would be a better measure. Better yet, one can ~~triangulate the~~ objective ~~measure with~~ [[subjective evaluation measures]].	+	Aside from the typical recsys measures such as accuracy and precision, in live experiments several other '''objective evaluation measures''' can be taken. These typically concern the users' behavior with the system: number of log-ins, session length, clicks, item views, and purchases. Objective evaluations provide a ground truth for the effect of the system on its users. However, it is sometimes hard to interpret differences in user behavior. For example, if users of a video recommender system click on more clips to watch, does this mean that the user experience is better (more consumption) or worse (more browsing)? In this case, the number of clips watched from beginning to end would be a better measure. Better yet, one can combine objective and [[subjective evaluation measures]]. In [[quantitative user experiments or field trials]], this entails correlating the objective and subjective measures. In [[qualitative user-studies]] (specifically observational studies), this entails asking the user to think aloud while performing tasks/work.
	[[Category:Evaluation]]		[[Category:Evaluation]]

← Older revision		Revision as of 21:16, 21 February 2011
Line 1:		Line 1:
−	Aside from the typical recsys measures such as accuracy and precision, in live experiments several other objective evaluation measures can be taken. These typically concern the users' behavior with the system: number of log-ins, session length, clicks, item views, and purchases. Objective evaluations provide a ground truth for the effect of the system on its users. However, it is sometimes hard to interpret differences in user behavior. For example, if users of a video recommender system click on more clips to watch, does this mean that the user experience is better (more consumption) or worse (more browsing)? In this case, the number of clips watched from beginning to end would be a better measure. Better yet, one can triangulate the objective measure with [[subjective evaluation measures]].	+	Aside from the typical recsys measures such as accuracy and precision, in live experiments several other '''objective evaluation measures''' can be taken. These typically concern the users' behavior with the system: number of log-ins, session length, clicks, item views, and purchases. Objective evaluations provide a ground truth for the effect of the system on its users. However, it is sometimes hard to interpret differences in user behavior. For example, if users of a video recommender system click on more clips to watch, does this mean that the user experience is better (more consumption) or worse (more browsing)? In this case, the number of clips watched from beginning to end would be a better measure. Better yet, one can triangulate the objective measure with [[subjective evaluation measures]].
		+	[[Category:Evaluation]]