Difference between revisions of "Area Under the ROC Curve"
Jump to navigation
Jump to search
Zeno Gantner (talk | contribs) (Created page with "The '''area under the ROC curve''' ('''AUC''') is a measure for ranking quality. The best possible value is 1, and any ranking that makes sense would have an AUC > 0.5. AUC does ...") |
Zeno Gantner (talk | contribs) |
||
| (5 intermediate revisions by the same user not shown) | |||
| Line 1: | Line 1: | ||
| − | The '''area under the ROC curve''' ('''AUC''') is a measure for ranking quality. | + | The '''area under the ROC curve''' ('''AUC''') is a measure for [[ranking]] quality. |
| − | The best possible value is 1, and any ranking that makes sense would have an AUC > 0.5. | + | |
| + | In [[recommender systems]], we are often interested in how well method can rank a given set of [[item]]s. | ||
| + | The best possible value is 1, and any non-random ranking that makes sense would have an AUC > 0.5. | ||
| + | |||
| + | An intuitive explanation: | ||
| + | :''The AUC specifies the probability that, when we draw two examples at random, their predicted pairwise ranking is correct.'' | ||
| + | ::(adapted from [http://www.ecmlpkdd2006.org/challenge.html], which we found via [http://biit.cs.ut.ee/personal-blog-entry/2006/05/13/roc-area-under-curve-explained]) | ||
| + | |||
AUC does not give a higher weight to items higher up in the ranking. | AUC does not give a higher weight to items higher up in the ranking. | ||
| + | Some measures that put more weight on higher-ranking items are [[normalized discounted cumulative gain]] ([[NDCG]]) and [[mean average precision]] ([[MAP]]). | ||
| + | |||
| + | == External links == | ||
| + | * [[Wikipedia: Receiver operating characteristic]] | ||
| + | * [http://github.com/zenogantner/MyMediaLite/blob/master/src/MyMediaLite/Eval/Measures/AUC.cs C# implementation of AUC evaluation] for binary responses (part of the [[MyMediaLite]] library) | ||
| + | |||
| − | [[ | + | [[Category: Evaluation measure]] |
Latest revision as of 06:08, 24 September 2011
The area under the ROC curve (AUC) is a measure for ranking quality.
In recommender systems, we are often interested in how well method can rank a given set of items. The best possible value is 1, and any non-random ranking that makes sense would have an AUC > 0.5.
An intuitive explanation:
- The AUC specifies the probability that, when we draw two examples at random, their predicted pairwise ranking is correct.
AUC does not give a higher weight to items higher up in the ranking. Some measures that put more weight on higher-ranking items are normalized discounted cumulative gain (NDCG) and mean average precision (MAP).
External links
- Wikipedia: Receiver operating characteristic
- C# implementation of AUC evaluation for binary responses (part of the MyMediaLite library)
