Computer Science – Artificial Intelligence
Scientific paper
2011-07-29
Computer Science
Artificial Intelligence
Scientific paper
ROC curves and cost curves are two popular ways of visualising classifier performance, finding appropriate thresholds according to the operating condition, and deriving useful aggregated measures such as the area under the ROC curve (AUC) or the area under the optimal cost curve. In this note we present some new findings and connections between ROC space and cost space, by using the expected loss over a range of operating conditions. In particular, we show that ROC curves can be transferred to cost space by means of a very natural way of understanding how thresholds should be chosen, by selecting the threshold such that the proportion of positive predictions equals the operating condition (either in the form of cost proportion or skew). We call these new curves {ROC Cost Curves}, and we demonstrate that the expected loss as measured by the area under these curves is linearly related to AUC. This opens up a series of new possibilities and clarifies the notion of cost curve and its relation to ROC analysis. In addition, we show that for a classifier that assigns the scores in an evenly-spaced way, these curves are equal to the Brier Curves. As a result, this establishes the first clear connection between AUC and the Brier score.
Ferri Cèsar
Flach Peter
Hernández-Orallo José
No associations
LandOfFree
Technical Note: Towards ROC Curves in Cost Space does not yet have a rating. At this time, there are no reviews or comments for this scientific paper.
If you have personal experience with Technical Note: Towards ROC Curves in Cost Space, we encourage you to share that experience with our LandOfFree.com community. Your opinion is very important and Technical Note: Towards ROC Curves in Cost Space will most certainly appreciate the feedback.
Profile ID: LFWR-SCP-O-318769