Computer Science – Computer Vision and Pattern Recognition
Scientific paper
2011-07-10
Computer Science
Computer Vision and Pattern Recognition
25 Pages, 1 Figure, 10 Tables
Scientific paper
This work presents a systematic study of objective evaluations of abstaining classifications using Information-Theoretic Measures (ITMs). First, we define objective measures for which they do not depend on any free parameter. This definition provides technical simplicity for examining "objectivity" or "subjectivity" directly to classification evaluations. Second, we propose twenty four normalized ITMs, derived from either mutual information, divergence, or cross-entropy, for investigation. Contrary to conventional performance measures that apply empirical formulas based on users' intuitions or preferences, the ITMs are theoretically more sound for realizing objective evaluations of classifications. We apply them to distinguish "error types" and "reject types" in binary classifications without the need for input data of cost terms. Third, to better understand and select the ITMs, we suggest three desirable features for classification assessment measures, which appear more crucial and appealing from the viewpoint of classification applications. Using these features as "meta-measures", we can reveal the advantages and limitations of ITMs from a higher level of evaluation knowledge. Numerical examples are given to corroborate our claims and compare the differences among the proposed measures. The best measure is selected in terms of the meta-measures, and its specific properties regarding error types and reject types are analytically derived.
He Ran
Hu Bao-Gang
Yuan Xiao-Tong
No associations
LandOfFree
Information-Theoretic Measures for Objective Evaluation of Classifications does not yet have a rating. At this time, there are no reviews or comments for this scientific paper.
If you have personal experience with Information-Theoretic Measures for Objective Evaluation of Classifications, we encourage you to share that experience with our LandOfFree.com community. Your opinion is very important and Information-Theoretic Measures for Objective Evaluation of Classifications will most certainly appreciate the feedback.
Profile ID: LFWR-SCP-O-33556