Modeling Multiple Annotator Expertise in the Semi-Supervised Learning Scenario

Details Modeling Multiple Annotator Expertise in the Semi-Supervised Learning Scenario Modeling Multiple Annotator Expertise in the Semi-Supervised Learning Scenario

2012-03-15

arxiv.org/abs/1203.3529v1

Computer Science

Learning

Appears in Proceedings of the Twenty-Sixth Conference on Uncertainty in Artificial Intelligence (UAI2010)

Scientific paper

Learning algorithms normally assume that there is at most one annotation or label per data point. However, in some scenarios, such as medical diagnosis and on-line collaboration,multiple annotations may be available. In either case, obtaining labels for data points can be expensive and time-consuming (in some circumstances ground-truth may not exist). Semi-supervised learning approaches have shown that utilizing the unlabeled data is often beneficial in these cases. This paper presents a probabilistic semi-supervised model and algorithm that allows for learning from both unlabeled and labeled data in the presence of multiple annotators. We assume that it is known what annotator labeled which data points. The proposed approach produces annotator models that allow us to provide (1) estimates of the true label and (2) annotator variable expertise for both labeled and unlabeled data. We provide numerical comparisons under various scenarios and with respect to standard semi-supervised learning. Experiments showed that the presented approach provides clear advantages over multi-annotator methods that do not use the unlabeled data and over methods that do not use multi-labeler information.

Affiliated with

Also associated with

No associations

Rating

Modeling Multiple Annotator Expertise in the Semi-Supervised Learning Scenario does not yet have a rating. At this time, there are no reviews or comments for this scientific paper.

If you have personal experience with Modeling Multiple Annotator Expertise in the Semi-Supervised Learning Scenario, we encourage you to share that experience with our LandOfFree.com community. Your opinion is very important and Modeling Multiple Annotator Expertise in the Semi-Supervised Learning Scenario will most certainly appreciate the feedback.

Rate now

Comments { 0 }

Profile ID: LFWR-SCP-O-32400