Efficient Human Computation

Details Efficient Human Computation Efficient Human Computation

2009-03-05

arxiv.org/abs/0903.1125v1

Computer Science

Learning

8 pages, 1 figure

Scientific paper

Collecting large labeled data sets is a laborious and expensive task, whose scaling up requires division of the labeling workload between many teachers. When the number of classes is large, miscorrespondences between the labels given by the different teachers are likely to occur, which, in the extreme case, may reach total inconsistency. In this paper we describe how globally consistent labels can be obtained, despite the absence of teacher coordination, and discuss the possible efficiency of this process in terms of human labor. We define a notion of label efficiency, measuring the ratio between the number of globally consistent labels obtained and the number of labels provided by distributed teachers. We show that the efficiency depends critically on the ratio alpha between the number of data instances seen by a single teacher, and the number of classes. We suggest several algorithms for the distributed labeling problem, and analyze their efficiency as a function of alpha. In addition, we provide an upper bound on label efficiency for the case of completely uncoordinated teachers, and show that efficiency approaches 0 as the ratio between the number of labels each teacher provides and the number of classes drops (i.e. alpha goes to 0).

Affiliated with

Also associated with

No associations

Rating

Efficient Human Computation does not yet have a rating. At this time, there are no reviews or comments for this scientific paper.

If you have personal experience with Efficient Human Computation, we encourage you to share that experience with our LandOfFree.com community. Your opinion is very important and Efficient Human Computation will most certainly appreciate the feedback.

Rate now

Comments { 0 }

Profile ID: LFWR-SCP-O-134778