Load Balancing for MapReduce-based Entity Resolution

Details Load Balancing for MapReduce-based Entity Resolution Load Balancing for MapReduce-based Entity Resolution

2011-08-08

arxiv.org/abs/1108.1631v1

Computer Science

Distributed, Parallel, and Cluster Computing

Scientific paper

The effectiveness and scalability of MapReduce-based implementations of complex data-intensive tasks depend on an even redistribution of data between map and reduce tasks. In the presence of skewed data, sophisticated redistribution approaches thus become necessary to achieve load balancing among all reduce tasks to be executed in parallel. For the complex problem of entity resolution, we propose and evaluate two approaches for such skew handling and load balancing. The approaches support blocking techniques to reduce the search space of entity resolution, utilize a preprocessing MapReduce job to analyze the data distribution, and distribute the entities of large blocks among multiple reduce tasks. The evaluation on a real cloud infrastructure shows the value and effectiveness of the proposed load balancing approaches.

Affiliated with

Also associated with

No associations

Rating

Load Balancing for MapReduce-based Entity Resolution does not yet have a rating. At this time, there are no reviews or comments for this scientific paper.

If you have personal experience with Load Balancing for MapReduce-based Entity Resolution, we encourage you to share that experience with our LandOfFree.com community. Your opinion is very important and Load Balancing for MapReduce-based Entity Resolution will most certainly appreciate the feedback.

Rate now

Comments { 0 }

Profile ID: LFWR-SCP-O-189741