On Modeling Dependency between MapReduce Configuration Parameters and Total Execution Time

Details On Modeling Dependency between MapReduce Configuration Parameters and Total Execution Time On Modeling Dependency between MapReduce Configuration Parameters and Total Execution Time

2012-03-03

arxiv.org/abs/1203.0651v1

Computer Science

Distributed, Parallel, and Cluster Computing

Scientific paper

In this paper, we propose an analytical method to model the dependency between configuration parameters and total execution time of Map-Reduce applications. Our approach has three key phases: profiling, modeling, and prediction. In profiling, an application is run several times with different sets of MapReduce configuration parameters to profile the execution time of the application on a given platform. Then in modeling, the relation between these parameters and total execution time is modeled by multivariate linear regression. Among the possible configuration parameters, two main parameters have been used in this study: the number of Mappers, and the number of Reducers. For evaluation, two standard applications (WordCount, and Exim Mainlog parsing) are utilized to evaluate our technique on a 4-node MapReduce platform.

Affiliated with

Also associated with

No associations

Rating

On Modeling Dependency between MapReduce Configuration Parameters and Total Execution Time does not yet have a rating. At this time, there are no reviews or comments for this scientific paper.

If you have personal experience with On Modeling Dependency between MapReduce Configuration Parameters and Total Execution Time, we encourage you to share that experience with our LandOfFree.com community. Your opinion is very important and On Modeling Dependency between MapReduce Configuration Parameters and Total Execution Time will most certainly appreciate the feedback.

Rate now

Comments { 0 }

Profile ID: LFWR-SCP-O-345637