Sector and Sphere: Towards Simplified Storage and Processing of Large Scale Distributed Data

Details Sector and Sphere: Towards Simplified Storage and Processing of Large Scale Distributed Data Sector and Sphere: Towards Simplified Storage and Processing of Large Scale Distributed Data

2008-09-06

arxiv.org/abs/0809.1181v2

Computer Science

Distributed, Parallel, and Cluster Computing

Scientific paper

Cloud computing has demonstrated that processing very large datasets over commodity clusters can be done simply given the right programming model and infrastructure. In this paper, we describe the design and implementation of the Sector storage cloud and the Sphere compute cloud. In contrast to existing storage and compute clouds, Sector can manage data not only within a data center, but also across geographically distributed data centers. Similarly, the Sphere compute cloud supports User Defined Functions (UDF) over data both within a data center and across data centers. As a special case, MapReduce style programming can be implemented in Sphere by using a Map UDF followed by a Reduce UDF. We describe some experimental studies comparing Sector/Sphere and Hadoop using the Terasort Benchmark. In these studies, Sector is about twice as fast as Hadoop. Sector/Sphere is open source.

Affiliated with

Also associated with

No associations

Rating

Sector and Sphere: Towards Simplified Storage and Processing of Large Scale Distributed Data does not yet have a rating. At this time, there are no reviews or comments for this scientific paper.

If you have personal experience with Sector and Sphere: Towards Simplified Storage and Processing of Large Scale Distributed Data, we encourage you to share that experience with our LandOfFree.com community. Your opinion is very important and Sector and Sphere: Towards Simplified Storage and Processing of Large Scale Distributed Data will most certainly appreciate the feedback.

Rate now

Comments { 0 }

Profile ID: LFWR-SCP-O-5330