BlinkDB: Queries with Bounded Errors and Bounded Response Times on Very Large Data

Details BlinkDB: Queries with Bounded Errors and Bounded Response Times on Very Large Data BlinkDB: Queries with Bounded Errors and Bounded Response Times on Very Large Data

2012-03-25

arxiv.org/abs/1203.5485v1

Computer Science

Databases

Scientific paper

Today's web is predominantly data-driven. Corporations, businesses and netizens are increasingly depending on enormous amounts of data (spanning terabytes or even petabytes in size) to make intelligent business and personal decisions. Often the time it takes to make these decisions is critical. Unfortunately, quickly analyzing large volumes of data poses significant challenges. For instance, scanning 1TB of data may take minutes, even when the data is spread across hundreds of machines and read in parallel. In this paper, we present BlinkDB, a massively parallel, sampling-based approximate query engine for running interactive queries on large volumes of data. The key observation in BlinkDB is that one can make perfect decisions in the absence of perfect answers. For example, reliably detecting a malfunctioning server in a distributed collection of system logs does not require knowing every request the server processed. Based on this insight, BlinkDB allows one to tradeoff between query accuracy and response time, enabling interactive queries over massive data by running queries on data samples. To achieve this, BlinkDB uses two key ideas that differentiate it from previous sampling-based database systems: (1) an optimization framework to build a set of multi-dimensional, multi-resolution samples, and (2) a strategy that uses a set of small samples to dynamically estimate a query's error and response time at runtime. We have built a BlinkDB prototype, and validate its effectiveness using well-known benchmarks and a real-world workload derived from Conviva Networks. Our experiments show that BlinkDB can execute a range of queries over a real-world query trace on up to 17 TB of data and 100 nodes in 2 seconds, with an error of 2-10%.

Affiliated with

Also associated with

No associations

Rating

BlinkDB: Queries with Bounded Errors and Bounded Response Times on Very Large Data does not yet have a rating. At this time, there are no reviews or comments for this scientific paper.

If you have personal experience with BlinkDB: Queries with Bounded Errors and Bounded Response Times on Very Large Data, we encourage you to share that experience with our LandOfFree.com community. Your opinion is very important and BlinkDB: Queries with Bounded Errors and Bounded Response Times on Very Large Data will most certainly appreciate the feedback.

Rate now

Comments { 0 }

Profile ID: LFWR-SCP-O-74974