Petabyte Scale Data Mining: Dream or Reality?

Computer Science – Databases

Scientific paper

Rate now

  [ 0.00 ] – not rated yet Voters 0   Comments 0

Details

originals at http://research.microsoft.com/scripts/pubs/view.asp?TR_ID=MSR-TR-2002-84

Scientific paper

10.1117/12.461427

Science is becoming very data intensive1. Today's astronomy datasets with tens of millions of galaxies already present substantial challenges for data mining. In less than 10 years the catalogs are expected to grow to billions of objects, and image archives will reach Petabytes. Imagine having a 100GB database in 1996, when disk scanning speeds were 30MB/s, and database tools were immature. Such a task today is trivial, almost manageable with a laptop. We think that the issue of a PB database will be very similar in six years. In this paper we scale our current experiments in data archiving and analysis on the Sloan Digital Sky Survey2,3 data six years into the future. We analyze these projections and look at the requirements of performing data mining on such data sets. We conclude that the task scales rather well: we could do the job today, although it would be expensive. There do not seem to be any show-stoppers that would prevent us from storing and using a Petabyte dataset six years from today.

No associations

LandOfFree

Say what you really think

Search LandOfFree.com for scientists and scientific papers. Rate them and share your experience with other people.

Rating

Petabyte Scale Data Mining: Dream or Reality? does not yet have a rating. At this time, there are no reviews or comments for this scientific paper.

If you have personal experience with Petabyte Scale Data Mining: Dream or Reality?, we encourage you to share that experience with our LandOfFree.com community. Your opinion is very important and Petabyte Scale Data Mining: Dream or Reality? will most certainly appreciate the feedback.

Rate now

     

Profile ID: LFWR-SCP-O-699087

  Search
All data on this website is collected from public sources. Our data reflects the most accurate information available at the time of publication.