Column-Oriented Table Access Using STIL: Fast Analysis of Very Large Tables

Details Column-Oriented Table Access Using STIL: Fast Analysis of Very Large Tables Column-Oriented Table Access Using STIL: Fast Analysis of Very Large Tables

Aug 2008

adsabs.harvard.edu/cgi-bin/nph-data_query?bibcode=2008aspc..394..422t&link_type=abstract

Astronomical Data Analysis Software and Systems ASP Conference Series, Vol. 394, Proceedings of the conference held 23-26 Septem

Statistics

Applications

Scientific paper

By use of column-oriented storage and file mapping, great improvements in efficiency over more conventional methods can be made for some important kinds of access to large and very large tabular datasets. These techniques have been implemented in the STIL library, enabling their use in the established table analysis applications TOPCAT and STILTS. Benchmarks are presented which show certain common analysis tasks running 10--40 times faster than their MySQL equivalents. Applied to datasets in the range hundreds of Mbyte to hundreds of Gbyte this speedup can be put to good use both on the desktop and at the data center to bring new regimes of data exploration within practical reach.

Affiliated with

Also associated with

No associations

Rating

Column-Oriented Table Access Using STIL: Fast Analysis of Very Large Tables does not yet have a rating. At this time, there are no reviews or comments for this scientific paper.

If you have personal experience with Column-Oriented Table Access Using STIL: Fast Analysis of Very Large Tables, we encourage you to share that experience with our LandOfFree.com community. Your opinion is very important and Column-Oriented Table Access Using STIL: Fast Analysis of Very Large Tables will most certainly appreciate the feedback.

Rate now

Comments { 0 }

Profile ID: LFWR-SCP-O-1691195