A Framework for Analyzing Massive Astrophysical Datasets on a Distributed Grid

Details A Framework for Analyzing Massive Astrophysical Datasets on a Distributed Grid A Framework for Analyzing Massive Astrophysical Datasets on a Distributed Grid

Oct 2007

adsabs.harvard.edu/cgi-bin/nph-data_query?bibcode=2007aspc..376...69g&link_type=abstract

Astronomical Data Analysis Software and Systems XVI ASP Conference Series, Vol. 376, proceedings of the conference held 15-18 Oc

Computer Science

Performance

3

Scientific paper

Virtual observatories will give astronomers easy access to an unprecedented amount of data. Extracting scientific knowledge from these data will increasingly demand both efficient algorithms as well as the power of parallel computers. Such machines will range in size from small Beowulf clusters to large, massively parallel platforms (MPPs) to collections of MPPs distributed across a Grid, such as the NSF TeraGrid facility. Nearly all efficient analyses of large astronomical datasets use trees as their fundamental data structure. Writing efficient tree-based techniques, a task that is time-consuming even on single-processor computers, is exceedingly cumbersome on parallel or grid-distributed resources. We have developed a framework, Ntropy, that provides a flexible, extensible, and easy-to-use way of developing tree-based data analysis algorithms for both serial and parallel platforms. Our experience has shown that not only does our framework save development time, it also delivers an increase in serial performance. Furthermore, our framework makes it easy for an astronomer with little or no parallel programming experience to scale their application quickly to a distributed, multi-processor environment. By minimizing development time for efficient and scalable data analysis, we will enable wide-scale knowledge discovery on massive datasets.

Affiliated with

Also associated with

No associations

Rating

A Framework for Analyzing Massive Astrophysical Datasets on a Distributed Grid does not yet have a rating. At this time, there are no reviews or comments for this scientific paper.

If you have personal experience with A Framework for Analyzing Massive Astrophysical Datasets on a Distributed Grid, we encourage you to share that experience with our LandOfFree.com community. Your opinion is very important and A Framework for Analyzing Massive Astrophysical Datasets on a Distributed Grid will most certainly appreciate the feedback.

Rate now

Comments { 0 }

Profile ID: LFWR-SCP-O-955689