Astronomy and Astrophysics – Astrophysics
Scientific paper
2007-09-12
Astronomy and Astrophysics
Astrophysics
Accepted to Challenges of Large Applications in Distributed Environments (CLADE) 2007, Monterey, CA. 9 pages and 3 figures
Scientific paper
Virtual observatories will give astronomers easy access to an unprecedented amount of data. Extracting scientific knowledge from these data will increasingly demand both efficient algorithms as well as the power of parallel computers. Nearly all efficient analyses of large astronomical datasets use trees as their fundamental data structure. Writing efficient tree-based techniques, a task that is time-consuming even on single-processor computers, is exceedingly cumbersome on massively parallel platforms (MPPs). Most applications that run on MPPs are simulation codes, since the expense of developing them is offset by the fact that they will be used for many years by many researchers. In contrast, data analysis codes change far more rapidly, are often unique to individual researchers, and therefore accommodate little reuse. Consequently, the economics of the current high-performance computing development paradigm for MPPs does not favor data analysis applications. We have therefore built a library, called Ntropy, that provides a flexible, extensible, and easy-to-use way of developing tree-based data analysis algorithms for both serial and parallel platforms. Our experience has shown that not only does our library save development time, it can also deliver excellent serial performance and parallel scalability. Furthermore, Ntropy makes it easy for an astronomer with little or no parallel programming experience to quickly scale their application to a distributed multiprocessor environment. By minimizing development time for efficient and scalable data analysis, we enable wide-scale knowledge discovery on massive datasets.
Connolly Andrew
Gardner Jeffrey P.
McBride Cameron
No associations
LandOfFree
Enabling Rapid Development of Parallel Tree Search Applications does not yet have a rating. At this time, there are no reviews or comments for this scientific paper.
If you have personal experience with Enabling Rapid Development of Parallel Tree Search Applications, we encourage you to share that experience with our LandOfFree.com community. Your opinion is very important and Enabling Rapid Development of Parallel Tree Search Applications will most certainly appreciate the feedback.
Profile ID: LFWR-SCP-O-214315