Supporting dynamic pipeline changes using Class-Based Object Versioning in Astro-WISE

Details Supporting dynamic pipeline changes using Class-Based Object Versioning in Astro-WISE Supporting dynamic pipeline changes using Class-Based Object Versioning in Astro-WISE

Dec 2011

adsabs.harvard.edu/cgi-bin/nph-data_query?bibcode=2011exa...tmp..133m&link_type=abstract

Experimental Astronomy, Online First

Astronomy and Astrophysics

Astronomy

Scientific Computing, Object Versioning, Data Provenance

Scientific paper

Understanding the difference between data objects is a major problem especially in a scientific collaboration which allows scientists to collectively reuse data, modify and adapt scripts developed by their peers to process data while publishing the results to a centralized data store. Although data provenance has been significantly studied to address the origins of a data item, it does not however address changes made to the source code. Systems often appear as a large number of modules each containing hundreds of lines of code. It is, in general, not obvious which parts of source code contributed to the change in data object. The paper introduces the Class-Based Object Versioning framework, which overcomes some of the shortcomings of popular versioning systems (e.g. CVS, SVN) in maintaining data and code provenance information in scientific computing environments. The framework automatically identifies and captures useful fine-grained changes in the data and code of scripts that perform scientific experiments so that important information about intermediate stages (i.e. unrecorded changes in experiment parameters and procedures) can be identified and analyzed. The benefits of such a system include querying specific methods and code attributes for data items of interest, finding missing gaps of data lineage and implicit storage of intermediate data.

Affiliated with

Also associated with

No associations

Rating

Supporting dynamic pipeline changes using Class-Based Object Versioning in Astro-WISE does not yet have a rating. At this time, there are no reviews or comments for this scientific paper.

If you have personal experience with Supporting dynamic pipeline changes using Class-Based Object Versioning in Astro-WISE, we encourage you to share that experience with our LandOfFree.com community. Your opinion is very important and Supporting dynamic pipeline changes using Class-Based Object Versioning in Astro-WISE will most certainly appreciate the feedback.

Rate now

Comments { 0 }

Profile ID: LFWR-SCP-O-1607642