Facing "the Curse of Dimensionality": Image Fusion and Nonlinear Dimensionality Reduction for Advanced Data Mining and Visualization of Astronomical Images

Details Facing "the Curse of Dimensionality": Image Fusion and Nonlinear Dimensionality Reduction for Advanced Data Mining and Visualization of Astronomical Images Facing "the Curse of Dimensionality": Image Fusion and Nonlinear Dimensionality Reduction for Advanced Data Mining and Visualization of Astronomical Images

May 2009

adsabs.harvard.edu/cgi-bin/nph-data_query?bibcode=2009aas...21440402p&link_type=abstract

American Astronomical Society, AAS Meeting #214, #404.02; Bulletin of the American Astronomical Society, Vol. 41, p.667

Statistics

Computation

Scientific paper

The complexity of multitemporal/multispectral astronomical data sets together with the approaching petascale of such datasets and large astronomical surveys require automated or semi-automated methods for knowledge discovery. Traditional statistical methods of analysis may break down not only because of the amount of data, but mostly because of the increase of the dimensionality of data. Image fusion (combining information from multiple sensors in order to create a composite enhanced image) and dimension reduction (finding lower-dimensional representation of high-dimensional data) are effective approaches to "the curse of dimensionality,” thus facilitating automated feature selection, classification and data segmentation. Dimension reduction methods greatly increase computational efficiency of machine learning algorithms, improve statistical inference and together with image fusion enable effective scientific visualization (as opposed to mere illustrative visualization). The main approach of this work utilizes recent advances in multidimensional image processing, as well as representation of essential structure of a data set in terms of its fundamental eigenfunctions, which are used as an orthonormal basis for the data visualization and analysis. We consider multidimensional data sets and images as manifolds or combinatorial graphs and construct variational splines that minimize certain Sobolev norms. These splines allow us to reconstruct the eigenfunctions of the combinatorial Laplace operator by using only a small portion of the graph. We use the first two or three eigenfunctions for embedding large data sets into two- or three-dimensional Euclidean space. Such reduced data sets allow efficient data organization, retrieval, analysis and visualization. We demonstrate applications of the algorithms to test cases from the Spitzer Space Telescope. This work was carried out with funding from the National Geospatial-Intelligence Agency University Research Initiative (NURI), grant HM1582-08-1-0019 and with partial funding from NASA to the California Institute of Technology and the Jet Propulsion Laboratory.

Affiliated with

Also associated with

No associations

Rating

Facing "the Curse of Dimensionality": Image Fusion and Nonlinear Dimensionality Reduction for Advanced Data Mining and Visualization of Astronomical Images does not yet have a rating. At this time, there are no reviews or comments for this scientific paper.

If you have personal experience with Facing "the Curse of Dimensionality": Image Fusion and Nonlinear Dimensionality Reduction for Advanced Data Mining and Visualization of Astronomical Images, we encourage you to share that experience with our LandOfFree.com community. Your opinion is very important and Facing "the Curse of Dimensionality": Image Fusion and Nonlinear Dimensionality Reduction for Advanced Data Mining and Visualization of Astronomical Images will most certainly appreciate the feedback.

Rate now

Comments { 0 }

Profile ID: LFWR-SCP-O-1106563