On-line Machine Learning and Event Detection in Petascale Data Streams

Details On-line Machine Learning and Event Detection in Petascale Data Streams On-line Machine Learning and Event Detection in Petascale Data Streams

Jan 2012

adsabs.harvard.edu/cgi-bin/nph-data_query?bibcode=2012aas...21911304t&link_type=abstract

American Astronomical Society, AAS Meeting #219, #113.04

Computer Science

Learning

Scientific paper

Traditional statistical data mining involves off-line analysis in which all data are available and equally accessible. However, petascale datasets have challenged this premise since it is often impossible to store, let alone analyze, the relevant observations. This has led the machine learning community to investigate adaptive processing chains where data mining is a continuous process. Here pattern recognition permits triage and followup decisions at multiple stages of a processing pipeline. Such techniques can also benefit new astronomical instruments such as the Large Synoptic Survey Telescope (LSST) and Square Kilometre Array (SKA) that will generate petascale data volumes. We summarize some machine learning perspectives on real time data mining, with representative cases of astronomical applications and event detection in high volume datastreams.
The first is a "supervised classification" approach currently used for transient event detection at the Very Long Baseline Array (VLBA). It injects known signals of interest - faint single-pulse anomalies - and tunes system parameters to recover these events. This permits meaningful event detection for diverse instrument configurations and observing conditions whose noise cannot be well-characterized in advance. Second, "semi-supervised novelty detection" finds novel events based on statistical deviations from previous patterns. It detects outlier signals of interest while considering known examples of false alarm interference. Applied to data from the Parkes pulsar survey, the approach identifies anomalous "peryton" phenomena that do not match previous event models. Finally, we consider online light curve classification that can trigger adaptive followup measurements of candidate events. Classifier performance analyses suggest optimal survey strategies, and permit principled followup decisions from incomplete data. These examples trace a broad range of algorithm possibilities available for online astronomical data mining.
This talk describes research performed at the Jet Propulsion Laboratory, California Institute of Technology. Copyright 2012, All Rights Reserved. U.S. Government support acknowledged.

Affiliated with

Also associated with

No associations

Rating

On-line Machine Learning and Event Detection in Petascale Data Streams does not yet have a rating. At this time, there are no reviews or comments for this scientific paper.

If you have personal experience with On-line Machine Learning and Event Detection in Petascale Data Streams, we encourage you to share that experience with our LandOfFree.com community. Your opinion is very important and On-line Machine Learning and Event Detection in Petascale Data Streams will most certainly appreciate the feedback.

Rate now

Comments { 0 }

Profile ID: LFWR-SCP-O-1574052