Statistics – Machine Learning
Scientific paper
2009-12-15
Statistics
Machine Learning
37 pages, 1 figure, submitted
Scientific paper
The ratio of two probability densities can be used for solving various machine learning tasks such as covariate shift adaptation (importance sampling), outlier detection (likelihood-ratio test), and feature selection (mutual information). Recently, several methods of directly estimating the density ratio have been developed, e.g., kernel mean matching, maximum likelihood density ratio estimation, and least-squares density ratio fitting. In this paper, we consider a kernelized variant of the least-squares method and investigate its theoretical properties from the viewpoint of the condition number using smoothed analysis techniques--the condition number of the Hessian matrix determines the convergence rate of optimization and the numerical stability. We show that the kernel least-squares method has a smaller condition number than a version of kernel mean matching and other M-estimators, implying that the kernel least-squares method has preferable numerical properties. We further give an alternative formulation of the kernel least-squares estimator which is shown to possess an even smaller condition number. We show that numerical studies meet our theoretical analysis.
Kanamori Takafumi
Sugiyama Masashi
Suzuki Taiji
No associations
LandOfFree
Condition Number Analysis of Kernel-based Density Ratio Estimation does not yet have a rating. At this time, there are no reviews or comments for this scientific paper.
If you have personal experience with Condition Number Analysis of Kernel-based Density Ratio Estimation, we encourage you to share that experience with our LandOfFree.com community. Your opinion is very important and Condition Number Analysis of Kernel-based Density Ratio Estimation will most certainly appreciate the feedback.
Profile ID: LFWR-SCP-O-304861