Agnostic Learning of Monomials by Halfspaces is Hard

Details Agnostic Learning of Monomials by Halfspaces is Hard Agnostic Learning of Monomials by Halfspaces is Hard

2010-12-03

arxiv.org/abs/1012.0729v1

Computer Science

Computational Complexity

37 pages, Preliminary version appeared in FOCS 2009

Scientific paper

We prove the following strong hardness result for learning: Given a distribution of labeled examples from the hypercube such that there exists a monomial consistent with $(1-\eps)$ of the examples, it is NP-hard to find a halfspace that is correct on $(1/2+\eps)$ of the examples, for arbitrary constants $\eps > 0$. In learning theory terms, weak agnostic learning of monomials is hard, even if one is allowed to output a hypothesis from the much bigger concept class of halfspaces. This hardness result subsumes a long line of previous results, including two recent hardness results for the proper learning of monomials and halfspaces. As an immediate corollary of our result we show that weak agnostic learning of decision lists is NP-hard. Our techniques are quite different from previous hardness proofs for learning. We define distributions on positive and negative examples for monomials whose first few moments match. We use the invariance principle to argue that regular halfspaces (all of whose coefficients have small absolute value relative to the total $\ell_2$ norm) cannot distinguish between distributions whose first few moments match. For highly non-regular subspaces, we use a structural lemma from recent work on fooling halfspaces to argue that they are ``junta-like'' and one can zero out all but the top few coefficients without affecting the performance of the halfspace. The top few coefficients form the natural list decoding of a halfspace in the context of dictatorship tests/Label Cover reductions. We note that unlike previous invariance principle based proofs which are only known to give Unique-Games hardness, we are able to reduce from a version of Label Cover problem that is known to be NP-hard. This has inspired follow-up work on bypassing the Unique Games conjecture in some optimal geometric inapproximability results.

Affiliated with

Also associated with

No associations

Rating

Agnostic Learning of Monomials by Halfspaces is Hard does not yet have a rating. At this time, there are no reviews or comments for this scientific paper.

If you have personal experience with Agnostic Learning of Monomials by Halfspaces is Hard, we encourage you to share that experience with our LandOfFree.com community. Your opinion is very important and Agnostic Learning of Monomials by Halfspaces is Hard will most certainly appreciate the feedback.

Rate now

Comments { 0 }

Profile ID: LFWR-SCP-O-515035