On the Local Correctness of L^1 Minimization for Dictionary Learning

Details On the Local Correctness of L^1 Minimization for Dictionary Learning On the Local Correctness of L^1 Minimization for Dictionary Learning

2011-01-29

arxiv.org/abs/1101.5672v1

Computer Science

Information Theory

37 pages, 1 figure

Scientific paper

The idea that many important classes of signals can be well-represented by linear combinations of a small set of atoms selected from a given dictionary has had dramatic impact on the theory and practice of signal processing. For practical problems in which an appropriate sparsifying dictionary is not known ahead of time, a very popular and successful heuristic is to search for a dictionary that minimizes an appropriate sparsity surrogate over a given set of sample data. While this idea is appealing, the behavior of these algorithms is largely a mystery; although there is a body of empirical evidence suggesting they do learn very effective representations, there is little theory to guarantee when they will behave correctly, or when the learned dictionary can be expected to generalize. In this paper, we take a step towards such a theory. We show that under mild hypotheses, the dictionary learning problem is locally well-posed: the desired solution is indeed a local minimum of the $\ell^1$ norm. Namely, if $\mb A \in \Re^{m \times n}$ is an incoherent (and possibly overcomplete) dictionary, and the coefficients $\mb X \in \Re^{n \times p}$ follow a random sparse model, then with high probability $(\mb A,\mb X)$ is a local minimum of the $\ell^1$ norm over the manifold of factorizations $(\mb A',\mb X')$ satisfying $\mb A' \mb X' = \mb Y$, provided the number of samples $p = \Omega(n^3 k)$. For overcomplete $\mb A$, this is the first result showing that the dictionary learning problem is locally solvable. Our analysis draws on tools developed for the problem of completing a low-rank matrix from a small subset of its entries, which allow us to overcome a number of technical obstacles; in particular, the absence of the restricted isometry property.

Affiliated with

Also associated with

No associations

Rating

On the Local Correctness of L^1 Minimization for Dictionary Learning does not yet have a rating. At this time, there are no reviews or comments for this scientific paper.

If you have personal experience with On the Local Correctness of L^1 Minimization for Dictionary Learning, we encourage you to share that experience with our LandOfFree.com community. Your opinion is very important and On the Local Correctness of L^1 Minimization for Dictionary Learning will most certainly appreciate the feedback.

Rate now

Comments { 0 }

Profile ID: LFWR-SCP-O-573156