Simultaneous Codeword Optimization (SimCO) for Dictionary Update and Learning

Details Simultaneous Codeword Optimization (SimCO) for Dictionary Update and Learning Simultaneous Codeword Optimization (SimCO) for Dictionary Update and Learning

2011-09-24

arxiv.org/abs/1109.5302v3

Computer Science

Learning

13 pages

Scientific paper

We consider the data-driven dictionary learning problem. The goal is to seek an over-complete dictionary from which every training signal can be best approximated by a linear combination of only a few codewords. This task is often achieved by iteratively executing two operations: sparse coding and dictionary update. In the literature, there are two benchmark mechanisms to update a dictionary. The first approach, such as the MOD algorithm, is characterized by searching for the optimal codewords while fixing the sparse coefficients. In the second approach, represented by the K-SVD method, one codeword and the related sparse coefficients are simultaneously updated while all other codewords and coefficients remain unchanged. We propose a novel framework that generalizes the aforementioned two methods. The unique feature of our approach is that one can update an arbitrary set of codewords and the corresponding sparse coefficients simultaneously: when sparse coefficients are fixed, the underlying optimization problem is similar to that in the MOD algorithm; when only one codeword is selected for update, it can be proved that the proposed algorithm is equivalent to the K-SVD method; and more importantly, our method allows us to update all codewords and all sparse coefficients simultaneously, hence the term simultaneous codeword optimization (SimCO). Under the proposed framework, we design two algorithms, namely, primitive and regularized SimCO. We implement these two algorithms based on a simple gradient descent mechanism. Simulations are provided to demonstrate the performance of the proposed algorithms, as compared with two baseline algorithms MOD and K-SVD. Results show that regularized SimCO is particularly appealing in terms of both learning performance and running speed.

Affiliated with

Also associated with

No associations

Rating

Simultaneous Codeword Optimization (SimCO) for Dictionary Update and Learning does not yet have a rating. At this time, there are no reviews or comments for this scientific paper.

If you have personal experience with Simultaneous Codeword Optimization (SimCO) for Dictionary Update and Learning, we encourage you to share that experience with our LandOfFree.com community. Your opinion is very important and Simultaneous Codeword Optimization (SimCO) for Dictionary Update and Learning will most certainly appreciate the feedback.

Rate now

Comments { 0 }

Profile ID: LFWR-SCP-O-73919