Sparsity vs. Statistical Independence in Adaptive Signal Representations: A Case Study of the Spike Process

Details Sparsity vs. Statistical Independence in Adaptive Signal Representations: A Case Study of the Spike Process Sparsity vs. Statistical Independence in Adaptive Signal Representations: A Case Study of the Spike Process

2001-04-06

arxiv.org/abs/math/0104083v1

Mathematics

Numerical Analysis

39 pages, 7 figures, submitted to Annals of the Institute of Statistical Mathematics

Scientific paper

Finding a basis/coordinate system that can efficiently represent an input data stream by viewing them as realizations of a stochastic process is of tremendous importance in many fields including data compression and computational neuroscience. Two popular measures of such efficiency of a basis are sparsity (measured by the expected $\ell^p$ norm, $0 < p \leq 1$) and statistical independence (measured by the mutual information). Gaining deeper understanding of their intricate relationship, however, remains elusive. Therefore, we chose to study a simple synthetic stochastic process called the spike process, which puts a unit impulse at a random location in an $n$-dimensional vector for each realization. For this process, we obtained the following results: 1) The standard basis is the best both in terms of sparsity and statistical independence if $n \geq 5$ and the search of basis is restricted within all possible orthonormal bases in $R^n$; 2) If we extend our basis search in all possible invertible linear transformations in $R^n$, then the best basis in statistical independence differs from the one in sparsity; 3) In either of the above, the best basis in statistical independence is not unique, and there even exist those which make the inputs completely dense; 4) There is no linear invertible transformation that achieves the true statistical independence for $n > 2$.

Affiliated with

Also associated with

No associations

Rating

Sparsity vs. Statistical Independence in Adaptive Signal Representations: A Case Study of the Spike Process does not yet have a rating. At this time, there are no reviews or comments for this scientific paper.

If you have personal experience with Sparsity vs. Statistical Independence in Adaptive Signal Representations: A Case Study of the Spike Process, we encourage you to share that experience with our LandOfFree.com community. Your opinion is very important and Sparsity vs. Statistical Independence in Adaptive Signal Representations: A Case Study of the Spike Process will most certainly appreciate the feedback.

Rate now

Comments { 0 }

Profile ID: LFWR-SCP-O-201323