Computer Science – Information Theory
Scientific paper
2010-05-13
Computer Science
Information Theory
Submitted to IEEE Transactions on Information Theory. 29 pages.
Scientific paper
The notion of typical sequences plays a key role in the theory of information. Central to the idea of typicality is that a sequence $x_1, x_2, ..., x_n$ that is $P_X$-typical should, loosely speaking, have an empirical distribution that is in some sense close to the distribution $P_X$. The two most common notions of typicality are that of strong (letter) typicality and weak (entropy) typicality. While weak typicality allows one to apply many arguments that can be made with strongly typical arguments, some arguments for strong typicality cannot be generalized to weak typicality. In this paper, we consider an alternate definition of typicality, namely one based on the weak* topology and that is applicable to Polish alphabets (which includes $\reals^n$). This notion is a generalization of strong typicality in the sense that it degenerates to strong typicality in the finite alphabet case, and can also be applied to mixed and continuous distributions. Furthermore, it is strong enough to prove a Markov lemma, and thus can be used to directly prove a more general class of results than weak typicality. As an example of this technique, we directly prove achievability for Gel'fand-Pinsker channels with input constraints for a large class of alphabets and channels without first proving a finite alphabet result and then resorting to delicate quantization arguments. While this large class does not include Gaussian distributions with power constraints, it is shown to be straightforward to recover this case by considering a sequence of truncated Gaussian distributions.
No associations
LandOfFree
Typical Sequences for Polish Alphabets does not yet have a rating. At this time, there are no reviews or comments for this scientific paper.
If you have personal experience with Typical Sequences for Polish Alphabets, we encourage you to share that experience with our LandOfFree.com community. Your opinion is very important and Typical Sequences for Polish Alphabets will most certainly appreciate the feedback.
Profile ID: LFWR-SCP-O-640641