On entropy for mixtures of discrete and continuous variables

Details On entropy for mixtures of discrete and continuous variables On entropy for mixtures of discrete and continuous variables

2006-07-14

arxiv.org/abs/cs/0607075v2

Computer Science

Information Theory

10 pages, two-column

Scientific paper

Let $X$ be a discrete random variable with support $S$ and $f : S \to S^\prime$ be a bijection. Then it is well-known that the entropy of $X$ is the same as the entropy of $f(X)$. This entropy preservation property has been well-utilized to establish non-trivial properties of discrete stochastic processes, e.g. queuing process \cite{prg03}. Entropy as well as entropy preservation is well-defined only in the context of purely discrete or continuous random variables. However for a mixture of discrete and continuous random variables, which arise in many interesting situations, the notions of entropy and entropy preservation have not been well understood. In this paper, we extend the notion of entropy in a natural manner for a mixed-pair random variable, a pair of random variables with one discrete and the other continuous. Our extensions are consistent with the existing definitions of entropy in the sense that there exist natural injections from discrete or continuous random variables into mixed-pair random variables such that their entropy remains the same. This extension of entropy allows us to obtain sufficient conditions for entropy preservation in mixtures of discrete and continuous random variables under bijections. The extended definition of entropy leads to an entropy rate for continuous time Markov chains. As an application, we recover a known probabilistic result related to Poisson process. We strongly believe that the frame-work developed in this paper can be useful in establishing probabilistic properties of complex processes, such as load balancing systems, queuing network, caching algorithms.

Affiliated with

Also associated with

No associations

Rating

On entropy for mixtures of discrete and continuous variables does not yet have a rating. At this time, there are no reviews or comments for this scientific paper.

If you have personal experience with On entropy for mixtures of discrete and continuous variables, we encourage you to share that experience with our LandOfFree.com community. Your opinion is very important and On entropy for mixtures of discrete and continuous variables will most certainly appreciate the feedback.

Rate now

Comments { 0 }

Profile ID: LFWR-SCP-O-331227