Universal Estimation of Directed Information

Details Universal Estimation of Directed Information Universal Estimation of Directed Information

2012-01-11

arxiv.org/abs/1201.2334v1

Computer Science

Information Theory

Submitted to IEEE Transactions on Information Theory

Scientific paper

We propose four approaches to estimating the directed information rate between a pair of jointly stationary ergodic processes with the help of universal probability assignments. The four approaches yield estimators with different merits such as nonnegativity and boundedness. We establish consistency of these estimators in various senses and derive near-optimal rates of convergence in the minimax sense under mild conditions. The estimators carry over directly to estimating other information measures of stationary ergodic processes, such as entropy rate and mutual information rate, and provide alternatives to classical approaches in the existing literature. Guided by the theoretical results, we use context tree weighting as the vehicle for the implementations of the proposed estimators. Experiments on synthetic and real data are presented, demonstrating the potential of the proposed schemes in practice and the efficacy of directed information estimation as a tool for detecting and measuring causality and delay.

Affiliated with

Also associated with

No associations

Rating

Universal Estimation of Directed Information does not yet have a rating. At this time, there are no reviews or comments for this scientific paper.

If you have personal experience with Universal Estimation of Directed Information, we encourage you to share that experience with our LandOfFree.com community. Your opinion is very important and Universal Estimation of Directed Information will most certainly appreciate the feedback.

Rate now

Comments { 0 }

Profile ID: LFWR-SCP-O-604419