Estimating the entropy of binary time series: Methodology, some theory and a simulation study

Details Estimating the entropy of binary time series: Methodology, some theory and a simulation study Estimating the entropy of binary time series: Methodology, some theory and a simulation study

2008-02-29

arxiv.org/abs/0802.4363v1

Computer Science

Information Theory

34 pages, 3 figures

Scientific paper

Partly motivated by entropy-estimation problems in neuroscience, we present a detailed and extensive comparison between some of the most popular and effective entropy estimation methods used in practice: The plug-in method, four different estimators based on the Lempel-Ziv (LZ) family of data compression algorithms, an estimator based on the Context-Tree Weighting (CTW) method, and the renewal entropy estimator. **Methodology. Three new entropy estimators are introduced. For two of the four LZ-based estimators, a bootstrap procedure is described for evaluating their standard error, and a practical rule of thumb is heuristically derived for selecting the values of their parameters. ** Theory. We prove that, unlike their earlier versions, the two new LZ-based estimators are consistent for every finite-valued, stationary and ergodic process. An effective method is derived for the accurate approximation of the entropy rate of a finite-state HMM with known distribution. Heuristic calculations are presented and approximate formulas are derived for evaluating the bias and the standard error of each estimator. ** Simulation. All estimators are applied to a wide range of data generated by numerous different processes with varying degrees of dependence and memory. Some conclusions drawn from these experiments include: (i) For all estimators considered, the main source of error is the bias. (ii) The CTW method is repeatedly and consistently seen to provide the most accurate results. (iii) The performance of the LZ-based estimators is often comparable to that of the plug-in method. (iv) The main drawback of the plug-in method is its computational inefficiency.

Affiliated with

Also associated with

No associations

Rating

Estimating the entropy of binary time series: Methodology, some theory and a simulation study does not yet have a rating. At this time, there are no reviews or comments for this scientific paper.

If you have personal experience with Estimating the entropy of binary time series: Methodology, some theory and a simulation study, we encourage you to share that experience with our LandOfFree.com community. Your opinion is very important and Estimating the entropy of binary time series: Methodology, some theory and a simulation study will most certainly appreciate the feedback.

Rate now

Comments { 0 }

Profile ID: LFWR-SCP-O-352949