Computer Science – Information Theory
Scientific paper
2011-11-28
Computer Science
Information Theory
Manuscript (10 pages, 3 figures, 4 tables) Supplemental (6 pages, 5 figures) Version 2 (Corrected a label in Table 4)
Scientific paper
Information theory is widely accepted as a powerful tool for analyzing complex systems and it has been applied in many disciplines. Recently, some central components of information theory, multivariate information measures, have found expanded use in the study of several phenomena. Despite this widespread use, there is disagreement regarding the interpretation and use of these information measures. Due to the broad use of multivariate information measures, this problem prevents progress in many areas of study. Here, we seek to bring clarity to the situation by comparing the results from many proposed multivariate information measures for a simple system: Boolean logic gates. These logic gates represent the building blocks of computation and are well known across many disciplines. We find that a recently introduced measure, the partial information decomposition, provides the most complete description of the interactions present in the logic gates under examination. In addition, we apply the multivariate information measures to a dynamic system: a back-propagation network designed to learn the logic gates. Using the partial information decomposition, we find that logic gates which possess higher levels of synergy require more time for a back-propagation network to learn. Conversely, we find that logic gates which possess higher level of redundancy require less time for a back-propagation network to learn. This relationship was obscured when using the previously proposed information measures. Finally, we have made software available online which allows the user to calculate all of the information measures discussed within this paper, as well as software that can be used to create the back-propagation networks discussed herein.
Alford Wesley
Beggs John M.
Flecker Benjamin
Timme Nicholas
No associations
LandOfFree
Towards resolving the multivariate information controversy does not yet have a rating. At this time, there are no reviews or comments for this scientific paper.
If you have personal experience with Towards resolving the multivariate information controversy, we encourage you to share that experience with our LandOfFree.com community. Your opinion is very important and Towards resolving the multivariate information controversy will most certainly appreciate the feedback.
Profile ID: LFWR-SCP-O-68519