City Research Online

The Identity of Information: How Deterministic Dependencies Constrain Information Synergy and Redundancy

Chicharro, D. ORCID: 0000-0002-4038-258X, Pica, G. and Panzeri, S. (2018). The Identity of Information: How Deterministic Dependencies Constrain Information Synergy and Redundancy. Entropy, 20(3), 169. doi: 10.3390/e20030169

Abstract

Understanding how different information sources together transmit information is crucial in many domains. For example, understanding the neural code requires characterizing how different neurons contribute unique, redundant, or synergistic pieces of information about sensory or behavioral variables. Williams and Beer (2010) proposed a partial information decomposition (PID) that separates the mutual information that a set of sources contains about a set of targets into nonnegative terms interpretable as these pieces. Quantifying redundancy requires assigning an identity to different information pieces, to assess when information is common across sources. Harder et al. (2013) proposed an identity axiom that imposes necessary conditions to quantify qualitatively common information. However, Bertschinger et al. (2012) showed that, in a counterexample with deterministic target-source dependencies, the identity axiom is incompatible with ensuring PID nonnegativity. Here, we study systematically the consequences of information identity criteria that assign identity based on associations between target and source variables resulting from deterministic dependencies. We show how these criteria are related to the identity axiom and to previously proposed redundancy measures, and we characterize how they lead to negative PID terms. This constitutes a further step to more explicitly address the role of information identity in the quantification of redundancy. The implications for studying neural coding are discussed.

Publication Type: Article
Additional Information: This is an open access article distributed under the Creative Commons Attribution License which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.
Publisher Keywords: information theory; mutual information decomposition; synergy; redundancy
Subjects: Q Science > QA Mathematics > QA75 Electronic computers. Computer science
R Medicine > RC Internal medicine > RC0321 Neuroscience. Biological psychiatry. Neuropsychiatry
Departments: School of Mathematics, Computer Science & Engineering > Computer Science
Date available in CRO: 09 Nov 2021 11:28
Date deposited: 9 November 2021
Date of acceptance: 28 February 2018
Date of first online publication: 5 March 2018
URI: https://openaccess.city.ac.uk/id/eprint/27078
[img]
Preview
Text - Published Version
Available under License Creative Commons: Attribution International Public License 4.0.

Download (1MB) | Preview

Export

Downloads

Downloads per month over past year

View more statistics

Actions (login required)

Admin Login Admin Login