Predictive information, multi-information, and binding information
نویسندگان
چکیده
We introduce an information theoretic measure of dependency between multiple random variables, called ‘binding information’ and compare it with several previously proposed measures of statistical complexity, including excess entropy, Crutchfield et al’s entropy of causal states, Bialek et al’s predictive information, Dubnov’s information rate, and the multiinformation. We discuss and prove some of the properties of the binding information, particularly in relation to the multi-information, and show that, in the case of sets of binary random variables, the processes which maximises binding information are the ‘parity’ processes. Finally we discuss some of the implications this has for the use of the binding information as a measure of complexity. Predictive information, multi-information, and binding information Samer Abdallah ([email protected]) Mark Plumbley ([email protected]) December 9, 2010 Abstract We introduce an information theoretic measure of dependency between multiple random variables, called ‘binding information’ and compare it with several previously proposed measures of statistical complexity, including excess entropy, Crutchfield et al’s entropy of causal states, Bialek et al’s predictive information, Dubnov’s information rate, and the multiinformation. We discuss and prove some of the properties of the binding information, particularly in relation to the multi-information, and show that, in the case of sets of binary random variables, the processes which maximises binding information are the ‘parity’ processes. Finally we discuss some of the implications this has for the use of the binding information as a measure of complexity.We introduce an information theoretic measure of dependency between multiple random variables, called ‘binding information’ and compare it with several previously proposed measures of statistical complexity, including excess entropy, Crutchfield et al’s entropy of causal states, Bialek et al’s predictive information, Dubnov’s information rate, and the multiinformation. We discuss and prove some of the properties of the binding information, particularly in relation to the multi-information, and show that, in the case of sets of binary random variables, the processes which maximises binding information are the ‘parity’ processes. Finally we discuss some of the implications this has for the use of the binding information as a measure of complexity.
منابع مشابه
Feature Selection Using Multi Objective Genetic Algorithm with Support Vector Machine
Different approaches have been proposed for feature selection to obtain suitable features subset among all features. These methods search feature space for feature subsets which satisfies some criteria or optimizes several objective functions. The objective functions are divided into two main groups: filter and wrapper methods. In filter methods, features subsets are selected due to some measu...
متن کاملA measure of statistical complexity based on predictive information
We introduce an information theoretic measure of statistical structure, called ‘binding information’, for sets of random variables, and compare it with several previously proposed measures including excess entropy, Bialek et al.’s predictive information, and the multi-information. We derive some of the properties of the binding information, particularly in relation to the multi-information, and...
متن کاملApplicability of BAT Model for Children Information Search Behavior in some Preschools in Tehran
Background and Aim: This study is set to represent information search process in the selected preschool children in Tehran (namely Mahgol and Taranom) and compare the results with BAT model. Methods: This is an Applied and comparative study with qualitative approach based on grounded theory. Research population was preschool children in Tehran from two different regions of city. Sampling was do...
متن کامل3D Classification of Urban Features Based on Integration of Structural and Spectral Information from UAV Imagery
Three-dimensional classification of urban features is one of the important tools for urban management and the basis of many analyzes in photogrammetry and remote sensing. Therefore, it is applied in many applications such as planning, urban management and disaster management. In this study, dense point clouds extracted from dense image matching is applied for classification in urban areas. Appl...
متن کاملDecoherence effects on quantum Fisher information of multi-qubit W states
Quantum fisher information of a parameter characterizing the sensitivity of a state with respect to parameter changes. In this paper, we study the quantum fisher information of the W state for four, five, six and seven particles in decoherence channels, such as amplitude damping, phase damping and depolarizing channel. Using Krauss operators for decoherence channels components, we investigate t...
متن کامل