Information Storage Capacity

The storage of information in biological memory relies on changes in neuronal circuits, termed plasticity. Synaptic contributions to plasticity, which are an important component, may be divided into changes in existing synapses, and changes in interneuronal connectivity through formation and elimination of synapses. Interneuronal connectivity changes may be further divided into contributions associated with dendritic spines without major remodeling of dendritic or axonal arbors, and contributions associated with major arbor remodeling [1]. An important step in understanding brain function is to quantify the information storage capacity of a neural system. Investigating an information theoretic way of defining the information capacity of a storage device, one finds different definitions in the literature. One way of defining the information capacity is to treat the recorder channel as an ordinary communications channel, e.g. an additive white Gaussian noise channel, and to consider the information capacity as the ordinary channel capacity [2]. Another way of defining the information capacity of a memory is as the logarithm of the number of cases it can distinguish, without explicit consideration of noise [3]. The difference between these two definitions is analogous to the difference between the definitions of Shannon entropy [4] and the Hartley information measure [5], thus we will refer to these two definitions as Shannon capacity and Hartley capacity, respectively. The quantity that we call Hartley capacity is closely related to MacKay’s concepts of structural and metrical information [6]-[7]. The three classes of synaptic plasticity occur on distinct time scales, synaptic strength change within a minute, dendritic spine formation on the order of tens of minutes, and branch remodeling on the order of days [1]. For a basic look at the overall storage capacity we consider the information storage capacity arising from the three classes of synaptic plasticity separately. First let us consider the information capacity associated with formation and elimination of dendritic spines, following the analysis of [1]. In [1], Stepanyants et al. estimate the number of different interneuronal connectivity patterns that can be formed through dendritic spine formation/elimination without major arbor remodeling and calculate the Hartley capacity. They estimate the number of potential synapses, NP, through a geometrical analysis of anatomy and the number of actual synapses, N, through anatomical measurement. These two values may be used to combinatorially determine the Hartley capacity. In particular the Hartley capacity is