Storage capacity in symmetric binary perceptrons

Asymptotics of Input-Constrained Binary Symmetric Channel Capacity

We study the classical problem of noisy constrained capacity in the case of the binary symmetric channel (BSC), namely, the capacity of a BSC whose inputs are sequences chosen from a constrained set. Motivated by a result of Ordentlich and Weissman [In Proceedings of IEEE Information Theory Workshop (2004) 117–122], we derive an asymptotic formula (when the noise parameter is small) for the ent...

On the capacity of the binary-symmetric parallel-relay network

We investigate the binary-symmetric parallel-relay network where there is one source, one destination, and multiple relays in parallel. We show that forwarding relays, where the relays merely transmit their received signals, achieve the capacity in two ways: with coded transmission at the source and a finite number of relays, or uncoded transmission at the source and a sufficiently large number...

Capacity Bounds and Mapping Design for Binary Symmetric Relay Channels

Capacity bounds for a three-node binary symmetric relay channel with orthogonal components at the destination are studied. The cut-set upper bound and the rates achievable using decode-and-forward (DF), partial DF and compress-and-forward (CF) relaying are first evaluated. Then relaying strategies with finite memory-length are considered. An efficient algorithm for optimizing the relay function...

Storage capacity of the reversed wedge perceptron with binary connections

Storage Capacity of a Multilayer Neural Network with Binary Weights

Statistical mechanics is applied to estimate the maximal capacity per weight (a3 of a two-layer feed-forward network with binary weights, functioning as a parity machine of the hidden units. For K 2 2 hidden units, the maximal theoretical capacity is achieved, a, = 1, and the average overlap between different solutions is zero. These results agree with the simulations. At finite temperature one...