Single-iteration Threshold Hamming

'I' , :, , t The similarity subnet uses mn connections and performs a single iteration. The WTA subnet has m2 connections. With randomly generated input and memory patterns, it converges in 8(m In(mn)) iterations (Floreen 1991). Since m is exponential in n, the space and time complexity of the network is primarily due to the WTA subnet (Domany &; Orland 1987). We analyze the performance of the HN in the practical scenario where the input pattern is a distorted version of some stored memory vector. We show that it is possible to replace the original activation function of the neurons in the memory layer by a simple threshold function, and completely discard the WTA subnet. If the threshold is properly tuned, only the neuron standing for the 'correct' memory is likely to be activated. The resulting Threshold Hamming Network (THN) will perform correctly (with probability approaching 1) in a single iteration, using only O( m In m) connections instead of the O(m2) connections in the original HN. We identify the optimal threshold, and measure its performance relative to the original HN. 2 The Threshold Hamming Network We examine a HN storing m + 1 memory patterns £.", 1 ~ p ~ m + 1, each being an n-dimensional vector of :i:l. The input pattern z is generated by selecting some memory pattern £." (w.l.g., £.m+l), and letting each bit Zi be either £.r or -£.r with probabilities a and (1 a) respectively, where a > 0.5. To analyze this HN, we use some tight approximations to the binomial distribution. Due to space considerations, their proofs are omitted. Lemma 1. Let X'"'" Bin(n, p). If Zn are integers such that liTnn"'~~ = fJ E (p, 1), then 1-p ,8 1-,8 P(X ~ zn) ~ 1')_Q/1 Q\ exp{ -n(,8ln + (1 -.8) In 1 ]} (1) (1J)y21rnfJ(l-fJ) p -p . in the sense that the ratio between LHS and RHS converges to 1 88 n 00. For the special case p = !' let G (fJ) = In 2 + fJ In fJ + (1 fJ) In( 1 fJ), then Lemma 2. Let Xi '" Bin( n, !) be independent,