A Priori Estimation of Classification Circuit Complexity ∗

The paper aims at tight upper bounds for the size of pattern classification circuits that can be used for a priori parameter settings in a machine learning context. The upper bounds relate the circuit size S(C) to nL := log2 mL , where mL is the number of training samples. In particular, we show that there exist unbounded fanin threshold circuits with less than (a) SR cc := 2 · √ 2nL + 3 gates for unbounded depth, (b) SL cc := 34.8 · √ 2nL + 14 · nL − 11 · log2 nL + 2 gates for small bounded depth, where in both cases all mL samples are classified correctly. We note that the upper bounds do not depend on the length n of input (sample) vectors. Since nL << n in real-world problem settings, the upper bounds return values that are suitable for practical applications. We provide experimental evidence that the circuit size estimations work well on a number of pattern classification tasks. As a result, we formulate the conjecture that 1.25 · SR cc or 0.07 · SL cc gates are sufficient to achieve a high generalization rate of bounded-depth classification circuits.