(Input) Size Matters for CNN Classifiers

Fully convolutional neural networks (CNNs) can process input of arbitrary size by applying a combination downsampling and pooling. However, we find that fully image classifiers are not agnostic to the but rather show significant differences in performance: presenting same at different scales result outcomes. A closer look reveals there is no simple relationship between model performance (no ‘bigger better’), each network has preferred size, for which it shows best results. We investigate this phenomenon methods, including spectral analysis layer activations probe classifiers, showing characteristic features depending on architecture. From discriminatory critically influencing how inference distributed among layers. Based these findings able derive basic design guidelines optimizing architectures specific datasets.