Limitation of capsule networks

A recently proposed method in deep learning groups multiple neurons to capsules such that each capsule represents an object or part of object. Routing algorithms route the output from lower-level layers upper-level layers. In this paper, we prove state-of-the-art routing procedures decrease expressivity networks. More precisely, it is shown EM-routing and routing-by-agreement prevent networks distinguishing inputs their negative counterpart. Therefore, only symmetric functions can be expressed by networks, concluded they are not universal approximators. We also theoretically motivate empirically show limitation affects training negatively. present incremental improvement for solves aforementioned stabilizes