Sparse Coding with Invariance Constraints

We suggest a new approach to optimize the learning of sparse features under the constraints of explicit transformation symmetries imposed on the set of feature vectors. Given a set of basis feature vectors and invariance transformations, from each basis feature a family of transformed features is generated. We then optimize the basis features for optimal sparse reconstruction of the input pattern ensemble using the whole transformed feature family. If the predefined transformation invariance coincides with an invariance in the input data, we obtain a less redundant basis feature set, compared to sparse coding approaches without invariances. We demonstrate the application to a test scenario of overlapping bars and the learning of receptive fields in hierarchical visual cortex models.