Machine Learning and Non-Negative Compressive Sampling

The new emerging theory of compressive sampling demonstrates that by exploiting the structure of a signal, it is possible to sample a signal below the Nyquist rate—using random projections—and achieve perfect reconstruction. In this paper, we consider a special case of compressive sampling where the uncompressed signal is non-negative, and propose a number of sparse recovery algorithms—which utilise Non-negative Matrix Factorisation (NMF), Iteratively Reweighted Least Squares (IRLS) & Nonnegative Quadratic Programming (NQP)—for the recovery of minimum `p-norm solutions, 0 ≤ p ≤ 1. We examine the performance of NMF when applied to compressed data and discuss consequences for machine learning algorithms applied to random projections of the data to be analysed. As a necessary first step, we investigate the signal recovery performance of the proposed algorithms, and demonstrate that— for sufficiently sparse non-negative signals—the signals recovered by the sparse recovery algorithms and their least squares versions are essentially the same, which suggests that a non-negativity constraint is enough to recover sufficiently sparse signals. We build on our results and extend Non-negative Matrix Factorisation to compressively sampled data, where a sparse non-negative basis and corresponding non-negative coefficients for the original uncompressed matrix are learned indirectly in the compressed domain. Thus, demonstrating that machine learning algorithms that employ non-negative constraints can successfully learn uncompressed features from compressed data—by way of a simple modification of the generative model.