Convolutional neural networks excel in image recognition tasks, but this comes at the cost of high computational and memory complexity. To tackle this problem, [1] developed a tensor factorization framework to compress fully-connected layers. In this paper, we focus on compressing convolutional layers. We show that while the direct application of the tensor framework [1] to the 4-dimensional ke...

The dominant approach for many NLP tasks are recurrent neural networks, in particular LSTMs, and convolutional neural networks. However, these architectures are rather shallow in comparison to the deep convolutional networks which are very successful in computer vision. We present a new architecture for text processing which operates directly on the character level and uses only small convoluti...

Convolutional neural networks (CNNs) are a popular and highly performant choice for pixel-wise dense prediction or generation. One of the commonly required components in such CNNs is a way to increase the resolution of the network’s input. The lower resolution inputs can be, for example, low-dimensional noise vectors in image generation [7] or low resolution (LR) feature maps for network vis...

Machine learning techniques have become increasingly popular in the field of resting state fMRI (functional magnetic resonance imaging) network based classification. However, the application of convolutional networks has been proposed only very recently and has remained largely unexplored. In this paper we describe a convolutional neural network architecture for functional connectome classifica...

Model based iterative reconstruction (MBIR) algorithms for low-dose X-ray CT are computationally expensive. To address this problem, we recently proposed the world-first deep convolutional neural network (CNN) for low-dose X-ray CT and won the second place in 2016 AAPM Low-Dose CT Grand Challenge. However, some of the texture were not fully recovered. To cope with this problem, here we propose ...

In deep neural networks with convolutional layers, each layer typically has fixed-size/single-resolution receptive field (RF). Convolutional layers with a large RF capture global information from the input features, while layers with small RF size capture local details with high resolution from the input features. In this work, we introduce novel deep multi-resolution fully convolutional neural...

Motivated by the increasing interest in powerful short channel codes for low-latency ultra-reliable communications, we analyze the performance of tail-biting convolutional codes with different memories, block lengths and code rates over the additive white Gaussian noise channel. The analysis is carried out both through Monte Carlo simulations and by upper bounding the error probability via Polt...

Performance of automated tissue classification in medical imaging depends on the choice of descriptive features. In this paper, we show how restricted Boltzmann machines (RBMs) can be used to learn features that are especially suited for texture-based tissue classification. We introduce the convolutional classification RBM, a combination of the existing convolutional RBM and classification RBM,...

For most deep learning practitioners, sequence modeling is synonymous with recurrent networks. Yet recent results indicate that convolutional architectures can outperform recurrent networks on tasks such as audio synthesis and machine translation. Given a new sequence modeling task or dataset, which architecture should one use? We conduct a systematic evaluation of generic convolutional and rec...

It is well known that a convolutional code can be viewed as a linear system over a nite eld. In this paper we develop this viewpoint for convolutional codes using several recent innovations from the systems theory literature. In particular we deene codes as behaviors of a set of compact support time trajectories over a vector space. We also consider several diierent representations of codes, in...