This paper explores the possibility of using multiplicative gate to build two recurrent neural network structures. These two structures are called Deep Simple Gated Unit (DSGU) and Simple Gated Unit (SGU), which are structures for learning long-term dependencies. Compared to traditional Long Short-Term Memory (LSTM) and Gated Recurrent Unit (GRU), both structures require fewer parameters and le...

The recent success of Deep Neural Networks (DNNs) has drastically improved the state of the art for many application domains. While achieving high accuracy performance, deploying state-of-the-art DNNs is a challenge since they typically require billions of expensive arithmetic computations. In addition, DNNs are typically deployed in ensemble to boost accuracy performance, which further exacerb...

Online social networks like Instagram are places for communication. Also, these media produce rich metadata which are useful for further analysis in many fields including health and cognitive science. Many researchers are using these metadata like hashtags, images, etc. to detect patterns of user activities. However, there are several serious ambiguities like how much reliable are these informa...

We use deep feedforward artificial neural networks to approximate solutions of partial differential equations of advection and diffusion type in complex geometries. We derive analytical expressions of the gradients of the cost function with respect to the network parameters, as well as the gradient of the network itself with respect to the input, for arbitrarily deep networks. The method is bas...

Deep Neural Networks are built to generalize outside of training set in mind by using techniques such as regularization, early stopping and dropout. But considerations to make them more resilient to adversarial examples are rarely taken. As deep neural networks become more prevalent in mission critical and real time systems, miscreants start to attack them by intentionally making deep neural ne...

This project aims to build an accurate, smallfootprint, low-latency Speech Command Recognition system that is capable of detecting predefined keywords. Using the Speech Commands Dataset provided by Google’s TensorFlow and AIY teams, we have implemented different architectures using different machine learning algorithms. Our models include: Vanilla Single-Layer softmax model, Deep Neural Network...

On account of its many successes in inference tasks and imaging applications, Dictionary Learning (DL) related sparse optimization problems have garnered a lot research interest. In DL area, most solutions are focused on single-layer dictionaries, whose reliance handcrafted features achieves somewhat limited performance. With the rapid development deep learning, improved methods called Deep (DD...