In this paper, we explore the possibility of leveraging Residual Networks (ResNet), a powerful structure in constructing extremely deep neural network for image understanding, to improve recurrent neural networks (RNN) for modeling sequential data. We show that for sequence classification tasks, incorporating residual connections into recurrent structures yields similar accuracy to Long Short T...

Recent studies show that state-space dynamics of randomly initialized recurrent neural network (RNN) has interesting and potentially useful properties even without training. More precisely, when initializing RNN with small weights, recurrent unit activities reflect history of inputs presented to the network according to the Markovian scheme. This property of RNN is called Markovian architectura...

Clinical measurements that can be represented as time series constitute an important fraction of the electronic health records and are often both uncertain and incomplete. Recurrent neural networks are a special class of neural networks that are particularly suitable to process time series data but, in their original formulation, cannot explicitly deal with missing data. In this paper, we explo...

This paper presents a novel approach to recurrent neural network (RNN) regularization. Differently from the widely adopted dropout method, which is applied to forward connections of feed-forward architectures or RNNs, we propose to drop neurons directly in recurrent connections in a way that does not cause loss of long-term memory. Our approach is as easy to implement and apply as the regular f...

This paper presents a deep neural solver to automatically solve math word problems. In contrast to previous statistical learning approaches, we directly translate math word problems to equation templates using a recurrent neural network (RNN) model, without sophisticated feature engineering. We further design a hybrid model that combines the RNN model and a similarity-based retrieval model to a...

Although GNSS/IMU integration has been studied for decades, an efficient estimator of their integration has remained a challenge. In the statistical approaches, the observation model of sensors and distribution of data must be known beforehand. This paper proposes a deep learning based approach to integrate GPS and reduced IMU information. In contrast to statistical approaches, our approach lea...

Pulse compression technique combines the high energy characteristic of a longer pulse width with the high resolution characteristic of a narrower pulse width. The major aspects that are considered for a pulse compression technique are signal to sidelobe ratio (SSR), noise and Doppler shift performances. The traditional algorithms like autocorrelation function (ACF), recursive least square (RLS)...

In this paper, we present a novel architecture for audio chord estimation using a hybrid recurrent neural network. The architecture replaces hidden Markov models (HMMs) with recurrent neural network (RNN) based language models for modelling temporal dependencies between chords. We demonstrate the ability of feed forward deep neural networks (DNNs) to learn discriminative features directly from ...

In this paper, we explore different ways to extend a recurrent neural network (RNN) to a deep RNN. We start by arguing that the concept of depth in an RNN is not as clear as it is in feedforward neural networks. By carefully analyzing and understanding the architecture of an RNN, however, we find three points of an RNN which may be made deeper; (1) input-to-hidden function, (2) hidden-tohidden ...

The paper focuses on the experimental identification and validation of recurrent neural network (RNN) models for air-fuel ratio (AFR) estimation and control in spark-ignited engines. Suited training procedures and experimental tests are proposed to improve RNN precision and generalization in predicting AFR transients for a wide range of operating scenarios. The reference engine has been tested ...