This paper explores the possibility of using multiplicative gate to build two recurrent neural network structures. These two structures are called Deep Simple Gated Unit (DSGU) and Simple Gated Unit (SGU), which are structures for learning long-term dependencies. Compared to traditional Long Short-Term Memory (LSTM) and Gated Recurrent Unit (GRU), both structures require fewer parameters and le...

An efficient algorithm for recurrent neural network training is presented. The approach increases the training speed for tasks where a length of the input sequence may vary significantly. The proposed approach is based on the optimal batch bucketing by input sequence length and data parallelization on multiple graphical processing units. The baseline training performance without sequence bucket...

A hybrid recurrent neural network is shown to learn small initial mealy machines (that can be thought of as translation machines translating input strings to corresponding output strings, as opposed to recognition automata that classify strings as either grammatical or nongrammatical) from positive training samples. A well-trained neural net 1 is then presented once again with the training set ...

This paper presents a novel Wiener-type recurrent neural network with the observer/Kalman filter identification (OKID) algorithm for unknown dynamic nonlinear system identification. The proposed Wiener-type recurrent network resembles the conventional Wiener model that consists of a dynamic linear subsystem cascaded with a static nonlinear subsystem. The novelties of our approach include: (1) t...

Nowadays, network attacks are the most crucial problem of modern society. All networks, from small to large, vulnerable threats. An intrusion detection (ID) system is critical for mitigating and identifying malicious threats in networks. Currently, deep learning (DL) machine (ML) being applied different domains, especially information security, developing effective ID systems. These systems cap...