Recurrent neural network is a powerful model that learns temporal patterns in sequential data. For a long time, it was believed that recurrent networks are difficult to train using simple optimizers, such as stochastic gradient descent, due to the so-called vanishing gradient problem. In this paper, we show that learning longer term patterns in real data, such as in natural language, is perfect...

In this paper we present an algorithm for automatic generation of fuzzy neural networks (FNN). Fuzzy neural networks are concept that integrates some features of the fuzzy logic and the artificial neural networks theory. Based on analysis of several different fuzzy neural networks models, uniform representation method is presented, and two basic types are identified: FNN based on perception fra...

Evolutionary training methods for Artificial Neural Networks can escape local minima. Thus, they are useful to train recurrent neural networks for short-term weather forecasting. However, these algorithms are not guaranteed to converge fast or even converge at all due to their stochastic nature. In this paper, we present an algorithm that uses implicit gradient information and is able to train ...

Recent advances in sequential data modeling have suggested a class of models that combine recurrent neural networks with state space models. Despite the success, the huge model complexity has brought an important challenge to the corresponding inference methods. This paper introduces an structured inference algorithm to efficiently learn such models, including variants where the emission and tr...

runoff estimation is one of the main challenges encountered in water and watershed management. spatial and temporal changes of factors which influence runoff due to het-erogeneity of the basins explain the complicacy of relations. artificial neural network (ann) is one of the intelligence techniques which is flexible and doesn’t call for any much physically complex processes. these networks can...

This paper describes multidimensional neural preference classes and preference Moore machines as a principle for integrating different neural and/or symbolic knowledge sources. We relate neural preferences to multidimensional fuzzy set representations. Furthermore, we introduce neural preference Moore machines and relate traditional symbolic transducers with simple recurrent networks by using n...

We propose a novel recurrent neural network model, where the hidden state hₜ is obtained by permuting vector elements of previous hₜ₋₁ and adding output learned function β(xₜ) input xₜ at time t. In our prediction given second function, which applied to s(hₜ). The method easy implement, extremely efficient, does not suffer from vanishing nor exploding gradients. an extensive set experiments, sh...

Graph neural networks (GNNs) model nonlinear representations in graph data with applications distributed agent coordination, control, and planning among others. Current GNN architectures assume ideal scenarios ignore link fluctuations that occur due to environment, human factors, or external attacks. In these situations, the fails address its task if topological randomness is not considered acc...