We extend our earlier work on deep-structured conditional random field (DCRF) and develop deep-structured hidden conditional random field (DHCRF). We investigate the use of this new sequential deep-learning model for phonetic recognition. DHCRF is a hierarchical model in which the final layer is a hidden conditional random field (HCRF) and the intermediate layers are zero-th-order conditional r...

The multilayer perceptron (MLP) is a well established neural network model for supervised learning problems. Furthermore, it is well known that its performance for a given problem depends crucially on appropriately selecting the MLP architecture, which is typically achieved using cross-validation. In this work, we propose an incremental Bayesian methodology to address the important problem of a...

The recurrent neural network (RNN) is appropriate for dealing with temporal sequences. In this paper, we present a deep RNN with new features and apply it for online handwritten Chinese character recognition. Compared with the existing RNN models, three innovations are involved in the proposed system. First, a new hidden layer function for RNN is proposed for learning temporal information bette...

Feed forward neural networks (FFNN) with an unconstrained random number of hidden neurons deene exible non-parametric regression models. In M uller and Rios Insua (1998) we have argued that variable architecture models with random size hidden layer signiicantly reduce posterior mul-timodality typical for posterior distributions in neural network models. In this chapter we review the model propo...

a local scouring phenomenon is one of the important problems in hydraulic design of groynes. due to constriction and downward flow, the scouring can occur around the groynes. nowadays, the artificial neural networks have a lot of applications in various water engineering problems where there is not any specific relation between effective parameters. in this study, the artificial neural networks...

Abstract. Artificial neural networks can be used to solve inverse problems. One relevant problem in hydrologic optics is the estimatation of the single scattering albedo from the emitted surface radiation. The multi-layer perceptron (MLP) can be applied to determine the albedo from the measured radiation. The MLP is designed with one hidden layer, where the activation employs the sigmoid functi...

Some novel learning strategies based on set covering in Hamming geometrical space are presented and proved, which are related to the three-layer Boolean neural network (BNN) for implementing an arbitrary Boolean function with lowcomplexity. Each hidden neuron memorizes a set of learning patterns, then the output layer combines these hidden neurons for explicit output as a Boolean function. The ...

Despite their great success, there is still no comprehensive theoretical understanding of learning with Deep Neural Networks (DNNs) or their inner organization. Previous work [Tishby and Zaslavsky (2015)] proposed to analyze DNNs in the Information Plane; i.e., the plane of the Mutual Information values that each layer preserves on the input and output variables. They suggested that the goal of...

Artificial neural networks have been showed their effectiveness in many real world problems such as signal processing, pattern recognition, and classification problems. Although they provide highly generalized solutions, we find several unanswered problems in using artificial neural networks. Determining the most appropriate architecture of artificial neural network is identified as one of thos...