We present a novel approach to generating photo-realistic images of a face with accurate lip sync, given an audio input. By using a recurrent neural network, we achieved mouth landmarks based on audio features. We exploited the power of conditional generative adversarial networks to produce highly-realistic face conditioned on a set of landmarks. These two networks together are capable of produ...

In this paper we show several approximation results for folding networks { a generalization of partial recurrent neural networks such that not only time sequences but arbitrary trees can serve as input: Any measurable function can be approximated in probability. Any continuous function can be approximated in the maximum norm on inputs with restricted height, but the resources necessarily increa...

Learning to recognize or predict sequences using long-term context has many applications. However, practical and theoretical problems are found in training recurrent neural networks to perform tasks in which input/output dependencies span long intervals. Starting from a mathematical analysis of the problem, we consider and compare alternative algorithms and architectures on tasks for which the ...

Sensory adaptation is a phenomenon in which neurons are affected not only by their immediate input but also by the sequence of preceding inputs. In visual cortex, for example, neurons shift their preferred orientation after exposure to an oriented stimulus. This adaptation is traditionally attributed to plasticity. We show that a recurrent network generates tuning curve shifts observed in cat a...

2000 MSC: 92B20 34C23 37C27 93C80 Keywords: Recurrent neural networks Distributed delays Hopf bifurcation Frequency domain Generalized Nyquist stability criterion a b s t r a c t In this paper, a class of recurrent neural networks with distributed delays and a strong kernel is studied. It is shown that the Hopf bifurcation occurs as the bifurcation parameter, the mean delay, passes a critical v...

Sentence processing takes place in real-time. Previous words in the sentence can influence the processing of the current word in the timescale of hundreds of milliseconds. Recent neurophysiological studies in humans suggest that the fronto-striatal system (frontal cortex, and striatum--the major input locus of the basal ganglia) plays a crucial role in this process. The current research provide...

9 have long been used to model networks of interacting neurons (Er10 mentrout, 1998; PHASE PLANE ANALYSIS OF NEURAL NETS). The 11 activity of neuron i is represented by a single dynamical variable 12 xi, and its input-output characteristics by a single transfer function 13 f. There are more biophysically realistic descriptions of neural net14 works that include many dynamical variables per neur...

A method for learning phonetic features from speech data using connectionist networks is described. A temporal flow model is introduced in which sampled speech data flows through a parallel network from input to output units. The network uses hidden units with recurrent links to capture spectral/temporal characteristics of phonetic features. A supervised learning algorithm is presented which pe...

We consider problems of sequence processing and propose a solution based on a discrete state model in order to represent past context. We introduce a recurrent connectionist architecture having a modular structure that associates a subnetwork to each state. The model has a statistical interpretation we call Input/Output Hidden Markov Model (IOHMM). It can be trained by the EM or GEM algorithms,...