2 Details of the pathological synthetic problems 3 2.1 The addition, multiplication, and XOR problem . . . . . . . . . . . . 3 2.2 The temporal order problem . . . . . . . . . . . . . . . . . . . . . . 4 2.3 The 3-bit temporal order problem . . . . . . . . . . . . . . . . . . . . 4 2.4 The random permutation problem . . . . . . . . . . . . . . . . . . . 4 2.5 Noiseless memorization . . . . . . ...

It can be difficult to tell whether a trained generative model has learned to generate novel examples or has simply memorized a specific set of outputs. In published work, it is common to attempt to address this visually, for example by displaying a generated example and its nearest neighbor(s) in the training set (in, for example, the L metric). As any generative model induces a probability de...

We propose an approach to address two undesired properties of unsupervised GANs. First, since GANs use only a continuous latent distribution to embed multiple classes or clusters of a dataset, GANs often do not correctly handle the structural discontinuity between disparate classes in a latent space. Second, discriminators of GANs easily forget about past generated samples by generators, incurr...

Abstract: The roles played by learning and memorization represent an important topic in deep learning research. Recent work on this subject has shown that the optimization behavior of DNNs trained on shuffled labels is qualitatively different from DNNs trained with real labels. Here, we propose a novel permutation approach that can differentiate memorization from learning in deep neural network...

Deep Neural Networks (DNNs) are increasingly being used in software engineering and code intelligence tasks. These powerful tools that capable of learning highly generalizable patterns from large datasets through millions parameters. At the same time, their capacity can render them prone to memorizing data points. Recent work suggests memorization risk manifests especially strongly when trainin...

Recent studies show that the effect of digital tools have a direct impact on atten-tion, motivation, autonomy and academic success. In Morocco, considering extent to which students use tools, precisely scientific students, be-cause essence memory all learning, this article is part overall framework study attempts find link between process memorization. The factors can influence memorization are...