Large deep neural networks are powerful, but exhibit undesirable behaviors such as memorization and sensitivity to adversarial examples. In this work, we propose mixup, a simple learning principle to alleviate these issues. In essence, mixup trains a neural network on convex combinations of pairs of examples and their labels. By doing so, mixup regularizes the neural network to favor simple lin...

I argue that European schools of thought on memory and memorization were critical in enabling growth of the scientific method. After giving a historical overview of the development of the memory arts from ancient Greece through 17 th century Europe, I describe how the Baconian viewpoint on the scientific method was fundamentally part of a culture and a broader dialogue that conceived of memoriz...

Neural networks trained with backpropagation often struggle to identify classes that have been observed a small number of times. In applications where most class labels are rare, such as language modelling, this can become a performance bottleneck. One potential remedy is to augment the network with a fast-learning non-parametric model which stores recent activations and class labels into an ex...

In this chapter we present the formal language, stochastic process algebra (STOPA), to specify cognitive systems. In addition to the usual characteristics of these formalisms, this language features the possibility of including stochastic time. This kind of time is useful to represent systems where the delays are not controlled by fixed amounts of time, but are given by probability distribution...

We present a qualitative analysis of the (potentially erroneous) outputs contextualized embedding-based methods for detecting diachronic semantic change. First, we introduce an ensemble method outperforming previously described approaches. This is used as basis in-depth degrees change predicted English words across 5 decades. Our findings show that can often predict high scores which are not un...

Cognitive flexibility is defined as the ability to assimilate previously learned information and concepts to generate novel solutions to new problems. This skill is crucial for success within ill-structured domains such as biology, physiology, and medicine, where many concepts are simultaneously required for understanding a complex problem, yet the problem consists of patterns or combinations o...

The fruit fly Drosophila melanogaster has a sophisticated visual system and exhibits complex visual behaviors. Visual responses, vision processing and higher cognitive processes in Drosophila have been studied extensively. However, little is known about whether the retinal location of visual stimuli can affect fruit fly performance in various visual tasks. We tested the response of wild-type Be...

Researchers have proposed that immersion could have advantages for tasks involving abstract mental activities, such as conceptual learning; however, there are few empirical results that support this idea. We hypothesized that higher levels of immersion would benefit such tasks if the mental activity can be mapped to objects or locations in a 3D environment. To investigate this hypothesis, we pe...

We introduce an interactive visualization component for the JoBimText project. JoBimText is an open source platform for large-scale distributional semantics based on graph representations. First we describe the underlying technology for computing a distributional thesaurus on words using bipartite graphs of words and context features, and contextualizing the list of semantically similar words t...