Learning to Remember Rare Events
نویسندگان
چکیده
Despite recent advances, memory-augmented deep neural networks are still limited when it comes to life-long and one-shot learning, especially in remembering rare events. We present a large-scale life-long memory module for use in deep learning. The module exploits fast nearest-neighbor algorithms for efficiency and thus scales to large memory sizes. Except for the nearest-neighbor query, the module is fully differentiable and trained end-to-end with no extra supervision. It operates in a life-long manner, i.e., without the need to reset it during training. Our memory module can be easily added to any part of a supervised neural network. To show its versatility we add it to a number of networks, from simple convolutional ones tested on image classification to deep sequence-to-sequence and recurrent-convolutional models. In all cases, the enhanced network gains the ability to remember and do life-long one-shot learning. Our module remembers training examples shown many thousands of steps in the past and it can successfully generalize from them. We set new state-of-the-art for one-shot learning on the Omniglot dataset and demonstrate, for the first time, life-long one-shot learning in recurrent neural networks on a large-scale machine translation task.
منابع مشابه
Different Learning Levels in Multiple-choice and Essay Tests: Immediate and Delayed Retention
This study investigated the effects of different learning levels, including Remember an Instance (RI), Remember a Generality (RG), and Use a Generality (UG) in multiple-choice and essay tests on immediate and delayed retention. Three-hundred pre-intermediate students participated in the study. Reading passages with multiple-choice and essay questions in different levels of learning were giv...
متن کاملCarcinoma of the Breast Presenting As Deep Vein Thrombosis of the Axillary Vein
Background: Breast cancer is the most common type of malignancy in the female population, rarely is reported by thromboembolic events during the course of the disease. In this case, we present a rare case of breast cancer presenting as deep vein thrombosis of the axillary vein.Case Presentation: A 40-year- old female was admitted due to pain and swelling of her right arm and forearm of two mont...
متن کاملStatistical Monitoring of Rare Events in Healthcare
There are miscellaneous quality characteristics in healthcare which are interested to be monitored. However, monitoring each characteristic needs a special statistical method. There are some characteristics with very small incidence rates that it’s usually considered not to be necessary to monitoring them, since their incidence rates are so small that p and np charts are not able to monitor t...
متن کاملLearning to Predict Rare Events in Event Sequences
Learning to predict rare events from sequences of events with categorical features is an important, real-world, problem that existing statistical and machine learning methods are not well suited to solve. This paper describes timeweaver, a genetic algorithm based machine learning system that predicts rare events by identifying predictive temporal and sequential patterns. Timeweaver is applied t...
متن کاملEncoding of episodic information through fast task-irrelevant perceptual learning
The mechanisms guiding our learning and memory processes are of key interest to human cognition. While much research shows that attention and reinforcement processes help guide the encoding process, there is still much to know regarding how our brains choose what to remember. Recent research of task-irrelevant perceptual learning (TIPL) has found that information presented coincident with impor...
متن کاملذخیره در منابع من
با ذخیره ی این منبع در منابع من، دسترسی به آن را برای استفاده های بعدی آسان تر کنید
برای دانلود متن کامل این مقاله و بیش از 32 میلیون مقاله دیگر ابتدا ثبت نام کنید
ثبت ناماگر عضو سایت هستید لطفا وارد حساب کاربری خود شوید
ورودعنوان ژورنال:
- CoRR
دوره abs/1703.03129 شماره
صفحات -
تاریخ انتشار 2017