Homeostasis-Inspired Continual Learning: Learning to Control Structural Regularization

Learning continually without forgetting might be one of the ultimate goals for building artificial intelligence (AI). However, unless there are enough resources equipped, knowledge acquired in past is inevitable. Then, we can naturally pose a fundamental question about how to control what and much it forget improve overall accuracy. To give clear answer it, propose novel trainable network termed homeostatic meta-model . The proposed neuromorphic framework natural extension conventional concept Synaptic Plasticity (SP) further optimizing accuracy continual learning. In preceding works on SP its variations, though they seek important parameters structural regularization, care less intensity regularization (IoR). Per contra, this work reveals that careful selection IoR during training remarkably tasks. method balances between newly learned previously-acquired ones rather than biasing specific task or evenly balancing. obtain effective optimal IoRs real-time learning circumstances, homeostasis-inspired meta architecture. automatically controls by capturing from previous tasks current direction. We provide experimental results considering various types showing notably outperforms methods terms forgetting. also show relatively stable robust compared existing SP-based methods. Furthermore, generated our model interestingly appears proactively controlled within range, which resembles negative feedback mechanism homeostasis synapses.