Self-paced Weight Consolidation for Continual Learning

Continual learning algorithms which keep the parameters of new tasks close to that previous tasks, are popular in preventing catastrophic forgetting sequential task settings. However, 1) performance for continual learner will be degraded without distinguishing contributions previously learned tasks; 2) computational cost greatly increased with number since most existing need regularize all when tasks. To address above challenges, we propose a self-paced Weight Consolidation (spWC) framework attain robust via evaluating discriminative specific, develop regularization reflect priorities past measuring difficulty based on key indicator ( i.e ., accuracy). When encountering task, sorted from “difficult” “easy” priorities. Then selectively maintaining knowledge amongst more difficult could well overcome less cost. We adopt an alternative convex search iteratively update model and priority weights bi-convex formulation. The proposed spWC is plug-and-play, applicable xmlns:xlink="http://www.w3.org/1999/xlink">e.g EWC, MAS RCIL) different directions classification segmentation). Experimental results several public benchmark datasets demonstrate our can effectively improve compared other algorithms.