Transfer Learning with Adaptive Regularizers
نویسندگان
چکیده
The success of regularized risk minimization approaches to classification with linear models depends crucially on the selection of a regularization term that matches with the learning task at hand. If the necessary domain expertise is rare or hard to formalize, it may be difficult to find a good regularizer. On the other hand, if plenty of related or similar data is available, it is a natural approach to adjust the regularizer for the new learning problem based on the characteristics of the related data. In this paper, we study the problem of obtaining good parameter values for a `2-style regularizer with feature weights. We analytically investigate a moment-based method to obtain good values and give uniform convergence bounds for the prediction error on the target learning task. An empirical study shows that the approach can improve predictive accuracy considerably in the application domain of text classification.
منابع مشابه
A Probe into Adaptive Transfer across Writing Contexts: A Case of an EGAP Class
In an effort to expand the disciplinary discussions on transfer in L2 writing and because most studies have focused on transfer as reuse and not as an adequate adaptation of writing knowledge in new contexts, the present study as the first of its kind aimed to explore the issue of adaptive transfer in an English for General Academic Purposes (EGAP) writing course. The study thus focused on type...
متن کاملEfficient Sparse Recovery via Adaptive Non-Convex Regularizers with Oracle Property
The main shortcoming of sparse recovery with a convex regularizer is that it is a biased estimator and therefore will result in a suboptimal performance in many cases. Recent studies have shown, both theoretically and empirically, that non-convex regularizer is able to overcome the biased estimation problem. Although multiple algorithms have been developed for sparse recovery with non-convex re...
متن کاملSmoothing Regularizers for Projective Basis Function Networks
Smoothing regularizers for radial basis functions have been studied extensively, but no general smoothing regularizers for projective basis functions (PBFs), such as the widely-used sigmoidal PBFs, have heretofore been proposed. We derive new classes of algebraically-simplemth-order smoothing regularizers for networks of projective basis functions f(W;x) = PNj=1 ujg xTvj + vj0 + u0; with genera...
متن کاملAn Adaptive Congestion Alleviating Protocol for Healthcare Applications in Wireless Body Sensor Networks: Learning Automata Approach
Wireless Body Sensor Networks (WBSNs) involve a convergence of biosensors, wireless communication and networks technologies. WBSN enables real-time healthcare services to users. Wireless sensors can be used to monitor patients’ physical conditions and transfer real time vital signs to the emergency center or individual doctors. Wireless networks are subject to more packet loss and congestion. T...
متن کاملOptimization Approaches for Learning with Low-rank Regularization
Low-rank modeling has a lot of important applications in machine learning, computer vision and social network analysis. As direct rank minimization is NP hard, many alternative choices have been proposed. In this survey, we first introduce optimization approaches for two popular methods on rank minimization, i.e., nuclear norm regularization and rank constraint. Nuclear norm is the tightest con...
متن کامل