The Potential Benefits of Data Set Filtering and Learning Algorithm Hyperparameter Optimization
نویسندگان
چکیده
The quality of a model induced by a learning algorithm is dependent upon the training data and the hyperparameters supplied to the learning algorithm. Prior work has shown that a model’s quality can be significantly improved by filtering out low quality instances or by tuning the learning algorithm hyperparameters. The potential impact of filtering and hyperparameter optimization (HPO) is largely unknown. In this paper, we estimate the potential benefits of instance filtering and HPO. While both HPO and filtering significantly improve the quality of the induced model, we find that filtering has a greater potential effect on the quality of the induced model than HPO, motivating future work in filtering.
منابع مشابه
Two-Stage Transfer Surrogate Model for Automatic Hyperparameter Optimization
The choice of hyperparameters and the selection of algorithms is a crucial part in machine learning. Bayesian optimization methods have been used very successfully to tune hyperparameters automatically, in many cases even being able to outperform the human expert. Recently, these techniques have been massively improved by using metaknowledge. The idea is to use knowledge of the performance of a...
متن کاملUsing Machine Learning to Improve Stochastic Optimization
In many stochastic optimization algorithms there is a hyperparameter that controls how the next sampling distribution is determined from the current data set of samples of the objective function. This hyperparameter controls the exploration/exploitation trade-off of the next sample. Typically heuristic “rules of thumb” are used to set that hyperparameter, e.g., a pre-fixed annealing schedule. W...
متن کاملHyperparameter Optimization and Boosting for Classifying Facial Expressions: How good can a "Null" Model be?
One of the goals of the ICML workshop on representation and learning is to establish benchmark scores for a new data set of labeled facial expressions. This paper presents the performance of a “Null model” consisting of convolutions with random weights, PCA, pooling, normalization, and a linear readout. Our approach focused on hyperparameter optimization rather than novel model components. On t...
متن کاملThe Potential Benefits of Filtering Versus Hyper-Parameter Optimization
The quality of an induced model by a learning algorithm is dependent on the quality of the training data and the hyper-parameters supplied to the learning algorithm. Prior work has shown that improving the quality of the training data (i.e., by removing low quality instances) or tuning the learning algorithm hyper-parameters can significantly improve the quality of an induced model. A compariso...
متن کاملHyperband: Bandit-based Configuration Eval- Uation for Hyperparameter Optimization
Performance of machine learning algorithms depends critically on identifying a good set of hyperparameters. While recent approaches use Bayesian Optimization to adaptively select configurations, we focus on speeding up random search through adaptive resource allocation. We present HYPERBAND, a novel algorithm for hyperparameter optimization that is simple, flexible, and theoretically sound. HYP...
متن کامل