A Comparative Study on Regularization Strategies for Embedding-based Neural Networks
نویسندگان
چکیده
This paper aims to compare different regularization strategies to address a common phenomenon, severe overfitting, in embedding-based neural networks for NLP. We chose two widely studied neural models and tasks as our testbed. We tried several frequently applied or newly proposed regularization strategies, including penalizing weights (embeddings excluded), penalizing embeddings, reembedding words, and dropout. We also emphasized on incremental hyperparameter tuning, and combining different regularizations. The results provide a picture on tuning hyperparameters for neural NLP models.
منابع مشابه
Link Prediction using Network Embedding based on Global Similarity
Background: The link prediction issue is one of the most widely used problems in complex network analysis. Link prediction requires knowing the background of previous link connections and combining them with available information. The link prediction local approaches with node structure objectives are fast in case of speed but are not accurate enough. On the other hand, the global link predicti...
متن کاملSteganalysis of embedding in difference of image pixel pairs by neural network
In this paper a steganalysis method is proposed for pixel value differencing method. This steganographic method, which has been immune against conventional attacks, performs the embedding in the difference of the values of pixel pairs. Therefore, the histogram of the differences of an embedded image is di_erent as compared with a cover image. A number of characteristics are identified in the di...
متن کاملUsing Methods Based on Neural Networks to Predict and Manage Diseases (A Case Study of Forecasting the Trend of Corona Disease)
Aim and background: Forecasting methods are used in various fields; one of the most important fields is the field of health systems. This study aimed to use the Artificial Neural Network (ANN) method in forecasting Corona patients in Iran. Method: The present study is descriptive and analytical of a comparative type that uses past information to predict the future, the time series of Corona in...
متن کاملSaturated Neural Adaptive Robust Output Feedback Control of Robot Manipulators:An Experimental Comparative Study
In this study, an observer-based tracking controller is proposed and evaluatedexperimentally to solve the trajectory tracking problem of robotic manipulators with the torque saturationin the presence of model uncertainties and external disturbances. In comparison with the state-of-the-artobserver-based controllers in the literature, this paper introduces a saturated observer-based controllerbas...
متن کاملPredictive Abilities of Bayesian Regularization and Levenberg–Marquardt Algorithms in Artificial Neural Networks: A Comparative Empirical Study on Social Data
The objective of this study is to compare the predictive ability of Bayesian regularization with Levenberg–Marquardt Artificial Neural Networks. To examine the best architecture of neural networks, the model was tested with one-, two-, three-, four-, and five-neuron architectures, respectively. MATLAB (2011a) was used for analyzing the Bayesian regularization and Levenberg–Marquardt learning al...
متن کامل