Probing Linguistic Knowledge in Italian Neural Language Models across Language Varieties
نویسندگان
چکیده
In this paper, we present an in-depth investigation of the linguistic knowledge encoded by transformer models currently available for Italian language. particular, investigate how complexity two different architectures probing affects performance Transformers in encoding a wide spectrum features. Moreover, explore implicit varies according to textual genres and language varieties.
منابع مشابه
Linguistic Features Of Italian Blogs: Literary Language
Preliminary surveys show that the lan guage of blogs is not restricted to the more informal levels of expression. In stead blogs may include many kinds of written language: from simple personal notes to literary prose or poetry. The pa per presents a sample of Italian blogs and comments on the results of the search of literary forms in two Web corpora using search engine queries.
متن کاملGenerative Knowledge Transfer for Neural Language Models
In this paper, we propose a generative knowledge transfer technique that trains an RNN based language model (student network) using text and output probabilities generated from a previously trained RNN (teacher network). The text generation can be conducted by either the teacher or the student network. We can also improve the performance by taking the ensemble of soft labels obtained from multi...
متن کاملIntegrating Encyclopedic Knowledge into Neural Language Models
Neural models have recently shown big improvements in the performance of phrase-based machine translation. Recurrent language models, in particular, have been a great success due to their ability to model arbitrary long context. In this work, we integrate global semantic information extracted from large encyclopedic sources into neural network language models. We integrate semantic word classes...
متن کاملAutomatic Extraction of Language Models from a Linguistic Knowledge Base*
We present an algorithm for the extraction of language models from a semantic network that contains syntactic, semantic and pragmatic knowledge. The use of such language models in acoustic recognition processes results in much better system performance in speed as well as in quality of results. The automatic extraction process guarantees that the created models are always up to date and consist...
متن کاملMulti-Language Neural Network Language Models
In recent years there has been considerable interest in neural network based language models. These models typically consist of vocabulary dependent input and output layers and one, or more, hidden layers. A standard problem with these networks is that large quantities of training data are needed to robustly estimate the model parameters. This poses a challenge when only limited data is availab...
متن کاملذخیره در منابع من
با ذخیره ی این منبع در منابع من، دسترسی به آن را برای استفاده های بعدی آسان تر کنید
ژورنال
عنوان ژورنال: IJCoL
سال: 2022
ISSN: ['2499-4553']
DOI: https://doi.org/10.4000/ijcol.965