Lower Energy Large Language Models (LLMs)
نویسندگان
چکیده
This message offers ideas about how to reduce the energy consumption associated with large language models.
منابع مشابه
Large Margin Neural Language Models
Neural language models (NLMs) are generative, and they model the distribution of grammatical sentences. Trained on huge corpus, NLMs are pushing the limit of modeling accuracy. Besides, they have also been applied to supervised learning tasks that decode text, e.g., automatic speech recognition (ASR). By re-scoring the n-best list, NLM can select grammatically more correct candidate among the l...
متن کاملLarge Margin Neural Language Models
Neural language models (NLMs) are generative, and they model the distribution of grammatical sentences. Trained on huge corpus, NLMs are pushing the limit of modeling accuracy. Besides, they have also been applied to supervised learning tasks that decode text, e.g., automatic speech recognition (ASR). By re-scoring the n-best list, NLM can select grammatically more correct candidate among the l...
متن کاملNieme - Large-Scale Energy-Based Models Nieme: Large-Scale Energy-Based Models
In this paper we introduce Nieme, a machine learning library for large-scale classification, regression and ranking. Nieme relies on the framework of energy-based models (LeCun et al., 2006) which unifies several learning algorithms ranging from simple perceptrons to recent models such as the pegasos support vector machine or l1-regularized maximum entropy models. This framework also unifies ba...
متن کاملLarge Vocabulary SOUL Neural Network Language Models
This paper presents continuation of research on Structured OUtput Layer Neural Network language models (SOUL NNLM) for automatic speech recognition. As SOUL NNLMs allow estimating probabilities for all in-vocabulary words and not only for those pertaining to a limited shortlist, we investigate its performance on a large-vocabulary task. Significant improvements both in perplexity and word error...
متن کاملLarge Scale Hierarchical Neural Network Language Models
Feed-forward neural network language models (NNLMs) are known to improve both perplexity and word error rate performance for speech recognition compared with conventional ngram language models. We present experimental results showing how much the WER can be improved by increasing the scale of the NNLM, in terms of model size and training data. However, training time can become very long. We imp...
متن کاملذخیره در منابع من
با ذخیره ی این منبع در منابع من، دسترسی به آن را برای استفاده های بعدی آسان تر کنید
ژورنال
عنوان ژورنال: IEEE Computer
سال: 2023
ISSN: ['1558-0814', '0018-9162']
DOI: https://doi.org/10.1109/mc.2023.3278160