Hidden Markov Models for Induction of Morphological Structure of Natural Language
نویسندگان
چکیده
This paper presents initial results from an on-going project on automatic induction of morphological structure of natural language, from plain, un-annotated textual corpora. In previous work, this area has been shown to have interesting potential applications. One of our main goals is to reduce reliance on heuristics as far as possible, and rather to investigate to what extent the morphological structure is inherent in the language or text per se. We present a Hidden Markov Model trained with respect to a two-part code cost function. We discuss performance on corpora in highly-inflecting languages, problems relating to evaluation, and compare to results obtained with the Morfessor algorithm.
منابع مشابه
Introducing Busy Customer Portfolio Using Hidden Markov Model
Due to the effective role of Markov models in customer relationship management (CRM), there is a lack of comprehensive literature review which contains all related literatures. In this paper the focus is on academic databases to find all the articles that had been published in 2011 and earlier. One hundred articles were identified and reviewed to find direct relevance for applying Markov models...
متن کاملMAN-MACHINE INTERACTION SYSTEM FOR SUBJECT INDEPENDENT SIGN LANGUAGE RECOGNITION USING FUZZY HIDDEN MARKOV MODEL
Sign language recognition has spawned more and more interest in human–computer interaction society. The major challenge that SLR recognition faces now is developing methods that will scale well with increasing vocabulary size with a limited set of training data for the signer independent application. The automatic SLR based on hidden Markov models (HMMs) is very sensitive to gesture's shape inf...
متن کاملLogistic Normal Priors for Unsupervised Probabilistic Grammar Induction
We explore a new Bayesian model for probabilistic grammars, a family of distributions over discrete structures that includes hidden Markov models and probabilistic context-free grammars. Our model extends the correlated topic model framework to probabilistic grammars, exploiting the logistic normal distribution as a prior over the grammar parameters. We derive a variational EM algorithm for tha...
متن کاملUnsupervised Bayesian Parameter Estimation for Dependency Parsing
We explore a new Bayesian model for probabilistic grammars, a family of distributions over discrete structures that includes hidden Markov models and probabilitsic context-free grammars. Our model extends the correlated topic model framework to probabilistic grammars, exploiting the logistic normal prior as a prior over the grammar parameters. We derive a variational EM algorithm for that model...
متن کاملHidden Markov Models Suitable for Text Generation
The paper presents the application of Hidden Markov Models to text generation in Polish language. A program generating text, taking advantage of Hidden Markov Models was developed. The program uses a reference text to learn the possible sequences of letters. The results of text processing have been also discussed. The presented approach can be also helpful in speech recognition process. Key-Wor...
متن کامل