A Survey on Statistical Approaches to Natural Language Processing

This survey attempts to catch up with the recent increasing interests in statistical approach to natural language processing based on large corpora. First of all, a historical overview traces back to 1950s when Noam Chomsky proposed his phrase structure transformation grammar and rejected the Markov process natural language modeling. With the development of large corpora and language modeling in recent years, the statistical approach to natural language processing is revived and gains more attention among computational linguists. This survey rst addresses the most successful statistic approach on the part-of-speech tagging by using the hidden Markov model (HMM) and dynamic programming. It then brieey introduces the self-organized method that estimates the parameters of a language model. This is followed by the various statistic estimation methods and the related probability theory. Finally, the corpus-based approach on syntactic structure together with the statistical machine translation are presented. To conclude, we present a brief discussion of the future trends in statistical approaches to natural language processing. 1. Historial overview With the recent increasing interest in statistical approaches to natural language processing, corpus linguistics has become a hot topic. This is due to the fact that Texts are more available than ever before and easier to use for various data tasks. The success of part-of-speech tagging by using the Hidden Markov Model (HMM) also draws the attention of computational linguists to lexical analysis, language modeling and machine translation by various statistical methods. The 1950s style of empiricism is back into fashion.