Morphological Processing of Compounds for Statistical Machine Translation

Machine Translation denotes the translation of a text written in one language into another language performed by a computer program. In times of internet and globalisation, there has been a constantly growing need for machine translation. For example, think of the European Union, with its 24 official languages into which each official document must be translated. The translation of official documents would be less manageable and much less affordable without computer-aided translation systems. Most state-of-the-art machine translation systems are based on statistical models. These are trained on a bilingual text collection to “learn” translational correspondences of words (and phrases) of the two languages. The underlying text collection must be parallel, i.e. the content of one line must exactly correspond to the translation of this line in the other language. After training the statistical models, they can be used to translate new texts. However, one of the drawbacks of Statistical Machine Translation (SMT) is that it can only translate words which have occurred in the training texts. This applies in particular to SMT systems which have been designed for translating from and to German. It is widely known that German allows for productive word formation processes. Speakers of German can put together existing words to form new words, called compounds. An example is the German “Apfel + Baum = Apfelbaum” (= “apple + tree = apple tree”). Theoretically there is no limit to the length of a German compound. Whereas “Apfelbaum” (= “apple tree”) is a rather common German compound, “Apfelbaumholzpalettenabtransport” (= “apple|tree|wood|pallet|removal”) is a spontaneous new creation, which (probably) has not occurred in any text collection yet. The productivity of German compounds leads to a large number of distinct compound types, many of which occur only with low frequency in a text collection, if they occur at all. This fact makes German compounds a challenge for SMT systems, as only words which have occurred in the parallel training data can later be translated by the systems. Splitting compounds into their component words can solve this problem. For example, splitting “Apfelbaumholzpalettenabtransport” into its component words, it becomes in-