On the Law of Zipf-Mandelbrot for Multi-Wort Phrases

The paper studies the probabilities of the occurrence of m word phrases (m=2,3, ...) in relation with the probabilities of occurrence of the single words. It is well-known that, in the latter case, the law of Zipf is valid (i.e. a power law). We prove that in the case of m word phrases (m22) this is not the case. We present two independent proofs of this. We furthermore show that in case we want to approximate the found distribution by Zipfs law we obtain exponents p, in this power law for which the sequence (P,),, is strictly decreasing. This explains experimental findings of Smith and Devine, Hilberg and Meyer. 'Permanent address Acknowledgement The author is grateful to Prof Dr. R. Rousseau for interesting discussions on the topic of this paper. 2 Our results should be compared with a heuristic finding of Rousseau who states that the law of Zipf-Mandelbrot is valid for multi-word phrases. He, however, uses other less classical assumptions than we do.