Formal and functional theories of language seem disparate, because formal answer the question what a is, while functions it serves. We argue that information theory provides bridge between these two approaches, via principle minimization complexity under constraints. Synthesizing recent work, we show how information-theoretic characterizations lead directly to mathematical descriptions forms po...