Transducers with Origin Information

Call a string-to-string function regular if it can be realised by one of the following equivalent models: mso transductions, two-way deterministic automata with output, and streaming transducers with registers. This paper proposes to treat origin information as part of the semantics of a regular string-to-string function. With such semantics, the model admits a machine-independent characterisation, Angluin-style learning in polynomial time, as well as effective characterisations of natural subclasses such as one-way transducers or first-order definable transducers. This paper is about string-to-string functions which can be described by deterministic two-way automata with output [AU70]. As shown in [EH01], this model is equivalent to mso definable string transductions. Another equivalent model, used in [AC10], is a deterministic one-way automaton with registers that store parts of the output. Examples of such functions include: duplication w 7→ ww; reversing w 7→ w; a function w 7→ ww which maps an input to a palindrome whose first half is w; and a function which duplicates inputs of even length and reverses inputs of odd length. As witnessed by the multiple equivalent definitions, this class of string-to-string function is robust, and therefore, following [AC10], we call it the class of regular string-to-string functions. Regular string-to-string functions have good closure properties. For instance, if f and g are regular, then the composition w 7→ f(g(w)) is also regular, which is straightforward if the mso definition is used, but nontrivial if the two-way automata definition is used [CJ77]. Also the concatenation w 7→ f(w) · g(w) is regular, which is apparent in any of the three definitions. Equivalence of regular string-to-string functions is decidable, as was shown in [Gur82] using the two-way automata definition. Origins. The motivation of this paper is the simple observation that the models discussed above, namely deterministic two-way automata with output, mso definable string transductions, and automata with registers, provide more than just a function from strings to strings. In each case, one can also reconstruct ? Supported by ERC Starting Grant “Sosna” 1 Registers are similar to attributes in attribute grammars. The equivalence of mso definable transductions with a form of attribute grammars, in the tree-to-tree case, was shown in [BE00]. In the special case of string-to-string functions, the attribute grammars from [BE00] correspond to left-to-right deterministic automata with registers and regular lookahead. origin information, which says how positions of the output string originate from positions in the input string. How do we reconstruct the origin of a position x in an output string? In the case of a deterministic two-way automaton, this is the position of the head when x was output. In the case of an mso definable transducer, this is the position in which x is interpreted. In the case of an automaton with registers, this is the position in the input when the letter x was first loaded into a register. In other words, for a transducer we can consider two semantics: the standard semantics, where the output is a string, and the origin semantics, where the output is a string with origin information. The second semantics is finer in the sense that some transducers might be equivalent under the standard semantics, but not under the origin semantics. Tracking origin information for transducers has been studied before, for instance in the programming language community, see e.g. [vDKT93]. Origin information has also been used as a technical tool in the study of tree-to-tree transducers. Examples include [EM03], where origin information is used to characterise those macro tree transducers which are mso definable, and [LMN10], where origin information is used to get a Myhill-Nerode characterisation of deterministic top-down tree transducers. The novelty of this paper is that origin information is built into the semantics of a transducer. Origin semantics. To illustrate the difference between the two semantics (standard and origin) of a string-to-string transducer, consider a transducer which is the identity on the string ab, and which maps other strings to the empty string. If we care about origins, then this description is incomplete, and can be instantiated in four different ways depicted below.