Homer: A Higher-Order Observational Equivalence Model checkER

We present HOMER, an observational-equivalence model checker for the 3rd-order fragment of Idealized Algol (IA) augmented with iteration. It works by first translating terms of the fragment into a precise representation of their game semantics as visibly pushdown automata (VPA). The VPA-translates are then passed to a VPA toolkit (which we have implemented) to test for equivalence. Thanks to the fully abstract game semantics, observational equivalence of these IA-terms reduces to the VPA Equivalence Problem. Our checker is thus sound and complete; because it model checks open terms, our approach is also compositional. Further, if the terms are inequivalent, HOMER will produce both a game-semantic and an operational-semantic counter-example, in the form of a play and a separating context respectively. We showcase these features on a number of examples and (where appropriate) compare its performance with similar tools. To the best of our knowledge, HOMER is the first implementation of a model checker of 3rd-order programs. 1 Theory and Implementation Motivation Higher-order functions are commonly used in functional programming. The functions map and foldr are standard examples of 2nd-order programs. 3rd and higher-order functions arise naturally in language processors [10]. Higher-order programs also crop up in imperative / object-oriented languages. E.g. any algorithm or data structure parameterised by, say, a comparison function is 2nd-order. A program that relies on such a 2nd-order function (being defined in an external library, say) is 3rd-order. Perhaps the most significant higher-order program is Google’s MapReduce system [11]. Here we present the first model checker for 3rd-order programs. Reynold’s Idealized Algol (IA) [9] is a higher-order procedural language that combines imperative constructs (such as block-allocated assignable variables, sequencing and iteration) with higher-order functional features. It is essentially a call-by-name variant of (core) ML. E.g. the imperative term while !X > 0 do {Y := !Y ∗ !X ; X := !X−1; } and the lambda-term λf .λg .λx.f x (g x) are both valid in IA. Here we consider the fragment of IA containing up to 3rd-order terms over finite base types. I.e. we allow functions of types ((b1 → b2) → b3) → b4, say, where each bi is one of the base types: com (commands), exp and var (expressions and variables ⋆ We thank A. Murawski for useful discussions and Microsoft Research PhD Scholarship Programme for funding this work. HOMER builds on and extends Hopkins’ dissertation [7]. respectively, with values taken from a finite prefix of the natural numbers). In addition, we allow while-loops but not full recursion. We denote this fragment IA∗3. Two terms Γ ⊢ M1,M2 : A are observationally (or contextually) equivalent, written Γ ⊢ M1 ∼= M2, just if for every program context C[−] such that both C[M1] and C[M2] are closed terms of type com , C[M1] converges if and only if C[M2] converges. I.e. two terms are observationally equivalent whenever no program context can possibly distinguish them. An intuitively compelling notion of program equivalence, observational equivalence has a rich theory. For example, λx.newX in {X := x; !X} ∼= λx.x, because the internal workings of the function are not detectable from the outside. However, these terms are not equivalent to λx.if x thenx elsex — because expressions can have side-effects, the outcome of evaluating x twice may be different from evaluating x only once. A much less obvious equivalence is p : com → com ⊢ new x := 0 in {p (x := 1); if !x = 1 thenΩ else skip} ∼= p Ω where Ω is the term that immediately diverges. This example shows that “snapback” (i.e. a term that first evaluates its com-type arguments and then immediately undoes their side-effects) is not definable in IA. The above equivalence holds because in either case, if p ever evaluates its argument, the computation will diverge. Game Semantics The fully abstract1 game semantics [1] of IA has proved extremely powerful. In this model, a type A is interpreted as a game [[A]] between P and O, and a term Γ ⊢ M : A as a strategy [[Γ ⊢ M ]] for P to play in the game [[Γ ⊢ A]]. A strategy is just a set of plays (forming a playbook for how P should respond at each step he is to play), and a play is a sequence of moves, each is a question or an answer equipped with a pointer2 to an earlier move. A play is complete if every question in it has been answered, so the game has run to completion. The highly accurate game semantics characterises observational equivalence in terms of complete plays i.e. Γ ⊢ M1 ∼= M2 if and only if comp[[Γ ⊢ M1]] = comp[[Γ ⊢ M2]], where comp σ is the set of complete plays in strategy σ. Murawski and Walukiewicz [8] have shown that the complete plays in the strategy denotation of an IA∗3-term are recognisable by a visibly pushdown automaton. Visibly Pushdown Automata The visibly pushdown automata (VPA) [3] are a subclass of pushdown automata in which the stack actions (push, pop or no-op) are completely determined by the input symbol. They are more expressive than finite automata, yet enjoy many nice closure properties of the regular languages. Because they are closed under complementation and intersection, and have a decidable emptiness problem, the VPA Equivalence Problem (“Given two VPA, do they accept the same language?”) is decidable. So by representing the set of complete plays in a strategy denotation of an IA∗3-term as a VPA, it is decidable (in EXPTIME [8]) if a given pair of β-normal IA ∗ 3terms are observationally equivalent. Implementation Following the algorithm of Murawski and Walukiewicz [8], we have created a tool, called HOMER, for checking observational equivalence of IA∗3 terms1 Full abstraction is a strong goodness-of-fit measure. A denotational semantics is fully abstract if the theory of equality induced by the semantics coincides with observational equivalence. 2 which models the operand-to-operator, and variable-to-binder relation within a term. in-context. Given two such terms, it first translates them to their VPA representations. These are then fed into a VPA toolkit, which we have created, to check for equivalence by complementing, intersecting and emptiness-checking to test for inclusion in both directions. The complementation and intersection operations are straightforward implementations from [3]. Since the VPA-translates are deterministic by construction, they are complemented just by complementing the final states. Intersection is by a product construction (which works for VPA – but not general PDA – because the two automata always perform the same stack action). More complex is the emptiness test. We experimented on a few algorithms before settling on Schwoon’s pre* algorithm for general PDA, [12]. When the two terms are inequivalent, this will produce as a counter-example a play recognisable by exactly one of the two VPA-translates. The tool will use this play to generate a separating context a context that converges when its hole is filled by one term, but diverges when filled by the other. HOMER is written in about 8 KLOC of F#, including about 600 LOC for the VPA toolkit. 2 Evaluation and Tests All tests in the following have been run on a laptop with a 2.53GHz Intel Core 2 Duo processor and 4GB RAM under Windows Vista. The base type exp%N is { 0, 1, · · ·N− 1 }. Unless specified otherwise, exp coincides with exp%N, which defaults to N = 3. Sorting Verifying sorting programs is a challenging test for model checkers because of the complex interplay between data flow and control flow [2]. An implementation of bubble sort is given below. Input to the program takes the form of an array of elements of the set {0, 1, 2}; we evaluate the model for different values of n, the length of the array. The program communicates with its environment only through the non-local variable x. In a call-by-name setting, because x can represent any var -typed procedure, it is legitimate to use it as an input/output stream. The program initially populates the array by reading from x, and after sorting writes the values back to x in order. These reads and writes are the only actions visible from the outside. 1 x : var%3 |2 new a[N]%3 in 3 {new i%N+1 in while !i < N do {a[!i] := !x; i := (!i + 1)}}; 4 { 5 new flag%2 in 6 flag := 1; 7 while !flag do{ 8 new i%N in 9 flag := 0; 10 while !i < N-1 do{ 11 if !a[!i] > !a[!i + 1] then{ 12 new temp%3 in 13 flag := 1; 14 temp := !a[!i] ; 15 a[!i] := !a[!i + 1]; 16 a[!i + 1] := !temp 17 } 18 else skip; 19 i := !i + 1 20 } 21 } 22 }; 23 {new i%N+1 in while !i < N do {x:= !a[!i];i := !i + 1}} The automaton produced when n = 2 is shown in the following using Graphviz3. Since this is a 1st-order program, the VPA-translate degenerates to a deterministic finite automaton. It can be seen that there is a trace through the automaton for each of the 9 possible input combinations. 4