Efficient Dual-Numbers Reverse AD via Well-Known Program Transformations

Where dual-numbers forward-mode automatic differentiation (AD) pairs each scalar value with its tangent value, /reverse-mode/ AD attempts to achieve reverse using a similarly simple idea: by pairing backpropagator function. Its correctness and efficiency on higher-order input languages have been analysed Brunel, Mazza Pagani, but this analysis used custom operational semantics for which it is unclear whether can be implemented efficiently. We take inspiration from their use of /linear factoring/ optimise reverse-mode an algorithm that has the correct complexity enjoys efficient implementation in standard functional language support mutable arrays, such as Haskell. Aside linear factoring ingredient, our optimisation steps consist well-known ideas programming community. demonstrate practical technique providing performant differentiates most Haskell98.