Densities of Almost Surely Terminating Probabilistic Programs are Differentiable Almost Everywhere

We study the differential properties of higher-order statistical probabilistic programs with recursion and conditioning. Our starting point is an open problem posed by Hongseok Yang: what class have densities that are differentiable almost everywhere? To formalise problem, we consider Statistical PCF (SPCF), extension call-by-value real numbers, constructs for sampling give SPCF a sampling-style operational semantics la Borgstrom et al., associated weight (commonly referred to as density) function value on set possible execution traces. main result almost-surely terminating programs, generated from primitive functions (e.g. analytic functions) satisfying mild closure properties, almost-everywhere differentiable. use stochastic form symbolic reason about differentiability. A by-product this work deterministic (S)PCF parameters denote practical interest, differentiability density required hold correctness major gradient-based inference algorithms.