Abstract The universal approximation property of various machine learning models is currently only understood on a case-by-case basis, limiting the rapid development new theoretically justified neural network architectures and blurring our understanding current models’ potential. This paper works towards overcoming these challenges by presenting characterization, representation, construction me...

If X is a Banach space such that the isomorphism constant to `2 from n dimensional subspaces grows sufficiently slowly as n → ∞, then X has the approximation property. A consequence of this is that there is a Banach space X with a symmetric basis but not isomorphic to `2 so that all subspaces of X have the approximation property. This answers a problem raised in 1980 [8]. An application of the ...