Energetic Natural Gradient Descent

In this appendix we show that 1 2 ∆ F (θ)∆ is a second order Taylor approximation of D KL (p(θ)p(θ + ∆)). First, let g q (θ) :=D KL (qp(θ)) = ω∈Ω q(ω) ln q(ω) p(ω|θ). We begin by deriving equations for the Jacobian and Hessian of g q at θ: ∂g q (θ) ∂θ = ω∈Ω q(ω) p(ω|θ) q(ω) ∂ ∂θ q(ω) p(ω|θ) = ω∈Ω q(ω) p(ω|θ) q(ω) −q(ω) ∂p(ω|θ) ∂θ p(ω|θ) 2 = ω∈Ω − q(ω) p(ω|θ) ∂p(ω|θ) ∂θ , (4) and so: ∂ 2 g q (θ) ∂θ 2 = ∂ ∂θ ∂g q (θ) ∂θ = − ω∈Ω q(ω) ∂ ∂θ 1 p(ω|θ) ∂p(ω|θ) ∂θ = − ω∈Ω q(ω) p(ω|θ) ∂ 2 p(ω|θ) ∂θ 2 + ω∈Ω q(ω) p(ω|θ) 2 ∂p(ω|θ) ∂θ ∂p(ω|θ) ∂θ = − ω∈Ω q(ω) p(ω|θ) ∂ 2 p(ω|θ) ∂θ 2 + ω∈Ω q(ω) ∂ ln p(ω|θ) ∂θ ∂ ln p(ω|θ) ∂θ. (5) Next we compute a second order Taylor expansion of g q (θ + ∆) around g q (θ): g p(θ) (θ + ∆) Taylor 2 ≈ g p(θ) (θ) + ∆ ∂g p(θ) (θ) ∂θ (6) + 1 2 ∆ ∂ 2 g p(θ) (θ) ∂θ 2 ∆. Notice that g p(θ) (θ) = D KL (p(θ)p(θ)) = 0, and by (4) ∆ ∂g p(θ) (θ) ∂θ = − ∆ ω∈Ω p(ω|θ) p(ω|θ) ∂p(ω|θ) ∂θ = − ∆ ∂ ∂θ ω∈Ω p(ω|θ) (a) =0, where (a) holds because ω∈Ω p(ω|θ) = 1, so ∂ ∂θ ω∈Ω p(ω|θ) = ∂1 ∂θ = 0. (7) Thus, the first two terms on the right side of (6) are zero, and thus: g p(θ) (θ + ∆) Taylor 2 ≈ 1 2 ∆ ∂ 2 g p(θ) (θ) ∂θ 2 ∆. (8) Next we focus on the Hessian, (5), with q = p(θ): ∂ 2 g p(θ) (θ) ∂θ 2 = − ω∈Ω p(ω|θ) p(ω|θ) ∂ 2 p(ω|θ) ∂θ 2 (a) =0 + ω∈Ω p(ω|θ) ∂ ln p(ω|θ) ∂θ ∂ ln p(ω|θ) ∂θ =F (θ), where (a) comes from taking the derivative of both sides of (7) with respect to θ. Substituting this into (8) we have that g p(θ) (θ + ∆) Taylor 2 ≈ 1 2 ∆ F (θ)∆. In this section we show that ∆ E(θ)∆ is a second order Taylor approximation of D E (p(θ), p(θ + ∆)) 2. First, let g q (θ) :=D E (q, p(θ)) =2 …