Guaranteed Rank Minimization via Singular Value Projection: Supplementary Material

We first give a proof of Lemma 2.1 which bounds the error of the (t + 1)-st iterate (ψ(X t+1)) in terms of the error incurred by the t-th iterate and the optimal solution. Proof of Lemma 2.1 Recall that ψ(X) = 1 2 ∥A(X) − b∥ 2 2. Since ψ(·) is a quadratic function, we have ψ(X t+1) − ψ(X t) = ⟨∇ψ(X t), X t+1 − X t ⟩ + 1 2 ∥A(X t+1 − X t)∥ 2 2 ≤ ⟨A T (A(X t) − b), X t+1 − X t ⟩ + 1 2 · (1 + δ 2k) · ∥X t+1 − X t ∥ 2 F , (0.1) where the inequality follows from RIP applied to the matrix X t+1 − X t of rank at most 2k. Let Y t+1 = X t − 1 1+δ 2k A T (A(X t) − b) and f t (X) = ⟨A T (A(X t) − b), X − X t ⟩ + 1 2 · (1 + δ 2k) · ∥X − X t ∥ 2 F. Now, f t (X) = 1 2 (1 + δ 2k) [ ∥X − X t ∥ 2 F + 2 ⟨ A T (A(X t) − b) 1 + δ 2k , X − X t ⟩] = 1 2 (1 + δ 2k)∥X − Y t+1 ∥ 2 F − 1 2(1 + δ 2k) · ∥A T (A(X t) − b)∥ 2 F. Thus, by definition, P k (Y t+1) = X t+1 is the minimizer of f t (X) over all matrices X ∈ C(k) (of rank at most k). In particular, f t (X t+1) ≤ f t (X *) and, ψ(X t+1) − ψ(X t) ≤ f t (X t+1) ≤ f t (X *) = ⟨A T (A(X t) − b), X * − X t ⟩ + 1 2 (1 + δ 2k)∥X * − X t ∥ 2 F ≤ ⟨A T (A(X t) − b), X * − X t ⟩ + 1 2 · 1 + δ 2k 1 − δ 2k ∥A(X * − X t)∥ 2 2 (0.2) = ψ(X *) − ψ(X t) + δ 2k (1 − δ 2k) ∥A(X * − X t)∥ 2 2 , where inequality (0.2) follows from RIP applied to X * − X t. We now prove Theorem …