Learning Heteroscedastic Models by Convex Programming under Group Sparsity

and ν◦ tRt,:α ◦ = 0 for every t. It should be emphasized that relation (18) holds true only in the case where the solution satisfies mink |X :,Gkφ ◦ :,Gk |2 6= 0, otherwise one has to replace it by the condition stating that the null vector belongs to the subdifferential. Since this does not alter the proof, we prefer to proceed as if everything was differentiable. On the one hand, (φ◦,α◦) satisfies (18) if and only if ΠGk(diag(Y )Rα ◦ − Xφ◦) = λkX:,Gkφ ◦ Gk /|X:,Gkφ ◦ Gk |2 with ΠGk = X:,Gk(X > :,Gk X:,Gk) X:,Gk being the orthogonal projector onto the range of X:,Gk in R T . Taking the norm of both sides in the last equation, we get ∣∣ΠGk(diag(Y )Rα◦ −Xφ◦)∣∣2 ≤ λk. This tells us that (φ◦,α◦) satisfy (7). On the other hand, since the minimum of (6) is finite, one easily checks that Rt,:α ◦ 6= 0 and, therefore, ν◦ = 0. Replacing in (19) ν◦ by zero and setting v◦ t = 1/Rt,:α ◦ we get that (φ◦,α◦,v◦) satisfies (8), (9). This proves that the set of feasible solutions of the optimization problem defined in the ScHeDs is not empty.