Supplement: Efficient Decomposed Learning for Structured Prediction
نویسندگان
چکیده
This supplement provides the proof for all the theorems and corollaries in the main paper. We provide the proofs as well as the theorem statements for the ease of understanding while borrowing the notation from the main draft. Proof. Assume that we are given an > 0 for which the condition mentioned in the theorem statement holds true (for the sake of simplicity we assume the same for all w ∈ R d by taking a minimum over all possible values.) Since W * ⊆ W dec (Observation 3 in the paper), to show exactness, it is sufficient to show W dec ⊆ W *. We want to show that ∀w ∈ R d , if w ∈ W dec then w ∈ W *. Suppose there exists a w ∈ W dec with w / ∈ W * ; we will show by contradiction that no such w exists. Consider any w * ∈ W * (recall that we assume that W * is non-empty) and define w t = w * +tw for t ∈ [0, 1] By convexity of W dec (Observation 2), w t ∈ W dec ∀t ∈ [0, 1]. Define m = max{t ∈ [0, 1]|w t ∈ W * }. By closedness of W * (Observation 1) and by our assumption that w / ∈ W * , we get m < 1. Now we have that for all ∈ (m, 1], w m+ / ∈ W * and w m+ ∈ W dec. It is easy to verify that for a given , w m+ − w m ≤ for an appropriate value of (≤ w). Let the corresponding weight vector be w = w m+ ∈ W dec. Note that since w − w m ≤ , we get by the condition mentioned in the theorem statement that if ∃y with f (x j , y; w) + ∆(y j , y) > f (x j , y j ; w) then ∃y ∈ nbr(y j) with f (x j , y ; w) + ∆(y j , y) > f (x j , y j ; w). In order to prove by contradiction that our assumption is wrong and that no such w exists, it is sufficient to show that w / ∈ W dec. This easily follows as w / ∈ W * ⇒ l(w) > 0 (due to …
منابع مشابه
Efficient Decomposed Learning for Structured Prediction
Structured prediction is the cornerstone of several machine learning applications. Unfortunately, in structured prediction settings with expressive inter-variable interactions, exact inference-based learning algorithms, e.g. Structural SVM, are often intractable. We present a new way, Decomposed Learning (DecL), which performs efficient learning by restricting the inference step to a limited pa...
متن کاملEfficient Methods for Learning and Inference in Structured Output Prediction
Binary classification has dominated machine learning for decades. However, many modern applications of machine learning require modeling complex high-dimensional outputs. Although the output space in such problems is typically very large, it often has intrinsic structure which can be exploited to construct efficient algorithms. Indeed, in recent years structured output prediction has shown stat...
متن کاملIllinoisSL: A JAVA Library for Structured Prediction
IllinoisSL is a Java library for learning structured prediction models. It supports structured Support Vector Machines and structured Perceptron. The library consists of a core learning module and several applications, which can be executed from command-lines. Documentation is provided to guide users. In Comparison to other structured learning libraries, IllinoisSL is efficient, general, and ea...
متن کاملLearning and Inference in Structured Prediction Models
This tutorial targets AI researchers who are interested in designing and applying structured prediction models to problems with interdependent output variables. The tutorial will introduce the problem of structured prediction exemplifying it in multiple AI problems, and then cover recent developments in efficient inference and learning methods in discriminative structured models and outline fur...
متن کامل(Online) Subgradient Methods for Structured Prediction
Promising approaches to structured learning problems have recently been developed in the maximum margin framework. Unfortunately, algorithms that are computationally and memory efficient enough to solve large scale problems have lagged behind. We propose using simple subgradient-based techniques for optimizing a regularized risk formulation of these problems in both online and batch settings, a...
متن کامل