L2 Model reduction and variance reduction
نویسندگان
چکیده
In this contribution we examine certain variance properties of model reduction. The focus is on L2 model reduction, but some general results are also presented. These general results can be used to analyze various other model reduction schemes. The models we study are nite impulse respons (FIR) and output error (OE) models. We compare the variance of two estimated models. The rst one is estimated directly form data and the other is computed bt reducing a high order model by L2 model reduction. In the FIR case, se show that it is never better to estimate the model directly from data, compared to estimating it via L2 model reduction of a high order FIR model. For OE models we show that the reduced order model has the same variance as the directly estimated one if the reduced model class used contains thr true system.
منابع مشابه
Variance Aspects of L2-model Reduction When Undermodeling - the Output Error Case
In this contribution, variance properties of L2 model reduction are studied. That is, given an estimated model of high order we study the resulting variance of an L2 reduced approximation. The main result of the paper is showing that estimating a low order output error (OE) model via L2 model reduction of a high order model gives a smaller variance compared to estimating a low order model direc...
متن کاملStochastic Conjugate Gradient Algorithm with Variance Reduction
Conjugate gradient methods are a class of important methods for solving linear equations and nonlinear optimization. In our work, we propose a new stochastic conjugate gradient algorithm with variance reduction (CGVR) and prove its linear convergence with the Fletcher and Revves method for strongly convex and smooth functions. We experimentally demonstrate that the CGVR algorithm converges fast...
متن کاملNumerical Solution of Stochastic Differential Equations with Constant Diffusion Coefficients
We present Runge-Kutta methods of high accuracy for stochastic differential equations with constant diffusion coefficients. We analyze L2 convergence of these methods and present convergence proofs. For scalar equations a second-order method is derived, and for systems a method of order one-and-one-half is derived. We further consider a variance reduction technique based on Hermite expansions f...
متن کاملResearching (Non) Fluent L2 Speakers’ Oral Communication Deficiencies: A Psycholinguistic Perspective
Fluency in a second language (L2) involves a quintessentially cognitive processing system that operates quickly and effectively. The perceived importance of researching fluency through a psycholinguistic lens has motivated the related L2 research to resort to current cognitive speaking-specific models. This study, drawing on Levelt’s (1999a) psycholinguistic model, probed the deficiency sources...
متن کاملVariance Reduction in SGD by Distributed Importance Sampling
Humans are able to accelerate their learning by selecting training materials that are the most informative and at the appropriate level of difficulty. We propose a framework for distributing deep learning in which one set of workers search for the most informative examples in parallel while a single worker updates the model on examples selected by importance sampling. This leads the model to up...
متن کاملذخیره در منابع من
با ذخیره ی این منبع در منابع من، دسترسی به آن را برای استفاده های بعدی آسان تر کنید
برای دانلود متن کامل این مقاله و بیش از 32 میلیون مقاله دیگر ابتدا ثبت نام کنید
ثبت ناماگر عضو سایت هستید لطفا وارد حساب کاربری خود شوید
ورودعنوان ژورنال:
- Automatica
دوره 38 شماره
صفحات -
تاریخ انتشار 2002