Stochastic Optimization of PCA with Capped MSG
نویسندگان
چکیده
We study PCA as a stochastic optimization problem and propose a novel stochastic approximation algorithm which we refer to as “Matrix Stochastic Gradient” (MSG), as well as a practical variant, Capped MSG. We study the method both theoretically and empirically.
منابع مشابه
Online Robust PCA via Stochastic Optimization
Robust PCA methods are typically based on batch optimization and have to load all the samples into memory during optimization. This prevents them from efficiently processing big data. In this paper, we develop an Online Robust PCA (OR-PCA) that processes one sample per time instance and hence its memory cost is independent of the number of samples, significantly enhancing the computation and st...
متن کاملStochastic Power Control for Time-Varying Long-Term Fading Wireless Networks
A new time-varying (TV) long-term fading (LTF) channel model which captures both the space and time variations of wireless systems is developed. The proposed TV LTF model is based on a stochastic differential equation driven by Brownian motion. This model is more realistic than the static models usually encountered in the literature. It allows viewing the wireless channel as a dynamical system,...
متن کاملA New Stochastic Optimization Algorithm based on a Particle Collision Metaheuristic
In many complex optimization problems, the so-called stochastic optimization methods as genetic algorithms and simulated annealing have been applied successfully. However, these methods have drawbacks: simulated annealing is too sensitive to the choice of parameters and the canonical genetic algorithm is liable to suboptimal convergence. With the purpose of overcoming these drawbacks, a new sto...
متن کاملPCA-Enhanced Stochastic Optimization Methods
In this paper, we propose to enhance particle-based stochastic optimization methods (SO) by using Principal Component Analysis (PCA) to build an approximation of the cost function in a neighborhood of particles during optimization. Then we use it to shift the samples in the direction of maximum cost change. We provide theoretical basis and experimental results showing that such enhancement impr...
متن کاملA Fuzzy Multi-Objective Optimization Model for Production and Consumption Management in Energy Micro Smart Grids
Electricity is one of the most important carriers of energy used in buildings. By introducing energy smart grids (SG) and energy micro smart grids (MSGs) alongside smart buildings, a good platform has been provided for optimal planning of electricity production and consumption. In this paper, an MSG consists of renewable resources, diesel generators and cell batteries in bidirectional connectio...
متن کامل