Optimal Multilevel Feedback Policies for ABR Flow Control using Two Timescale SPSA
نویسندگان
چکیده
Optimal multilevel feedback control policies for rate based flow control in available bit rate (ABR) service in asynchronous transfer mode (ATM) networks are obtained in the presence of information and propagation delays, using a numerically efficient two timescale simultaneous perturbation stochastic approximation (SPSA) algorithm. Convergence analysis of the algorithm is presented. Numerical experiments demonstrate fast convergence even in the presence of significant delays and large number of parametrized policy levels.
منابع مشابه
Rate Based ABR Flow Control using Two Timescale SPSA
In this paper, a two timescale simultaneous perturbation stochastic approximation (SPSA) algorithm is developed and applied to closed loop rate based available bit rate (ABR) ow control. The relevant convergence results are stated and explained. Numerical experiments demonstrate fast convergence even in the presence of signiicant delays and a large number of parameterized policy levels.
متن کاملMultiscale Chaotic SPSA and Smoothed Functional Algorithms for Simulation Optimization
The authors propose a two-timescale version of the one-simulation smoothed functional (SF) algorithm with extra averaging. They also propose the use of a chaotic simple deterministic iterative sequence for generating random samples for averaging. This sequence is used for generating the N independent and identically distributed (i.i.d.), Gaussian random variables in the SF algorithm. The conver...
متن کاملA Simulation-Based Algorithm for Ergodic Control of Markov Chains Conditioned on Rare Events
We study the problem of long-run average cost control of Markov chains conditioned on a rare event. In a related recent work, a simulation based algorithm for estimating performance measures associated with a Markov chain conditioned on a rare event has been developed. We extend ideas from this work and develop an adaptive algorithm for obtaining, online, optimal control policies conditioned on...
متن کاملModeling and Simulation of an ABR Flow Control Algorithm Using a Virtual Source/Virtual Destination Switch
The Available Bit Rate (ABR) service class of Asynchronous Transfer Mode networks uses a feedback control mechanism to adapt to varying link capacities. The Virtual Source/Virtual Destination (VS/VD) technique offers the possibility to segment the otherwise end-to-end ABR control loop into separate loops. The improved feedback delay and the control of ABR traffic inside closed segments provide ...
متن کاملReinforcement Learning Based Algorithms for Average Cost Markov Decision Processes
This article proposes several two-timescale simulation-based actor-critic algorithms for solution of infinite horizon Markov Decision Processes with finite state-space under the average cost criterion. Two of the algorithms are for the compact (non-discrete) action setting while the rest are for finite-action spaces. On the slower timescale, all the algorithms perform a gradient search over cor...
متن کاملذخیره در منابع من
با ذخیره ی این منبع در منابع من، دسترسی به آن را برای استفاده های بعدی آسان تر کنید
عنوان ژورنال:
دوره شماره
صفحات -
تاریخ انتشار 1999