Discrete Stochastic Approximation for Cross-layer Transmission Control in Wireless Communications

This paper proposes the use of discrete stochastic approximation (DSA) algorithm for a crosslayer adaptive modulation problem in wireless communications. In this system, the throughput in the physical (PHY) layer and quality of service (QoS) incurred by the queueing effects in the data link layer are required to be optimized simultaneously and in the long run. By assuming Markov decision process (MDP) modeling, we prove that the optimal transmission policy is characterized by a queue threshold vector and can be determined by a multivariate discrete convex optimization problem if the dynamic programming (DP) is submodular. We then propose to use DSA, a subgradient-based stochastic approximation (SA) algorithm with proven convergence rate, for approximating the optimal value of the queue threshold vector. By an application in a cross-layer transmission control problem in a network-coded two way relay channel (NC-TWRC), we compare the performance of DSA with that of simultaneous perturbation stochastic approximation (SPSA), the commonly used SA algorithm for threshold policy optimization problems. The results show that DSA converges faster than SPSA resulting in lower and controllable computational cost. Index Terms discrete stochastic approximation, dynamic programming, L-convexity, Markov decision process, optimal monotonic/threshold policy, submodularity. The authors are with the Research School of Engineering, College of Engineering and Computer Science, the Australian National University (ANU), Canberra, ACT 0200, Australia (email: {ni.ding, rodney.kennedy, parastoo.sadeghi}@anu.edu.au). January 17, 2014 DRAFT DSA FOR CROSS-LAYER TRANSMISSION CONTROL IN WIRELESS COMMUNICATIONS 2 f (t) scheduler FIFO queue transmitter wireless fading channel data link layer PHY layer Fig. 1. Cross-layer transmission control system. A scheduler adapts the transmission rate according to the optimization concerns in both layers, e.g., packet/symbol delay and queue overflow in data link layer and transmission error and spectral efficiency in physical (PHY) layer, in the long run.