Structural Results on Optimal Transmission Scheduling: a Constrained Markov Decision Process Approach

The problem of transmission scheduling over a correlated time-varying wireless channel is formulated as a Constrained Markov Decision Process. The model includes a transmission buffer and finite state Markov model for time-varying radio channel and incoming traffic. The resulting cross-layer optimization problem is formulated to minimize the transmission cost under the constraint on a buffer cost such as the transmission delay. Under the assumptions on submodularity and convexity of the cost function it is shown that the optimal randomized policy is monotonically increasing with the increase of the buffer state. Furthermore, the influence of the channel and traffic correlation matrices on the optimal transmission cost is investigated. It is shown that comparison between optimal transmission costs of two different channels can be performed by considering the stochastic dominance relation of their conditional probability distributions. As an example of this result, channels with smaller scattering and the same mean can achieve smaller average transmission cost for the same average buffer cost.