This paper presents a discrete-time single-server finite buffer N threshold policy queue with renewal input and discreteMarkovian service process. The server terminates service whenever the system becomes empty, and recommencesservice as soon as the number of waiting customers in the queue is N. We obtain the system-length distributionsat pre-arrival and arbitrary epochs using the supplementary...

We describe how multi-stage non-Markovian decision problems can be solved using actor-critic reinforcement learning by assuming that a discrete version of CohenGrossberg node dynamics describes the node-activation computations of a neural network (NN). Our NN (i.e., agent) is capable of rendering the process Markovian implicitly and automatically in a totally model-free fashion without learning...

This paper introduces an analytical method for approximating the performance of a two-class priority M/M/1 system. The system is fully non-preemptive. More specifically, the prioritized class-1 jobs are real-time and served with the non-preemptive earliest-deadline-first (EDF) policy, but despite their priority cannot preempt any non real-time class-2 job. The waiting class-2 jobs can only be s...

Deterministic fluid models are developed to provide simple first-order performance descriptions for multi-server queues with abandonment under heavy loads. Motivated by telephone call centers, the focus is on multi-server queues with a large number of servers and non-exponential service-time and time-to-abandon distributions. The first fluid model serves as an approximation for the G/GI/s + GI ...

A single server retrial system having several operation modes is considered. The modes are distinguished by the transition rate of the batch Markovian arrival process (BMAP), kernel of the semi-Markovian (SM) service process and the intensity of retrials. Stationary state distribution is calculated under the fixed value of the multi-threshold control strategy. Dependence of the cost criterion, ...

This paper provides an accurate model of the General Packet Radio Service (GPRS). GPRS is modeled as a single server queue in a Markovian environment. The queueing performance of data packets is evaluated by matrix geometric methods. The arrival process is assumed to follow a two state Markov modulated Poisson process (MMPP), and the service rate fluctuates based on voice loading. The analytica...

the national bridge stock of iran consists of about 330,000 bridges, of which around 50% are older than 30 years. since 2010, iran road maintenance & transportation organization has started implementing a comprehensive bridge management system in order to manage this aged stock efficiently. to predict future conditions of bridge stock, a heuristic numerical method is presented. this methodology...

In this paper we introduce the concept of a Quasi-Birth-and-Death process (QBD) with Rational Arrival Process components. We use the physical interpretation of a Rational Arrival Process (RAP), developed by Asmussen and Bladt, to consider such a Markov process. We exploit this interpretation to develop an analytic method for such a process, that parallels the analysis of a traditional QBD. We d...