In the last few years Ayache, Esser and Hamonier introduced a new Multifractional Process with Random Exponent (MPRE) obtained by replacing Hurst parameter in moving average representation of Fractional Brownian Motion through Wiener integral an adapted Hölder continuous stochastic process indexed integration variable. Thus, this MPRE can be expressed as Itô which is considerable advantage resp...

We consider a nonhomogeneous stochastic infinite horizon optimization problem whose objective is to minimize the overall average cost per-period of an infinite sequence of actions (average optimality). Optimal solutions to such problems will in general be non-stationary. Moreover, a solution which initially makes poor decisions, and then selects wisely thereafter, can be average optimal. Howeve...

In this paper we study the extremal behavior of stationary mixed moving average processes Y (t) = ∫ R+×R f(r, t − s) dΛ(r, s) for t ∈ R, where f is a deterministic function and Λ is an infinitely divisible independently scattered random measure, whose underlying driving Lévy process is regularly varying. We give sufficient conditions for the stationarity of Y and compute the tail behavior of ce...

(1-2) Pj(t) = Pj(t + d) , qt = qt+d and {E,} is a normal white noise with zero mean value and a variance a. Such process is natural analogy of the periodic autoregressive process (see e.g. [ l ] , [5], [6], [7]) and therefore the idea originates to use the estimation technique described in [3] for the treatment of the multiple moving average models (the same approach to the multiple autoregress...

Abstrac t . Some simple models are introduced which may be used for modelling or generating sequences of dependent discrete random variables with generalized Poisson marginal distribution. Our approach for building these models is similar to that of the Poisson ARMA processes considered by Al-Osh and Alzaid (1987, J. Time Ser. Anal., 8, 261-275; 1988, Statist. Hefte, 29, 281-300) and McKenzie (...

Reinforcement learning (RL) has become a central paradigm for solving learning-control problems in robotics and artificial intelligence. R L researchers have focussed almost exclusively on problems where the controller has to maximize the discounted sum of payoffs. However, as emphasized by Schwartz (1$X)3), in many problems, e.g., those for which the optimal behavior is a limit cycle, it is mo...

We consider how state similarity in average reward Markov decision processes (MDPs) may be described by pseudometrics. Introducing the notion of adequate pseudometrics which are well adapted to the structure of the MDP, we show how these may be used for state aggregation. Upper bounds on the loss that may be caused by working on the aggregated instead of the original MDP are given and compared ...

In this paper, we give a summary of recent development of simulation-based algorithms for average cost MDP problems, which are different from those for discounted cost problems or shortest path problems. We introduce both simulation-based policy iteration algorithms and simulation-based value iteration algorithms for average cost problem, and give the pros and cons of each algorithm.