This paper introduces AntNet, a novel adaptive approach to routing tables learning in connectionless communications networks. AntNet is inspired by the stigmergy communication model observed in ant colonies. We compare AntNet with the current Internet routing algorithm (OSPF), some old Internet routing algorithms (SPF and distributed adaptive Bellman-Ford), and recently proposed forms of asynch...

In this paper we present two versions of AntNet, a novel approach to adaptive learning of routing tables in wide area best-effort datagram networks. AntNet is a distributed multi-agent system inspired by the stigmergy model of communication observed in ant colonies. We report simulation results for AntNet on realistically sized networks using as performance measures throughput, packet delays an...

Applied into a various area of domains, the graph theory and its applications allow the determination of the shortest route. The common algorithm to solve this problem is Bellman-Kalaba, based on the matrix multiplying operation. If the graph is very large (e.g., the dimension of the associated incidence matrix is big) one of the main problems is to reduce the calculus time. This paper presents...

An Adaptive Dynamic Programming algorithm for nonlinear systems with unknown dynamics is developed. The algorithm is initialized with a positive definite cost functional / stabilizing control law pair (V0, k0) (coupled via the Hamilton Jacobi Bellman Equation). Given (Vi, ki), one runs the system using control law ki recording the state and control trajectories, with these trajectories used to ...

We consider the strong form of the John-Nirenberg inequality for the L2-based BMO. We construct explicit Bellman functions for the inequality in the continuous and dyadic settings and obtain the sharp constant, as well as the precise bound on the inequality’s range of validity, both previously unknown. The results for the two cases are substantially different. The paper not only gives another i...

We investigate the Cauchy problem for a nonlinear parabolic partial differential equation of Hamilton–Jacobi–Bellman type and prove some regularity results, such as Lipschitz continuity and semiconcavity, for its unique viscosity solution. Our method is based on the possibility of representing such a solution as the value function of the associated stochastic optimal control problem. The main f...

We propose local error estimates together with algorithms for adap-tive a-posteriori grid and time reenement in reinforcement learning. We consider a deterministic system with continuous state and time with innnite horizon discounted cost functional. For grid re-nement we follow the procedure of numerical methods for the Bellman-equation. For time reenement we propose a new criterion, based on ...

There are many kinds of distance{vector algorithms for adaptive routing in wide{area computer networks, ranging from the classical Distributed Bellman{Ford to several recent algorithms that have better performance. However, these algorithms have very complicated behaviors and their analyses in the literature has been incomplete (and operational). In this paper, we present a stepwise assertional...

A stochastic Ramsey model is studied with the Cobb-Douglas production function maximizing the expected discounted utility of consumption. We transformed the Hamilton-Jacobi-Bellman (HJB) equation associated with the stochastic Ramsey model so as to transform the dimension of the state space by changing the variables. By the viscosity solution method, we established the existence of viscosity so...

The aim of this paper is to study the stochastic monotonicity and continuity properties of the extinction time of Bellman–Harris branching processes depending on their reproduction laws. Moreover, we show their applications in an epidemiological context, obtaining an optimal criterion to establish the proportion of susceptible individuals in a given population that must be vaccinated in order t...