Markov Decision Process (MDP) is a well-known framework for devising the optimal decision making strategies under uncertainty. Typically, the decision maker assumes a stationary environment which is characterized by a time-invariant transition probability matrix. However, in many real-world scenarios, this assumption is not justified, thus the optimal strategy might not provide the expected per...

The main objective of electric power dispatch is to provide electricity to the customers at low cost and high reliability. Transmission line failures constitute a great threat to the electric power system security. We use a Markov decision process (MDP) approach to model the sequential dispatch decision making process where demand level and transmission line availability change from hour to hou...

The semi-Markov decision model is a powerful tool in analyzing sequential decision processes with random decision epochs. In this paper, we have built the semi-Markov decision process (SMDP) for the maintenance policy optimization of condition-based preventive maintenance problems, and have presented the approach for joint optimization of inspection rate and maintenance policy. Through numerica...

In a network of low-powered wireless sensors, it is essential to capture as many environmental events as possible while still preserving the battery life of the sensor node. This paper focuses on a real-time learning algorithm to extend the lifetime of a sensor node to sense and transmit environmental events. A common method that is generally adopted in ad-hoc sensor networks is to periodically...

Predictive maintenance is based on observing an indicator of the state of a system, at different intervals of time which gives the decision maker some information about the exact state. The problem is to obtain an optimal replacement policy minimizing the long run expected cost per unit of time and to formulate it as a partially observable Markov decision process.

The Distributed Cooperative Modeling System (DCMS) solves complex decision problems involving a lot of participants with different viewpoints by network based distributed modeling and multi-template aggregation. This thesis aims at extending the system with support for dynamic decision making process. First, the thesis presents a discussion of characteristics and optimal policy finding Markov D...

Learning the enormous number of parameters is a challenging problem in model-based Bayesian reinforcement learning. In order to solve the problem, we propose a model-based factored Bayesian reinforcement learning (F-BRL) approach. F-BRL exploits a factored representation to describe states to reduce the number of parameters. Representing the conditional independence relationships between state ...

Meta-level control manages the allocation of limited resources to deliberative actions. This paper discusses efforts in adding meta-level control capabilities to a Markov Decision Process (MDP)-based scheduling agent. The agent’s reasoning process involves continuous partial unrolling of the MDP state space and periodic reprioritization of the states to be expanded. The meta-level controller ma...

Planning in Large Domains: The Markov decision process (MDP) formalism has emerged as a powerful representation for control and planning domains that are subject to stochastic effects. In particular, MDPs model situations in which an agent can exactly observe all relevant aspects of the world’s state but in which the effects of the agent’s actions are nondeterministic. Though the theory of MDPs...

Partially observable Markov decision processes have been widely used to provide models for real-world decision making problems. In this paper, we will provide a method in which a slightly different version of them called Mixed observability Markov decision process, MOMDP, is going to join with our problem. Basically, we aim at offering a behavioural model for interaction of intelligent agents w...