The reinforcement learning literature typically assumes fixed state transition functions for the sake of tractability. However, in many real-world tasks, the state transition function changes over time, and this change may be governed by exogenous variables outside of the control loop. This can make policy learning difficult. In this paper, we propose a new algorithm to address the aforemention...

This paper introduces a graphical notation called algebraic state transition diagrams (ASTD), which allows for the combination of state transition diagrams using classical process algebra operators like sequence, iteration, parallel composition, quantified choice and quantified synchronization. It is inspired from automata, statecharts and process algebras. Hence, it combines the strength of al...

This paper introduces a graphical notation called algebraic state transition diagrams (ASTD), which allows for the combination of state transition diagrams using classical process algebra operators like sequence, iteration, parallel composition, quantified choice and quantified synchronization. It is inspired from automata, statecharts and process algebras. Hence, it combines the strength of al...

We are interested in the problem of transition reduction of nondeterministic automata. We present some results on the reduction of the automata recognizing the language L(En) denoted by the regular expression En = (1 + ε) · (2 + ε) · (3 + ε) · · · (n+ ε). These results can be used in the general case of the transition reduction problem.

Opacity is a confidentiality property that characterizes whether a ‘‘secret’’ of a system can be inferred by an outside observer called an ‘‘intruder’’. In this paper, we consider the problem of enforcing opacity in systems modeled as partially-observed finite-state automata. We propose a novel enforcement mechanism based on the use of insertion functions. An insertion function is a monitoring ...

In this paper we propose a queueing model with a batch service with both event and time counter. Its lower power consumption is an improvement over the batch service with only an event counter which probably causes a longer mean response time. If there are few events in the queue, the event counter cannot reach the lower boundary of the number of events, and thus the computer system will requir...

An optimum channel input-traffic control (OCIC) scheme is proposed and investigated for slotted frequency-hopped spread spectrum multiple access communication systems. The conditional throughput is obtained, the state transition probability is derived, the steady-state throughput is obtained, and the mean packet delay is calculated. It is shown that the performance of the OCIC scheme is better ...

We study on a special class of one dimensional, three neighborhood, two state, linear non-group Cellular Automata called LNGCA. We find property related to state transition diagram of this particular class of CA. The results have been verified with computer experiments.

When studying a biological regulatory network, it is usual to use boolean network models. In these models, boolean variables represent the behavior of each component of the biological system. Taking in account that the size of these state transition models grows exponentially along with the number of components considered, it becomes important to have tools to minimize such models. In this pape...

Two-level phonology, as currently practiced, has two severe limitations. One is that phonological generalizations are generally expressed in terms of transition tables of finite-state automata, and these tables are cumbersome to develop and refine. The other is that lexical idiosyncrasy is encoded by introducing arbitrary diacritics into the spelling of a morpheme. This paper explains how phono...