A central challenge in ad hoc networks is the design of routing protocols that can adapt their behavior to frequent and rapid changes at the network level. Choosing between reactive, proactive, or hybrid routing regimes and selecting appropriate configuration parameters for a chosen protocol are difficult tasks. This paper introduces a framework, called TAF, for seamlessly adapting between proa...

The Semantic Web envisions a distributed environment with well-defined data that can be understood and used by machines. This machine-understandable knowledge allows intelligent agents to automatically take decisions and perform tasks on our behalf. In the past, different Semantic Web policy languages have been developed as a powerful means to describe a system’s behavior by defining statements...

Environmental change in general, and climate change in particular, can lead to changes in distribution of fish stocks. When such changes involve transboundary fish stocks, the countries sharing the stock need to reconsider their harvesting policies. We investigate the effects of changing stock distribution on the optimal fishing policies in a two players’ non-cooperative game. We compare reacti...

As data continues to be generated at exponentially growing rates in heterogeneous formats, fast analytics to extract meaningful information is becoming increasingly important. Systems widely use in-memory caching as one of their primary techniques to speed up data analytics. However, caches in data analytics systems cannot rely on simple caching policies and a fixed data layout to achieve good ...

Ubiquitous computing is resulting in a proliferation of cyber-physical systems that host or manage valuable physical and digital assets. These assets can be harmed by malicious agents through both cyber-enabled or physically-enabled attacks, particularly ones that exploit the often ignored interplay between the cyber and physical world. The explicit representation of spatial topology is key to ...

We propose and study a new model for reinforcement learning with rich observations, generalizing contextual bandits to sequential decision making. These models require an agent to take actions based on observations (features) with the goal of achieving long-term performance competitive with a large set of policies. To avoid barriers to sample-efficient learning associated with large observation...

There are several shortcomings in the existing data migration techniques in Hierarchical Storage Systems (HSS). The first and the most important among them is that data migration policies are user defined hence static and reactive. Secondly, data migration at the host side is not yet completely explored. The other major drawbacks are that each storage tier is modelled as an agent; the data migr...

Security mechanisms are at the base of modern computer systems, demanded to be more and more reactive to changing environments and malicious intentions. Security policies unable to change in time are destined to be exploited and thus, system security compromised. However, the ability to properly change security policies is only possible once the most effective mechanism to adopt under specific ...

We present Policy Gradient Actor-Critic (PGAC), a new model-free Reinforcement Learning (RL) method for creating limited-memory stochastic policies for Partially Observable Markov Decision Processes (POMDPs) that require long-term memories of past observations and actions. The approach involves estimating a policy gradient for an Actor through a Policy Gradient Critic which evaluates probabilit...

Reactive (memoryless) policies are sufficient in completely observable Markov decision processes (MDPs), but some kind of memory is usually necessary for optimal control of a partially observable MDP. Policies with finite memory can be represented as finite-state automata. In this paper, we extend Baird and Moore’s VAPS algorithm to the problem of learning general finite-state automata. Because...