In many real-world data analysis problems observations arrive sequentially in time and it is required to perform inference on-line. Sequential learning provides us with techniques to fuse information, learn policies, analyse risks, forecast outcomes and make decisions in such a way that a current model is updated as new information becomes available. This framework of sequential learning is par...

We present a system based on sequential decision making for the online summarization of massive document streams, such as those found on the web. Given an event of interest (e.g. “Boston marathon bombing”), our system is able to filter the stream for relevance and produce a series of short text updates describing the event as it unfolds over time. Unlike previous work, our approach is able to j...

We describe a method for time-critical de­ cision making involving sequential tasks and stochastic processes. The method employs several iterative refinement routines for solv­ ing different aspects of the decision mak­ ing problem. This paper concentrates on the meta-level control problem of delibera­ tion scheduling, allocating computational re­ sources to these routines. We provide dif­ fere...

Solving sequential decision making problems in computational sustainability often requires simulators of ecology, weather, fire, or other complex phenomena. The extreme computational expense of these simulators stymie optimization and interactive visualization of decision rules (policies). This work presents our results in creating an interactive visualization for a wildfire management problem ...

Following work of Stroud and Saeger (Proceedings of ISI, Springer Verlag, New York, 2006) and Anand et al. (Proceedings of Computer, Communication and Control Technologies, 2003), we formulate a port of entry inspection sequencing task as a problem of finding an optimal binary decision tree for an appropriate Boolean decision function. We report on new algorithms for finding such optimal trees ...

Sequential decision making from experience, or reinforcement learning (RL), is a paradigm that is well-suited for agents seeking to optimize longterm gain as they carry out sensing, decision, and action in an unknown environment. RL tasks are commonly formulated as Markov Decision Problems (MDPs). Learning in finite MDPs enjoys several desirable properties, such as convergence, sample-efficienc...

This paper formalizes the notion of flexibility in sequential decision making and investigates conditions under which the use of flexibility as an additional criterion may be justified. The correlations between flexibility and value, and flexibility and risk, are studied under various assumptions of uncertainty and information. A number of approaches to constructing a multiple objective decisio...

Dynamic Programming

This paper introduces a hybrid model that combines connectionist, symbolic, and reinforcement learning for tackling reactive sequential decision tasks by a situated agent. Both procedural skills and high-level symbolic representations are acquired through an agent's experience interacting with the world, in a bottom-up direction. It deals with on-line learning, that is, learning continuously fr...