PRUDENT: A Sequential-Decision-Making Framework for Solving Industrial Planning

While network query engines make it possible to gather andcombine data from multiple Web sources, these systemsprimarily focus on efficient query execution and do notsolve some of the more complicated problems of onlineinformation gathering. Such problems require alternativetypes of control flow and better integration with the externalworld; the unique nature of the Web requires query plans beexpressive enough to accommodate these demands. In thispaper, we describe an information gathering plan languagethat is expressive and promotes efficient execution.Through its support for subplans, recursion, and a unique setof operators, the language allows plans that can interactivelygather data over a series of pages, monitor remote sources,and asynchronously notify users of updates and results. Wealso present a execution system that efficiently implementsthe plan language using a dataflow-style executor capable ofpipelining data between operators.