Towards a Protocol for Benchmark Selection in IPC
نویسندگان
چکیده
The planning competition has traditionally played an important role in motivating research and advances in Planning & Scheduling techniques. Despite its pivotal role in the planning community, some aspects of the competition have not been engineered yet. This is the case for the protocol for selecting benchmark instances. Benchmarks are of critical importance, since they can significantly affect competition results. In this paper we describe desirable properties of a selection protocol, discuss methods exploited in past SAT and planning competitions, and identify challenges that organisers of future competitions have to address in order to improve reliability and usefulness of the insights gained by looking at competitions’ results.
منابع مشابه
RT-IPC: An IPC Extension for Real-Time Mach
Interprocess communication (IPC) provides the fundamental mechanism for the Mach microkernel to be extensible and exible. Mach IPC provides eecient communication mechanisms for many applications. However, it does not provide suucient functionality for real-time applications which have rigid timing constraints among threads. In Real-Time Mach (RT-Mach), we have extended Mach IPC to be priority i...
متن کاملSimulation-based inter-professional education to improve attitudes towards collaborative practice: a prospective comparative pilot study in a Chinese medical centre
OBJECTIVES Inter-professional education (IPE) builds inter-professional collaboration (IPC) attitude/skills of health professionals. This interventional IPE programme evaluates whether benchmarking sharing can successfully cultivate seed instructors responsible for improving their team members' IPC attitudes. DESIGN Prospective, pre-post comparative cross-sectional pilot study. SETTING/PART...
متن کاملCOPLAS 2010 Proceedings of the Workshop on Constraint Satisfaction
Introduction of explicit time and resources into planning that typically focuses on causal relations between actions is an important step towards modelling real-life problems. In this paper we propose a suboptimal domain-independent planning system Filuta that focuses on planning, where time plays a major role and resources are constrained. We benchmark Filuta on the planning problems from the ...
متن کاملAn Evaluation of Stratified Sampling of Microarchitecture Simulations
Recent research advocates applying sampling to accelerate microarchitecture simulation. Simple random sampling offers accurate performance estimates (with a high quantifiable confidence) by taking a large number (e.g., 10,000) of short performance measurements over the full length of a benchmark. Simple random sampling does not exploit the often repetitive behaviors of benchmarks, collecting ma...
متن کاملExperiments in Automatic Benchmark Synthesis
In the nineteen-eighties, synthetic workloads such as Whetstone and Dhrystone fell out of favor as benchmarks of computer performance because they became unrepresentative of the performance of continuously-evolving applications. Hand-coded synthetic benchmarks take work to develop and maintain, are language feature specific, and are subject to compiler optimizations that eliminate code meant to...
متن کامل