Software Engagement with Sleeping CPUs

Idle CPUs may enter power-saving hardware sleeps by, for instance, lowering the operating voltage and flushing the caches. However, wakeup delays that reach one hundred μSecs or more are disrupting the operations of fast devices like solid-state disks and tightly integrated accelerators. On the other hand, maximal power savings on modern multicores are only realized through continuous, simultaneous CPU sleeps. We argue that strong software engagement (at the OS and applications) is needed to maximize the power saving while maintaining the desired performance. Specifically, we present anticipatory CPU wakeups for latency-sensitive operations on fast devices. We also explore power-saving sleep shaping opportunities through non-work-conserving scheduling on smartphones and staged bursts on servers. 1 Motivation and Approach An idle CPU can enter hardware sleep states that power down various resources to conserve energy. The wakeup of a sleeping CPU incurs a delay due to the restoration of the hardware operating condition and state. For instance, an Intel E5-2620 v3 Haswell socket can save a few dozen Watts of power by sleeping at the ACPI C6 state (cycles halted; clocks shut off; L1 / L2 cache flushed; core voltage removed). On the other hand, its wakeup causes 106μSecs of extra delay for a simple network ping-pong service. Emerging hardware and workload trends have highlighted the challenges of the CPU sleep management. From the performance perspective, wakeup delays of dozens or hundreds of μSecs could cause excessive disruptions. Commodity solid-state disks are already operating at 100μSec-level latencies (particularly for reads). Emerging integrated accelerators like GPUs (exemplified by Intel IvyBridge and Haswell, AMD Fusion APUs, and NVIDIA Denver) allow fast CPU / accelerator interactions and facilitate efficient acceleration of fine-grained tasks. A slow CPU wakeup produces significant slowdown for these device operations. In a further example, latency-sensitive network services (e.g., a memorycached hash table) inside a data center may suffer multifold slowdown if a wakeup from a CPU deep sleep is involved. From the power perspective, the benefit of CPU sleeping is highly influenced by the sleep pattern. First, it is desirable for a CPU to sleep continuously to minimize the energy costs during active / sleeping state transitions. Second, due to aggressive resource sharing, modern multicores realize maximal power saving only when all CPU cores/threads sleep simultaneously. Unfortunately, continuous, simultaneous CPU sleeps are rare under the conventional work-conserving CPU scheduling when finegrained units of work may activate one or a few CPUs in intermittent fashions. This paper argues for strong software engagement (at both OS and application levels) of CPU sleep management to maintain high performance when needed and maximize the energy saving when possible. Specifically in low-latency operating conditions, the CPU should start waking up early (before the work-triggering interrupt) such that the CPU is immediately ready for work when needed. Such anticipatory wakeups are best requested by the software layers with knowledge of or ability to model the time of future work resumption. The OS should aggregate anticipatory wakeup requests and maintain a CPU sleep plan with high efficiency and (if desirable) proper energy accounting. On the other hand, many system and application contexts manifest a high degree of slacks in their quality-ofservice requirements. For example, due to the long operation time of the wide area network and other peripheral devices in smartphones, some tasks can be slowed substantially without compromising the user experience. Also, requests in a web server application may be delayed as long as they are all completed within a specified latency threshold. Such quality-of-service flexibility presents opportunities for shaping the CPU idleness patterns (through delaying, staging, and consolidating work) toward continuous, simultaneous CPU sleeps with high power saving. Much of prior attention on energy-efficient CPU management has targeted the processor frequency control [13, 22, 23]. Among the most seminal, Weiser et