A Study of Multithreaded Benchmarks on the Hewlett-Packard X- and V-Class Architectures

The Hewlett-Packard Xand V-Class ccNUMA systems appear well suited to exploiting coarse and ne-grained parallelism, using multithreading techniques. This paper brie y summarizes the multilevel memory subsystem for the Xand V-Class platforms. Typical MPP distributed memory programming concerns for the codes under investigation, such as explicit memory localization and load balancing, are compared to relevant issues when porting and tuning for the Xand V-Class. This paper uses two small benchmarks as the basis for investigating di erences running multithreaded codes in SPP-UX and HP-UX environments. One code is from the Command, Control, Communication and Intelligence (C3I) Parallel Benchmark suite, shown to have the potential for large-scale parallelization with straightforward multithreading techniques. The second benchmark exhibits the computationally dynamic behavior of a thermally-driven explosion model. Both codes are shown to stress the HP systems' ability to keep memory close to processors and appropriate threads of execution. 1 System Architecture High-Level Overview The Hewlett Packard X-Class and V-Class servers are symmetric multiprocessor (SMP) cache coherent nonuniform memory-access (ccNUMA) systems, providing the illusion of simple, integrated memory. The fundamental building block of the X and V-Class systems is the hypernode. Each hypernode is an SMP, containing multiple processors, a crossbar, caches and synchronous DRAM (SDRAM) distributed across multiple memory boards. Hypernodes are connected through a ring interface, referred to as Coherent Toroidal Interconnect (CTI). 1.1 Globally Shared Memory All processors in X and V-Class servers share memory within a hypernode (local memory) and across the entire collection of hypernodes (remote memory). The global shared memory (GSM) subsystem is two-level; each of which is tuned for a particular class of data sharing. Level one includes a crossbar, connecting memory, processors and I/O within a hypernode. The crossbar provides high bandwidth, low latency nonblocking access from processors and I/O channels to hypernode local memory. Level two encompasses the interconnection between hypernodes through the use of CTI rings. The CTI is a collection of rings used to access remote memory across hypernodes. CTI is specially designed to enable extremely high-bandwidth (15.3 Gigabytes/second) data movement between processors, I/O devices and memory on a node. Memory references not satis ed by level one memory subsystem requests use the crossbar and CTI (level two) interconnection, to access data not in hypernode local memory. 1.2 Globally Shared Memory and Cache Coherence All processor references to memory, cause copies of the accessed data to be encached into either the instruction or data cache of each processor. Processor local cache holds the most frequently accessed data. When cache misses are encountered, an attempt is made to retrieve data from node local SDRAM. When requested data is not resident in local cache nor local memory, the data resides in memory of another node. Such remote data is obtained via the CTI interconnect. Since remote memory accesses come with a high latency cost, each node has local memory dedicated to CTI cache. The CTI cache is responsible for caching data accessed by other nodes, thus reducing the time to retrieve remote memory. When a processor modi es data within its data cache, and another processor references the same data, stale data conditions exist. The Xand V-Class hardware supports cache coherence, thus each processor is assured caches always contain the latest data values. Hardware supported cache coherence relieves the programmer from explicitly ushing cache and tending to expensive synchronization details. Maintaining coherent copies of processor cache is achieved by adherence to the following rules: Any number of read encachements on a cache line can occur concurrently. A cache line can be readshared in multiple caches. A processor must exclusively \own" a cache line in order to write data into a cache line. Modi ed cache lines must be written back to memory from the cache before overwriting occurs. Particular hardware characteristics for the platforms used in our benchmarking are listed in Table 1. System CPU CPUs/Node Nodes Clock Speed Data/Instr Cache Memory/Node OS V2250 PA-8200 16 2 240 MHz 2MB/2MB 8GB HP-UX 11.01 X2000 PA-8000 16 16 180 MHz 1MB/1MB 4GB SPP-UX 5.3 Tab. 1: Platform Speci cs 2 The Benchmark Problems The U.S. Air Force Rome Laboratory C3I Parallel Benchmark Suite [1] consists of eight problems chosen to compactly represent the essential elements of real C3I applications. The C3I representative benchmark discussed in this paper is Threat Analysis|a time-stepped simulation involving the trajectories of incoming ballistic threats, with computation of options for intercepting the threats. The benchmark includes input data sets, a sequential C program, and output sets to validate correctness. Threat Analysis is computationally intensive, compact, and involves non-trivial data and control structures. It should make a good test for compiler parallelization e ectiveness and multithreaded performance analysis on the Xand V-Class architectures.