Multistage bus network (MBN): an interconnection network for cache coherent multiprocessors

Single bus multiprocessor systems do not scale well due to the limited bandwidth of the bus. Hierarchical bus interconnections are scalable, but unfortunately the top level bus becomes a bottleneck for larger systems. In this paper we present a novel interconnection network called the Multistage Bus Network(MBN). The MBN consists of multiple stages of buses and it preserves the bandwidth properties of a conventional Multistage Interconnection Network(M1N). At the same time the MBN also retains the economic design and fast broadcast advantages of the bus based systems. We show that the MBN performs almost as good as a MIN and is much more cost effective than the MIN. We also design and study a multistage snooping cache protocol for the MBN. 1 Introduction The efficiency of a large multiprocessor system depends to a great extent on the underlying interconnection network. When the interconnection network becomes more complex, the memory access time increases and severely degrades the system performance. A large amount of research has been done on the design of efficient interconnection networks for such multiproces-sors. Among various interconnection networks, the hierarchical buses[l2, 131 and Multistage Interconnection Networks(MINs)[3, 8, 101 show a good promise for use in future. These networks usually operate under synchronous packet switched environments[2,3]. The processor submits a memory request and waits till it is satisfied before submitting another request. In a strictly hierarchical system[l2, 131, there are a number of buses connected in the form of a tree between the processors and the memories. When the system size grows, the buses in the upper levels, especially the top level bus, form the system bottleneck. Thus, the scalability of a hierarchical bus system becomes limited by the bandwidth of the top level bus. However, the simplicity of the bus based designs and the availability of a fast broadcasting mechanism are factors that make these interconnections attractive. Moreover, bus based networks efficiently support cache coherence protocols due to their snooping capabilities[l, 5, 121. On the other hand, the MINs offer uniform band-width across all stages of the network. The bandwidth of the network increases proportionally with the increase in system size, making the MIN a highly scal-able interconnection. Several commercial machines and research projects are based on such MINs[2, 6, 111. The switches in a MIN are made up of small crossbar switches. When the system size grows, bigger switches can be used t o keep the number …