Parallel out-of-core sorting and fast accesses to disks

نویسندگان

  • Christophe Cérin
  • Olivier Cozette
  • Gil Utard
  • Hazem Fkaier
  • Mohamed Jemni
چکیده

The paper addresses two problems. We investigate the problem of parallel external sorting in the context of a form of heterogeneous clusters then we investigate the impact of efficient disk remote accesses on the performance of external sorting. We explore three techniques to show how they can be deployed for clusters with proportional processor performances. We also validate the READ library, an efficient implementation of remote SCSI disk accesses. We derive a new parallel sorting algorithm that is adapted to the READ interface. The expected gain of using READ is compared to the measured gain for one external sorting implementation.

برای دانلود متن کامل این مقاله و بیش از 32 میلیون مقاله دیگر ابتدا ثبت نام کنید

ثبت نام

اگر عضو سایت هستید لطفا وارد حساب کاربری خود شوید

منابع مشابه

Sorting and Selection on Parallel Disk Models

Data explosion is an increasingly prevalent problem in every field of science. Traditional out-of-core models that assume a single disk have been found inadequate to handle voluminous data. As a result, models that employ multiple disks have been proposed in the literature. For example, the Parallel Disk Systems (PDS) model assumes D disks and a single computer. It is also assumed that a block ...

متن کامل

Duality between Prefetching and Queued Writing with Parallel Disks

Parallel disks promise to be a cost effective means for achieving high bandwidth in applications involving massive data sets, but algorithms for parallel disks can be difficult to devise. To combat this problem, we define a useful and natural duality between writing to parallel disks and the seemingly more difficult problem of prefetching. We first explore this duality for applications involvin...

متن کامل

A survey of out-of-core algorithms in numerical linear algebra

This paper surveys algorithms that efficiently solve linear equations or compute eigenvalues even when the matrices involved are too large to fit in the main memory of the computer and must be stored on disks. The paper focuses on scheduling techniques that result in mostly sequential data accesses and in data reuse, and on techniques for transforming algorithms that cannot be effectively sched...

متن کامل

Performing Out-of Core FFTs on Parallel Disk Systems

The Fast Fourier Transform (FFT) plays a key role in many areas of computational science and engineering. Although most one-dimensional FFT problems can be solved entirely in main memory, some important classes of applications require out-of-core techniques. For these, use of parallel I/O systems can improve performance considerably. This paper shows how to perform one-dimensional FFTs using a ...

متن کامل

A high-performance sorting algorithm for multicore single-instruction multiple-data processors

Many sorting algorithms have been studied in the past, but there are only a few algorithms that can effectively exploit both SIMD instructions and thread-level parallelism. In this paper, we propose a new high-performance sorting algorithm, called Aligned-Access sort (AA-sort), for exploiting both the SIMD instructions and thread-level parallelism available on today's multicore processors. Our ...

متن کامل

ذخیره در منابع من


  با ذخیره ی این منبع در منابع من، دسترسی به آن را برای استفاده های بعدی آسان تر کنید

برای دانلود متن کامل این مقاله و بیش از 32 میلیون مقاله دیگر ابتدا ثبت نام کنید

ثبت نام

اگر عضو سایت هستید لطفا وارد حساب کاربری خود شوید

عنوان ژورنال:
  • IJHPCN

دوره 3  شماره 

صفحات  -

تاریخ انتشار 2005