Docker-Enabled Build and Execution Environment (BEE): an Encapsulated Environment Enabling HPC Applications Running Everywhere
نویسندگان
چکیده
Variations in High Performance Computing (HPC) system software configurations mean that applications are typically configured and built for specific HPC environments. Building applications can require a significant investment of time and effort for application users and requires application users to have additional technical knowledge. Container technologies like Docker bring great benefits to the application development, build and deployment processes. While much cloud computing infrastructure is already designed to support Docker, little work has been done to support production Docker deployment on HPC systems. In this work, we propose a Docker-enabled Build and Execution Environment (BEE) for HPC systems and detail a standard backend for BEE using virtual machines, the BEE-VM. This brings many of the benefits of Docker to existing HPC machines in user-space without the requirement of specialized pre-installed software and with no system administrator configuration. We show that current HPC application can be easily configured to run within BEE, eliminating the need to reconfigure and rebuild applications for different systems while preserving comparable performance.
منابع مشابه
Contain This, Unleashing Docker for HPC
Containers are a lightweight virtualization method for running multiple isolated Linux systems under a common host operating system. Container-based computing is revolutionizing the way applications are developed and deployed. A new ecosystem has emerged around the Docker platform to enable container based computing. However, this revolution has yet to reach the HPC community. In this paper, we...
متن کاملBuilding a Virtual HPC Cluster with Auto Scaling by the Docker
Solving the software dependency issue under the HPC environment has always been a difficult task for both computing system administrators and application scientists. This work would like to tackle the issue by introducing the modern container technology, the Docker, to be specific. By integrating the auto-scaling feature of service discovery with the light-weight virtualization tool, the Docker...
متن کاملNGSeasy: a next generation sequencing pipeline in Docker containers [version 1; referees: 3 approved with reservations]
Bioinformatic pipelines often use large numbers of components Motivation and deploying them incurs substantial configuration and maintenance burden that remains a significant barrier to reproducible research. Our aim is to define a new paradigm and best practices for developing, distributing and running pipelines encapsulated in Docker containers (lightweight virtualization), with a focus on ne...
متن کاملHPC Containers in Use
Linux containers in the commercial world are changing the landscape for application development and deployments. Container technologies are also making inroads into HPC environments, as exemplified by NERSC’s Shifter and LBL’s Singularity. While the first generation of HPC containers offers some of the same benefits as the existing open container frameworks, like CoreOS or Docker, they do not a...
متن کاملThe Dockstore: enabling modular, community-focused sharing of Docker-based genomics tools and workflows
As genomic datasets continue to grow, the feasibility of downloading data to a local organization and running analysis on a traditional compute environment is becoming increasingly problematic. Current large-scale projects, such as the ICGC PanCancer Analysis of Whole Genomes (PCAWG), the Data Platform for the U.S. Precision Medicine Initiative, and the NIH Big Data to Knowledge Center for Tran...
متن کاملذخیره در منابع من
با ذخیره ی این منبع در منابع من، دسترسی به آن را برای استفاده های بعدی آسان تر کنید
برای دانلود متن کامل این مقاله و بیش از 32 میلیون مقاله دیگر ابتدا ثبت نام کنید
ثبت ناماگر عضو سایت هستید لطفا وارد حساب کاربری خود شوید
ورودعنوان ژورنال:
- CoRR
دوره abs/1712.06790 شماره
صفحات -
تاریخ انتشار 2017