Crossroads - Spring 2010

14 Spring 2010/ Vol. 16, No. 3 www.acm.org/crossroads Crossroads Besides public data repositories, scientific collaborations maintain community-wide data resources. For example, in gravitational-wave physics, the Laser Interferometer Gravitational-Wave Observatory [3] maintains geographically distributed repositories holding time-series data collected by the instruments and their associated metadata. Along with the large increase in online data, the need to process these data is growing. In addition to traditional high performance computing (HPC) centers, a nation-wide cyberinfrastructure—a computational environment, usually distributed, that hosts a number of heterogeneous resources; cyberinfrastructure could refer to both grids and clouds or a mix of the two—is being provided to the scientific community, including the Open Science Grid (OSG) [36] and the TeraGrid [47]. These infrastructures, also known as grids [13], allow access to highperformance resources over wide area networks. For example, the TeraGrid is composed of computational and data resources at Indiana University, Louisiana University, University of Illinois, and others. These resources are accessible to users for storing data and performing parallel and sequential computations. They provide remote login access as well as remote data transfer and job scheduling capabilities. Scientific workflows are used to bring together these various data and compute resources and answer complex research questions. Work flows describe the relationship of the individual computational components and their input and output data in a declarative way. In astronomy, scientists are using workflows to generate science-grade mosaics of the sky [26], to examine the structure of galaxies [46], and, in general, to understand the structure of the universe. In bioinformatics, researchers are using workflows to understand the underpinnings of complex diseases [34, 44]. In earthquake science, workflows are used to predict the magnitude of earthquakes within a geographic area over a period of time [10]. In physics, workflows are used to search for gravitational waves [5] and model the structure of atoms [40]. In ecology, scientists use workflows to explore the issues of biodiversity [21]. Today, workflow applications are running on national and international cyberinfrastructures such as OSG, TeraGrid, and EGEE [11]. The broad spectrum of distributed computing provides unique opportunities for large-scale, complex scientific applications in terms of resource selection, performance optimization, and reliability. In addition to the large-scale cyberinfrastructure, applications can target campus clusters, or utility computing platforms such as commercial [1, 17] and academic clouds [31]. However, these opportunities also bring with them many challenges. It’s hard to decide which resources to use and how long they will be needed. It’s hard to determine what the cost-benefit tradeoffs are when running in a particular environment. And it’s difficult to achieve good performance and reliability for an application on a given system. Clouds have recently appeared as an option for on-demand computing. Originating in the business sector, clouds can provide computational and storage capacity when needed, which can result in infrastructure savings for a business. One idea driving cloud computing is that businesses can plan only for a sustained level of capacity while reaching out to the cloud for resources in times of peak demand. When using the cloud, consumers pay only for what they use in terms of computational resources, storage, and data transfer in and out of the cloud. Although clouds were built primarily with business computing needs in mind, they are also being considered in science. In this article we focus primarily on workflow-based scientific applications and describe how they can benefit from the new computing paradigm.