A UNICORE Globus Interoperability Layer

We approach the issue of defining the set of minimal Grid services by considering the problem of interoperability of different Grid systems. On the one hand if we have identified the minimal set of Grid services then any well-composed resource request expressed in terms of this minimal set will be honoured by other Grids even if some translation may need to be made between different resource description mechanisms. On the other, if we are seeking to identify such a minimal set then looking at practical issues of interoperability can help since the minimal set can be defined as the intersection of all interoperability sets between objects that are considered to be Grids. This is a constructionist approach and avoids a precise definition of what a Grid is by refining from systems commonly agreed to be Grids. This case study uses two well-established systems, Globus and UNICORE, that implement reasonably complete solutions for the metacomputing Grid problem. An additional benefit of our approach, providing interoperability between Globus and UNICORE, is that it would result in an advanced set of Grid services that gain strength from each other, addressing the problem of what is a usable as opposed to minimal Grid. Here we gain pointers to augmenting each system to utilise features not initially contained in the other, i.e. beyond the minimal set. This can be considered as the union (in the set-theoretic sense) as opposed to * Fujitsu European Laboratories., {snelling,svdb}@fle.fujitsu.com ††† Argonne National Laboratory, [email protected] *** Forschungszentrum Juelich GmbH, [email protected] † University of Manchester, {jon.maclaren,john.brooke}@man.ac.uk ** University of Southampton, [email protected] †† Pallas GmbH, [email protected] intersection of two Grid implementations. This paper outlines some of these parallels and differences as they relate to the development of an interoperability layer between Unicore and Globus. Given the increasing ubiquity of Globus, what emerges is the desire for a hybridised facility that utilises the Unicore work-flow management of complex, multi-site tasks, but that can run on either Unicoreor Globus-enabled resources. The technical challenge in achieving this, addressed in this paper, consists of mapping resource descriptions from both grid environments to an abstract format appropriate to work-flow preparation, and then the instantiation of work-flow tasks on the target systems. Other issues such as reconciling disparate security models are addressed, authentication being considered as part of the minimal set. 1. The Grid Interoperability Problem This Use Case paper addresses the current concern of the GPA WG in identifying a minimal set of Grid services. It proposes to use the interoperability issue to shed light on this question, since two Grids must have a common set of services via which they interoperate (although these may be described in different languages in either Grid). A possible way of approaching the minimal services problem is to define the minimal set as the intersection of all such common sets of interoperable common sets, provided that the intersection is extended to give it “nice” properties of closure (e.g. services are consistent and complete in the sense that appeal does not have to be made outside the set for providing the defined minimal functionality). Even if this concept turns out to be a little too idealistic, in a practical sense it can be seen that identifying services which are used consistently by a number of differing Grid systems is an indication that they are somehow capturing the essentials of what a Grid is. We note that these considerations lead very directly to considering the ontology problem for Grid services, solving this problem is one of the very important steps to a true Semantic Grid. The very dynamic nature of Grid computing has meant that a number of different definitions of a computational Grid have been given. This is not surprising, for comparison in the field of nonlinear systems there a number of different usages of the term “chaos” which are not completely compatible even though there is a deep definition of a chaotic attractor which is mathematically precise. In [Fost98] reference was made to the electrical Power Grid model and in [Fost01] Grids were envisaged as dynamically constituted Virtual Organizations, leading to the Grid Services concept presented in [Fost02] and known as OGSA. The VO concept is interesting in that a very similar concept was proposed quite independently and termed a miniGrid [Broo00]. OGSA has the potential of reconciling some of the differing Grid concepts in the manner that the precise mathematical definition of “chaos” unifies the looser, more case-specific, usages widespread in mathematics and physics literature. For our study the Grid Services and Virtual Organization viewpoints are particularly apt in that they focus attention on the concept of the joining of organizations and groups of services, which is where interoperability is highlighted. For the purposes of our study we wish to highlight another important aspect common to most usages of Grid. This is the expectation that there exist two important groupings used within the VO concept. One is the group of Resource Requestors (RR) and the other the group of Resource Providers (RP). In the metacomputing usage of the Grid a resource requestor would normally be a users “job”, i.e. a request for various processing, storage and transfer operations associated with some well defined task usually initiated from a single point (e.g. a specific workstation). This is a one-to-many mapping from the space of RR to the space of RP. In more sophisticated scenarios the RP space can recursively 1 A topologically transitive set containing a dense set of periodic orbits. These two conditions imply the often quoted “sensitivity to initial conditions” which latter is not sufficient to describe chaotic attractors. cast itself as an object in the RR space by passing on the resource request (onward transfer of service requests). The Globus and UNICORE Grid middleware systems that we examine in this case study have a particular orientation to this mode of Grid usage. We note that the human usage of the Grid comes in here as an “invisible” factor. The one-to-many concept comes from a single person submitting a complex request. In the case of collaborative working, however, there is a many-to-many model and the interactions of the RR and RP spaces are highly dynamic. The Grid abstraction is particularly useful for examining such complex usage patterns since it allows each physical resource in the Grid to be used in either an RR or RP context. It is in this sense that one can genuinely consider the Access Grid [Agdoc1] as a Grid in which each AG node is simultaneously and persistently in both the RR and RP spaces (full many-to-many interaction). The RR and RP spaces come naturally from the consideration of the implications of resource sharing in a Virtual Organization. We use the work “spaces” loosely, whether these can be considered as mathematically well-defined spaces (e.g. vector spaces) needs to be investigated. These considerations are interesting and worthy of future study. However for this current Use Case paper we revert to the more traditional (i.e. over 10 years old) metacomputing usage of Grid. Much current work can be cast into this framework (for example most of the various computational and data Grids) and it is to this area that the two interoperable systems Globus and UNICORE are primarily directed in their current versions. We will not fully address the newer OGSA framework although it has been shown elsewhere that the Grid middleware described here can be adapted to use OGSA [Snell02]. Globus has been described as a Toolkit for the Grid and Unicore as a grid enabled workflow environment. This distinction probably best describes the differences in approach. Globus provides a collection of tools and an architecture that constitute an infrastructure for Grid enabled application development. Unicore on the other hand provides a vertical solution for performing work-flow task management across multiple disparate resources. Unicore is engineered to allow existing applications and their users to become mobile [Erwi01]. This mobility requires management of computational and data resources and their coordination. The Unicore work-flow services (see below) support the management of the complex, multi-site tasks that arise in this mobile environment. In addition to EUROGRID [Euro02], the European Commission has recently funded the Grid Interoperability Project, to investigate the issues arising from interoperability between Grids. This paper outlines some of the parallels and differences as they relate to the development of an interoperability layer between Unicore and Globus. Given the established and desirable high-level workflow GUI that Unicore provides, and the pervasiveness of the Globus Toolkit on high-end compute resources, it is natural to hybridise the two by fitting the high-level Unicore functionality over the lower-level components of the Globus Toolkit. Our interoperability case study now becomes clearly defined: 2 GRid Interoperability Project, IST-20001-32257. “Can a workflow request constructed in the RR space of UNICORE be realised in the RP space of Globus?” More colloquially, “Can Globus run a UNICORE job?”. Since Globus and UNICORE are well-constructed systems that have been used in demanding real environments we know that each can map from RR to RP space within their own resource description mechanisms and can handle issues such as authentication and instantiation of workflow requests.