Nilpotent Orbits, Normality, and Hamiltonian Group Actions

Let M be a G-covering of a nilpotent orbit in 0 where G isa complex semisimple Lie group and g = Lie(G). We prove that under Poisson bracket the space R[2] of homogeneous functions on M of degree 2 is the unique maximal semisimple Lie subalgebra of R = R{M) containing g . The action of g' ~ R[2] exponentiates to an action of the corresponding Lie group G' on a G'-cover M' of a nilpotent orbit in g' such that M is open dense in M'. We determine all such pairs (g C g') • The theory of coadjoint orbits of Lie groups is central to a number of areas in mathematics. A list of such areas would include ( 1 ) group representation theory, (2) symmetry-related Hamiltonian mechanics and attendant physical theories, (3) symplectic geometry, (4) moment maps, and (5) geometric quantization. From many points of view the most interesting cases arise when the group G in question is semisimple. For semisimple G the most familar of the orbits are of semisimple elements. In that case the associated representation theory is pretty much understood (Borel-Weil-Bott and noncompact analogues, e.g., Zuckerman functors). The isotropy subgroups are reductive and the orbits are in one form or another related to flag manifolds and their natural generalizations. A totally different experience is encountered with nilpotent orbits of semisimple groups. Here the associated representation theory (unipotent representations) is poorly understood and there is a loss of reductivity of isotropy subgroups. To make matters worse (or really more interesting) orbits are no longer closed and there can be a failure of normality for orbit closures. In addition simple connectivity is generally gone but more seriously there may exist no invariant polarizations. The interest in nilpotent orbits of semisimple Lie groups has increased sharply over the last two decades. This perhaps may be attributed to the reoccuring experience that sophisticated aspects of semisimple group theory often leads one to these orbits (e.g., the Springer correspondence with representations of the Weyl group). In this note we announce new results concerning the symplectic and algebraic geometry of the nilpotent orbits O and the symmetry groups of that geometry. The starting point is the recognition (made also by others) that the ring R of regular functions on any G-cover M of O is not only a Poisson algebra (the case for any coadjoint orbit) but that R is also naturally graded. The key theme is that the same nilpotent orbit may be "shared" by more than one simple Received by the editors March 29, 1991 and, in revised form, May 29, 1991. 1991 Mathematics Subject Classification. Primary 22E46; Secondary 58F05, 58F06, 32M05, 14L30. The first author is an Alfred P. Sloan Fellow. The work of the second author was supported in part by NSF Grant DMS-8703278. ©1992 American Mathematical Society 0273-0979/92 $1.00+ $.25 per page 269 270 RANEE BRYLINSKI AND BERTRAM KOSTANT group, and the key result is the determination of all pairs of simple Lie groups having a shared nilpotent orbit. Furthermore there is then a unique maximal such group and this group is encoded in the symplectic and algebraic geometry of the orbit. Remarkably a covering of nilpotent orbit of a classical group may "see" an exceptional Lie group as the maximal symmetry group of this sympectic manifold. A beautiful instance of this is that Gi is the symmetry group of the simply connected covering of the maximal nilpotent orbit of SL(3, C) and that this six-dimensional space "becomes" the minimal nilpotent orbit of Gi. Our work began with a desire to thoroughly investigate a striking discovery of Levasseur, Smith, and Vogan. They found that the failure of the closure of the eight-dimensional nilpotent orbit of G2 to be a normal variety may be "remedied" by refinding this orbit as the minimal nilpotent orbit of SO(7, C). The failure has a lot to do with the seven-dimensional representation of G2 . In general given M we have found that there exists a unique minimal representation n (containing the adjoint) wherein M may be embedded with normal closure. It was the study of n that led to the discovery of the maximal symmetry group G'. Using a new general transitivity result for coadjoint orbits we prove that, modulo a possible normal Heisenberg subgroup (and that occurs in only one case), G' is semisimple. Past experience has shown that the action of a subgroup H on a coadjoint orbit of G is a strong prognostigator as to how the corresponding representation L of G decomposes under H. If this continues to hold for unipotent representations our classification result should yield all cases where L remains irreducible (or decomposes finitely) under a semisimple subgroup. 1. THE MAXIMAL SYMMETRY GROUP AND "SHARED" ORBITS Let G be a simply connected complex semisimple Lie group and g the Lie algebra of G. Let e e g be nilpotent and assume (for simplicity of exposition but with no real loss) that e has nonzero projection in every simple component of g. Let O be the adjoint orbit of e and let v. M —> O be a G-covering. Let R = R(M) be the ring of regular functions on M and let R[g] be the copy of 0 in R defining the moment map M —► g*. Identify g* ~ g in a G-equivariant way. Now R carries a G-invariant ring grading R = ©/t>0 R[k] ( k e Z ) such that R[g] c R[2]. Then R[0] = C. The Poisson bracket satisfies [R[k], R[l]] c R[k + / 2], for all k, I (see also [7]). Hence R[2] is a finite-dimensional Lie subalgebra of R under Poisson bracket. Our first main result is