On the cuspidality criterion for the Asai transfer to GL(4)

Let F be a number field and K a quadratic algebra over F , i.e., either F × F or a quadratic field extension of F . Denote by G the F -group defined by GL(2)/K. Then, given any cuspidal automorphic representation π of G(AF ), one has (cf. [8], [9]) a transfer to an isobaric automorphic representation Π of GL4(AF ) corresponding to the L-homomorphism LG → LGL(4). Usually, Π is called the Rankin-Selberg product when K = F × F , and the Asai transfer when K is a quadratic extension. (See also [4]) In the former case, π is a pair (π1, π2) of cuspidal automorphic representations of GL2(AF ), and Π is denoted π1 π2, while in the latter case, Π is denoted AsK/F (π). The main purpose of this Note is the following. In the Appendix of [10], a criterion is given for deciding when Π is cuspidal, which is correct for non-dihedral forms π, but has to be modified for the dihedral ones. This error was encountered in the key Asai case by the first author in his work with Anandavardhanan [1]. Here we give two different proofs of the corrected cuspidality criterion. The first one is due to the second author, which slightly modifies and adapts his original arguments in [8], [10], while the second one, due to the first author, is different and may generalize to other situations. There is hardly any difference in the Rankin-Selberg case, so we give a unified single proof in that case. We hope that it is appropriate to present the proofs here as an appendix because Krishnamurthy also needs the corrected criterion for use in his work presented in [5]. The criterion has a natural analogue when F is a local field, where by a cuspidal representation we will mean (as in [7]) a discrete series representation. In order to treat the local and global cases simultaneously, let us write CF for F ∗, resp. the idele class group AF /F ∗, in the former, resp. latter, case. In each case class field theory furnishes a natural isomorphism of CF with the abelianization of the Weil group WF , and we will, by abuse of notation, use the same letter to denote the corresponding characters of CF and WF . Let An(F ) denote the set of irreducible isobaric automorphic, resp. admissible, representations of GLn(AF ), resp. GLn(F ), when F is global, resp. local. Then one knows (cf. [7], [6]) that for every π in An(F ), there is a unique partition n1 + · · ·+ nr, and cuspidal representations πj of GL(nj), 1 ≤ j ≤ r, such that there is, in the sense of Langlands (cf. [7], [6]), an isobaric sum decomposition π = j=1 πj , which means in particular that the L-function (resp. ε-factor) of π is the product of the corresponding ones of the πj . We will say that π is of (isobaric) type (n1, . . . , nr), and call each πj an isobaric summand of π. We will normalize the order so that ni ≥ nj if i ≥ j. For convenience, we write down the full cuspidality criterion in the Rankin-Selberg and Asai situations, though a correction is needed only in the dihedral case. Recall that a cuspidal representation π of GL2(AF ) is dihedral iff π ≃ π ⊗ δ for a quadratic character δ of CF . In this case, if E denotes the quadratic extension of F over which δ becomes trivial, there is a character χ of CE ∗Supported by the NSF grant DMS-1001916