betaFeedback

WorksMathematics

Classification of GL1(D)×GLn−1(D){\rm GL}_1(D)\times {\rm GL}_{n-1}(D)-distinguished representations

Abstract

Let DD be a quaternion division algebra over a non-Archimedean local field FF of characteristic zero. We prove the conjecture of Dagar, Sharma, and Verma classifying irreducible smooth representations of GLn(D)\mathrm{GL}_n(D) distinguished by GL1(D)×GLn−1(D)\mathrm{GL}_1(D)\times\mathrm{GL}_{n-1}(D) for n>2n>2. These are precisely the trivial representation and the irreducible normalized parabolic inductions \one_{n-2}\timesτ, where ττ is an infinite-dimensional irreducible representation of GL2(D)\mathrm{GL}_2(D) distinguished by GL1(D)×GL1(D)\mathrm{GL}_1(D)\times\mathrm{GL}_1(D). The proof combines a choice of cuspidal inducing factor with orbit conditions and multisegment methods.

Introduction

Distinction problems study the existence of invariant linear forms on representations under the action of a subgroup. For reductive groups and their symmetric subgroups, these questions are closely related to harmonic analysis on homogeneous spaces. Linear periods provide an important example of this connection. In this paper, we classify representations of general linear groups over a quaternion division algebra admitting a linear period with respect to a block-diagonal subgroup of type (1,n−1)(1, n - 1).

Let FF be a non-Archimedean local field of characteristic zero, and let DD be a quaternion division algebra over FF. For n≥1n \ge1, set Gn=GLn(D)G_n = \mathrm{GL}_n(D). Given nonnegative integers pp, qq with p+q=np + q = n, let

Hp,q={diag⁡(g1,g2):g1∈Gp, g2∈Gq}⊆Gn.H_{p,q} = \{ \operatorname{diag}(g_1,g_2) : g_1 \in G_p,\ g_2 \in G_q \} \subseteq G_n.

An irreducible smooth complex representation π\pi of GnG_n is called Hp,qH_{p,q}-distinguished if Hom⁡Hp,q(π,1)≠0\operatorname{Hom}_{H_{p,q}}(\pi, \mathbf{1}) \ne0. The elements of this space are called linear periods. Our concern is the subgroup Hn=H1,n−1H_n = H_{1,n-1}.

The theory of linear periods involves both uniqueness and classification questions. For split general linear groups, Jacquet and Rallis [8] established the uniqueness of linear periods for block-diagonal subgroups. Chen and Sun [5] studied twisted linear periods for the pair (GL2n(F),GLn(F)×GLn(F))(\mathrm{GL}_{2n}(F), \mathrm{GL}_n(F) \times\mathrm{GL}_n(F)) and their relation to twisted Shalika periods. On the classification side, Yang [16] characterized the irreducible unitary representations of GL2n(F)\mathrm{GL}_{2n}(F) admitting linear periods with respect to GLn(F)×GLn(F)\mathrm{GL}_n(F) \times\mathrm{GL}_n(F). These results illustrate the interaction between distinction, parabolic induction, and the structure of the irreducible dual.

For quaternionic general linear groups, the rank-two problem was studied by Raghuram [12] and Gan–Takeda [7]. Their results describe the representations of G2G_2 admitting an H1,1H_{1,1}-period. In higher ranks, Anandavardhanan, Lu, Matringe, Sécherre, and Yang [1] established results on multiplicities and vanishing of linear periods. In particular, their results exclude HnH_n-distinction for essentially square-integrable representations of GnG_n when n>2n > 2. This makes the analysis of parabolically induced representations central to the classification problem.

Dagar, Sharma, and Verma [6] proposed that HnH_n-distinction in every rank n>2n > 2 is governed by distinguished representations of G2G_2. They proved their conjecture for n=3n = 3 and n=4n = 4. We recall its statement, writing 1r\mathbf{1}_r for the trivial representation of GrG_r and ×\times for normalized parabolic induction.

Conjecture 1.1 ([6 Conjecture 1.1]). Let n>2n > 2, and let π∈Irr⁡(Gn)\pi\in\operatorname{Irr}(G_n). Then π\pi is HnH_n-distinguished if and only if either π≃1n\pi\simeq\mathbf{1}_n or

π≃1n−2×τ,\pi\simeq\mathbf{1}_{n-2} \times\tau,

where τ\tau is an infinite-dimensional irreducible H1,1H_{1,1}-distinguished representation of G2G_2.

The induction in the second alternative is necessarily irreducible, since it is isomorphic to π\pi. Thus the conjecture describes every nontrivial irreducible HnH_n-distinguished representation by adjoining a trivial factor to distinguished data of rank two.

The main result of this paper establishes the conjecture for all n>2n > 2.

Theorem A. Conjecture 1.11.1 holds for every n>2n > 2.

Outline of the proof

Our proof combines the orbit conditions of [6] with the structure of representations described by multisegments. The relevant classification originates in the work of Zelevinsky [17] for split general linear groups. For general linear groups over division algebras, we use the results of Tadić [15] and Mínguez–Sécherre [10]. We also use the Mœglin–Waldspurger algorithm [11], extended to GL⁡n(D)\operatorname{GL}_n(D) by Badulescu and Renard [3], to pass between the Zelevinsky and Langlands parametrizations.

A key step is the choice of a cuspidal inducing factor. For n≥5n \ge5, we prove that every irreducible non-cuspidal representation π\pi, after replacement by π∨\pi^\vee if necessary, admits a quotient presentation ρ×η↠π\rho\times\eta\twoheadrightarrow\pi, where ρ\rho is cuspidal, η\eta is irreducible, and ρ≄νn−3\rho\not\simeq\nu^{n-3} as a representation of G1G_1. This follows by comparing the endpoints of the segments parametrizing π\pi and π∨\pi^\vee. The resulting choice excludes one closed-orbit condition and permits a uniform argument in higher ranks.

Suppose that π\pi is distinguished. Its period pulls back to ρ×η\rho\times\eta, so the orbit conditions apply to this induced representation. If ρ\rho has rank at least two, cuspidality excludes the open orbit, while the remaining closed-orbit condition and the vanishing results of [1] force its rank to be exactly two. This leads to the representations predicted by Conjecture 1.11.1.

For a rank-one factor, the analysis reduces to the open orbit. Frobenius reciprocity then gives an embedding η↪ρ∨×1n−2\eta\hookrightarrow\rho^\vee\times\mathbf{1}_{n-2}, so that π\pi is an irreducible constituent of In(ρ)=ρ×ρ∨×1n−2I_n(\rho) = \rho\times\rho^\vee\times\mathbf{1}_{n-2}. We determine the possible distinguished constituents of this family by separating the boundary values ρ=ν±(n−1)\rho= \nu^{\pm(n-1)} from the remaining cases. At the boundary, the Mœglin–Waldspurger algorithm and a vanishing statement for inductions involving νn−2St⁡2\nu^{n-2}\operatorname{St}_2 exclude every nontrivial constituent. Away from the boundary, linked-segment arguments and the irreducibility of unitary parabolic induction [13] reduce the problem to distinction in rank two. The converse follows from the sufficient closed-orbit condition of [6].

Organization of the paper

Section 2 fixes the notation and recalls the necessary background on representations and linear periods. Section 3 constructs quotient presentations with suitable cuspidal first factors. Section 4 analyzes the distinguished constituents of the induced representations arising from the orbit conditions, including the boundary cases. Section 5 combines these results to complete the proof of Theorem A.

Acknowledgements

Quanyong Chen expresses his gratitude to Kei Yuen Chan for valuable discussions and comments. The authors acknowledge the assistance of ChatGPT in preparing this manuscript. The authors have reviewed and verified the AI-assisted content and take full responsibility for the mathematical arguments, results.

Conventions and preliminaries

Fix a non-Archimedean local field FF of characteristic zero, and let DD be a quaternion division algebra over FF, so that dim⁡FD=4\dim_F D = 4. For each n∈Nn \in\mathbb{N}, write

Gn=GL⁡n(D).G_n = \operatorname{GL}_n(D).

Let Nrd⁡D/F:Gn→F×\operatorname{Nrd}_{D/F}: G_n \to F^\times denote the reduced norm map, and define

νn(g)=∣Nrd⁡D/F(g)∣F,g∈Gn.\nu_n(g) = \lvert\operatorname{Nrd}_{D/F}(g)\rvert_F,\quad g \in G_n.

For p,q≥0p,q \ge0 with p+q=np+q=n, let Hp,qH_{p,q} be the block-diagonal subgroup

Hp,q={(g100g2):g1∈Gp,g2∈Gq}⊆Gn.H_{p,q} = \left\{ \begin{pmatrix} g_1 & 0 \\ 0 & g_2 \end{pmatrix} : g_1 \in G_p, g_2 \in G_q \right\} \subseteq G_n.

This subgroup is the fixed-point group of the involution

θ(g)=δp,qgδp,q−1,δp,q=diag⁡(Ip,−Iq).\theta(g) = \delta_{p,q}g\delta_{p,q}^{-1},\quad\delta_{p,q} = \operatorname{diag}(I_p,-I_q).

We equip Hp,qH_{p,q} with the family of characters χs\chi_s defined by

χs(diag⁡(g1,g2))=ν(g1)2sν(g2)−2s,s∈R.\chi_s(\operatorname{diag}(g_1,g_2)) = \nu(g_1)^{2s}\nu(g_2)^{-2s},\quad s \in\mathbb{R}.

Equivalently, if αs:F×→C×\alpha_s:F^\times\to\mathbb{C}^\times is given by αs(x)=∣x∣F2s\alpha_s(x)=\lvert x\rvert_F^{2s}, then χs\chi_s is the composition of αs\alpha_s with the homomorphism

Hp,q⟶F×,diag⁡(g1,g2)⟼Nrd⁡(g1)Nrd⁡(g2)−1.H_{p,q} \longrightarrow F^\times,\quad\operatorname{diag}(g_1,g_2) \longmapsto\operatorname{Nrd}(g_1)\operatorname{Nrd}(g_2)^{-1}.

Representations of GnG_n

In this subsection, as in [4], we review the requisite facts on the representation theory of GnG_n. Let Rep⁡(Gn)\operatorname{Rep}(G_n) denote the category of smooth complex representations of GnG_n of finite length. We denote by Irr⁡(Gn)\operatorname{Irr}(G_n) the set of equivalence classes of irreducible representations, and by C(Gn)\mathcal{C}(G_n) the subset consisting of cuspidal representations. In this paper, all representations are smooth. For π∈Rep⁡(Gn)\pi\in\operatorname{Rep}(G_n), we denote its contragredient by π~\widetilde{\pi}.

In this paper, whenever the value of nn is clear from the context, we write GG for GnG_n. Let PP be a parabolic subgroup of GnG_n with Levi decomposition P=MNP = MN, where MM is a Levi subgroup and NN the unipotent radical. Let ρ\rho be a representation of MM. By inflation, we regard ρ\rho as a representation of PP, with the unipotent radical NN acting trivially. The normalized parabolic induction of ρ\rho to GG is defined by Ind⁡PG(δP1/2⊗ρ)\operatorname{Ind}_{P}^{G}(\delta_{P}^{1/2} \otimes\rho), where δP\delta_P denotes the modular character of PP. Let α=(n1,…,nr)\alpha= (n_1,\ldots,n_r) be a partition of nn, and let Mα≅Gn1×⋯×GnrM_\alpha\cong G_{n_1} \times\cdots\times G_{n_r} be the corresponding Levi subgroup of GnG_n. Every representation of MαM_\alpha can be written in the form

ρ1⊗⋯⊗ρr,ρi∈Rep⁡(Gni).\rho_1 \otimes\cdots\otimes\rho_r,\qquad\rho_i \in\operatorname{Rep}(G_{n_i}).

We use the notation ρ1×⋯×ρr\rho_1 \times\cdots\times\rho_r for the normalized parabolic induction Ind⁡PαGn(ρ1⊗⋯⊗ρr)\operatorname{Ind}_{P_\alpha}^{G_n}(\rho_1 \otimes\cdots\otimes\rho_r). Parabolic induction is exact and preserves finite length. Permuting inducing factors preserves their Jordan–Hölder constituents, but does not in general preserve which constituent is a subrepresentation or a quotient.

We next recall the Jacquet functor. Let (π,V)(\pi,V) be a representation of GG, and let NN be the unipotent radical of PP. Define V(N)=Span⁡{π(n)v−v:n∈N,v∈V}V(N) = \operatorname{Span}\{\pi(n)v-v:n\in N,v\in V\}, and set VN=V/V(N)V_N = V/V(N). The group PP acts VNV_N by

πN(p)(v+V(N))=δP−1/2(p)π(p)v+V(N),p∈P.\pi_N(p)(v+V(N))=\delta_P^{-1/2}(p)\pi(p)v+V(N),\qquad p\in P.

Since δP\delta_P is trivial on NN, this action factors through the quotient P/NP/N. Hence VNV_N may be viewed as a representation of the Levi subgroup M≅P/NM \cong P/N. The resulting representation (πN,VN)(\pi_N,V_N) is called the normalized Jacquet module of π\pi. For the Levi subgroup M=Gl×Gn−lM=G_l\times G_{n-l}, we denote the corresponding Jacquet module by rl,n−l(π)r_{l,n-l}(\pi).

Segments and multisegments

For a cuspidal representation ρ\rho of GdG_d, write νρ=νs(ρ)\nu_\rho=\nu^{s(\rho)} for its reducibility character. For a≤ba\leq b with b−a∈Z≥0b-a\in\mathbb{Z}_{\geq0}, a segment is a set

[a,b](ρ)={νρaρ,νρa+1ρ,…,νρbρ}.[a,b]_{(\rho)}=\{\nu_\rho^a\rho,\nu_\rho^{a+1}\rho,\ldots,\nu_\rho^b\rho\}.

Its degree is d(b−a+1)d(b-a+1).

Following [6 Section 2], we denote by Z(Δ)Z(\Delta) the unique irreducible subrepresentation of the ascending cuspidal induction attached to Δ\Delta, and by L(Δ)L(\Delta) its unique irreducible quotient. On the trivial cuspidal line of G1G_1, one has ν1=ν2\nu_1=\nu^2. To distinguish ordinary ν\nu-exponents from νρ\nu_\rho-exponents, we write

[a,b]2={νa,νa+2,…,νb},[a]2={νa}.[a,b]_2=\{\nu^a,\nu^{a+2},\ldots,\nu^b\},\qquad[a]_2=\{\nu^a\}.

where b−a∈2Z≥0b-a\in2\mathbb{Z}_{\geq0}. In particular,

Z([1−r,r−1]2)=1r,L([c−1,c+1]2)=νcSt⁡2.(1)Z([1-r,r-1]_2)=1_r,\qquad L([c-1,c+1]_2)=\nu^c\operatorname{St}_2. \tag*{(1)}

Let Δ1=[a1,b1](ρ1)\Delta_1=[a_1,b_1]_{(\rho_1)} and Δ2=[a2,b2](ρ2)\Delta_2=[a_2,b_2]_{(\rho_2)} be two segments. We say that Δ1\Delta_1 precedes Δ2\Delta_2 if, after concatenating the terms of Δ1\Delta_1 and Δ2\Delta_2 in this order, one can select a subsequence which forms a segment of length strictly larger than both ℓ(Δ1)\ell(\Delta_1) and ℓ(Δ2)\ell(\Delta_2). The segments Δ1\Delta_1 and Δ2\Delta_2 are called linked if Δ1\Delta_1 precedes Δ2\Delta_2 or Δ2\Delta_2 precedes Δ1\Delta_1.

Let S\mathcal{S} denote the set of segments and M(S)\mathcal{M}(\mathcal{S}) the set of finite multisegments. Let m={Δ1,…,Δs}\mathfrak{m}=\{\Delta_1,\ldots,\Delta_s\} be a multisegment, and define λ(m)=Z(Δ1)×⋯×Z(Δs)\lambda(\mathfrak{m})=Z(\Delta_1)\times\cdots\times Z(\Delta_s), where Δi\Delta_i does not precede Δj\Delta_j for i<ji<j. Then λ(m)\lambda(\mathfrak{m}) has a unique irreducible subrepresentation Z(Δ1,…,Δs)Z(\Delta_1,\ldots,\Delta_s). This representation depends only on the multisegment m\mathfrak{m}. Similarly, set π(m)=L(Δ1)×⋯×L(Δs)\pi(\mathfrak{m})=L(\Delta_1)\times\cdots\times L(\Delta_s). The unique irreducible quotient of π(m)\pi(\mathfrak{m}) is denoted by L(m)L(\mathfrak{m}).

We shall use the following consequences of the multisegment classification; see [6 Section 2] and [3, 10, 15]. The following lemma provides a realization of Z(m)Z(\mathfrak{m}) as the unique irreducible quotient of a suitably ordered parabolic induction.

Lemma 2.1. [9 Theorem 2.4] Let m={Δ1,…,Δt}\mathfrak{m}=\{\Delta_1,\ldots,\Delta_t\} be a multisegment, ordered so that Δj\Delta_j does not precede Δi\Delta_i whenever i<ji<j. Then the normalized parabolic induction

Z(Δ1)×⋯×Z(Δt)Z(\Delta_1)\times\cdots\times Z(\Delta_t)

has Z(m)Z(\mathfrak{m}) as its unique irreducible quotient. In particular, for Δ=[a,b](ρ)\Delta=[a,b]_{(\rho)}, there is a surjection

νρbρ×νρb−1ρ×⋯×νρaρ↠Z(Δ).\nu_\rho^b\rho\times\nu_\rho^{b-1}\rho\times\cdots\times\nu_\rho^a\rho\twoheadrightarrow Z(\Delta).

We will also use the following compatibility between the multisegment parametrization and contragredients.

Lemma 2.2. [10 Proposition 5.10] For every multisegment m={Δ1,…,Δr}\mathfrak{m}=\{\Delta_1,\ldots,\Delta_r\},

Z(m)∨≃Z(m∨),Z(\mathfrak{m})^\vee\simeq Z(\mathfrak{m}^\vee),

where

m∨={Δ1∨,…,Δr∨},[a,b](ρ)∨=[−b,−a](ρ∨).\mathfrak{m}^\vee=\{\Delta_1^\vee,\ldots,\Delta_r^\vee\},\qquad[a,b]_{(\rho)}^\vee=[-b,-a]_{(\rho^\vee)}.

In particular, Z(Δ)∨≃Z(Δ∨)Z(\Delta)^\vee\simeq Z(\Delta^\vee).

We next recall the standard exact sequences associated with a pair of linked segments.

Lemma 2.3. [10 Lemma 5.12] Let Δ≺Δ′\Delta\prec\Delta' be linked segments and set U=Δ∪Δ′U=\Delta\cup\Delta', V=Δ∩Δ′V=\Delta\cap\Delta'. If V=∅V=\varnothing, we agree that Z(V)Z(V) is the trivial representation of G0G_0, which may be omitted in inductions. Then there are short exact sequences

0⟶Z(U)×Z(V)⟶Z(Δ)×Z(Δ′)⟶Z(Δ,Δ′)⟶0(2)0\longrightarrow Z(U)\times Z(V)\longrightarrow Z(\Delta)\times Z(\Delta')\longrightarrow Z(\Delta,\Delta')\longrightarrow0 \tag*{(2)}

and

0⟶Z(Δ,Δ′)⟶Z(Δ′)×Z(Δ)⟶Z(U)×Z(V)⟶0.(3)0 \longrightarrow Z(\Delta,\Delta') \longrightarrow Z(\Delta') \times Z(\Delta) \longrightarrow Z(U) \times Z(V) \longrightarrow0. \tag*{(3)}

For m,n∈M(S)\mathbf{m},\mathbf{n} \in\mathcal{M}(S), write m≤n\mathbf{m} \leq\mathbf{n} if there is a finite chain m=m0,m1,⋯ ,mr=n\mathbf{m}=\mathbf{m}_0,\mathbf{m}_1,\cdots,\mathbf{m}_r=\mathbf{n} such that each mi+1\mathbf{m}_{i+1} is obtained from mi\mathbf{m}_i by replacing two linked segments by their union and intersection, with an empty omitted. Such a replacement is called an elementary operation. This relation defines a partial order on M(S)\mathcal{M}(S). We recall the following properties of the multisegment classification from [10].

Lemma 2.4. Let m={Δ1,…,Δr}\mathbf{m}=\{\Delta_1,\ldots,\Delta_r\}, and set

I(m)=Z(Δ1)×⋯×Z(Δr),I(\mathbf{m})=Z(\Delta_1)\times\cdots\times Z(\Delta_r),

where the factors may be taken in any order. For a multisegment n\mathbf{n}, denote by [I(m):Z(n)][I(\mathbf{m}):Z(\mathbf{n})] the multiplicity of Z(n)Z(\mathbf{n}) as a composition factor of I(m)I(\mathbf{m}). Then the following statements hold:

(i) [I(m):Z(n)]>0[I(\mathbf{m}):Z(\mathbf{n})]>0 if and only if n≤m\mathbf{n}\leq\mathbf{m}.

(ii) [I(m):Z(m)]=1[I(\mathbf{m}):Z(\mathbf{m})]=1.

(iii) If the segments of m\mathbf{m} are pairwise unlinked, then I(m)≃Z(m)I(\mathbf{m})\simeq Z(\mathbf{m}) is irreducible.

(iv) If π1=Z(m)\pi_1=Z(\mathbf{m}) and π2=Z(n)\pi_2=Z(\mathbf{n}) are supported on two distinct cuspidal lines, then

π1×π2≃Z(m+n)\pi_1\times\pi_2\simeq Z(\mathbf{m}+\mathbf{n})

is irreducible.

Mœglin–Waldspurger algorithm

The Mœglin–Waldspurger algorithm provides a combinatorial description of the correspondence between the Zelevinsky and Langlands parametrizations. Originally established in [11], it was extended to GLn(D)\mathrm{GL}_n(D) by Badulescu and Renard [3]. An additional account of the algorithm and its applications to Speh representations can be found in [2]. The algorithm associates to each multisegment m\mathbf{m} a multisegment mt\mathbf{m}^t satisfying Z(m)≃L(mt)Z(\mathbf{m})\simeq L(\mathbf{m}^t) and (mt)t=m(\mathbf{m}^t)^t=\mathbf{m}. We recall the recursive construction below.

First suppose that all segments of m\mathbf{m} lie on a single cuspidal line, with endpoints expressed in powers of νρ\nu_\rho. For a segment Δ=[a,b](ρ)\Delta=[a,b]_{(\rho)}, set Δ−=[a,b−1](ρ)\Delta^-=[a,b-1]_{(\rho)}, interpreting Δ−\Delta^- as empty when a=ba=b. Let dd be the largest right endpoint occurring in m\mathbf{m}. Among the segments ending at dd, choose Δ0\Delta_0 with the largest left endpoint. Having chosen Δs−1\Delta_{s-1}, choose Δs\Delta_s with the largest left endpoint among those segments that end at d−sd-s and precede Δs−1\Delta_{s-1}. Stop when no such segment exists. If the resulting chain is Δ0,Δ1,…,Δr\Delta_0,\Delta_1,\ldots,\Delta_r, record the segment Γ=[d−r,d](ρ)\Gamma=[d-r,d]_{(\rho)}.

Form a new multisegment m′\mathbf{m}' by replacing each selected occurrence of Δs\Delta_s by Δs−\Delta_s^-, discarding empty segments and leaving all other occurrences unchanged. The construction is then repeated on m′\mathbf{m}'. Equivalently, mt={Γ}+(m′)t\mathbf{m}^t=\{\Gamma\}+(\mathbf{m}')^t, where ++ denotes segments addition. The total length decreases at each step, so the procedure terminates. For a multisegment supported on several cuspidal lines, the construction is carried out separately on each line, and the resulting multisegments are combined.

Finite-length arguments

In this subsection, we record two elementary facts about finite-length representations. These allow us to extract irreducible inducing factors while retaining a given irreducible quotient, and concerns the irreducible quotients of a representation with a unique irreducible quotient up to isomorphism.

Proposition 2.5. Let ρ\rho be irreducible, let XX have finite length, and let θ\theta be irreducible. If ρ×X↠θ\rho\times X \twoheadrightarrow\theta, then there is an irreducible constituent η\eta of XX such that ρ×η↠θ\rho\times\eta\twoheadrightarrow\theta.

Proof. Choose a composition series 0=X0⊂X1⊂⋯⊂Xt=X0=X_0\subset X_1\subset\cdots\subset X_t=X. For a fixed surjection f:ρ×X→θf:\rho\times X\to\theta, take the least ii such that ff is nonzero on ρ×Xi\rho\times X_i. That restriction is surjective because θ\theta is irreducible, and it vanishes on ρ×Xi−1\rho\times X_{i-1}. Exactness of induction gives the required surjection from ρ×(Xi/Xi−1)\rho\times(X_i/X_{i-1}). □

Proposition 2.6. Suppose a finite-length representation MM has a unique irreducible quotient up to isomorphism, equal to LL. Every nonzero quotient of MM has an irreducible quotient isomorphic to LL.

Proof. Every nonzero finite-length representation has an irreducible quotient. Any irreducible quotient of a quotient of MM is itself a quotient of MM. □

Results about distinction

The external period results needed here are the following.

(P1) If an irreducible π\pi is HnH_n-distinguished, so is π∨\pi^\vee [6 Lemma 2.6].

(P2) An essentially square-integrable representation of GrG_r, r>2r>2, has no H1,r−1H_{1,r-1}-period against any character [1 Theorem 3.8].

(P3) For an infinite-dimensional τ∈Irr⁡(G2)\tau\in\operatorname{Irr}(G_2),

Hom⁡H1,1(τ,χs)≠0⟺Hom⁡H1,1(τ,1)≠0s∈R.(4)\operatorname{Hom}_{H_{1,1}}(\tau,\chi_s)\ne0 \Longleftrightarrow\operatorname{Hom}_{H_{1,1}}(\tau,1)\ne0 \qquad s\in\mathbb{R}. \tag*{(4)}

We use the formulation of [6 Remark 3.1]. The rank-two period theory is developed in [12, 7].

(P4) Normalized induction of an irreducible unitary representation of a Levi subgroup of GrG_r is irreducible [13].

Let π1∈Irr⁡(Gk)\pi_1\in\operatorname{Irr}(G_k) and π2∈Irr⁡(Gn−k)\pi_2\in\operatorname{Irr}(G_{n-k}). The necessary orbit conditions of [6 Lemma 2.8] say that π1×π2\pi_1\times\pi_2 can be HnH_n-distinguished only if at least one of the following holds:

(a) π1=νn−k−21k\pi_1=\nu^{n-k-2}1_k, and π2\pi_2 is (H1,n−k−1,χk/2)(H_{1,n-k-1},\chi_{k/2})-distinguished.

(b) π1\pi_1 is (H1,k−1,χ(k−n)/2)(H_{1,k-1},\chi_{(k-n)/2})-distinguished, and π2=ν2−k1n−k\pi_2=\nu^{2-k}1_{n-k}.

(c) Define Ln,k=Gk−1×Δ(G1)×Gn−k−1L_{n,k}=G_{k-1}\times\Delta(G_1)\times G_{n-k-1}, Then one has

Hom⁡Ln,k(r(k−1,1)(π1)⊗r(1,n−k−1)(π2),νn−k−1⊠1⊠ν−(k−1))≠0,(5)\operatorname{Hom}_{L_{n,k}}\left(r_{(k-1,1)}(\pi_1)\otimes r_{(1,n-k-1)}(\pi_2),\nu^{n-k-1}\boxtimes1\boxtimes\nu^{-(k-1)}\right)\ne0, \tag*{(5)}

where Δ(G1)={(g,g):g∈G1}⊂G1×G1\Delta(G_1)=\{(g,g):g\in G_1\}\subset G_1\times G_1. The two closed conditions also give sufficient conditions for distinction [6 Lemma 2.9].

Proposition 2.7. Let m=n−2m=n-2, σ∈Irr⁡(G1)\sigma\in\operatorname{Irr}(G_1), and η∈Irr⁡(Gm+1)\eta\in\operatorname{Irr}(G_{m+1}). If the open orbit contributes to σ×η\sigma\times\eta, then

η↪σ∨×1.\eta\hookrightarrow\sigma^\vee\times1.

Proof. Denote by SS and VV the spaces of σ\sigma and η\eta, respectively, and set

J=r(1,m)(η).J=r_{(1,m)}(\eta).

Denote the action of G1×GmG_1\times G_m on JJ by jj. For a rank-one first factor, the open-orbit condition gives a nonzero linear functional

ℓ:S⊗J⟶C\ell:S\otimes J\longrightarrow\mathbb{C}

such that

ℓ(σ(a)s⊗j(a,B)x)=ℓ(s⊗x)\ell(\sigma(a)s\otimes j(a,B)x)=\ell(s\otimes x)

for all a∈G1a\in G_1, B∈GmB\in G_m, s∈Ss\in S, and x∈Jx\in J.

Every irreducible smooth representation of D×D^\times is finite-dimensional, so its smooth dual equals its algebraic dual. Define

T:J⟶S∨,T(x)(s)=ℓ(s⊗x).T:J\longrightarrow S^\vee,\qquad T(x)(s)=\ell(s\otimes x).

The invariance of ℓ\ell implies

T(j(a,B)x)(s)=ℓ(s⊗j(a,B)x)=ℓ(σ(a−1)s⊗x)=(σ∨(a)T(x))(s).T(j(a,B)x)(s)=\ell(s\otimes j(a,B)x)=\ell(\sigma(a^{-1})s\otimes x)=(\sigma^\vee(a)T(x))(s).

Thus TT is a homomorphism

T:J⟶σ∨⊠1m.T:J\longrightarrow\sigma^\vee\boxtimes1_m.

Moreover, T≠0T\ne0 since T=0T=0 would imply ℓ=0\ell=0.

We now make the Frobenius reciprocity map explicit. Let G=Gm+1G=G_{m+1} and let

P=P(1,m)={(ay0B):a∈D×, B∈Gm, y∈M1,m(D)}.P=P_{(1,m)}=\left\{\left(\begin{matrix}a&y\\0&B\end{matrix}\right):a\in D^\times,\ B\in G_m,\ y\in M_{1,m}(D)\right\}.

Write P=MNP=MN, where M≃G1×GmM\simeq G_1\times G_m is the block-diagonal Levi subgroup and NN is the unipotent radical. Let q:V⟶Jq:V\longrightarrow J be the natural quotient map. By the normalization of the Jacquet module,

q(η(p)v)=δP(p)1/2j(a,B)q(v),p=(ay0B).q(\eta(p)v)=\delta_P(p)^{1/2}j(a,B)q(v),\qquad p=\left(\begin{matrix}a&y\\0&B\end{matrix}\right).

Define

Φ(v)(g)=T(q(η(g)v)),v∈V,g∈G.\Phi(v)(g)=T(q(\eta(g)v)),\qquad v\in V,\quad g\in G.

Then

Φ(v)(pg)=δP(p)1/2σ∨(a)Φ(v)(g),\Phi(v)(pg)=\delta_P(p)^{1/2}\sigma^\vee(a)\Phi(v)(g),

which is precisely the transformation law for the normalized induction σ∨×1m\sigma^\vee\times1_m. If an open compact subgroup fixes vv, it also fixes Φ(v)\Phi(v) under right translation, so Φ(v)\Phi(v) is smooth. Furthermore,

Φ(η(g0)v)(g)=Φ(v)(gg0),\Phi(\eta(g_0)v)(g)=\Phi(v)(gg_0),

hence Φ\Phi is a GG-homomorphism. It is nonzero. Indeed, since qq is surjective and T≠0T\ne0, some v∈Vv\in V satisfies

Φ(v)(1)=T(q(v))≠0.\Phi(v)(1)=T(q(v))\ne0.

Since Φ≠0\Phi\ne0 and η\eta is an irreducible representation, we have η↪σ∨×1m\eta\hookrightarrow\sigma^\vee\times1_m.

Distinguished constituents of induced representations

In this section, we study the distinguished irreducible constituents of ρ×ρ∨×1N−2\rho\times\rho^\vee\times1_{N-2}. A vanishing result for inductions involving a Steinberg twist allows us to treat the boundary cases.

Choosing the first cuspidal factor

For an irreducible non-supercuspidal representation π\pi of GnG_n, define

CL⁡(π)={ρ∈Cusp⁡(Gr):1≤r<n, ρ×η↠π for some η∈Irr⁡(Gn−r)}.\operatorname{CL}(\pi)=\{\rho\in\operatorname{Cusp}(G_r):1\le r<n,\ \rho\times\eta\twoheadrightarrow\pi\text{ for some }\eta\in\operatorname{Irr}(G_{n-r})\}.

Proposition 3.1. Let π=Z(m)∈Irr⁡(Gn)\pi=Z(\mathfrak{m})\in\operatorname{Irr}(G_n) be non-cuspidal. On any cuspidal line occurring in m\mathfrak{m}, choose a segment Δ=[a,b](ρ)\Delta=[a,b]_{(\rho)} with minimal ending. Then νρbρ∈CL⁡(π)\nu_\rho^{b}\rho\in\operatorname{CL}(\pi).

Proof. Let ρ∈C(Gd)\rho\in\mathcal{C}(G_d). Any segment preceding Δ\Delta must lie on the same cuspidal line and have a strictly smaller ending. Thus no segment of m\mathfrak{m} precedes Δ\Delta.

We may therefore order the segments as Δ1=Δ,Δ2,…,Δt\Delta_1=\Delta,\Delta_2,\ldots,\Delta_t, with endings nondecreasing on each cuspidal line. No later segment precedes an earlier one. By Lemma 2.1, the quotient form of the multisegment classification gives

Z(Δ)×Z(Δ2)×⋯×Z(Δt)↠π.Z(\Delta)\times Z(\Delta_2)\times\cdots\times Z(\Delta_t)\twoheadrightarrow\pi.

Using the surjection

νρbρ×⋯×νρaρ↠Z(Δ)\nu_\rho^{b}\rho\times\cdots\times\nu_\rho^{a}\rho\twoheadrightarrow Z(\Delta)

and exactness of induction, we obtain

f:ρ0×X↠π,ρ0=νρbρ,f:\rho_0\times X\twoheadrightarrow\pi,\qquad\rho_0=\nu_\rho^{b}\rho,

where

X=νρb−1ρ×⋯×νρaρ×Z(Δ2)×⋯×Z(Δt),X=\nu_\rho^{b-1}\rho\times\cdots\times\nu_\rho^{a}\rho\times Z(\Delta_2)\times\cdots\times Z(\Delta_t),

with empty lists omitted. This is a finite-length representation of Gn−dG_{n-d}, since normalized parabolic induction preserves finite length.

Choose a composition series

0=X0⊂X1⊂⋯⊂Xr=X,0 = X_0 \subset X_1 \subset\cdots\subset X_r = X,

and let ii be minimal such that f(ρ0×Xi)≠0f(\rho_0 \times X_i) \ne0. Irreducibility of π\pi implies that this image is all of π\pi, while minimality gives f(ρ0×Xi−1)=0f(\rho_0 \times X_{i-1}) = 0. Exactness of induction therefore yields a surjection

ρ0×(Xi/Xi−1)↠π.\rho_0 \times(X_i/X_{i-1}) \twoheadrightarrow\pi.

The constituent Xi/Xi−1X_i/X_{i-1} is irreducible. Finally, d<nd < n. otherwise m\mathbf{m} would consist of a single segment of length one, making π\pi cuspidal. Since ρ0\rho_0 is cuspidal, the displayed surjection proves ρ0∈CL(π)\rho_0 \in\mathrm{CL}(\pi).

Proposition 3.2. Let n≥5n \ge5, and let π∈Irr⁡(Gn)\pi\in\operatorname{Irr}(G_n) be non-cuspidal. For at least one of π\pi and π∨\pi^\vee, there is a presentation

ρ×η↠πorρ×η↠π∨\rho\times\eta\twoheadrightarrow\pi\qquad\text{or} \qquad\rho\times\eta\twoheadrightarrow\pi^\vee

with ρ\rho cuspidal, η\eta irreducible, and ρ≄νn−3\rho\not\simeq\nu^{n-3} as a representation of G1G_1.

Proof. We argue by contradiction and assume that the conclusion fails. Proposition 3.1 shows that both sets of possible first factors are nonempty, and hence

CL(π)=CL(π∨)={νn−3}.(6)\mathrm{CL}(\pi) = \mathrm{CL}(\pi^\vee) = \{\nu^{n-3}\}. \tag*{(6)}

Apply Proposition 3.1 separately to each cuspidal line of the multisegment m\mathbf{m} for π\pi. Every such line must contain νn−3\nu^{n-3}. Thus all segments lie on the single step-two line containing this character. Set A=n−3A = n - 3, and write m={[ai,bi]2}i=1t\mathbf{m} = \{[a_i,b_i]_2\}_{i=1}^t in ordinary ν\nu-exponents. Minimal endings for π\pi, followed by minimal endings for π∨=Z({[−bi,−ai]2}i=1t)\pi^\vee= Z(\{[-b_i,-a_i]_2\}_{i=1}^t), give

min⁡ibi=A,−max⁡iai=A.\min_i b_i = A, \qquad-\max_i a_i = A.

Every segment therefore contains [−A,A]2[-A,A]_2, and has degree at least A+1=n−2A+1 = n-2. Since 2(n−2)>n2(n-2) > n, there is only one segment. Its endpoints must then be exactly −A,A-A,A, giving total degree n−2n-2, a contradiction.

Remark 3.3. Lemma 3.2 is independent of distinction and of the central character. Its role is to eliminate orbit (a) for a suitable cuspidal first factor. This is why the proof below does not require an inductive classification of twisted periods in smaller ranks.

Vanishing and the boundary configuration

In this subsection, we establish a vanishing result for inductions involving νN−2St2\nu^{N-2}\mathrm{St}_2. Combining this with the orbit conditions and the Mœglin–Waldspurger algorithm, we show that the trivial representation is the only HNH_N-distinguished irreducible constituent of ν−(N−1)×1N−2×νN−1\nu^{-(N-1)} \times1_{N-2} \times\nu^{N-1}.

Proposition 3.4. Let N>2N > 2 and η∈Irr⁡(GN−2)\eta\in\operatorname{Irr}(G_{N-2}). Then

Hom⁡HN(νN−2St2×η,1)=0.\operatorname{Hom}_{H_N}(\nu^{N-2}\mathrm{St}_2 \times\eta,1)=0.

Proof. Let τ=νN−2St2\tau=\nu^{N-2}\mathrm{St}_2. Suppose that τ×η\tau\times\eta is HNH_N-distinguished. The three-orbit criterion with k=2k=2 then requires at least one of the following conditions.

Closed orbit (a) requires

τ≃νN−412.\tau\simeq\nu^{N-4}1_2.

which is impossible because τ\tau is infinite-dimensional.

Closed orbit (b) requires

Hom⁡H1,1(τ,χ(2−N)/2)≠0.\operatorname{Hom}_{H_{1,1}}(\tau,\chi_{(2-N)/2})\ne0.

However, for z∈F×z \in F^\times,

χ(2−N)/2(zI2)=1,τ(zI2)=∣z∣F4(N−2)id⁡,\chi_{(2-N)/2}(zI_2)=1,\qquad\tau(zI_2)=|z|_F^{4(N-2)}\operatorname{id},

since St2\mathrm{St}_2 has trivial central character. As N>2N>2, these central characters differ, so this Hom space vanishes.

For the open orbit, by [10 Proposition 4.10], the normalized Jacquet module is

r(1,1)(τ)=(νN−2⊠νN−2)⊗(ν⊠ν−1)=νN−1⊠νN−3.r_{(1,1)}(\tau)=(\nu^{N-2}\boxtimes\nu^{N-2})\otimes(\nu\boxtimes\nu^{-1})=\nu^{N-1}\boxtimes\nu^{N-3}.

Consequently, the first G1G_1-factor of G1×ΔG1×GN−3G_1\times\Delta G_1\times G_{N-3} acts on the source by νN−1\nu^{N-1}, whereas the target character restricts to νN−3\nu^{N-3}. These characters are distinct, so the open-orbit Hom space is trivial.

Thus none of the three necessary conditions holds, we complete the proof.

We now apply the preceding vanishing result to determine the distinguished irreducible constituents in the following boundary case.

Proposition 3.5. Let N≥4N\geq4 and a=−(N−1)a=-(N-1), b=N−1b=N-1, and set

BN=νa×Z([a+2,b−2]2)×νb=νa×1N−2×νb.B_N=\nu^a\times Z([a+2,b-2]_2)\times\nu^b=\nu^a\times1_{N-2}\times\nu^b.

The only HNH_N-distinguished irreducible constituent of BNB_N is 1N1_N.

Proof. The elementary operations starting from {[a]2,[a+2,b−2]2,[b]2}\{[a]_2, [a+2,b-2]_2, [b]_2\} can attach the left endpoint, the right endpoint, or both endpoints to the middle segment. The possible constituents are therefore precisely

TN=Z([a,b]2)=1N,UN=Z([a]2,[a+2,b]2),VN=Z([a,b−2]2,[b]2)=UN∨,WN=Z([a]2,[a+2,b−2]2,[b]2).(7)\begin{aligned} T_N &= Z([a,b]_2)=1_N,\\ U_N &= Z([a]_2,[a+2,b]_2),\\ V_N &= Z([a,b-2]_2,[b]_2)=U_N^\vee,\\ W_N &= Z([a]_2,[a+2,b-2]_2,[b]_2). \tag*{(7)} \end{aligned}

No multiplicity assertion is needed here.

Excluding UNU_N and VNV_N. Set E=Z([a]2,[a+2,b−2]2)E=Z([a]_2,[a+2,b-2]_2), an irreducible representation of GN−1G_{N-1}. The Mœglin–Waldspurger algorithm gives

UN=L([b]2,[b−2]2,…,[a+4]2,[a,a+2]2),(8)U_N=L([b]_2,[b-2]_2,\ldots,[a+4]_2,[a,a+2]_2), \tag*{(8)}
E=L([b−2]2,…,[a+4]2,[a,a+2]2).(9)E=L([b-2]_2,\ldots,[a+4]_2,[a,a+2]_2). \tag*{(9)}

To see the algorithm explicitly, remove the right endpoints of the long segment one at a time. Each step produces a singleton until the final remaining endpoint is combined with [a]2[a]_2 to form [a,a+2]2[a,a+2]_2. The two computations differ only by the initial singleton [b]2[b]_2.

Let

R=νb−2×νb−4×⋯×νa+4×L([a,a+2]2).R=\nu^{b-2}\times\nu^{b-4}\times\cdots\times\nu^{a+4}\times L([a,a+2]_2).

The factors are in Langlands order. Equations (8)–(9) say that RR has unique irreducible quotient EE, while νb×R\nu^b\times R has unique irreducible quotient UNU_N. Exactness gives a nonzero quotient νb×R↠νb×E\nu^b\times R\twoheadrightarrow\nu^b\times E. According to Proposition 2.6, we have

νb×E↠UN.(10)\nu^b\times E\twoheadrightarrow U_N. \tag*{(10)}

If UNU_N were distinguished, so would be the representation on the left of (10). Since the first inducing factor is νb=νN−1\nu^b=\nu^{N-1}, neither closed-orbit condition can hold, as they require this factor to be νN−3\nu^{N-3} and ν−(N−1)\nu^{-(N-1)}, respectively. By Proposition 2.7, the open-orbit embedding would require

E↪ν−b×1N−2=νa×Z([a+2,b−2]2).(11)E\hookrightarrow\nu^{-b}\times1_{N-2}=\nu^a\times Z([a+2,b-2]_2). \tag*{(11)}

The induction on the right has length two and unique irreducible socle Z([a,b−2]2)Z([a,b-2]_2), by the linked-segment rule. Its other constituent is EE, a different irreducible representation. Hence (11) is impossible. This excludes UNU_N, and property (P1) excludes VN=UN∨V_N=U_N^\vee.

Excluding WNW_N. The algorithm gives

WN=L([b−2,b]2,[b−4]2,[b−6]2,…,[a+4]2,[a,a+2]2).(12)W_N=L([b-2,b]_2,[b-4]_2,[b-6]_2,\ldots,[a+4]_2,[a,a+2]_2). \tag*{(12)}

The first step joins [b]2[b]_2 to the right endpoint of the middle segment. Subsequent steps remove singletons, and the final step joins the left endpoint to [a]2[a]_2. When N=4N=4, the list of middle singletons in (12) is empty.

Thus WNW_N is a quotient of an induction L([b−2,b]2)×XL([b-2,b]_2)\times X with XX of finite length. From Proposition 2.5, replaces XX by an irreducible representation η\eta. But

L([b−2,b]2)=νb−1St⁡2=νN−2St⁡2,L([b-2,b]_2)=\nu^{b-1}\operatorname{St}_2=\nu^{N-2}\operatorname{St}_2,

so Proposition 3.4 excludes distinction of WNW_N. Only TN=1NT_N=1_N remains.

Example 3.6 (The boundary formulas in rank five). For N=5N=5, the boundary cuspidal exponents are −4,−2,0,2,4-4,-2,0,2,4, and

U5=Z([−4]2,[−2,4]2)=L([4]2,[2]2,[−4,−2]2),U_5=Z([-4]_2,[-2,4]_2)=L([4]_2,[2]_2,[-4,-2]_2),
E=Z([−4]2,[−2,2]2)=L([2]2,[−4,−2]2),E=Z([-4]_2,[-2,2]_2)=L([2]_2,[-4,-2]_2),
W5=Z([−4]2,[−2,2]2,[4]2)=L([2,4]2,[0]2,[−4,−2]2).W_5=Z([-4]_2,[-2,2]_2,[4]_2)=L([2,4]_2,[0]_2,[-4,-2]_2).

In particular W5W_5 is a quotient of ν3St⁡2×11×ν−3St⁡2\nu^3\operatorname{St}_2\times1_1\times\nu^{-3}\operatorname{St}_2, to which Lemma 3.4 applies after extracting an irreducible tail.

The rank-one open-orbit family

We study distinction for 1m×τ1_m\times\tau in terms of H1,1H_{1,1}-periods on τ∈Irr⁡(G2)\tau\in\operatorname{Irr}(G_2). Together with the boundary case treated above, this criterion allows us to determine the distinguished irreducible constituents of ρ×ρ∨×1N−2\rho\times\rho^\vee\times1_{N-2}.

Proposition 3.7. For m≥2m\ge2 and τ∈Irr⁡(G2)\tau\in\operatorname{Irr}(G_2),

Hom⁡H1,m+1(1m×τ,1)≠0\operatorname{Hom}_{H_{1,m+1}}(1_m\times\tau,1)\ne0

if and only if τ\tau is infinite dimensional and H1,1H_{1,1}-distinguished.

Proof. Apply the orbit conditions with n=m+2n=m+2 and k=mk=m. Orbit (b) would require

Hom⁡H1,m−1(1m,χ−1)≠0,\operatorname{Hom}_{H_{1,m-1}}(1_m,\chi^{-1})\ne0,

which is false. For orbit (c), by the definition of the normalized Jacquet functor [14 Definition 1.22], we have

rP(1m)=(ν−11m−1)⊠νm−1.r_P(1_m)=(\nu^{-1}1_{m-1})\boxtimes\nu^{m-1}.

The first factor would have to be ν1m−1\nu1_{m-1}. The characters are different since m−1≥1m-1\ge1. Hence only orbit (a) can contribute, and it requires Hom⁡H1,1(τ,χm/2)≠0\operatorname{Hom}_{H_{1,1}}(\tau,\chi_{m/2})\ne0. Every finite-dimensional irreducible smooth representation of G2G_2 is a character. Such a character restricts to H1,1H_{1,1} with the same character on both factors, so it cannot equal νm⊠ν−m\nu^m\boxtimes\nu^{-m}. Thus τ\tau is infinite dimensional, and (4) gives its H1,1H_{1,1}-distinction. Conversely, (4) and the sufficiency of closed orbit (a) give the desired period on 1m×τ1_m\times\tau.

The following proposition describes the possible distinguished constituents of IN(ρ)I_N(\rho).

Proposition 3.8. Let N≥4N \ge4, and let ρ∈Irr⁡(G1)\rho\in\operatorname{Irr}(G_1). Every HNH_N-distinguished irreducible constituent of

IN(ρ)=ρ×ρ∨×1N−2I_N(\rho)=\rho\times\rho^\vee\times\mathbf{1}_{N-2}

is either 1N\mathbf{1}_N, or an irreducible induction 1N−2×τ\mathbf{1}_{N-2}\times\tau with τ∈Irr⁡(G2)\tau\in\operatorname{Irr}(G_2) infinite dimensional and H1,1H_{1,1}-distinguished.

Proof. Set m=N−2m=N-2, and write

Δm=[−(m−1),m−1]2,Z(Δm)=1m.\Delta_m=[-(m-1),m-1]_2,\qquad Z(\Delta_m)=\mathbf{1}_m.

If ρ=ν±(m+1)\rho=\nu^{\pm(m+1)}, invariance of semisimplified parabolic induction under permutation of the factors shows that IN(ρ)I_N(\rho) has the same irreducible constituents as BNB_N. The assertion then follows from Proposition 3.5.

Assume henceforth that ρ≠ν±(m+1)\rho\ne\nu^{\pm(m+1)}. A singleton is linked to Δm\Delta_m only when it is immediately adjacent to one of its endpoints, namely [−(m+1)]2[-(m+1)]_2 or [m+1]2[m+1]_2. Consequently, neither [ρ][\rho] nor [ρ∨][\rho^\vee] is linked to Δm\Delta_m. The corresponding factors can therefore be interchanged, giving

IN(ρ)≃1m×Q,Q=ρ×ρ∨.I_N(\rho)\simeq\mathbf{1}_m\times Q,\qquad Q=\rho\times\rho^\vee.

If QQ is irreducible, the three segments [ρ][\rho], [ρ∨][\rho^\vee], and Δm\Delta_m are pairwise unlinked. Thus IN(ρ)I_N(\rho) is irreducible, and the assertion follows from Proposition 3.7 with τ=Q\tau=Q.

Suppose now that QQ is reducible. The reducibility criterion gives

ρ∨≃νρερ,ε∈{1,−1}.\rho^\vee\simeq\nu_\rho^\varepsilon\rho,\qquad\varepsilon\in\{1,-1\}.

Set σ=νρε/2ρ\sigma=\nu_\rho^{\varepsilon/2}\rho. Then

σ∨≃νρ−ε/2ρ∨≃νρε/2ρ=σ.\sigma^\vee\simeq\nu_\rho^{-\varepsilon/2}\rho^\vee\simeq\nu_\rho^{\varepsilon/2}\rho=\sigma.

Twisting does not change the reducibility step, so νσ=νρ\nu_\sigma=\nu_\rho, and therefore

{ρ,ρ∨}={νσ−1/2σ,νσ1/2σ}.\{\rho,\rho^\vee\}=\{\nu_\sigma^{-1/2}\sigma,\nu_\sigma^{1/2}\sigma\}.

Moreover, QQ has length two; write its composition series as

0⟶τ1⟶Q⟶τ2⟶0,τ1,τ2∈Irr⁡(G2).0\longrightarrow\tau_1\longrightarrow Q\longrightarrow\tau_2\longrightarrow0,\qquad\tau_1,\tau_2\in\operatorname{Irr}(G_2).

If σ≄11\sigma\not\simeq\mathbf{1}_1, the cuspidal line containing ρ\rho and ρ∨\rho^\vee is disjoint from the line supporting 1m\mathbf{1}_m. Indeed, an intersection would imply σ≃νt\sigma\simeq\nu^t for some t∈Rt\in\mathbb{R}. Self-duality would then give νt=ν−t\nu^t=\nu^{-t}, forcing t=0t=0, contrary to σ≄11\sigma\not\simeq\mathbf{1}_1. Since both τi\tau_i have cuspidal support {ρ,ρ∨}\{\rho,\rho^\vee\}, disjointness of cuspidal lines implies that each 1m×τi\mathbf{1}_m\times\tau_i is irreducible.

If σ≃11\sigma\simeq\mathbf{1}_1, then νσ=ν2\nu_\sigma=\nu^2 and ρ=ν±1\rho=\nu^{\pm1}. In this case

{τ1,τ2}={12,St⁡2}.\{\tau_1,\tau_2\}=\{\mathbf{1}_2,\operatorname{St}_2\}.

Both 1m×12\mathbf{1}_m\times\mathbf{1}_2 and 1m×St⁡2\mathbf{1}_m\times\operatorname{St}_2 are irreducible by the irreducibility of normalized induction of irreducible unitary representations.

Thus, in either case, exactness of normalized parabolic induction gives

0⟶1m×τ1⟶IN(ρ)⟶1m×τ2⟶00 \longrightarrow\mathbf{1}_m \times\tau_1 \longrightarrow I_N(\rho) \longrightarrow\mathbf{1}_m \times\tau_2 \longrightarrow0

with irreducible outer terms. These are therefore precisely the irreducible constituents of IN(ρ)I_N(\rho). Finally, by Proposition 3.7, a constituent 1m×τi\mathbf{1}_m \times\tau_i is distinguished only if τi\tau_i is infinite-dimensional and H1,1H_{1,1}-distinguished.

proof of the conjecture

In this section, we combine the preceding results to prove Conjecture 1.1. A suitable choice of cuspidal first factor allows us to apply the orbit conditions and reduce necessity to the cases already treated. The sufficiency follows from the closed-orbit criterion.

Theorem 4.1. Let n>2n > 2, and let θ∈Irr⁡(Gn)\theta\in\operatorname{Irr}(G_n) be HnH_n-distinguished. Then θ\theta belongs to the list in Conjecture 1.1.

Proof. For n=3,4n = 3, 4, use [6 Theorems 3.3 and 4.3]. Assume n≥5n \ge5. Property (P2) implies that θ\theta is not cuspidal. By Proposition 3.2, and after replacing θ\theta by θ∨\theta^\vee if necessary, we have

ρ×η↠θ,ρ∈C(Gr),η∈Irr⁡(Gn−r),ρ≄νn−3.(13)\rho\times\eta\twoheadrightarrow\theta,\qquad\rho\in\mathcal{C}(G_r),\qquad\eta\in\operatorname{Irr}(G_{n-r}),\qquad\rho\not\simeq\nu^{n-3}. \tag*{(13)}

The replacement preserves distinction by (P1). The period on θ\theta pulls back through (13), so the orbit conditions apply to ρ×η\rho\times\eta.

First case: r≥2r \ge2. Orbit (a) is impossible because ρ\rho is cuspidal and is not a character. Orbit (c) is impossible because r(r−1,1)(ρ)=0r_{(r-1,1)}(\rho) = 0. Thus orbit (b) gives

η=ν2−r1n−r,Hom⁡H1,r−1(ρ,χ(r−n)/2)≠0.(14)\eta= \nu^{2-r}\mathbf{1}_{n-r},\qquad\operatorname{Hom}_{H_{1,r-1}}\left(\rho,\chi_{(r-n)/2}\right) \ne0. \tag*{(14)}

By (P2), this forces r=2r = 2. Then η=1n−2\eta= \mathbf{1}_{n-2}, and (2.4) shows that the cuspidal ρ\rho is H1,1H_{1,1} distinguished. The induction ρ×1n−2\rho\times\mathbf{1}_{n-2} is irreducible, since its two factors have disjoint cuspidal lines. Therefore θ≃1n−2×ρ\theta\simeq\mathbf{1}_{n-2} \times\rho, as required.

Second case: r=1r = 1. The choice in (13) excludes orbit (a). If orbit (b) held, then

ρ=ν−(n−1),η=ν1n−1.\rho= \nu^{-(n-1)},\qquad\eta= \nu\mathbf{1}_{n-1}.

In the notation of Proposition 3.5, this gives

ρ×η=νa×Z([a+2,b]2).\rho\times\eta= \nu^a \times Z([a+2,b]_2).

Its unique irreducible quotient is UnU_n, by the linked-segment rule. Hence θ≃Un\theta\simeq U_n, contradicting Proposition 3.5. Thus orbit (b) is also impossible.

The open orbit must therefore contribute. By Proposition 2.7, we have η↪ρ∨×1n−2\eta\hookrightarrow\rho^\vee\times\mathbf{1}_{n-2}, and exact induction gives

ρ×η↪ρ×ρ∨×1n−2.\rho\times\eta\hookrightarrow\rho\times\rho^\vee\times\mathbf{1}_{n-2}.

Since θ\theta is a quotient of the left-hand side, it is an irreducible constituent of the right-hand side. It follows from Proposition 3.8 that θ\theta belongs to the required list.

Finally, the list is stable under contragredients. The dual of 1n\mathbf{1}_n is 1n\mathbf{1}_n, and

(1n−2×τ)∨≃1n−2×τ∨(\mathbf{1}_{n-2}\times\tau)^\vee\simeq\mathbf{1}_{n-2}\times\tau^\vee

The representation τ∨\tau^\vee is again infinite dimensional and H1,1H_{1,1}-distinguished by (P1). Thus the same conclusion holds for the original θ\theta if we made a contragredient replacement.

We now prove the sufficiency. The trivial representation is distinguished by definition. Suppose θ=1n−2×τ\theta=\mathbf{1}_{n-2}\times\tau is irreducible, with τ\tau infinite dimensional and H1,1H_{1,1}-distinguished. By (4),

Hom⁡H1,1(τ,χ(n−2)/2)≠0.\operatorname{Hom}_{H_{1,1}}(\tau,\chi_{(n-2)/2})\ne0.

Apply the sufficient closed-orbit condition [6 Lemma 2.9(1)] with k=n−2k=n-2. Its first inducing factor is νn−k−21k=1n−2\nu^{n-k-2}\mathbf{1}_k=\mathbf{1}_{n-2}, and its remaining period is exactly the one just displayed. Hence Hom⁡Hn(θ,1)≠0\operatorname{Hom}_{H_n}(\theta,\mathbf{1})\ne0. □

Remark 4.2 (Quotient directions used in the argument). Throughout the proof, we pull periods back along surjective homomorphisms without assuming that they descend from an induced representation to each of its irreducible quotients.

The University of Hong Kong, China

Email address: [email protected]

Harbin Engineering University, Harbin, China

Email address: [email protected]

References

  1. [1]U. K. Anandavardhanan, H. Lu, N. Matringe, V. Sécherre, and C. Yang, The sign of linear periods, preprint, arXiv:2402.12106v3 (2024). https://arxiv.org/abs/2402.12106.
  2. [2]A. I. Badulescu, On p-adic Speh representations, Bull. Soc. Math. France 142 (2014), no. 2, 255–267.arxiv.org/abs/1110.5080
  3. [3]A. I. Badulescu and D. Renard, Zelevinsky involution and Mœglin–Waldspurger algorithm for GLₙ(D), in Functional Analysis IX, Various Publications Series (Aarhus), vol. 48, Aarhus University, Aarhus, 2007, pp. 9–15.
  4. [4]I. N. Bernstein and A. V. Zelevinsky, Representations of the group GL(n, F), where F is a local nonarchimedean field, Russian Math. Surveys 31 (1976), no. 3, 1–68.
  5. [5]F. Chen and B. Sun, Uniqueness of twisted linear periods and twisted Shalika periods, Sci. China Math. 63 (2020), no. 1, 1–22. https://doi.org/10.1007/s11425-018-9502-y.
  6. [6]P. Dagar, H. Sharma, and M. K. Verma, On representations of GLₙ distinguished by GL₁ × GLₙ₋₁ over a quaternion division algebra, J. Algebra 711 (2027), 405–420. https://doi.org/10.1016/j.jalgebra.2026.07.003.
  7. [7]W. T. Gan and S. Takeda, On Shalika periods and a theorem of Jacquet–Martin, Amer. J. Math. 132 (2010), no. 2, 475–528.arxiv.org/abs/0705.1576
  8. [8]H. Jacquet and S. Rallis, Uniqueness of linear periods, Compositio Math. 102 (1996), no. 1, 65–123.
  9. [9]E. Lapid and A. Mínguez, On parabolic induction on inner forms of the general linear group over a non-archimedean local field, Selecta Math. (N.S.) 22 (2016), no. 4, 2347–2400. https://doi.org/10.1007/s00029-016-0281-7.
  10. [10]A. Mínguez and V. Sécherre, Représentations banales de GLₘ(D), Compos. Math. 149 (2013), no. 4, 679–704. https://doi.org/10.1112/S0010437X12000590.
  11. [11]C. Mœglin and J.-L. Waldspurger, Sur l’involution de Zelevinski, J. Reine Angew. Math. 372 (1986), 136–177.DOI
  12. [12]A. Raghuram, On the restriction to D× × D× of representations of p-adic GL₂(D), Canad. J. Math. 59 (2007), no. 5, 1050–1068.
  13. [13]V. Sécherre, Proof of the Tadić conjecture (UO) on the unitary dual of GL_m(D), J. Reine Angew. Math. 626 (2009), 187–203. https://doi.org/10.1515/CRELLE.2009.007.
  14. [14]M. Solleveld, Group algebras of reductive p-adic groups, their representations and their noncommutative geometry, preprint, arXiv:2510.17260 (2025). https://arxiv.org/abs/2510.17260.
  15. [15]M. Tadić, Induced representations of GL(n, A) for p-adic division algebras A, J. Reine Angew. Math. 405 (1990), 48–77. https://doi.org/10.1515/crll.1990.405.48.
  16. [16]C. Yang, Linear periods for unitary representations, Math. Z. 302 (2022), no. 4, 2253–2284. https://doi.org/10.1007/s00209-022-03136-y.
  17. [17]A. V. Zelevinsky, Induced representations of reductive p-adic groups. II. On irreducible representations of GL(n), Ann. Sci. École Norm. Sup. (4) 13 (1980), no. 2, 165–210. https://doi.org/10.24033/asens.1379.

Paper details

vlnc:W_ynugnydz
Version
v1 · latest
Published
Fresh — added to the Hub within the last 30 days
Added
License
CC BY 4.0
Usage

views · 3

downloads · 0

A view counts once per browser per day, a download each time the PDF is requested.

  • Mathematics

ChatGPT

View statement
Looking for related papers…
Contents