Harper's beyond square-root conjecture for random matrices

Introduction

Background

Harper’s theorem on “better than square-root cancellation” in partial sums of random multiplicative functions [6] marked a major advance in probabilistic number theory, revealing a surprising connection with critical multiplicative chaos. We refer to [8, 9] for surveys of these ideas and their number theoretic significance.

A particularly attractive feature of Harper’s theorem is that it also sheds light on fundamental deterministic problems in number theory. For example, using a derandomization argument, Harper [7] proved that for a prime modulus $r$,

whenever $\min(x,r/x)\to\infty$. An analogous result holds for the continuous family $n^{it}$, averaged over $|t|\le T$, when $\min(x,T/x)\to\infty$. A further application to Möbius cancellation is particularly striking: Harper proved that the same phenomenon holds in the same range after twisting by the Möbius function $\mu$, and he conjectured that this should hold for $x\le r^{100}$. It turns out that this seemingly unimportant summation range $x\le r^{100}$ is crucial. Indeed, it is not hard to show that if Harper’s conjecture holds, then Möbius cancellation holds in the form

as long as $y\gg\sqrt{x}/W(x)$ for some $W(x)$ tending to infinity. In fact, Harper’s conjecture is quantitative, and as a corollary, one can take $W(x)=(\log\log x)^c$ for any fixed $0<c<1/4$. This would break the famous square-root barrier, which remains out of reach even under the Riemann Hypothesis. See also [7, 9]. In [17], Wang and Xu managed to prove the discrete version of Harper’s conjecture under strong assumptions, namely the Ratios conjecture and the Generalized Riemann Hypothesis.

In this paper, we establish Harper’s conjecture unconditionally in the random matrix setting, which one may also view as a function field analogue. Since Möbius twists occur as coefficients in the Dirichlet series of reciprocals of $L$-functions, the relevant random matrix quantities are the coefficients

Wang and Xu conjectured that the corresponding first absolute moment tends to 0 as $k\to\infty$, provided $k\le BN$ for a fixed $B>0$ [17 Conjecture 1.5]. In the arithmetic analogy, $k$ corresponds to $\log x$ and $N$ to the logarithm of the conductor, so a bounded ratio $k/N$ corresponds to sums of length at most a fixed power of the conductor. We prove their conjecture in a stronger form, i.e., with no restriction on $k/N$.

Main result

We write $\mathbb{E}_{U(N)}$ for integration against the normalized Haar measure on $U(N)$, and $\mathbb{E}$ for expectation over the auxiliary random variables introduced below.

Connections with function fields and multiplicative chaos

The relation with function fields is explicit. For a primitive odd Dirichlet character $\chi$ modulo a monic squarefree polynomial $Q \in\mathbb{F}_q[T]$, with $\deg Q \ge2$, write $L(u,\chi) = \det(I - \sqrt{q}u\Theta_\chi)$, where $\Theta_\chi\in U(\deg Q - 1)$. Then

Katz’s equidistribution theorem [10] shows that averages over these characters converge to Haar averages as $q \to\infty$ at fixed $k$ and conductor degree. This gives a function field interpretation of our theorem, with the field size tending to infinity first. See also work of Gorodetsky [4].

The relation between random multiplicative functions and critical chaos has been instrumental in many recent works. See [16 Section 1.6] for a recent account. This becomes especially transparent through the multiplicative chaos model studied by Soundararajan and Zaman [15]. Let $X(j)$ be independent standard complex Gaussian random variables, and define $A(k)$ by

The random power series in (2) is also studied as critical holomorphic multiplicative chaos by Najnudel, Paquette and Simm [12]. These coefficients arise as large-field limits of normalized sums of Steinhaus random multiplicative functions over polynomials of fixed degree [15]. For fixed $k$, they also arise as the limits in distribution of $h_k(U_N)$ as $N \to\infty$, by the trace limit theorem of Diaconis and Shahshahani [3]. Soundararajan and Zaman proved

matching the saving in Harper’s theorem under the correspondence of $k$ with $\log x$. An asymptotic refinement of this estimate was conjectured and supported numerically in [1]. Together with the second moment identity, the fixed $k$ convergence gives the iterated limit

It does not, however, provide the uniformity needed when $k$ grows with $N$.

For the ordinary characteristic polynomial, whose coefficient of $z^k$ is $(-1)^k\operatorname{tr}(\wedge^k U_N)$, with $\wedge^k$ denoting the $k$th exterior power, Najnudel, Paquette and Simm [12, Theorem 1.2] proved convergence to zero in distribution as $k \to\infty$ with $k \le N/2$. Since the second moment equals one, the first absolute moment also tends to zero. Our theorem concerns the symmetric powers arising from the reciprocal characteristic polynomial and allows arbitrary relative growth of $k$ and $N$. These coefficient problems are also related to the harmonic analysis of Gaussian multiplicative chaos measures (see [13]). A further connection between our proof and critical chaos is explained in § 3.3.

Notation and preliminaries

Partitions and symmetric functions

A partition is a finite nonincreasing sequence $\lambda=(\lambda_1,\ldots,\lambda_\ell)$ of positive integers. Write $\ell(\lambda)=\ell$ for its length, $|\lambda|=\sum_i\lambda_i$ for its size and $m_j(\lambda)=\#\{i:\lambda_i=j\}$ for the number of parts equal to $j$. We write $\lambda\vdash n$ if $|\lambda|=n$ and include the empty partition for $n=0$. Put

For variables $x_1,\ldots,x_N$, consider the power sums $p_j$ and the complete homogeneous symmetric polynomials $h_n$, defined by

Set $h_0=1$ and $h_n=0$ for $n<0$. Their generating function identity is

For algebraic calculations, regard the $p_j$ as indeterminates with weighted degree $j$. Expanding the exponential and grouping terms by the multiplicities of the parts gives

If the $x_i$ are the eigenvalues of $U\in U(N)$, then $p_j(U)=\operatorname{tr}U^j$ and $h_n(U)=\operatorname{tr}(\operatorname{Sym}^n U)$ as in (1.1).

A Gaussian inner product

Recall that the $X(j)$ are independent standard complex Gaussians. For $\mathbf{g}=(\sqrt{j}X(j))_{j\geq1}$, write $P(\mathbf{g})$ for the specialization $p_j=\sqrt{j}X(j)$. In particular,

Since $\mathbb{E}[X(j)^a\overline{X(j)}^b]=\mathbf{1}_{a=b}a!$, independence gives $\mathbb{E}[p_\lambda(\mathbf{g})\overline{p_\rho(\mathbf{g})}]=\mathbf{1}_{\lambda=\rho}z_\lambda$. Thus $\langle P,Q\rangle=\mathbb{E}[P(\mathbf{g})\overline{Q(\mathbf{g})}]$ is the Hermitian extension of the Hall inner product to complex symmetric functions.

Schur polynomials

We use three classical facts about symmetric polynomials, recalled here in the forms needed below. See, e.g., [11 Chapter I].

For a partition $\lambda$ of length $\ell$, define its Schur function by the determinant

In particular $s_{(k)}=h_k$ for $k\geq1$, with $s_{\varnothing}=h_0=1$. For each $n$, the $s_\lambda$ with $\lambda\vdash n$ form a basis of the homogeneous polynomials of weighted degree $n$ in the $p_j$ and are orthonormal for the Hall inner product. In $N$ variables $s_\lambda$ vanishes if $\ell(\lambda)>N$. For $\ell(\lambda)\leq N$, $s_\lambda(U)$ is an irreducible character of $U(N)$, so the specializations of these Schur functions are also orthonormal under Haar probability.

The Young diagram of $\lambda$ has $\lambda_i$ boxes in row $i$. Pieri’s rule states that

A horizontal $k$-strip means that $\rho$ is obtained from $\lambda$ by adding $k$ boxes, with no two added boxes in the same column. Equivalently,

The essential consequence is $\ell(\rho) \le\ell(\lambda)+1$. Indeed, two new nonempty rows would both require an added box in column one. This bound is independent of the number $k$ of boxes added.

Proof of thm:main

Exact moment comparison

We first state a lemma which allows us to transfer weighted moments. The identity follows from a classical moment formula for complete symmetric functions, for instance as in [4 Section 3]. It can also be deduced by coefficient extraction from the unitary ratios formula of Conrey, Farmer, and Zirnbauer [2]. We include a short proof for completeness.

In this statement, $p(U)$ stands for $(p_j(U))_{j\ge1}$.

Proof. Expand $P$ and $Q$ in the Schur basis. Only partitions of size at most $D$ occur, and these have length at most $D$. By Pieri’s rule (4), every Schur function occurring in $h_kP$ or $h_kQ$ therefore has length at most $D+1 \le N$. These Schur functions are orthonormal for the Hall inner product and, after evaluation at the eigenvalues of $U$, for the Haar inner product too. Consequently

Since $h_k(g)=A(k)$, this is the desired identity. □

The case $P=Q=1$ gives $\mathbb{E}|A(k)|^2=\mathbb{E}_{U(N)}|h_k|^2=1$. At $k=0$ the comparison reduces to the classical trace moment identity [3]. Higher moments depend on $k$. To control the first moment, we instead apply the method of moments to suitable weighted measures.

Let $\mu_N$ be Haar probability and let $\nu_{N,k}$ be the probability measure defined by $d\nu_{N,k}=|h_k|^2d\mu_N$. Fix $m$ and write

where $\Theta$ is independent and uniform on $[0,2\pi)$.

Proof. Using (2) and the complex Gaussian identity $\mathbb{E}e^{aX(j)+b\overline{X(j)}}=e^{ab}$, we obtain the following identity of formal power series:

Fix a monomial $t^\alpha s^\beta$ with $\sum_j j\alpha_j,\sum_j j\beta_j\le D$. Every term $x^a y^b$ from the last exponential that contributes to this coefficient has $a,b\le D$. For $k\ge D$, we therefore have

For the chosen coefficient, extracting the coefficient of $x^k y^k$ consequently has the same effect as retaining the factor $\exp(\sum_{j\le m}t_js_j)$ and averaging the last exponential with $x=e^{-i\theta}$ and $y=e^{i\theta}$. By Fourier orthogonality, the resulting coefficient is that of $t^\alpha s^\beta$ in

Conditioning on $\Theta$ and applying the Gaussian identity again identifies this expression with

Comparing coefficients and multiplying by $\alpha!\beta!$ gives $\mathbb{E}[|A^{(k)}|^2X_m^\alpha\overline{X}_m^\beta]=\mathbb{E}[W_m^\alpha\overline{W}_m^\beta]$. Applying Theorem 3.1 with the polynomials $\prod_{j\le m}(p_j/\sqrt{j})^{\alpha_j}$ and $\prod_{j\le m}(p_j/\sqrt{j})^{\beta_j}$ proves the identity for monomials, and hence for $P$ by linearity. □

Completion of the proof of Theorem 1.1

Let $\gamma_m$ be the law of $X_m$ and write $H_m=\sum_{j\le m}j^{-1}$. The density of a fixed translate $X_m+v$ relative to $\gamma_m$ is $\exp(2\Re\sum_j\overline{z_j}v_j-\sum_j|v_j|^2)$. Averaging this identity over $v_j=e^{ij\Theta}/\sqrt{j}$ and changing $\theta$ to $-\theta$ in the integral shows that $W_m$ has density $M_m$ relative to $\gamma_m$, where

In particular, $M_m$ is positive and continuous, and $\mathbb{E}M_m(X_m)=1$.

For fixed $m$, Theorem 3.1 with $k=0$ shows that every fixed mixed moment of $T_m$ under $\mu_N$ agrees with the corresponding moment of $X_m$ for all sufficiently large $N$. By Theorem 3.2, every fixed mixed moment under $\nu_{N,k}$ agrees with the corresponding moment of $W_m$ for all sufficiently large $N$ and $k$. The second moments imply tightness, while higher even moments imply uniform integrability of every fixed polynomial. Consequently, every subsequential limit has the moments of $X_m$ or $W_m$, respectively. These laws are determined by their moments, since their real and imaginary coordinates have joint moment generating functions in a neighbourhood of the origin. Thus, for every bounded continuous $f:\mathbb{C}^m\to\mathbb{C}$, as $\min(N,k)\to\infty$,

We next establish the qualitative estimate needed to complete the proof.

Proof. Realize all the vectors $W_m$ using the same Gaussian sequence and the same independent uniform variable $\Theta$, and put $R_m=M_m(W_m)^{-1}$. Since $W_m$ has density $M_m$ relative to $\gamma_m$, for $n\ge m$ and every bounded measurable $f:\mathbb{C}^m\to\mathbb{C}$,

Thus $(R_m)$ is a nonnegative martingale of mean one for the filtration $(\sigma(W_m))_{m\ge1}$, and consequently converges almost surely to a finite limit.

We claim that $\limsup_m M_m(W_m)=\infty$ almost surely. By rotation invariance, the sequence $(M_m(W_m))_{m\ge1}$ has the same law as

Set

Using periodicity, restricting the integral in (6) to $[-1/m,1/m]$ and applying Jensen’s inequality gives

for an absolute constant $c>0$. Here we used $a_{j,m}=1+O((j/m)^2)$ and $H_m=\log m+O(1)$. The variable $G_m$ is a centered Gaussian with

Hence, for each real $A$, $\mathbb{P}(G_m>A)\to1/2$, which implies $\mathbb{P}(\sup_m G_m=\infty)\ge1/2$. Changing finitely many of the variables $X(j)$ changes the sequence $(G_m)$ by an amount bounded uniformly in $m$. Thus $\{\sup_m G_m=\infty\}$ is a tail event, and Kolmogorov’s zero-one law gives it probability one. This proves the claim.

It follows that $\liminf_m R_m=0$ almost surely. Since $(R_m)$ converges, we have $R_m\to0$ almost surely. Finally, the density identity and bounded convergence give

as desired. □

Set $g_m=(1+M_m)^{-1}$ and $g=g_m(T_m)$. The Cauchy–Schwarz inequality gives

Here we used that $\mu_N$ and $\nu_{N,k}$ are probability measures, and $0\le g\le1$. By (7), both expectations on the right tend to $\mathbb{E}[M_m(X_m)/(1+M_m(X_m))]$. Therefore

Now let $m\to\infty$. The right side tends to zero by Theorem 3.3, proving Theorem 1.1.

Connection with critical chaos

The random variable $M_m(X_m)$ is the mean one total mass of the Fourier approximation to critical Gaussian multiplicative chaos on the circle. This gives the connection with the critical chaos model discussed in the introduction.

One can also deduce Theorem 3.3 directly from the low moment estimates for the total mass of the chaos, established by Soundararajan and Zaman. Indeed, as in [15], write $F_m(z)=\exp\left(\sum_{j\le m}\frac{X(j)z^j}{\sqrt{j}}\right)$. By (6),

The low moment estimates of [15 Proposition 3.2], with $K=m$, $r=1$ and $q=1/2$, gives

since $H_m = \log m + O(1)$. The inequality $x/(1+x) \le\sqrt{x}$ then implies the desired result.

Acknowledgments. This work was initiated during the first author’s visit to Tsinghua University, whose hospitality he gratefully acknowledges. We thank Victor Wang for helpful discussions and for pointing out the connection between Theorem 3.1 and the theorem of Conrey, Farmer, and Zirnbauer.

References

Department of Mathematics, University of Toronto, Toronto, ON, Canada

Email address: [email protected]

Yau Mathematical Sciences Center, Tsinghua University, Beijing, 100084, China

Beijing Institute of Mathematical Sciences and Applications, Beijing, 101408, China

Email address: [email protected], [email protected]