WorksPhysics and Astronomy
Band mixing in the quantum anomalous Hall regime of twisted semiconductor bilayers
Abstract
Remarkable recent experiments have observed fractional quantum anomalous Hall (FQAH) effects at zero field and unusually high temperatures in twisted semiconductor bilayer MoTe. Intriguing observations in these experiments such as the absence of integer Hall effects at twist angles where a fractional Hall effect is observed, do however remain unexplained. The experimental phase diagram as a function of twist angle remains to be established. By comprehensive numerical study, we show that band mixing has large qualitative and quantitative effects on the energetics of competing states and their energy gaps throughout the twist angle range . This lays the ground for the detailed realistic study of a rich variety of strongly correlated twisted semiconductor multilayers and an understanding of the phase diagram of these fascinating systems.
Introduction
Recent years have witnessed the advent of moiré superlattices, made of atomically-thin transition metal dichalcogenides (TMDs), as highly tunable platforms where experiments have established the existence of a plethora of strongly correlated phases phenomena such as correlated insulators, ferromagnetism and quantum criticality [1–7].
Due to the emergence of narrow flat topological bands, moiré superlattices have become an arena for an intricate interplay between topology and correlations. The hallmark example of topological phases is the quantum Hall effect [8, 9], both integer and fractional, occurring in Landau levels which arise when two dimensional electrons are subject to strong magnetic fields [10]. There has been numerous theoretical and experimental research into lattice analogues of the quantum Hall effect in the absence of external magnetic fields, known as the quantum anomalous Hall effect, as these might provide an experimentally feasible way to realize high temperature topological phases [11–19]. Materials exhibiting fractional quantum anomalous Hall effect (FQAH) effect have often been called fractional Chern insulators (FCIs) in the literature [15, 20].
The integer quantum anomalous Hall effect (QAH) has been already experimentally observed in different moiré systems [22, 23] [21]. In addition, local incompressibility measurements [24] on twisted bilayer graphene point towards the existence of fractional quantum Hall states that survive down to small magnetic fields (T), below which, topologically trivial charge density wave (CDW) states are found instead.
In a very recent exciting development, the first evidence of FQAH states has been observed in twisted MoTe based on both thermodynamic and transport measurements [25, 26, 27, 28]. Twisted transition metal dichalcogenide bilayers host topological moiré bands with spin/valley contrasting Chern numbers [29]. At small twist angles, strong exchange and correlated effects in topological narrow bands [30] drive spontaneous spin/valley ferromagnetism and FQAH states [31, 32].
Motivated by the current state of art, we provide a detailed study of the many-body interacting problem of AA stacked twisted TMD homobilayers at both integer () and fractional ( and ) hole doping of the underlying moiré valence bands. Using unbiased exact diagonalization techniques (ED) that includes multiple bands, we uncover novel significant effects of band mixing on various topological and correlated phases, which also enables a new electronic phase.
For integer filling , we demonstrate robust QAH states over a wide range of twist angles. However, for realistic interaction strength at dielectric constant , we find the interacting QAH energy gap as a function of twist angle to be strongly renormalized compared to the non-interacting case, highlighting ubiquitous correlation effects due to band mixing. Remarkably, we also find that strong interactions could drive spontaneous layer polarization (i.e. ferroelectricity) that is only enabled by multiband effects.
Moving to fractional fillings and where previous single-band projected ED studies [32, 33, 34, 35, 36] have predicted various correlated phases such as FQAH and CDW states, we further show that band mixing plays a crucial role in the stability of the aforementioned phases. In particular, band mixing weakens the FQAH state at in favor of the competing CDW state, while at , FQAH states remain robustly present over a wide range of twist angles. We perform the entanglement spectrum study to characterize and distinguish FQAH and CDW states in finite-size systems.
Model
To model the K valence bands of twisted bilayer TMDs, we use a continuum description [29]. The non-interacting Hamiltonian around a single valley (or spin due to spin-valley locking [37]) written in the basis of the two layers is given by
Where captures the intralayer moiré potential and represents the interlayer tunneling. Fourier expanding both functions to the lowest harmonics and taking into account symmetry constraints restrict their form to take
In the Hamiltonian , the corners of the moiré Brillouin zone are chosen to be and the moiré reciprocal lattice vectors are given by for where is the moiré lattice constant for twist angle and is the lattice constant of the monolayer. The Hamiltonian around the opposite valley is related to by a time-reversal transformation.
Focusing on twisted MoTe (tMoTe), we take Å and use the following parameters obtained from fitting the continuum model to DFT calculations [33],
We are concerned with the problem of interacting holes in the moiré valence bands of . The full momentum space many-body Hamiltonian reads
with
where () are creation (annihilation) operators of holes in a Bloch state where and are band and spin indices respectively. is the single-particle energies and are the two-body interaction matrix elements between the different Bloch states. Our two-body interaction is the dual-gated screened Coulomb interaction for dielectric constant which controls the interaction strength. Unless stated otherwise, we choose the distance from the sample to the gates to be nm.
We diagonalize the Hamiltonian (4) on different equal-aspect-ratio clusters [38] keeping a finite number of valence bands. By the variational principle, our calculations provide an upper bound on the exact ground state energy of Eq. (4) that becomes tighter upon increasing . We utilize generic twisted boundary conditions,
parameterized by along the two axes of the cluster. Ferromagnetism across a broad range of filling factors has been observed experimentally [25, 26] and in previous numerical studies [31, 33]. With this motivation, we perform all calculations in the full spin-valley sector. However, the presence of full spin/valley polarization in ground and low-lying excited states throughout the entire twist angle range studied in this work should not be taken for granted and requires further study.
Results. We begin our analysis by investigating integer hole filling where previous studies have mainly used self-consistent Hartree-Fock methods [39, 40]. For weak interaction strength , we find the many-body ground state to be a QAH state for twist angles spanning from to . The interacting QAH state is smoothly connected to the non-interacting limit with holes completely filling the first valence band which has a Chern number . The many-body spectrum shown in Figure 1(c) at features a single many-body ground state at total momentum as expected from a Slater determinant state for a certain band and spin . However, as shown in Figure 1(a), the energy gap of the interacting QAH state generically differs from the noninteracting energy gap and can be either smaller or larger depending on the twist angle.
Figure 1. (a)-(b) at for (a) where and (b) where for and for . is the noninteracting band gap between the first two bands. (c)-(d) ED spectrum at in (c) the quantum anomalous Hall phase and (d) the ferroelectric phase. Calculations were done on the 9 site cluster [38] with periodic boundary conditions, . The index labels the different momentum points.
As the interaction strength is increased to the realistic value , we observe the appearance of a ferroelectric phase which exhibits strong layer polarization. The signs of ferroelectricity stem from the existence of two-degenerate many-body ground states (Figure 1(d)) which are manifestation of spontaneously breaking symmetry and realize two layer-polarized states on either the top or the bottom layer.
The ferroelectric phase dominates over the QAH for smaller twist angles. In Figure 1(b), we plot as a function of twist angle for realistic interaction strength and find that it is non-vanishing up until , after which, the system undergoes a transition to a QAH state.
Similar to [41], which focused on smaller twist angles, the emergence of layer polarization can be intuitively understood from a real-space picture also in the regime that
we consider. Here, the first three valence bands have a total Chern number and admit a real-space description in terms of three Wannier orbitals which are maximally localized on the high symmetry positions, , , and forming a triangular lattice of three sublattices [40]. denotes atomic positions in the moiré unit cell where the atom (metal or chalcogen ) of the top layer is aligned with the atom. The two Wannier orbitals centered on the and sites are mainly localized in the top and bottom layers respectively while the orbital at the site carries equal weight in both layers. When the interaction strength is strong, it becomes energetically favorable for the holes to minimize repulsion by localizing on one of the two layer-polarized sublattices.
The emergence of ferroelectric phase strongly affects the phase space of QAH state. For realistic interaction strength , the QAH state now appears at . Interestingly, its topological gap (within the fully polarized sector) increases monotonously (see Figure 1(b)) with the twist angle, at least up to . This contrasts sharply with the noninteracting case where the band gap decreases in this angle range. For comparison, we also performed ED calculation using a different set of continuum model parameters in the literature [34], and found that at the QAH phase only appears in a narrow twist angle range for , and is entirely absent for [38].
Next, we consider fractional hole fillings and . For , single-band exact diagonalization studies [33, 35, 36] have revealed an intriguing interplay between FQAH and CDW states as a function of twist angle while more robust FQAH states for a wider range of twist angles were found at .
To contrast the two fillings, we focus first on twist angle and . As shown in Figure 2(a) and 2(c), the many-body spectrum in both cases displays three quasi-degenerate ground states at three distinct total momentum sectors. In this cluster geometry, the ground state momentum sectors expected of FQAH and CDW states happen to be identical and, therefore, distinguishing between the two candidate phases requires further analysis.
Figure 2. ED results at twist angle and . (a) and (c) show the many-body spectrum in the CDW and FQAH phases respectively while (b) and (d) show the PES and the projected (onto the first valence band) PES respectively obtained by keeping particles. There are 18 states and 42 states below the line in (b) and (d), consistent with a CDW and a FQAH state respectively. Calculations were done on the 12 site cluster [38] using periodic boundary conditions in (a)-(b) and anti-periodic boundary conditions, in (c)-(d). The index labels the different momentum points.
In order to pin-point the underlying phase, we calculate the particle entanglement spectrum (PES) [42, 43] of the quasi-degenerate states. The density matrix is defined as where () denotes the three quasi-degenerate ground states. We perform a cut in the particle-space corresponding to minority particles in a single band, holes for and electrons for . By tracing out particles and keeping ones, the entanglement spectrum consists of the eigenvalues of where is the reduced density matrix. In order to trace out electrons from the first valence band and avoid additional entanglement structure from the filled higher bands, we calculate the entanglement spectrum obtained from projecting the density matrix onto the first valence band.
As evident from Figure 2(b) and 2(d), the PES exhibits a well-separated low-lying spectrum at both fillings. For , we find the number of low-lying states to be consistent with a Laughlin-like FQAH which satisfies a counting rule (1,3) of the number of admissible configurations (at most 1 particle in each 3 consecutive orbitals [44, 45, 46]). In contrast, the counting of the fewer low-lying states at is consistent with a CDW state [47].
Moreover, we find the CDW state at and the FQAH state at to exist for a wide range of twist angles. In Figure 3, we plot the energy gap as a function of the twist angle for both and . In addition to signatures of a weak FQAH state at for [38], we observe robust CDW and FQAH states at fillings and respectively for , using the realistic interaction strength . For weak interaction , we find the system at to become metallic for .
Figure 3. as a function of twist angle at and for and . Calculations were done on the 12 site cluster [38] using periodic boundary conditions in (a)-(b) and anti-periodic boundary conditions, in (c)-(d).
We also study the stability of the discussed phases for various twisted boundary conditions. While, we find the phases at and to be insensitive to the choice of boundary conditions, we observe significant effects of twisting the boundary conditions at , see Figure 4 where we fix . Although FQAH states should be insensitive to changes in the boundary conditions in the thermodynamic limit, these states may exhibit a strong dependence in the small systems available
Figure 4. Effect of twisting the boundary conditions for , , and . (a) The many-body spectrum with periodic boundary conditions, . (b) The evolution of the many-body spectrum upon twisting the boundary condition in one direction and keeping the other direction unchanged . The colors blue, green and orange label the three different momentum sectors where FQAH are expected while the color red labels the rest of the sectors. (c) The projected particle entanglement spectrum at calculated from the two quasi-degenerate ground states (see (a)). (d) The projected particle entanglement spectrum at calculated from the three quasi-degenerate ground states. particles are kept in (c) and (d). There are 42 states below the line in (c) and (d), consistent with a FQAH state.
to exact diagonalization. At zero flux (standard periodic boundary conditions) we find an apparent two-fold ground state degeneracy, seemingly at odds with FQAH expectations, see Figure 4(a). Crucially, however, we notice that a three-fold set of states originating from the predicted FQAH momenta transform into each other under twisting boundary conditions while evolving essentially separately from the rest of the spectrum as shown in Figure 4(b). Moreover, with anti-periodic boundary conditions in one direction (), these three states are separated from the rest of the spectrum and their corresponding PES shows a well developed entanglement gap with the predicted FQAH counting below the gap as shown in Figure 4(d). In fact, this non-trivial counting persists if we consider the PES resulting only the two low lying states at as evident from Figure 4(c). We take this as strong evidence for a FQAH state in the large system limit.
Discussion. Inspired by recent experiments we have shown that multiband effects are crucial for understanding integer and fractional QAH states as well as their competitors at in twisted semiconductor bilayer MoTe.
Our results have several important implications. First, it follows that the optimum twist angle for QAH states is filling dependent, i.e. there is no unique magic angle for all fillings. For instance, at twist angles , an integer QAH state is missing at (Figure 1(b)) while a fractional FAQH state at prevails (Figure 3(d)). Second, a new intriguing phase is enabled by band mixing: spontaneous layer polarized state at (Figure 1(b),(Figure 1d)). Third, in addition to the particle-hole symmetry breaking within a band, multiband effects provide a second key ingredient in understanding why twisted bilayer may exhibit the FAQH effect at but not at (Figure 2 and Figure 3). Fourth, the theoretical multiband effects uncovered here, and their relation to experimental results provide important means of distinguishing between the greatly varying available sets of model parameters obtained via first principles calculations.
We expect that these effects of multiband mixing will carry over mutatis mutandis to related setups. This thus provides key input for future studies of a rich variety of strongly correlated twisted semiconductor multilayers.
Note added. During the preparation of this work, a related study of twisted bilayer reported two-band ED results at [48]. Upon submission a related preprint appeared [49].
Acknowledgments
Acknowledgements. A.A. and E.J.B. were supported by the Swedish Research Council (VR, grant 2018-00313), the Wallenberg Academy Fellows program (2018.0460) and and the Göran Gustafsson Foundation for Research in Natural Sciences and Medicine. A.A. is also supported by the Wallenberg scholarship program (2022.0348). The work at Massachusetts Institute of Technology is supported by the U.S. Army DEVCOM ARL Army Research Office through the MIT Institute for Soldier Nanotechnologies under Cooperative Agreement number W911NF-23-2-0121 and the Simons Foundation.1
Supplemental Material
Exact Diagonalization
In this paper, we employ exact diagonalization techniques on finite clusters. Let and be two basis vectors for the moiré superlattice. The finite cluster is spanned by two vectors and given by
where , , and are integers. Generic twisted boundary conditions are imposed by requiring and where is the translation operator and is a Bloch state of momentum , band and spin .
In momentum space, this amounts to a discrete set of momentum points given by for integer and where and are the discretization vectors of the momentum grid, given by and with and are the reciprocal lattice basis vectors. In the main text, we use a 9 site cluster and a 12 site cluster shown in Figure S1.
Figure S1. Momentum points of the (a) 9 site cluster and (b) 12 site cluster for
Particle Entanglement Spectrum at ,
Here, we provide evidence for a FQAH state at for twist angles (c.f Figure 3(a)-(Figure 3b)) from particle entanglement spectroscopy. We take as an example. As shown in Figure S2, the PES has a spectral gap, below which the number of states matches the counting expected from a Laughlin type FQAH. This stands in constrast to the counting in the CDW phase, shown in Figure 2(b) which has fewer states.
Figure S2. PES calculated at , and for (a) and (b) . particles are kept. There are 42 states below the line, consistent with a FQAH state.
Results using Ref wang2023fractional parameters
In this sections, we study the continuum model of tMoTe using the parameters introduced in Ref [34]. We focus at filling . As shown in Figure S3, we find signs of QAH states only for weak interaction at twist angles while we find robust ferroelectricity for and for the realistic interaction at all twist angles between and .
Figure S3. (a)-(d) The many body spectrum on the 9 site cluster S1(a) using [34] parameters at and . (e)-(f) Energy gap as a function of twist angle where in (e), for and for while in (f).
References
- [22]K. Watanabe, T. Taniguchi, L. Balents, and A. Young, Intrinsic quantized anomalous hall effect in a moiré het- erostructure, Science 367, 900 (2020).DOI
- [23]Z. Tao, B. Shen, S. Jiang, T. Li, L. Li, L. Ma, W. Zhao, J. Hu, K. Pustunova, K. Watanabe, T. Taniguchi, T. F. Heinz, K. F. Mak, and J. Shan, Valley-coherent quantum anomalous Hall state in AB-stacked MoTe2/WSe2 bilayers (2022), arXiv:2208.07452 [cond-mat].arxiv.org/abs/2208.07452
- [24]Y. Xie, A. T. Pierce, J. M. Park, D. E. Parker, E. Khalaf, P. Ledwith, Y. Cao, S. H. Lee, S. Chen, P. R. Forrester, et al., Fractional chern insulators in magic-angle twisted bilayer graphene, Nature 600, 439 (2021).DOI
- [25]J. Cai, E. Anderson, C. Wang, X. Zhang, X. Liu, W. Holtzmann, Y. Zhang, F. Fan, T. Taniguchi, K. Watanabe, et al., Signatures of fractional quantum anomalous hall states in twisted mote2, Nature , 1 (2023).DOI
- [26]Y. Zeng, Z. Xia, K. Kang, J. Zhu, P. Knüppel, C. Vaswani, K. Watanabe, T. Taniguchi, K. F. Mak, and J. Shan, Thermodynamic evidence of fractional chern insulator in moiré mote2, Nature , 1 (2023).DOI
- [27]H. Park, J. Cai, E. Anderson, Y. Zhang, J. Zhu, X. Liu, C. Wang, W. Holtzmann, C. Hu, Z. Liu, et al., Observation of fractionally quantized anomalous hall effect, Nature , 1 (2023).DOI
- [28]F. Xu, Z. Sun, T. Jia, C. Liu, C. Xu, C. Li, Y. Gu, K. Watanabe, T. Taniguchi, B. Tong, J. Jia, Z. Shi, S. Jiang, Y. Zhang, X. Liu, and T. Li, Observation of integer and fractional quantum anomalous hall effects in twisted bilayer mote2, Phys. Rev. X 13, 031037 (2023).DOI
- [29]F. Wu, T. Lovorn, E. Tutuc, I. Martin, and A. MacDonald, Topological insulators in twisted transition metal dichalcogenide homobilayers, Physical review letters 122, 086402 (2019).DOI
- [30]T. Devakul, V. Crépel, Y. Zhang, and L. Fu, Magic in twisted transition metal dichalcogenide bilayers, Nature communications 12, 6730 (2021).DOI
- [31]V. Crépel and L. Fu, Anomalous hall metal and fractional chern insulator in twisted transition metal dichalcogenides, Physical Review B 107, L201109 (2023).DOI
- [32]H. Li, U. Kumar, K. Sun, and S.-Z. Lin, Spontaneous fractional chern insulators in transition metal dichalcogenide moiré superlattices, Physical Review Research 3, L032070 (2021).DOI
- [33]A. P. Reddy, F. Alsallom, Y. Zhang, T. Devakul, and L. Fu, Fractional quantum anomalous hall states in twisted bilayer mote2 and wse2, Phys. Rev. B 108, 085117 (2023).DOI
- [34]C. Wang, X.-W. Zhang, X. Liu, Y. He, X. Xu, Y. Ran, T. Cao, and D. Xiao, Fractional chern insulator in twisted bilayer mote -2, arXiv preprint arXiv:2304.11864 (2023).DOI
- [35]N. Morales-Durán, J. Wang, G. R. Schleder, M. Angeli, Z. Zhu, E. Kaxiras, C. Repellin, and J. Cano, Pressure-enhanced fractional chern insulators along a magic line in moiré transition metal dichalcogenides, Physical Review Research 5, L032022 (2023).
- [36]A. P. Reddy and L. Fu, Toward a global phase diagram of the fractional quantum anomalous hall effect, arXiv preprint arXiv:2308.10406 (2023).arxiv.org/abs/2308.10406
- [37]Z. Y. Zhu, Y. C. Cheng, and U. Schwingenschlögl, Giant spin-orbit-induced spin splitting in two-dimensional transition-metal dichalcogenide semiconductors, Phys. Rev. B 84, 153402 (2011).DOI
- [38]See supplemental material for details.
- [39]T. Wang, T. Devakul, M. P. Zaletel, and L. Fu, Topo-
- [40]W.-X. Qiu, B. Li, X.-J. Luo, and F. Wu, Interaction-driven topological phase diagram of twisted bilayer mote -2, arXiv preprint arXiv:2305.01006 (2023).DOI
- [41]A. Abouelkomsan, E. J. Bergholtz, and S. Chatterjee, Multiferroicity and topology in twisted transition metal dichalcogenides, arXiv preprint arXiv:2210.14918 (2022).
- [42]H. Li and F. D. M. Haldane, Entanglement spectrum as a generalization of entanglement entropy: Identification of topological order in non-abelian fractional quantum hall effect states, Phys. Rev. Lett. 101, 010504 (2008).DOI
- [43]A. Sterdyniak, N. Regnault, and B. A. Bernevig, Extracting excitations from model state entanglement, Phys. Rev. Lett. 106, 100405 (2011).
- [44]F. D. M. Haldane, “fractional quantum” statistics in arbitrary dimensions: A generalization of the pauli principle, Phys. Rev. Lett. 67, 937 (1991).pubmed.ncbi.nlm.nih.gov/10045028
- [45]B. A. Bernevig and F. D. M. Haldane, Model fractional quantum hall states and jack polynomials, Phys. Rev. Lett. 100, 246802 (2008).pubmed.ncbi.nlm.nih.gov/18643608
- [46]E. J. Bergholtz and A. Karlhede, Quantum hall system in tao-thouless limit, Physical Review B 77, 155308 (2008).DOI
- [47]B. A. Bernevig and N. Regnault, Thin-torus limit of fractional topological insulators, arXiv preprint arXiv:1204.5682 (2012).arxiv.org/abs/1204.5682
- [48]C. Xu, J. Li, Y. Xu, Z. Bi, and Y. Zhang, Maximally localized wannier orbitals, interaction models and fractional quantum anomalous hall effect in twisted bilayer mote2, arXiv preprint arXiv:2308.09697 (2023).DOI
- [49]J. Yu, J. Herzog-Arbeitman, M. Wang, O. Vafek, B. A. Bernevig, and N. Regnault, Fractional chern insulators vs. non-magnetic states in twisted bilayer mote2, arXiv preprint arXiv:2309.14429 (2023).arxiv.org/abs/2309.14429