betaFeedback

WorksPhysics and Astronomy

End-state-assisted tunneling at a topological insulator STM tip

Abstract

Recent spin-sensitive scanning tunneling experiments with topological-insulator tips motivate a microscopic understanding of how tip geometry controls current delivery. We show that topological end states mediate resonant tunneling through the gap opened by interedge hybridization in a quantum spin Hall tip. An asymmetric termination gives oppositely spin-polarized helical paths unequal effective lengths and resonance widths. Within an independent-path description, their transmitted-current imbalance vanishes at a matched resonance and becomes finite away from it. Coupling these directional channels to a dd-wave altermagnet yields a spin-dependent interference contrast. Hybridized end states along a stepped termination provide multiple tunneling resonances. Our results establish end-state-mediated tunneling as a transport mechanism for STM with strongly confined topological tips.

Introduction

Spin-polarized scanning tunneling microscopy (STM) extends local imaging and spectroscopy [1–3] to magnetic contrast [4], exchange-split electronic states [5], and atomic-scale antiferromagnetism [6]. Spin-resolved tunneling and spin-selective Andreev reflection also test candidate Majorana zero modes in superconducting chains and vortices [7, 8, 9]. In these applications, the electronic states of the tip and their coupling to the sample determine how spin information enters the measured signal.

Compensated magnets provide a demanding application because momentum-dependent spin splitting and spin-dependent transport can coexist with vanishing net magnetization [10, 11, 12, 13, 14, 15, 16], as exemplified by altermagnetism [17, 18]. Recent experiments using topological insulator tips have reported bias-reversible magnetic contrast [19, 20, 21, 22] and spin-sensitive quasiparticle interference (QPI) in KV2_2Se2_2O [20, 22]. The probes are interpreted in terms of distinct boundary states: helical edge or hinge channels in Bi4_4Br4_4 [20] and spin–momentum-locked surface states in SmB6_6 [22]. Interpreting such measurements requires understanding how these extended boundary states supply electrons to the local tunneling junction. For a helical tip, a sufficiently narrow apex can gap the boundary channels and suppress their connection to the sample even while the wider tip remains conducting. Identifying the states that sustain tunneling through this confined region is therefore essential, because their spectrum and coupling determine the energy-dependent transmission through which the sample is probed.

A quantum spin Hall (QSH) tip provides a minimal setting for addressing this problem Figure 1(a). When its terminal width becomes comparable to the edge-state penetration length [23, 24], opposite edges hybridize and open a gap [25, 26], suppressing direct subgap transmission. Localized states supported by the termination can instead connect the conducting edges to the sample through resonant tunneling Figure 1(b). Their coupling to the supplying channels depends on the boundary geometry. Since ideal helical modes on opposite edges carry opposite spins at a fixed propagation direction [27], unequal path transmissions can produce a directional spin polarization. Time-reversal symmetry nevertheless ensures that equally occupied Kramers partners give zero equilibrium spin density. Whether the directional imbalance appears in a charge measurement therefore depends on how the contact selects the helical contributions Figure 1(c).

Figure 1. End-state-assisted tunneling at a confined QSH apex. (a) Helical-edge-mediated STM of an altermagnet (AM), motivated by the Bi₄Br₄-tip measurements [20]. (b) Interedge hybridization opens an apex confinement gap; a localized end state (purple) mediates resonant tunneling. (c) Asymmetric termination, with the question mark denoting the microscopic channel-selection problem. (d) The supplying edges couple to a common end state through unequal paths, ℓ+>ℓ−\ell_{+} > \ell_{-}, producing different escape amplitudes. Yellow shading marks the hybridization-gapped region. Red and blue label ideal helical branches, with arrows specifying their propagation directions. The two supplying sides belong to the same boundary perimeter; each supports counterpropagating Kramers partners.

In this Letter, we show that topological end states mediate resonant tunneling through a strongly confined QSH apex. Row-end orbitals supplied by dimensional reduction [28] hybridize along a stepped termination and acquire substantial apex weight (End Matter A). These states provide transmission resonances within the gap that suppresses direct edge transport. For an asymmetric termination, the supplying edges reach the localized end state through unequal effective paths Figure 1(d). Their different resonance widths generate unequal transmitted weights away from resonance. The resulting directional polarization requires the contact to preserve the association between spin and path. Coupling these channels to a dd-wave altermagnet yields a spin-dependent interference contrast, with the shared end-state propagator connecting the path weights to the sample response. The theory thus relates the terminal spectrum and path-dependent resonance widths to STM current delivery and the spatial response of a spin-split sample.

Model

We employ a minimal four-band QSH model to capture the helical edge transport and its modification at the atomic-scale apex [25, 29, 30, 31, 32, 33, 34]

HA/B(k)=[MA/B−4Ba2+2Ba2(cos⁡kxa+cos⁡kya)]τzσz−γa(σysin⁡kxa−σxsin⁡kya),(1)H_{A/B}(k)=\left[M_{A/B}-\frac{4B}{a^2}+\frac{2B}{a^2}\left(\cos k_xa+\cos k_ya\right)\right]\tau_z\sigma_z -\frac{\gamma}{a}\left(\sigma_y\sin k_xa-\sigma_x\sin k_ya\right), \tag*{(1)}

where k2=kx2+ky2k^2=k_x^2+k_y^2 and k±=kx±ikyk_{\pm}=k_x\pm ik_y. σi\sigma_i and τi\tau_i are Pauli matrices acting in the spin and orbital spaces, respectively. In the numerical calculations, we take B=−30B=-30 eV·Å², γ=3\gamma=3 eV·Å, and the lattice constant a=1a=1 nm [33, 34, 35].

To isolate the contribution of terminal states, we compare two QSH parameter regimes with matched confinement gaps. We take MA=−0.05M_{\mathrm{A}}=-0.05 eV, associated with the widely studied (Bi,Sb)₂Te₃ model [33, 34, 35], as an end-state-free regime, and MB=−1.15M_{\mathrm{B}}=-1.15 eV as the end-state-assisted regime. The two models have identical transverse probability profiles and finite-size gaps at kx=0k_x=0, with the same leading linear edge dispersion; their finite-kxk_x corrections differ (Secs. SI and SII of the Supplemental Material [36]). Their distinction emerges upon dimensional reduction: for the row Hamiltonian and termination considered here, HAH_{\mathrm{A}} is trivial, whereas HBH_{\mathrm{B}} supports localized end states protected by its chiral structure. This distinction requires more than time-reversal symmetry alone. For a tapered STM tip (Figure 2(a)), finite-size hybridization suppresses the extended-edge contribution near the apex in HAH_{\mathrm{A}}. In contrast, HBH_{\mathrm{B}} supports states at the apex and step terminations (End Matter A), providing additional channels for resonant tunneling.

Figure 2. (a) Schematic of a tapered QSH tip coupled to a sample. Hybridization between opposite edges opens a finite-size gap near the apex (yellow region), while a localized end state (purple) can mediate resonant tunneling through this region. The upper sketches illustrate the gapless edge spectrum, the hybridization-induced gap, and an end state inside the gap. Red and blue channels denote opposite spin, with arrows indicating their propagation directions. The lower profile shows the effective mass barrier: m(s)=mWe−W/λφm(s)=m_{\mathrm{W}}e^{-W/\lambda_{\varphi}} for 0<s<L0<s<L and m(s)=0m(s)=0 outside, where ss is the longitudinal coordinate, L=60L=60 nm is the barrier length, and WW is the effective transverse width of the tip. (b–d) Transmission versus Fermi energy EFE_{\mathrm{F}} for W=30W=30, 15, and 10 nm, respectively. Violet solid curves show the end-state-assisted transmission TBT_{\mathrm{B}}, while orange dashed curves with open square markers show the end-state-free transmission TAT_{\mathrm{A}}. (e) Transmission at EF=0E_{\mathrm{F}}=0 versus WW. As the tip narrows, direct transmission is strongly suppressed, whereas the end-state-assisted channel retains unit transmission at resonance.

Dirac quantum dot model.— Using the edge theory (Sec. SII of the Supplemental Material [36]), we derive an effective model for transport along one boundary of a tapered QSH tip, accounting for hybridization with the opposite boundary and coupling to a localized end

ory (Sec. SII of the Supplemental Material [36]), we derive an effective model for transport along one boundary of a tapered QSH tip, accounting for hybridization with the opposite boundary and coupling to a localized end state, as follows:

HA=∫ds ψ†(s)[−iγρx∂s+m(s)ρz]ψ(s),(2)H_A = \int ds\, \psi^\dagger(s)\left[-i\gamma\rho_x\partial_s + m(s)\rho_z\right]\psi(s), \tag*{(2)}
HB=HA+ϵdd†d+V0ψ†(s0)d+V0†d†ψ(s0).(3)H_B = H_A + \epsilon_d d^\dagger d + V_0\psi^\dagger(s_0)d + V_0^\dagger d^\dagger\psi(s_0). \tag*{(3)}

where ss is transport direction Figure 2(a) and ρi\rho_i act on the opposite boundaries of the QSH tip. For m(s)=0m(s)=0, the effective Hamiltonian describes a gapless Dirac spectrum, corresponding to conducting edge channels without interedge hybridization. A nonzero mass describes the gapped Dirac spectrum produced by finite-size hybridization between the opposite boundaries.

To model the finite-size-gapped region of a tapered tip, we approximate it as a segment of uniform transverse width WW and longitudinal length LL. The mass is therefore taken to be constant within 0<s<L0 < s < L and zero outside: m(s)=mWΘ(s)Θ(L−s)m(s)=m_W\Theta(s)\Theta(L-s), with mW=m∗e−W/λψm_W=m_*e^{-W/\lambda_\psi} denoting the width-dependent effective mass induced by hybridization between opposite edges. We take V0=30 meV nm1/2V_0=30\ \mathrm{meV\,nm}^{1/2} for the end-state-assisted case, and V0=0V_0=0 for the end-state-free case. The massless regions on either side serve as conducting leads. Here, m∗=2∣γMA∣γ2−4BMAm_*=\frac{2|\gamma M_A|}{\sqrt{\gamma^2-4BM_A}} is the value of the finite gap inside the narrow segment and λψ=2∣B∣∣γ∣−γ2−4BMA\lambda_\psi=\frac{2|B|}{|\gamma|-\sqrt{\gamma^2-4BM_A}} is the transverse localization length of the edge-state wave functions. The width WW denotes the separation between the transverse hard walls. For the parameters given above, we have λψ=4.73 nm\lambda_\psi=4.73\ \mathrm{nm}. We note that finite-size effects become significant when W∼λψW\sim\lambda_\psi and grow exponentially as WW decreases.

In Sec. SIII A of the Supplemental Material [36], we derive the transmission coefficients of the two systems. For the end-state-free case HAH_A and the end-state-assisted case HBH_B, respectively, the subgap transmissions are

TA(EF)=[1+mW2mW2−EF2sinh⁡2(κL)]−1,T_A(E_F)=\left[1+\frac{m_W^2}{m_W^2-E_F^2}\sinh^2(\kappa L)\right]^{-1},
TB(EF)=[1+mW2mW2−EF2sinh⁡2(κL)R(EF)]−1.(4)T_B(E_F)=\left[1+\frac{m_W^2}{m_W^2-E_F^2}\sinh^2(\kappa L)R(E_F)\right]^{-1}. \tag*{(4)}

Here, κ(EF)=mW2−EF2/(ℏv)\kappa(E_F)=\sqrt{m_W^2-E_F^2}/(\hbar v) is the inverse decay length inside the mass barrier, with ℏv=∣γ∣\hbar v=|\gamma|. The resonant-contact factor is R(EF)=[δ+ΓcEFtanh⁡(κL/2)/(ℏvκ)]2/(δ2+Γc2)R(E_F)=[\delta+\Gamma_cE_F\tanh(\kappa L/2)/(\hbar v\kappa)]^2/(\delta^2+\Gamma_c^2), where δ=EF−ϵd\delta=E_F-\epsilon_d and Γc=V02/(2ℏv)\Gamma_c=V_0^2/(2\hbar v). When V0=0V_0=0, the end state decouples, giving R(EF)=1R(E_F)=1 and TB=TAT_B=T_A.

It is noticed that the model possesses chiral symmetry, which pins the isolated topological end states to zero energy in the absence of hybridization. However, we emphasize that our tunneling mechanism does not require zero-energy pinning and also applies to end states at nonzero energies within the gap.

Figures 2(b–d) compare the transmission with and without end-state assistance as the strip narrows from W=30W=30 to 10 nm10\ \mathrm{nm}. Stronger hybridization between opposite edges progressively suppresses the end-state-free transmission TAT_A. In contrast, TBT_B retains a pronounced

Figure 3. (a,b) Transmission along the two paths for W=15W=15 and 10 nm10\ \mathrm{nm}, respectively, with ℓ+=70 nm\ell_+=70\ \mathrm{nm} and ℓ−=50 nm\ell_-=50\ \mathrm{nm}. Red solid and blue dashed curves denote TB(EF,ℓ−)T_B(E_F,\ell_-) and TB(EF,ℓ+)T_B(E_F,\ell_+), respectively. (c,d) Corresponding spin polarizations for electron tunneling from the tip to the sample (P+P_+, violet solid) and in the reverse direction (P−P_-, ochre dashed). Both polarizations vanish at the zero-energy resonance. The two columns in (a–d) use different energy scales. (e,f) Polarizations versus the path-length difference Δℓ=ℓ+−ℓ−\Delta\ell=\ell_+-\ell_- for W=15W=15 and 10 nm10\ \mathrm{nm}, respectively, evaluated away from resonance. The shorter path is fixed at ℓ−=50 nm\ell_-=50\ \mathrm{nm}, while ℓ+=ℓ0+Δℓ\ell_+=\ell_0+\Delta\ell varies. We take EF=0.2 meVE_F=0.2\ \mathrm{meV} in (e) and (f).

resonance at zero energy, whose peak height remains unity while its width decreases. Figure 2(e) highlights this contrast: the zero-energy transmission rapidly decreases as the strip narrows in the end-state-free case, but remains unity in the end-state-assisted case. These results show that a localized end state can sustain resonant tunneling through the finite-size-gapped region even when direct edge transmission is strongly suppressed.

In Sec. B of the End Matter, we perform numerical transport calculations within the tight-binding approximation, using the Landauer–Büttiker formula [37, 38, 39] and the recursive Green’s function method [40, 41]. The numerical results are qualitatively consistent with the analytical predictions, showing suppressed transport in HAH_A and pronounced in-gap resonant peaks in HBH_B.

Asymmetric boundaries and directional spin polarization

The Bi4_4Br4_4 experiment associates spin selectivity with tunneling dominated by one helical edge [20]. In the present model, geometric asymmetry gives the two edge-to-end-state paths unequal effective lengths. The resulting resonance widths differ, so that the shorter path contributes more strongly away from resonance. End-state-assisted tunneling thus connects the asymmetry of the supplying paths to their transmitted-current imbalance.

To quantify this mechanism, we describe tunneling from the two helical edges in Figure 1(d) using effective paths of lengths ℓ+\ell_+ and ℓ−\ell_-, with mass profiles m±(s)=mWΘ(s)Θ(ℓ±−s)m_{\pm}(s) = m_W \Theta(s)\Theta(\ell_{\pm} - s). Within an independent-path approximation, the transmission coefficients along the two edge-state paths are TB(E,L=ℓ+)T_B(E,L=\ell_+) and TB(E,L=ℓ−)T_B(E,L=\ell_-), respectively, as given by Eq. (4).

For propagation from the tip toward the sample, the incoming branches on the ℓ+\ell_+ and ℓ−\ell_- paths carry opposite spin projections, ↓\downarrow and ↑\uparrow Figure 1(d). For equal incident fluxes and a contact that preserves these directional contributions, the independent-path approximation gives the energy-resolved polarization

P+(E)=TB(E,ℓ−)−TB(E,ℓ+)TB(E,ℓ+)+TB(E,ℓ−).(5)P_+(E) = \frac{T_B(E,\ell_-) - T_B(E,\ell_+)}{T_B(E,\ell_+) + T_B(E,\ell_-)}. \tag*{(5)}

For reciprocal paths at the same energy and scattering potential, reversing the selected direction exchanges the spin labels, giving P−(E)=−P+(E)P_-(E) = -P_+(E).

Figures 3(a)-3(b) compare the transmissions for ℓ+=70\ell_+ = 70 nm and ℓ−=50\ell_- = 50 nm at W=15W = 15 and 1010 nm. Both paths reach unit transmission at the matched zero-energy resonance. Away from resonance, the shorter path retains a larger transmission because its resonance is broader. This difference produces the spin polarizations in Figs. 3(c)-3(d). The shorter path carries spin-up electrons toward the sample, giving P+>0P_+ > 0; the reciprocal selected current has the opposite polarization. At resonance, equal path transmissions give P±=0P_{\pm} = 0. Reducing WW narrows both peaks and the central region of weak polarization, while increasing the polarization magnitude away from resonance.

To examine the effect of geometric asymmetry, we fix the shorter path at ℓ−=ℓ0=50\ell_- = \ell_0 = 50 nm and increase the longer path as ℓ+=ℓ0+Δℓ\ell_+ = \ell_0 + \Delta\ell, with Δℓ≥0\Delta\ell\ge0. Figures 3(e,f) show the resulting polarizations for W=15W = 15 and 1010 nm, respectively. Equal path lengths give equal transmission weights and zero polarization. As the length difference increases, the longer-path contribution is progressively suppressed, and the polarization magnitude grows. Over the plotted range, this increase is modest for W=15W = 15 nm, whereas the W=10W = 10 nm case approaches P+=1P_+ = 1 and P−=−1P_- = -1. These results illustrate how unequal path lengths can produce single-edge-dominated tunneling through the localized end state.

Spin-polarized quasiparticle interference in an altermagnet

We simulate the scanning conductance of a QSH tip coupled to a collinear dd-wave altermagnet, with the spin axis parallel to the sample Néel vector. The sample Hamiltonian is

HAM=∑kck†[ϵks0+dksz]ck+U∑σc0σ†c0σ,(6)H_{\mathrm{AM}} = \sum_{\mathbf{k}} c_{\mathbf{k}}^\dagger[\epsilon_{\mathbf{k}}s_0 + d_{\mathbf{k}}s_z]c_{\mathbf{k}} + U\sum_{\sigma}c_{0\sigma}^\dagger c_{0\sigma}, \tag*{(6)}

where crσc_{\mathbf{r}\sigma} annihilates an electron on the square lattice of spacing aa, sis_i act on physical spin, ϵk=−2t(cos⁡kxa+cos⁡kya)−4t′cos⁡kxacos⁡kya−μ\epsilon_{\mathbf{k}} = -2t(\cos k_x a + \cos k_y a) - 4t'\cos k_x a\cos k_y a - \mu, and dk=−Δd(cos⁡kxa−cos⁡kya)d_{\mathbf{k}} = -\Delta_d(\cos k_x a - \cos k_y a), which is chosen to mimic the dd-wave altermagnetic band structure of the experimentally studied material KV2_2Se2_2O [20].

Figure 4(a) displays the characteristic dd-wave altermagnetic spin texture. The spin-up and spin-down constant-energy contours extend along the kxk_x and kyk_y directions, respectively, and are related by a 90∘90^\circ rotation. The spin splitting therefore changes sign under this rotation and vanishes along the diagonals kx=±kyk_x = \pm k_y, where the two contours intersect. This momentum-dependent, alternating spin polarization coexists with zero net spin polarization after momentum integration, illustrating a key feature of altermagnetism. It also produces distinct spin-resolved scattering patterns that can be probed by the directionally polarized tip.

Figure 4. Spin-dependent conductance difference of an altermagnet probed by an asymmetric QSH tip. (a) Spin-up (red solid) and spin-down (blue dashed) constant-energy contours at EF=0.12tE_F = 0.12t. (b) Δg(r)=gV+(r)−gV−(r)\Delta g(\mathbf{r}) = g_{V+}(\mathbf{r}) - g_{V-}(\mathbf{r}) on a 49×4949 \times49 lattice window centered on a scalar impurity. (c) Real part of the unnormalized Fourier transform Δg~(q)\Delta\widetilde{g}(\mathbf{q}). Sample parameters are W=10W = 10 nm, t=100t = 100 meV, t′/t=−0.25t'/t = -0.25, μ/t=−1\mu/t = -1, Δd/t=0.8\Delta_d/t = 0.8, U/t=2U/t = 2, and tc/t=0.06t_c/t = 0.06; the tip-relative energy is ω=0.12\omega= 0.12 meV. The peak ∣Δg∣|\Delta g| is 0.127 e2/h0.127\,e^2/h.

We describe the tip using four effective Dirac path models, indexed by spin σ=±1\sigma= \pm1 and boundary b=±b = \pm:

hσb=−iσℏvρx∂s+mb(s)ρz,(7)h_{\sigma b} = -i\sigma\hbar v\rho_x\partial_s + m_b(s)\rho_z, \tag*{(7)}

with mb(s)=mWΘ(s)Θ(ℓb−s)m_b(s) = m_W\Theta(s)\Theta(\ell_b-s). Here σ=+1\sigma= +1 (−1-1) denotes spin up (down), while b=+b = + (−-) labels the upper (lower) boundary path. The two boundaries are distinguished by their mass-segment lengths ℓ+\ell_+ and ℓ−\ell_-. We retain the localized end state and its coupling to the paths as in Eq. (3). At scanning position r\mathbf{r}, the end state couples to the sample through Htun(r)=∑σ(tcdσ†crσ+H.c.)H_{\mathrm{tun}}(\mathbf{r}) = \sum_\sigma(t_c d_\sigma^\dagger c_{\mathbf{r}\sigma} + \mathrm{H.c.}), where dσd_\sigma annihilates the spin-σ\sigma end-state component shared by the two paths. We incorporate the continuous paths and the impurity-containing lattice sample through their self-energies on the end state. The channel transmissions Tσ,b(r)T_{\sigma,b}(\mathbf{r}) are calculated from the resulting Green function, retaining the contact-induced energy shifts and broadenings. The self-energy construction and numerical details are given in Sec. SIII B of the Supplemental Material [36].

To connect with Eq. (5), we compare two directional channel sets at a common energy EE. The forward set contains the spin-up lower-boundary channel (↑,−\uparrow,-) and the spin-down upper-boundary channel (↓,+\downarrow,+), associated with the right-pointing arrows in Figure 2(a). The reciprocal set contains (↓,−\downarrow,-) and (↑,+\uparrow,+). All return channels remain in the end-state self-energy. For each prescribed selection, the channel-resolved conductance coefficient is

gV+(r)≡e2h[T↑,−(r)+T↓,+(r)],g_{V+}(\mathbf{r}) \equiv\frac{e^2}{h}\left[T_{\uparrow,-}(\mathbf{r}) + T_{\downarrow,+}(\mathbf{r})\right],
gV−(r)≡e2h[T↓,−(r)+T↑,+(r)].(8)g_{V-}(\mathbf{r}) \equiv\frac{e^2}{h}\left[T_{\downarrow,-}(\mathbf{r}) + T_{\uparrow,+}(\mathbf{r})\right]. \tag*{(8)}

The relation to complete reservoir injection is specified in Sec. SIII B of the Supplemental Material [36].

Scanning yields Δg(r)=gV+(r)−gV−(r)\Delta g(\mathbf{r}) = g_{V+}(\mathbf{r}) - g_{V-}(\mathbf{r}) and its Fourier transform Δg~(q)=∑re−iq⋅rΔg(r)\Delta\widetilde{g}(\mathbf{q}) = \sum_{\mathbf{r}}e^{-i\mathbf{q}\cdot\mathbf{r}}\Delta g(\mathbf{r}). Figure 4(b) shows alternating positive and negative conductance modulations around the impurity. The real-space contrast and signed Fourier features in Figure 4(c) reverse sign under a 90∘90^\circ rotation, consistent with the exchange of the spin-up and spin-down dispersions in Figure 4(a).

The contrast combines unequal helical-channel weights with the spin-dependent sample response. Comparison with opposite-bias STM maps additionally requires the bias dependence of occupations and level alignment.

Role of topological end states

An isolated trivial localized state can also mediate resonant tunneling, but its contribution is concentrated near a particular resonance whose energy depends on the local confining potential. The advantage of topological end states in the present geometry lies in their repeated occurrence at the appropriate boundary terminations. For the symmetries considered here, the nontrivial one-dimensional structure obtained by dimensional reduction of HBH_B supports end-state orbitals at the row ends. The stepped tip boundary therefore naturally provides multiple such orbitals without requiring a separate local defect at each termination.

These end-state orbitals hybridize with one another and mix with extended edge states, producing a set of in-gap levels with substantial step-end weight distributed over a broad energy interval, as shown in Figure 5 of the End Matter. When coupled to the conducting tip channels and the sample, the hybridized states with appreciable contact overlap can each contribute a tunneling resonance. The resulting collection of resonances provides multiple opportunities for transmission through the finite-size-gapped region as the energy alignment varies. This mechanism offers a route to tunneling that is less dependent on the precise alignment of a single localized level, providing a practical advantage of topological end states in a stepped QSH tip.

Figure 5. Spatial and spectral signatures of step-end states in a tapered QSH tip. (a) Geometry used for both HAH_A and HBH_B. Purple sites mark the row terminations along the stepped boundary, including the apex, and define the set SS. The inset enlarges a section of the stepped boundary. (b,c) Energy spectra of HAH_A and HBH_B, respectively. Dashed lines mark the bulk band edges at ±50\pm50 meV. In-gap states are colored by their total probability WstepW_{\mathrm{step}} on SS, while states outside the gap are shown in gray. The eigenvalues are ordered by energy, with n=1n = 1 denoting the lowest positive-energy state. (d,e) Spatial probability distributions of the circled states in (b,c), respectively. The insets enlarge the apex and a section of the stepped boundary. Colors encode the local probability ∣ψ(i)∣2|\psi(i)|^2, with separate color scales for (d) and (e); each inset uses the scale of its parent panel.

Acknowledgments

We would like to thank Zhenyu Wang and Chui-Zhen Chen for helpful discussions. This work was supported by the NSFC (under Grants Nos. 12474151, 92565103, 12304195, 12674198, 12547101, and U25D8012). D.H.X. acknowledges the Beijing National Laboratory for Condensed Matter Physics (No. 2024BNLCMPKF025) and the Fundamental Research Funds for the Central Universities (Grant No. 2025CDJAIJSYB-032). R.C. acknowledges the Chutian Scholars Program in Hubei Province, the Hubei Provincial Natural Science Foundation (Grant No. 2025AFA081), the key project of Hubei provincial department of education (under Grant No. D20241004), and the original seed program of Hubei University. B.Z. acknowledges the Wuhan city key R&D program (under Grant No. 2025050602030069).

Step-end states in a tapered tip

To examine the spatial origin of end-state-assisted tunneling, we compare the spectra and wave-function distributions of HAH_A and HBH_B using the same tapered geometry shown in Figure 5(a). The purple sites mark the terminal sites of the lattice rows along the stepped boundary, including the apex. We denote this set by SS and quantify the probability concentrated on these sites as

Wstep=∑i∈S∣ψ(i)∣2,(9)W_{\mathrm{step}} = \sum_{i \in S} |\psi(i)|^2, \tag*{(9)}

where ψ\psi is a normalized eigenstate and ∣ψ(i)∣2|\psi(i)|^2 is the probability on site ii, summed over the internal degrees of freedom.

Figure 5(b) and 5(c) reveal a clear contrast between the two models. Within the bulk gap, the edge-derived states of HAH_A carry little weight on the step-end sites. In contrast, HBH_B exhibits more in-gap states with substantial probability concentrated at the step ends. This distinction is directly visible in the lowest positive-energy states shown in Figure 5(d) and Figure 5(e). For HAH_A, the probability is distributed along the boundary, with little weight at the row terminations or the apex. For HBH_B, pronounced local concentrations appear at the step ends, including finite weight at the apex.

The HBH_B levels with appreciable step-end weight span approximately 82.9 meV. Their spread in energy reflects hybridization among the row-end orbitals and their mixing with extended edge states, as analyzed in the Supplemental Material [36]. The stepped boundary thus supports a set of end-state-derived levels distributed across a broad portion of the bulk gap. These states can provide resonant tunneling channels when a step termination or the apex is coupled to the sample.

Transport

We investigate the quantum transport properties of the system based on the specific tip geometry shown in Figure 6(a). The two-terminal conductance is calculated by using the Landauer-Büttiker formula [37, 38, 39] combined with the recursive Green’s function method [40, 41] (see Sec. SIII C of Supplemental Material [36] for more details). The setup consists of a tapered QSH STM tip and

Figure 6. (a) Geometry of the QSH STM tip and sample used in the tight binding calculations. (b) Two-terminal conductance gg as a function of the tip angle α\alpha at a fixed Fermi energy EF=1E_F = 1 meV. (c) Conductance gg as a function of the Fermi energy EFE_F at a fixed tip angle α=π/3\alpha= \pi/3. (d) Conductance map in the (EF,α)(E_F,\alpha) plane for the end-state-assisted QSH model.

a flat QSH sample. The angle α\alpha controls the geometry of the tip apex.

As shown in Figure 6(b), when α=0\alpha= 0, both QSH models exhibit an identical quantized conductance of g=2e2/hg = 2e^2/h. This is because the uniform wide ribbon allows the helical edge modes to transport without experiencing the finite-size effect. However, as α\alpha becomes nonzero, the transport behaviors of the two models diverge sharply. The conductance of the end-state-free QSH model is immediately suppressed once the tip begins to taper. This behavior reflects the fact that the helical edge mode is interrupted by the large finite-size gap opened at the atomic-scale apex and therefore cannot tunnel into the sample. In sharp contrast, the end-state-assisted QSH model retains a finite conductance over a broad range of α\alpha, despite having the same wide-region edge dispersion as the end-state-free model. In Sec. SIII C of the Supplemental Material [36], we provide further numerical results and demonstrate that this conclusion remains robust against variations in the mass parameter MM and tip width.

To further explore the transport behavior, Figure 6(c) shows the conductance as a function of the Fermi energy at a fixed angle α=π/3\alpha= \pi/3. The conductance displays sharp resonant peaks inside the finite-size gap. These peaks are the hallmark of resonant tunneling. As we will show below, we attribute this resonant transport to the emergent end states at the tip apex. This feature is further supported by the full conductance map in the (EF,α)(E_F,\alpha) plane for the end-state-assisted model shown in Figure 6(d). Notably, the resonant tunneling behavior becomes more pronounced when α\alpha is relatively large.

As α\alpha increases, the conductance spectrum transforms from a smooth transport window into sharp resonance branches that split across the energy range. We provide a detailed analysis of this behavior in Sec. SIII C of the Supplemental Material [36]. Specifically, an increase in α\alpha geometrically extends the length of the hybridized transition region at the tip apex, which increases the spatial extent of the gapped region. This reduces the lifetime broadening of the in-gap states, causing the resonant tunneling paths to turn into sharp lines at larger angles.

Transport simulation

To make a qualitative connection with the spin-resolved scanning-tunneling experiment [20], we also modeled an STM tip within the tight-binding framework and performed scanning transport calculations using the device geometry shown in Figure 7(a). The setup consists of an end-state-assisted QSH tip coupled to a two-dimensional dd-wave altermagnetic device. The microscopic Hamiltonian, the lead configuration, and the numerical implementation are given in Sec. III D of the Supplemental Material [36]. This geometry allows us to calculate the two-terminal spin-resolved conductance, denoted by gσ(r)g^\sigma(\mathbf{r}), as a function of the tip position r\mathbf{r}, with

Figure 7. (a) Schematic of the qualitative scanning-transport simulation of a QSH tip coupled to a dd-wave altermagnetic device. A QSH tip is scanned over the AM region connected to external leads. The spin-resolved altermagnetic spectrum is shown in the device. (b) Fourier-space spin contrast extracted from the scanning transmission maps. The opposite signs along the qxq_x and qyq_y directions reflect the dd-wave altermagnetic spin texture. (c) and (d) Real-space spin-resolved transmission modulations δg↑\delta g^{\uparrow} and δg↓\delta g^{\downarrow}, defined as the deviation of the calculated local transmission from its spatial average, δgσ(r)=gσ(r)−⟨gσ⟩r\delta g^{\sigma}(\mathbf{r}) = g^{\sigma}(\mathbf{r}) - \langle g^{\sigma}\rangle_{\mathbf{r}} with σ=↑,↓\sigma= \uparrow,\downarrow. The two spin channels show mutually orthogonal quasi-one-dimensional modulations.

σ=↑,↓\sigma= \uparrow,\downarrow denoting spin.

Figures 7(c) and 7(d) show the spin-resolved transport modulations, denoted by δg↑\delta g^{\uparrow} and δg↓\delta g^{\downarrow}, respectively. Here the modulation is defined by subtracting the spatial average from the calculated transport map, δgσ(r)=gσ(r)−⟨gσ(r)⟩r\delta g^{\sigma}(\mathbf{r}) = g^{\sigma}(\mathbf{r}) - \langle g^{\sigma}(\mathbf{r})\rangle_{\mathbf{r}}, where ⟨gσ(r)⟩r\langle g^{\sigma}(\mathbf{r})\rangle_{\mathbf{r}} denotes the spatial average of gσ(r)g^{\sigma}(\mathbf{r}) over all tip positions r\mathbf{r} in the scanning region. This subtraction removes the uniform background transport and highlights the position-dependent scanning contrast.

The two maps exhibit approximately orthogonal anisotropic patterns. This behavior can be directly understood from the spin-resolved spectrum shown in Figure 7(a). Its constant-energy contours contain nearly parallel segments along orthogonal directions. Specifically, for the spin-up sector (colored by cyan), the relevant contour segments are nearly parallel to the kyk_y direction and are mainly separated along kxk_x. Therefore, the dominant scattering wave vector selected by the QSH tip is oriented along qxq_x, with q↑≃(Δkx,0)\mathbf{q}_{\uparrow} \simeq(\Delta k_x, 0). The corresponding real-space transport modulation can be written schematically as δg↑(r)∼cos⁡(q↑⋅r)=cos⁡(qxx)\delta g^{\uparrow}(\mathbf{r}) \sim\cos(\mathbf{q}_{\uparrow}\cdot\mathbf{r}) = \cos(q_xx). Thus, δg↑\delta g^{\uparrow} varies primarily along the xx direction and is relatively uniform along the yy direction, giving rise to the stripe-like pattern in Figure 7(c).

In contrast, for the spin-down sector, the corresponding contour segments are rotated by approximately 90∘90^\circ. They are nearly parallel to the kxk_x direction and are mainly separated along kyk_y, giving a dominant scattering wave vector q↓≃(0,Δky)\mathbf{q}_{\downarrow} \simeq(0,\Delta k_y). The resulting real-space response is then δg↓(r)∼cos⁡(q↓⋅r)=cos⁡(qyy)\delta g^{\downarrow}(\mathbf{r}) \sim\cos(\mathbf{q}_{\downarrow}\cdot\mathbf{r}) = \cos(q_yy). Therefore, δg↓\delta g^{\downarrow} varies primarily along the yy direction and produces the orthogonal pattern shown in Figure 7(d), with strong Fourier components mainly along the qyq_y direction.

To further expose the momentum-space information contained in the scanning signal, we Fourier transformed the real-space transport maps and plotted the spin contrast δg(q)\delta g(\mathbf{q}) in Figure 7(b). The Fourier-space contrast shows opposite signs for the qxq_x- and qyq_y-oriented scattering features, consistent with the dd-wave form factor of the altermagnetic spin texture. Thus, our simulation captures the same qualitative fingerprint as in the experiments: spin-selective, mutually orthogonal real-space modulations and a sign-changing dd-wave contrast in momentum space [20].

We also performed additional calculations in Sec. III D of the Supplemental Material [36]. First, we replaced the end-state-assisted QSH tip by an end-state-free QSH tip. In this case, the characteristic spin-selective scanning pattern disappears. The reason is that, at the atomic-scale tip apex, finite-size hybridization opens a local gap for the helical edge states. Without the localized zero-dimensional end state at the apex, the helical channel is blocked and cannot efficiently mediate tunneling into the AM device. By contrast, the end-state-assisted QSH tip supports resonant tunneling through the localized apex state, so that the helical spin selectivity survives in the scanning transport signal.

Second, we replaced the AM device by a normal-metal device while keeping the QSH tip and the lead geometry unchanged. The characteristic real-space images and Fourier-space spin contrast are again absent. This control calculation shows that the QSH tip alone is insufficient to generate the observed pattern. A normal metal does not possess dd-wave spin splitting or symmetry-related spin-polarized X/YX/Y-valley quasiparticle channels; therefore, there is no underlying altermagnetic spin texture for the helical tip to select. Any residual interference generated by the finite device geometry or by the attached leads is not locked to the alternating AM spin texture and therefore cannot reproduce the orthogonal spin-resolved maps or the sign-changing Fourier contrast.

Taken together, these control calculations support the proposed mechanism within the device geometry and parameter regime studied here. End-state assistance sustains tunneling through the narrow tip apex, while the altermagnetic band structure produces the characteristic spin-dependent spatial modulations. The resulting scanning contrast is qualitatively consistent with the experimental observations [20]. These calculations illustrate how an end-state-assisted QSH tip can probe altermagnetic spin textures, while the microscopic structure of the experimental apex remains to be established.

Figure 8. Energy spectra of the strip model as a function of the transverse width LyL_y. (a) and (b) represent the calculated results using the effective model of Bi₄Br₄ with Lx=40L_x = 40 nm and the realistic effective model of HgTe with Lx=500L_x = 500 nm, respectively. The green points denote bulk states, and the orange points correspond to edge states at large transverse widths LyL_y and localized end states at small LyL_y.

Emergent end state in Bi₄Br₄

To further verify our findings, we calculate the energy spectra using the realistic effective models for Bi₄Br₄ [42,

43] and HgTe [31, 44, 45, 46]. The detailed model Hamiltonians and parameters are provided in Sec. IV of the Supplemental Material [36]. We observe that when the system is dimensionally reduced to a one-dimensional chain, localized end states emerge in the Bi₄Br₄ model [Fig. 8(a)]. In contrast, the HgTe model becomes fully gapped without any end states under the same one-dimensional reduction [Fig. 8(b)].

Therefore, this intrinsic property makes Bi₄Br₄ a natural candidate for realizing end-state-assisted tunneling at the extreme atomic limit, providing a possible explanation for the recent experimental observation [20]. Furthermore, the dimensional reduction mechanism may provide an explanation why such topology-driven tip applications have remained elusive in prototypical topological materials like HgTe [31, 45, 46] or Bi₂Te₃ [35, 47]. In those systems, the inevitable finite-size hybridization at the atomic apex trivially gaps out the edge modes without generating any intermediate end states, thereby blocking the topological transport channels.

References

  1. [5]M. Bode, M. Getzlaff, and R. Wiesendanger, “Spin-polarized vacuum tunneling into the exchange-split surface state of gd(0001)”, Phys. Rev. Lett. 81, 4256 (1998).DOI
  2. [6]S. Heinze, M. Bode, A. Kubetzka, O. Pietzsch, X. Nie, S. Blügel, and R. Wiesendanger, “Real-space imaging of two-dimensional antiferromagnetism on the atomic scale”, Science 288, 1805 (2000).pubmed.ncbi.nlm.nih.gov/10846158
  3. [7]S. Jeon, Y. Xie, J. Li, Z. Wang, B. A. Bernevig, and A. Yazdani, “Distinguishing a majorana zero mode using spin-resolved measurements”, Science 358, 772 (2017).DOI
  4. [8]J. J. He, T. K. Ng, P. A. Lee, and K. T. Law, “Selective equal-spin andreev reflections induced by majorana fermions”, Phys. Rev. Lett. 112, 037001 (2014).pubmed.ncbi.nlm.nih.gov/24484161
  5. [9]H.-H. Sun, S.-J. Zhang, J.-W. Hu, C. Li, G.-Y. Wang, H.-Y. Ma, et al., “Majorana zero mode detected with spin selective andreev reflection in the vortex of a topological superconductor”, Phys. Rev. Lett. 116, 257003 (2016).DOI
  6. [10]M. Naka, S. Hayami, H. Kusunose, Y. Yanagi, Y. Motome, and H. Seo, “Spin current generation in organic antiferromagnets”, Nat. Commun. 10, 4305 (2019).
  7. [11]K.-H. Ahn, A. Hariki, K.-W. Lee, and J. Kuněs, “Antiferromagnetism in RuO2 as d-wave Pomeranchuk instability”, Phys. Rev. B 99, 184432 (2019).DOI
  8. [12]S. Hayami, Y. Yanagi, and H. Kusunose, “Momentum-dependent spin splitting by collinear antiferromagnetic ordering”, J. Phys. Soc. Jpn. 88, 123702 (2019).DOI
  9. [13]L.-D. Yuan, Z. Wang, J.-W. Luo, E. Rashba, and A. Zunger, “Giant momentum-dependent spin splitting in centrosymmetric low-z antiferromagnets”, Phys. Rev. B 102, 014422 (2020).DOI
  10. [14]I. I. Mazin, K. Koepĕrnik, M. D. Johannes, R. González-Hernández, and L. Šmejkal, “Prediction of unconventional magnetism in doped FeSb2”, Proc. Nat. Acad. Sci. 118, e2108924118 (2021).DOI
  11. [15]H.-Y. Ma, M. Hu, N. Li, J. Liu, W. Yao, J.-F. Jia, and J. Liu, “Multifunctional antiferromagnetic materials with giant piezomagnetism and noncollinear spin current”, Nat. Commun. 12, 2846 (2021).
  12. [16]L. Šmejkal, A. H. MacDonald, J. Sinova, S. Nakat­sujii, and T. Jungwirth, “Anomalous Hall antiferromagnets”, Nat. Rev. Mat. 7, 482 (2022).arxiv.org/abs/2107.03321
  13. [17]L. Šmejkal, J. Sinova, and T. Jungwirth, “Beyond conventional ferromagnetism and antiferromagnetism: A phase with nonrelativistic spin and crystal rotation symmetry”, Phys. Rev. X 12, 031042 (2022).DOI
  14. [18]L. Šmejkal, J. Sinova, and T. Jungwirth, “Emerging research landscape of altermagnetism”, Phys. Rev. X 12, 040501 (2022).arxiv.org/abs/2204.10844
  15. [19]A. Aishwarya, Z. Cai, A. Raghavan, M. Romanelli, X. Wang, X. Li, et al., “Spin-selective tunneling from nanowires of the candidate topological kondo insulator smb6”, Science 377, 1218 (2022).DOI
  16. [20]Z. Wang, S. Yu, X. Cheng, X. Xiao, W. Ma, F. Quan, et al., “Atomic-scale spin sensing of a 2d d-wave altermagnet via helical tunneling”, Nature (2026), https://doi.org/10.1038/s41586-026-11257-1.
  17. [21]S. Banerjee, A. Aishwarya, F. Liu, L. Jiao, V. Madhavan, E. J. Mele, and P. Coleman, “Axionic tunneling from a topological kondo insulator”, arXiv preprint arXiv:2512.05057 (2025).arxiv.org/abs/2512.05057
  18. [22]G. Yang, C. Li, C. Wang, X. Zhao, Y. Wan, H. Gui, et al., “Visualizing spin-polarization of an altermagnet kv2se2o via spin-selective tunneling”, arXiv:2603.21969 (2026).arxiv.org/abs/2603.21969
  19. [23]W. Li, J. Zang, and Y. Jiang, “Size effects on transport properties in topological anderson insulators”, Phys. Rev. B 84, 033409 (2011).DOI
  20. [24]H. Jiang, H. Liu, J. Feng, Q. Sun, and X. C. Xie, “Transport discovery of emerging robust helical surface states in Z2 = 0 systems”, Phys. Rev. Lett. 112, 176601 (2014).pubmed.ncbi.nlm.nih.gov/24836263
  21. [25]S.-Q. Shen, Topological Insulators–Dirac Equation in Condensed Matter (Springer Singapore, 2017).
  22. [26]B. Brüne, H.-Z. Lu, R.-L. Chu, S.-Q. Shen, and Q. Niu, “Finite Size Effects on Helical Edge States in a Quantum Spin-Hall System”, Phys. Rev. Lett. 101, 246807 (2008).
  23. [27]C. Brüne, A. Roth, H. Buhmann, E. M. Hankiewicz, L. W. Molenkamp, J. Maciejko, X.-L. Qi, and S.-C. Zhang, “Spin polarization of the quantum spin hall edge states”, Nature Physics 8, 485 (2012).DOI
  24. [28]D. J. Klaassen, L. Eek, A. N. Rudenko, E. D. van’t Westende, C. Castenmiller, Z. Zhang, et al., “Realization of a one-dimensional topological insulator in ultrathin germanene nanoribbons”, Nature communications 16, 2059 (2025).DOI
  25. [29]C. L. Kane and E. J. Mele, “Quantum spin hall effect in graphene”, Phys. Rev. Lett. 95, 226801 (2005).pubmed.ncbi.nlm.nih.gov/16384250
  26. [30]B. A. Bernevig and S.-C. Zhang, “Quantum Spin Hall Effect”, Phys. Rev. Lett. 96, 106802 (2006).pubmed.ncbi.nlm.nih.gov/16605772
  27. [31]M. König, S. Wiedmann, C. Brüne, A. Roth, H. Buhmann, L. W. Molenkamp, X.-L. Qi, and S.-C. Zhang, “Quantum spin Hall insulator state in HgTe quantum wells”, Science 318, 766 (2007).pubmed.ncbi.nlm.nih.gov/17885096
  28. [32]X.-L. Qi, Y.-S. Wu, and S.-C. Zhang, “Topological quantization of the spin hall effect in two-dimensional paramagnetic semiconductors”, Phys. Rev. B 74, 085308 (2006).DOI
  29. [33]H.-Z. Lu, W.-Y. Shan, W. Yao, Q. Niu, and S.-Q. Shen, “Massive Dirac fermions and spin physics in an ultrathin film of topological insulator”, Phys. Rev. B 81, 115407 (2010).DOI
  30. [34]H.-Z. Lu, A. Zhao, and S.-Q. Shen, “Quantum transport in magnetic topological insulator thin films”, Phys. Rev. Lett. 111, 146802 (2013).DOI
  31. [35]C. Chang, J. Zhang, X. Feng, J. Shen, Z. Zhang, M. Guo, et al., “Experimental observation of the quantum anomalous Hall effect in a magnetic topological insulator”, Science 340, 167 (2013).DOI
  32. [36]See Supplemental Material for more details.
  33. [37]R. Landauer, “Electrical resistance of disordered one-dimensional lattices”, Philos. Mag. 21, 863 (1970).DOI
  34. [38]M. Büttiker, “Absence of backscattering in the quantum Hall effect in multiprobe conductors”, Phys. Rev. B 38, 9375 (1988).pubmed.ncbi.nlm.nih.gov/9945751
  35. [39]D. S. Fisher and P. A. Lee, “Relation between conductivity and transmission matrix”, Phys. Rev. B 23, 6851 (1981).DOI
  36. [40]A. MacKinnon, “The calculation of transport properties and density of states of disordered solids”, Z. Phys. B 59, 385 (1985).DOI
  37. [41]G. Metalidis and P. Bruno, “Green’s function technique for studying electron flow in two-dimensional mesoscopic samples”, Phys. Rev. B 72, 235304 (2005).DOI
  38. [44]M. König, H. Buhmann, L. W. Molenkamp, T. Hughes, C.-X. Liu, X.-L. Qi, and S.-C. Zhang, “The quantum spin hall effect: Theory and experiment”, J. Phys. Soc. Jpn. 77, 031007 (2008).
  39. [45]B. A. Bernevig, T. L. Hughes, and S.-C. Zhang, “Quantum spin Hall effect and topological phase transition in HgTe quantum wells”, Science 314, 1757 (2006).
  40. [46]A. Roth, C. Brüne, H. Buhmann, L. W. Molenkamp, J. Maciejko, X.-L. Qi, and S.-C. Zhang, “Nonlocal transport in the quantum spin Hall state”, Science 325, 294 (2009).DOI
  41. [47]R. Yu, W. Zhang, H.-J. Zhang, S.-C. Zhang, X. Dai, and Z. Fang, “Quantized anomalous Hall effect in magnetic topological insulators”, Science 329, 61 (2010).

Paper details

vlnc:W_8c46f8rn
Version
v1 · latest
Published
Fresh — added to the Hub within the last 30 days
Added
License
CC BY 4.0
Usage

views · 0

downloads · 0

A view counts once per browser per day, a download each time the PDF is requested.

  • Physics and Astronomy

Not recorded for this paper

View statement
Looking for related papers…
Contents