Introduction. —The exploration of novel quantum states, such as quantum spin liquids (QSLs) [1 , 2 , 3 , 4 , 5 ] , relies heavily on understanding anisotropic magnetic interactions beyond the Heisenberg model. Historically, the theoretical framework for such bond-dependent anisotropy was established by the quantum compass model [6 , 7 , 8 , 9 ] . This concept has found its most profound realization in the Kitaev model on a honeycomb lattice [10 , 11 , 12 , 13 ] , which has become increasingly important as a universal quantum interaction mechanism. The most representative example is the Kitaev interaction in two-dimensional (2D) honeycomb lattices constructed from edge-sharing octahedral coordination, e.g. , α-RuCl₃ [5 , 14 , 15 , 16 , 17 ] , which is closely related to frustrated states such as QSLs and Majorana fermions [18 , 19 , 20 ] . However, in another significant class of systems possessing strong electron correlation and rich quantum phenomena—edge-sharing tetrahedral coordination structures like 2D iron-based superconductors—whether similar critical bond-dependent interactions exist and how to characterize them remains an unexplored topic. This is closely related to whether the magnetic disorder in iron-based superconductor parent compounds originates from, or partially involves, bond-dependent interactions, necessitating urgent research into such interactions.
Generalizing bond-dependent interactions from octahedral to tetrahedral coordination structures, and ultimately to other lattice geometries, has been hindered by the lack of a unified calculation scheme to quantitatively characterize their underlying rules. The “Energy mapping combined with magnetic anisotropy energy (MAE)” calculation scheme [21 , 22 , 23 ] developed by our group provides a basis for establishing bond-dependent interaction models; its principle is that differences in the MAE of various collinear magnetic configurations can reflect the existence and laws of bond-dependent interactions.
Bond-dependent interactions are inherently tied to spin-orbit coupling (SOC) [24 , 25 , 26 , 27 ] , yet conventional theories often underestimate the influence of SOC on magnetism in iron-based superconductors [28 , 29 , 30 , 31 ] . However, the iron-based superconductor parent compound FeTe incorporates the heavy ligand element Te, which induces substantial SOC [21 , 32 , 33 , 34 , 35 ] ; similarly, the ligand element Se in FeSe also imparts non-negligible SOC [22 , 36 , 37 , 38 ] . This suggests that novel bond-dependent interaction mechanisms driven by SOC, which have not yet been fully appreciated, may lie hidden within FeTe and FeSe parent compounds, with their strength and anisotropy inherently governed by the local crystal structure (especially bond directionality).
In this Letter, we demonstrate the presence of Kitaevlike bond-dependent interactions induced by chalcogen ligands (Te/Se) in the edge-sharing tetrahedral environments of 2D iron-based superconductors FeX (X=Te, Se). Such distinctive interactions are intrinsically tied to specific bond directions, offering a framework to understand their unique magnetic and superconducting phenomena. We first construct a Kitaev-like model based on the characteristics of the edge-sharing tetrahedral geometry and the physics of the Ising model, and derive a minimal microscopic spin model—incorporating Heisenberg, Dzyaloshinskii-Moriya (DM), Kitaev-like, and off-diagonal exchange interactions—via strong-coupling perturbation theory. Through first-principles calculations combining energy mapping with MAE [21 , 22 , 23 ] , we evaluate the angle-dependent MAE for various collinear magnetic configurations and parameterize this bond-dependent anisotropic model. By isolating the contribution from SIA, we accurately extract the Kitaev-like interaction parameters. Despite their similar tetragonal layered structures, FeTe and FeSe exhibit fundamentally distinct microscopic origins of MAE: the Kitaev-like interaction dominates in FeTe, whereas in FeSe it becomes comparable to SIA. Furthermore, we demonstrate that these Kitaev-like interactions in iron-based superconductor parent compounds can induce single-site spin frustration at Fe sites. This finding highlights the universality of bond-dependent models in tetragonal systems, providing a fresh perspective toward addressing key questions in high-temperature superconductivity.
The core issue of this work is how to construct and describe bond-dependent interactions in monolayer FeX (X=Te, Se) in the 2D edge-sharing tetrahedral coordination structure. As shown in Figure 1 (a) and 1(b), the basic structural unit of monolayer FeX consists of one Fe atomic layer sandwiched between two X atomic layers. As illustrated in Figure 1 (c), each Fe atom is located at the center of a tetrahedron formed by four surrounding X atoms, forming Fe–X–Fe–X planes with the X atoms. Adjacent planes without shared edges [faces of the same color in Figure 1 (c)] are mutually perpendicular, while those with shared edges form a specific angle. These mutually perpendicular planes give rise to single-site spin frustration induced by bond-dependent interactions. However, the lack of a natural orthogonal basis along the bond directions within the structure complicates the construction of a bond-dependent model. Nevertheless, we find that for any two of the four Fe–X bonds formed by an Fe atom and its four surrounding X atoms, their vector difference is perpendicular to the other two bonds as well as to the Fe–X–Fe(–X) plane between 1NN (2NN) sites [Figure 1 (d) and 1(e)], providing a crucial prerequisite for constructing bond-dependent interactions.
Figure 1. Structure of monolayer FeX(X=Te, Se) and schematic of Kitaev-like interactions. Fe is represented by brown atoms, upper-layer X by gray atoms, and lower-layer X by black atoms. (a) Top view of monolayer FeX and schematic of first- and second-nearest-neighbor (1NN and 2NN) Fe atoms. The unit cell is a square formed by 1NN connections. (b) Side view of monolayer FeX, where α \alpha α , β \beta β , and γ \gamma γ denote the Kitaev-like basis vectors for the corresponding planes, while x ~ \tilde{x} x ~ , y ~ \tilde{y} y ~ , and z ~ \tilde{z} z ~ represent the local spin quantization axes. (c) Structural diagram of monolayer FeX. (d), (e) 1NN and 2NN Kitaev-like interactions, respectively. w p w_p w p (w ⃗ p \vec{w}_p w p ) (p = 1 , 2 , 3 , 4 p=1,2,3,4 p = 1 , 2 , 3 , 4 ) represent the Fe–X bond directions (where w p w_p w p and w ⃗ p \vec{w}_p w p only distinguish different sites within the same FeX unit cell). Taking 1NN as an example, the Kitaev-like interaction basis directions in different planes are along the two in-plane bond directions w p w_p w p , w p + 1 w_{p+1} w p + 1 (α \alpha α or β \beta β ) and the vector difference of the two out-of-plane bond directions w ( p + 2 ) − ( p + 3 ) = w p + 2 − w p + 3 w_{(p+2)-(p+3)}=w_{p+2}-w_{p+3} w ( p + 2 ) − ( p + 3 ) = w p + 2 − w p + 3 (γ \gamma γ , the Ising axis direction, indicated by yellow arrows); the Fe–Fe bond direction W ( p + 2 ) − ( p + 3 ) W_{(p+2)-(p+3)} W ( p + 2 ) − ( p + 3 ) is perpendicular to the Ising axis direction. The subscript p + q p+q p + q (q = 1 , 2 , 3 q=1,2,3 q = 1 , 2 , 3 ) denotes a shift in the index, with cyclic symmetry (looping back to 1 when exceeding 4).
In the following, combining second-order strong-coupling perturbation theory, we formulate the Kitaev-like bond-dependent interaction model and derive its microscopic spin Hamiltonian in an ideal 2D edge-sharing tetrahedral structure [25 , 26 , 27 , 39 , 40 , 41 , 42 ] . Unlike the conventional Kitaev model where the Kitaev local spin basis
α \alpha α , β \beta β , and γ \gamma γ on a given plane are mutually orthogonal, we denote the two non-orthogonal Fe–X bonds in the Fe–X–Fe(–X) plane corresponding to a given Fe–Fe bond as α \alpha α and β \beta β [Figure 1 (b)], while γ \gamma γ corresponds to direction perpendicular to this plane, which can be described by the vector difference between two out-of-plane Fe–X bonds; specific setups for different bonds are detailed in the captions of Figure 1 and Sec. S1 of the Supplemental Material (SM) [43 ] . Consequently, we can model the bond-dependent Kitaev-like interactions by leveraging the inherent bond orientations of the 2D edge-sharing tetrahedral geometry.
However, as shown in Figure 1 (d) and 1(e), the normal vectors of the planes between 1NN and 2NN differ, making it impossible to describe the superexchange processes between different Fe–Fe atom pairs within a single global coordinate system and its permutations. Furthermore, with the non-normality, the Kitaev-like local basis α \alpha α , β \beta β , and γ \gamma γ cannot be directly treated as local quantization axes. Therefore, we establish local coordinate systems (x ~ , y ~ , z ~ \tilde{x},\tilde{y},\tilde{z} x ~ , y ~ , z ~ ) for different Fe–X–Fe(–X) planes to unify the superexchange processes across all Fe–Fe bonds Figure 1 (b) (see SM Sec. S2 for details [43 ] ). In the microscopic derivation, we consider the Kanamori interaction [40 ] for Fe in the hole representation alongside crystal-field splitting to construct the tight-binding Hamiltonian. Under strong coupling, treating the tight-binding hopping integrals as perturbations, we analyze the indirect superexchange process between Fe atoms mediated by ligand X-atom SOC. Finally, mapping the spin operators from local coordinates back to the global Cartesian coordinate system yields the minimal spin Hamiltonian originating from 1NN and 2NN superexchange in the 2D edge-sharing system:
H s = ∑ ⟨ i , j ⟩ 1 , 2 1 2 [ J i j ( S i ⋅ S j ) ] + ∑ ⟨ i , j ⟩ 1 , 2 1 2 [ K i j S i γ S j γ + Γ i j ( S i α S j β + S i β S j α ) ] + ∑ ⟨ i , j ⟩ 2 1 2 D i j [ ( z × d i j ) ⋅ ( S i × S j ) ] + ∑ i A k [ 1 − ( S i ⋅ k ) 2 ] , (1) H_s = \sum_{\langle i,j\rangle_{1,2}} \frac{1}{2}\left[J_{ij}(\mathbf{S}_i\cdot\mathbf{S}_j)\right]
+ \sum_{\langle i,j\rangle_{1,2}} \frac{1}{2}\left[K_{ij}S_i^\gamma S_j^\gamma+\Gamma_{ij}(S_i^\alpha S_j^\beta+S_i^\beta S_j^\alpha)\right]
+ \sum_{\langle i,j\rangle_2}\frac{1}{2}D_{ij}\left[(\mathbf{z}\times\mathbf{d}_{ij})\cdot(\mathbf{S}_i\times\mathbf{S}_j)\right]
+ \sum_i A_k\left[1-(\mathbf{S}_i\cdot\mathbf{k})^2\right], \tag*{(1)} H s = ⟨ i , j ⟩ 1 , 2 ∑ 2 1 [ J ij ( S i ⋅ S j ) ] + ⟨ i , j ⟩ 1 , 2 ∑ 2 1 [ K ij S i γ S j γ + Γ ij ( S i α S j β + S i β S j α ) ] + ⟨ i , j ⟩ 2 ∑ 2 1 D ij [ ( z × d ij ) ⋅ ( S i × S j ) ] + i ∑ A k [ 1 − ( S i ⋅ k ) 2 ] , (1)
where the terms represent the Heisenberg interaction J i j J_{ij} J ij , the Kitaev-like interaction K i j K_{ij} K ij , the off-diagonal exchange Γ i j \Gamma_{ij} Γ ij , the DM interaction D i j D_{ij} D ij . In addition, a single-ion anisotropy (SIA) term A k A_k A k is incorporated [27 , 37 ] . Here, ⟨ i , j ⟩ \langle i,j\rangle ⟨ i , j ⟩ represent Fe atom neighbor pairs, where subscripts 1 and 2 denote the 1NNs and 2NNs, respectively, and S \mathbf{S} S is the normalized spin operator. The local Kitaev-like basis { α , β , γ } \{\alpha,\beta,\gamma\} { α , β , γ } are the spin component directions of the planar basis in Figure 1 (b), Figure 1 (d), and Figure 1 (e). The explicit form of the Hamiltonian formulated using the bond directions is given in Eq. (62) in the Supplemental Material [43 ] . Here, α \alpha α and β \beta β form an angle of 109.47° and are not perpendicular. i , j , k \mathbf{i},\mathbf{j},\mathbf{k} i , j , k are the basis vectors of the Cartesian coordinate system. Specifically, the Kitaev-like and off-diagonal interactions originate from spin-flip processes involving ligand SOC during virtual hoppings along both Fei → _i\rightarrow i → Fej _j j and Fej → _j\rightarrow j → Fei _i i paths [27 ] ; the DM interaction arises from superexchange processes where SOC acts during only a single virtual hopping step; and the Heisenberg interaction receives contributions from both spin-flip processes and spin-conserving processes without SOC.
In the edge-sharing tetrahedral coordination structure, the Fe atom is at the center of the crystal field of the tetrahedron formed by Te atoms, and the five degenerate d d d orbitals split into higher-energy triply degenerate t 2 t_2 t 2 and lower-energy doubly degenerate e e e orbitals. Analysis of the superexchange processes reveals that, in the global coordinate system, the d x z d_{xz} d x z (d y z d_{yz} d y z ) orbitals of the central Fe0 _0 0 atom and the d y z d_{yz} d y z (d x 2 − y 2 d_{x^2-y^2} d x 2 − y 2 ) orbitals of Fe1 _1 1 mediate the 1NN Kitaev-like interaction via the p p p orbitals of Te. Meanwhile, the d x z d_{xz} d x z (d y z d_{yz} d y z ) orbitals of Fe0 _0 0 and the d x z d_{xz} d x z (d y z d_{yz} d y z ) orbitals of Fe2 _2 2 mediate the 2NN Kitaev-like interaction via the p p p orbitals of Te Figure 2 (a) and Figure 2 (b). Here, only representative spin-flip superexchange paths involving σ \sigma σ channels are presented; additional processes are detailed in Sec. S5 of the SM [43 ] .
Figure 2. Superexchange spin-flip paths in the global coordinate system. Paths connecting different atoms correspond to hopping integrals, while paths on the same Te atom represent p p p -orbital transformations under SOC. In the schematic of Te atomic orbitals, the blue regions denote hybrid orbitals simultaneously incorporating p x / p z p_x/p_z p x / p z or p y / p z p_y/p_z p y / p z interactions. Spin-flip superexchange processes between (a) d x z d_{xz} d x z and d y z d_{yz} d y z orbitals for 1NN, d x z d_{xz} d x z and d x z d_{xz} d x z orbitals for 2NN; (b) d y z d_{yz} d y z and d x 2 − y 2 d_{x^2-y^2} d x 2 − y 2 orbitals for 1NN, d y z d_{yz} d y z and d y z d_{yz} d y z orbitals for 2NN.
Naturally, how to intuitively and concisely extract and parameterize the edge-sharing tetrahedral bond-dependent Kitaev-like interactions in DFT calculations and decompose the interaction parameters in Eq. (1) becomes the focus of our subsequent discussion. To further analyze and validate our model via DFT calculations [44 , 45 , 46 , 47 , 48 ] , we obtained the electronic structures of monolayer FeX (see Sec. S3 of the SM [43 ] ). The band structure shows that in the presence of SOC, band gaps open at crossing points that exist without SOC. This indicates that the Te/Se ligands induce strong SOC [21 , 32 ] , which serves as the entry point for Kitaev-like interactions.
Based on previous work, Kitaev interactions can be directly reflected in the differences of MAE among various collinear magnetic orders [21 , 22 , 23 ] . Because the magnetic moments are parallel or antiparallel in these collinear configurations, isotropic Heisenberg and DM interactions do not contribute to the collinear MAE. Therefore, to verify the existence of Kitaev-like interactions in monolayer FeX, we need to calculate diverse and reasonable magnetic order structures. Previous DFT studies on FeTe have primarily focused on typical collinear configurations such as FM, Néel AFM, and BAFM [49 , 50 , 51 ] , as shown in Figure 3 . Among them, BAFM order has been widely determined by calculations to be the magnetic ground state of the FeTe system [49 ] . For the FeSe system, the magnetic ground state remains a subject of ongoing debate and theoretical challenge [49 , 52 , 53 ] .
Figure 3. Calculation and formula fitting results of M A E ( θ , φ ) \mathrm{MAE}(\theta,\varphi) MAE ( θ , φ ) for monolayer FeTe and FeSe under various magnetic configurations. (a)–(c) show M A E ( θ , φ ) \mathrm{MAE}(\theta,\varphi) MAE ( θ , φ ) for FeTe under FM, Néel AFM, and BAFM magnetic orders, respectively. (d)–(f) show the corresponding results for FeSe. Blue and red discrete points correspond to DFT calculation results (taken every 5 ∘ 5^\circ 5 ∘ ) for out-of-plane and in-plane M A E ( θ , φ ) \mathrm{MAE}(\theta,\varphi) MAE ( θ , φ ) , respectively; blue and red curves are the fitted curves obtained after substituting the coefficients.
For specific FeX systems, we can obtain the parameters for Kitaev-like and SIA interactions by fitting the theoretically derived formulas to DFT calculation results using the least-squares method. Substituting these parameters into the formulas to generate MAE curves, selfconsistency with calculation results would verify the correctness of the model [21 , 22 , 23 ] .
Based on this, we performed first-principles DFT calculations to evaluate the angular dependence of both out-of-plane and in-plane collinear M A E ( θ , φ ) \mathrm{MAE}(\theta,\varphi) MAE ( θ , φ ) for monolayer FeX in FM, Néel AFM, and BAFM orders (discrete points in Figure 3 ). If the anisotropy were governed solely by SIA, the profiles would be strictly identical across all magnetic structures and confined to the out-of-plane component.
Therefore, simple SIA is far from sufficient to explain the complexity of the DFT results in Figure 3 . Taking FeTe as an example, the calculated M A E ( θ , φ ) \mathrm{MAE}(\theta,\varphi) MAE ( θ , φ ) reveals a strong dependence on the magnetic order: the out-of-plane trends for FM and Néel AFM are diametrically opposite [Figs. 3(a) vs 3(b)]. Furthermore, the BAFM order [Fig. 3(c)] exhibits a non-zero in-plane anisotropy that is strictly forbidden in a pure SIA model. These contradictions provide clear evidence that bond-dependent anisotropic interactions must exist beyond SIA in monolayer FeX.
Notably, a theoretical model restricted to SIA and 1NN interactions predicts a vanishing in-plane MAE for the BAFM order. This contrasts the in-plane calculation results for FeTe, which show a clear sin 2 φ \sin^2\varphi sin 2 φ oscillation. This indicates that introducing 1NN Kitaev-like interactions is insufficient to capture the full magnetic anisotropy. Based on the results from perturbation theory, the superexchange interaction between 2NN Fe ions holds equal status to that of the 1NN. Therefore, we naturally introduced 2NN Kitaev-like interactions, which resolved this issue.
By fitting the calculation results of collinear MAE under different magnetic configurations to the theoretical formulas using the least-squares method, as shown in Figure 4 (a), we extract the full set of magnetic interaction parameters. As shown in Figure 3 , the fitted curves utilizing these parameters perfectly match the calculated values, demonstrating the correctness of our proposed Kitaev-like model.
Figure 4. (a) Fitted magnetic interaction parameters derived from the MAE mapping for monolayer FeTe (left) and FeSe (right). (b), (c) Schematic visualization of competing anisotropy terms acting on the central Fe atom for (b) FeTe and (c) FeSe. The vectors represent the local easy axes induced by different mechanisms: SIA (blue), 1NN Kitaev interaction (yellow), and 2NN Kitaev interaction (pink). Arrow lengths indicate the relative strength of the influence on spin orientation; spins tend to align with the dominantly longer arrows.
For FeTe, the magnetic anisotropy is dominated by the 1NN Kitaev-like interaction, K 1 = − 1.63 K_1=-1.63 K 1 = − 1.63 meV/Fe. This term is significantly larger than both the SIA (A k = − 0.73 A_k=-0.73 A k = − 0.73 meV/Fe) and the off-diagonal exchange terms. In contrast, for monolayer FeSe, the lighter Se ligand induces a weaker SOC, resulting in generally reduced anisotropy scales. Nevertheless, K 1 K_1 K 1 retains the leading role with a value of − 0.37 -0.37 − 0.37 meV/Fe. Crucially, however, this term faces strong competition from the A k A_k A k term, which possesses a comparable magnitude of 0.27 0.27 0.27 meV/Fe but acts in the opposite sign. Furthermore, the coefficients of the Kitaev-like terms are generally larger than those of the off-diagonal interaction terms, indicating that the Kitaev-like interactions in monolayer FeTe and FeSe are primarily Ising-type.
Additionally, the parameters for FeTe are generally larger than those for FeSe [Fig. 4(a)], consistent with the SOC effects of Te and Se, confirming that SOC plays a key role in bond-dependent interactions. Furthermore, we examined the additional off-diagonal interaction Γ ′ \Gamma' Γ ′ [11 , 23 , 27 ] , which are subleading relative to the dominate Kitaev interaction. Their inclusion changes the fitted K K K , Γ \Gamma Γ and A k A_k A k parameters by less than (1%) and does not alter our main conclusions (see § 5.3.4 of the SM [43 ] .
To understand the physical implications of these extracted parameters, we visualize their spatial arrangement with respect to the lattice [Figs. 4(b) and 4(c)]. As discussed in Figure 1 , the directions of the Kitaev-like interaction for 1NN and 2NN are perpendicular to the Fe-X-Fe planes and match the directions of the tetrahedron edges formed by the four X atoms surrounding the central Fe atom [indicated by red dashed lines in Figs. 1(d) and 1(e)]. Therefore, these directions are mutually perpendicular or form angles in space [Figs. 4(b) and 4(c), with each favoring the central spin alignment along its own axis. This results in the spin being unable to simultaneously satisfy the energy minimization of all interactions, giving rise to ground-state degeneracy and spin frustration at single lattice sites. Such spin frustration induced by structure-compatible Kitaev-like bond-dependent interactions provides a plausible microscopic mechanism for understanding the magnetic disorder observed in parent compounds of iron-based superconductors, offering a new perspective beyond conventional exchange models.
Notably, the 2NN DM interaction parameters in FeTe and FeSe reach D 2 = − 7.56 D_{2} = -7.56 D 2 = − 7.56 and − 4.98 -4.98 − 4.98 meV/Fe, respectively. When considering DM interactions alone, the spins on each Fe sublattice tend to form short-period noncollinear helical orders propagating along the ( 1 1 ˉ 0 ) (1\bar{1}0) ( 1 1 ˉ 0 ) and equivalent directions, indicating potential chiral magnetic competition (see SM Sec. S4 of the SM [43 ] ). This adds to the magnetic complexity and provides a symmetry foundation for multi-q q q chiral magnetic states driven by external magnetic fields, magnetic anisotropy, or higher-order exchange interactions [54 , 55 , 56 , 57 ] , making these materials promising candidate platforms for exploring topological spin textures under external fields, strain, or interface engineering.
In summary, we have established a universal bond-dependent magnetic interaction model and derived its Hamiltonian for non-honeycomb, 2D edge-sharing tetrahedral iron-based superconductors, exemplified by monolayer FeTe and FeSe. Our work points out that magnetic interactions related to bond directionality universally exist in lattices with SOC. This not only lays the foundation for extending Kitaev physics to 2D tetrahedral and even more general lattice structures, but also opens a promising research platform for iron-based superconducting materials: their unique crystal structure and electronic properties provide rich possibilities for exploring novel interaction mechanisms deeply linked to superconducting pairing, quantum magnetism, and topological states, offering a fresh perspective toward understanding the mechanism of high-temperature superconductivity.
**Acknowledgments.—**This work was supported by the National Natural Science Foundation of China (NSFC) under Grant Nos. 11204131, 11974181, and 12471455, and the Natural Science Foundation of Jiangsu Higher Education Institutions of China (Grant No. 25KJB140017). We thank Associate Professors Qiang Luo from NUAA and Hongjun Xiang from Fudan University for their assistance and helpful discussions.
References
Supplemental Material for “Modeling Bond-Dependent Kitaev-like interaction in 2D Edge-Sharing Tetrahedral Magnets: FeX (X=Te, Se)”
Mengdong Li; Can Huang; Bingjie Liu; Zhixin Liu; Yanfei Pan; Jiyu Fan; Chunlan Ma; Daning Shi; Yan Zhu
Basis-vector selection rules for the Kitaev(-like) model
Kitaev interactions in octahedral coordination
Taking the hexagonal lattice 1T-CrI3 as an exemplary system, the Kitaev spin model is intimately correlated with its distinctive crystal structure and spin-orbit coupling effects [24 , 25 , 26 , 27 ] . As illustrated in Figure S1 , 1T-CrI3 exhibits a characteristic sandwich-type layered architecture, wherein the Cr atomic layer is sandwiched between two I atomic layers, forming a Cr-I-Cr heterogeneous interlayer stacking configuration. From a crystallographic perspective, each Cr atom occupies the center of a near-perfect octahedral coordination environment constituted by six nearest-neighbor I atoms. Particularly noteworthy is that adjacent Cr atoms achieve magnetic coupling through nearly square Cr-I-Cr-I coplanar motifs, and the three Cr-I-Cr-I planes within the octahedral coordination field are mutually orthogonal. This unique geometric configuration breaks the spatial isotropy of spin exchange interactions, thereby inducing spin frustration at individual lattice sites—that is, the orientational preferences of neighboring spins cannot be simultaneously satisfied, leading to high ground-state degeneracy and strongly correlated quantum fluctuations in the system.
Figure S1. Monolayer 1T-CrI3 structure and schematic illustration of the Kitaev model. Cr is represented by blue atoms, and I by purple atoms. X X X , Y Y Y , Z Z Z and x x x , y y y , z z z correspond to the red, green, and blue Cr-Cr bond directions and Kitaev basis directions, respectively, where X X X (Y Y Y , Z Z Z ) and x x x (y y y , z z z ) are mutually perpendicular.
The Kitaev interaction in 1T-CrI3 fundamentally originates from the strong spin-orbit coupling(SOC) effect induced by the heavy coordinating element I. Specifically, the heavy halogen element I, possessing a large atomic number, exhibits pronounced SOC effects that render the magnetic interactions between Cr-Cr pairs anisotropic, generating Ising-type exchange interactions perpendicular to the Cr-I-Cr-I planes (i.e., the Kitaev interaction term). Meanwhile, within the Cr-I-Cr-I planes, the spin magnetic moments of Cr ions produce in-plane off-diagonal exchange interaction terms (i.e., the Γ \Gamma Γ term) mediated by the p p p orbitals of I atoms along the Cr-I bond directions. The geometric representation of the Kitaev interaction is depicted in Figure S1 , where the uppercase letters X, Y, and Z correspond to the three types of Cr-Cr bond directions marked in red, green, and blue, respectively (representing the bond orientations of Kitaev interactions), while the lowercase letters x, y, and z denote the local coordinate basis vectors of the Kitaev spins. These satisfy the orthogonality constraint that the X (Y, Z) axes are perpendicular to the x (y, z) axes, respectively. Based on these definitions, the Hamiltonian of the Kitaev model in the 1T-CrI3 _3 3 system can be formulated as:
H k i t = ∑ ⟨ i , j ⟩ 1 2 [ K i j S i γ S j γ + Γ i j ( S i α S j β + S i β S j α ) ] , (2) H_{\mathrm{kit}}=\sum_{\langle i,j\rangle}\frac{1}{2}\left[K_{ij}S_i^\gamma S_j^\gamma+\Gamma_{ij}\left(S_i^\alpha S_j^\beta+S_i^\beta S_j^\alpha\right)\right], \tag*{(2)} H kit = ⟨ i , j ⟩ ∑ 2 1 [ K ij S i γ S j γ + Γ ij ( S i α S j β + S i β S j α ) ] , (2)
where ⟨ i , j ⟩ \langle i,j\rangle ⟨ i , j ⟩ denotes the nearest-neighbor Cr atom pairs, and α \alpha α , β \beta β , γ \gamma γ correspond to the spin component directions along the X, Y, Z bonds, respectively. For the three mutually orthogonal Cr-I-Cr-I planes, cyclic permutation symmetry exists among α \alpha α , β \beta β , and γ \gamma γ .
Kitaev-like interactions in tetrahedral coordination
In monolayer FeX (X=Te, Se) systems, the Kitaev-like interactions are similarly dependent on the unique tetrahedral coordination structure. As illustrated in Figure 1 (a)-(c), the fundamental structural unit of monolayer FeX consists of a single Fe atomic layer sandwiched between two X atomic layers. Each Fe atom occupies the center of a tetrahedral coordination environment formed by four surrounding X atoms, giving rise to four distinct Fe-X-Fe-X planes. Among these planes, two adjacent planes without shared edges (indicated by identical colors) are mutually perpendicular, whereas planes sharing common edges are oriented at specific angles relative to each other. Analogous to the octahedral coordination configuration, this geometric arrangement induces spin frustration at individual lattice sites.
A brief geometric proof of the aforementioned orthogonality relation is provided as follows: considering the two blue Fe-X-Fe-X planes within the unit cell as an example, the three Fe atoms are collinear and lie within the planes, while being simultaneously perpendicular to the lines connecting the X atoms within each plane (indicated by red dashed lines in the Figure 1 (c)). According to the geometric properties of the regular tetrahedron, the lines connecting the X atoms in the two planes are also mutually perpendicular. By applying spatial geometry theorems, it can be deduced that these two planes are orthogonal to each other. The same reasoning can be applied to verify that the remaining pairs of adjacent planes without shared edges are likewise mutually perpendicular.
Based on the unique lattice structure of the FeX system, we construct a corresponding bond-dependent interaction model. Under the SOC effect induced by the X atoms, the magnetic moment interactions between the central Fe atom and its four nearest-neighbor(1NN) Fe atoms generate components perpendicular to the Fe-X-Fe-X planes, constituting the Kitaev-like interaction term. Meanwhile, within the planes, the spin magnetic moments of Fe ions along the Fe-X bond directions are mediated by the orbitals of X ions, giving rise to in-plane off-diagonal exchange interaction terms. For convenience, we denote the directions from the central Fe atom along the four Fe-X bonds pointing toward the X atoms as w p w_p w p , w ˉ p \bar{w}_p w ˉ p (p = 1 , 2 , 3 , 4 p = 1, 2, 3, 4 p = 1 , 2 , 3 , 4 ). The local coordinate basis vectors of different Fe-X-Fe-X planes can be expressed as linear combinations of these bond directions. As shown in Figure S2 , the local geometry of the 1NN Kitaev-like interaction is depicted. The vector difference of the two bonds lying out of the Fe-X-Fe-X plane is perpendicular to that plane and is denoted w 3 − 4 w_{3-4} w 3 − 4 ; it is identified as the γ \gamma γ bond of the local frame. The remaining two vectors w 1 w_1 w 1 and w 2 w_2 w 2 correspond to the α \alpha α and β \beta β bonds, respectively. The triad α \alpha α , β \beta β , γ \gamma γ thus constitutes the basis of the Kitaev-like interaction of the plane.
Figure S2. Schematic of local Kitaev-like interactions. Brown spheres denote Fe atoms, labeled 0, 1 and 2 for the central, 1NN and 2NN sites, respectively. Black (gray) spheres denote lower- (upper-) layer X (X=Te, Se) atoms. Blue arrows indicate Fe-X bonds; yellow arrows indicate the vector differences of the two bonds, with the yellow arrow perpendicular to the plane under consideration. (a) Kitaev-like interaction between the central Fe atom and the 1NN Fe atom at (1,1). (b) Kitaev-like interaction between the central Fe atom and the 2NN Fe atom at (-2,0).
Kitaev-like interactions also exist between the central Fe atom and its second nearest-neighbor (2NN) Fe atoms, involving two mutually perpendicular Fe-X-Fe planes. Similarly, under the SOC effect induced by the X atoms, the magnetic moment interactions between the central Fe atom and its four 2NN Fe atoms generate components perpendicular to the planes, constituting the Kitaev-like interaction term. Meanwhile, the in-plane spin magnetic moments of Fe ions along the Fe-X bond directions are mediated by X ions, producing in-plane off-diagonal exchange interaction terms. The local basis vectors of different planes can likewise be expressed in terms of the bond directions. As shown in Figure S2 , the vector difference of the two bonds lying out of the Fe-X-Fe plane is perpendicular to that plane and is denoted w 4 − 2 w_{4-2} w 4 − 2 ; it is identified as the γ \gamma γ bond of the local frame. The remaining two vectors w 3 w_3 w 3 and w 1 w_1 w 1 correspond to the α \alpha α and β \beta β bonds, respectively. The triad α \alpha α , β \beta β , γ \gamma γ thus constitutes the Kitaev-like interaction basis of the plane. Note that the interactions of the central Fe atom with the pair of opposite 2NN Fe atoms lie in the same plane and therefore give identical Kitaev-like contributions; they are nevertheless treated separately for clarity.
The local basis vectors defined on the planes that contain the various Fe–Fe bonds are listed in the Table S1 ; they correspond to the models shown in Figure 1 (d) and (e).
Table S1. Local Kitaev-like interaction basis-vector directions for the distinct Fe-X-Fe(-X) planes.
Taken together, the model conventions imply that, on each Fe-X-Fe(-X) plane under study, the γ \gamma γ bond stands perpendicular to the plane while the α \alpha α and β \beta β bonds lie within it. Their in-plane angle is not 90 ∘ 90^\circ 9 0 ∘ but 109.47 ∘ 109.47^\circ 109.4 7 ∘ , as illustrated in Figure S2 . A new coordinate frame must therefore be constructed when the effective spin Hamiltonian is analyzed; this construction is discussed in detail below. It should also be noted that the bond-direction expressions given here apply to only one of the two sublattices; the corresponding expressions on the other sublattice are modified accordingly.
Microscopic derivation of the Kitaev-like model
In studies of the Kitaev model on ideal honeycomb lattices with edge-sharing octahedra, microscopic derivations are typically performed via the Kanamori interaction and a tight-binding model [24 , 40 ] ; the effective spin interactions are obtained by describing virtual hopping processes between magnetic atoms M1 _1 1 and M2 _2 2 . The microscopic origin of the Kitaev interaction is the strong spin–orbit coupling (SOC) of ligands X1 _1 1 and X2 _2 2 , which drives spin-flip virtual transitions and thereby yields bond-dependent anisotropic exchange. In conventional derivations for such ideal octahedral systems, however, only Heisenberg and Kitaev terms appear; off-diagonal interactions require further local symmetry lowering induced by lattice distortions.
For the two-dimensional edge-sharing tetrahedral structure investigated here, applying the same Kanamori interaction and tight-binding framework reveals that the Kitaev-like interaction also originates from ligand-SOC-induced spin-flip virtual processes. Crucially, even in the absence of any lattice distortion, off-diagonal exchange interactions emerge spontaneously in the ideal edge-sharing tetrahedral geometry. Below, taking FeTe as an example, we derive explicitly the effective magnetic exchange interactions for this ideal geometry based on the Kanamori interaction and tight-binding model.
Kanamori interaction and tight-binding model
In monolayer FeTe, each Fe atom sits at the center of a regular tetrahedron formed by four surrounding Te atoms; the edge-sharing tetrahedral network is composed primarily of Fe d d d orbitals and Te p p p orbitals. The Fe d d d shell is more than half-filled, corresponding to a d 6 d^6 d 6 configuration, whereas the Te p p p orbitals are fully occupied (p 6 p^6 p 6 ). Thus, the total Hamiltonian of the system comprises the Kanamori interaction on the Fe sites and the tight-binding Hamiltonian.
However, because the Fe d d d orbitals are more than half-filled, we adopt the hole picture to simplify the model. The Fe d 6 d^6 d 6 electron configuration is then mapped onto a d 4 d^4 d 4 hole configuration. Consequently, the local Hamiltonian on each Fe site, expressed in the hole basis, is given by the Kanamori interaction together with crystal-field splitting, and takes the following form [40 ] :
H e e = U ∑ α n α ↑ n α ↓ + U ′ 2 ∑ α ≠ β , σ , σ ′ n α σ n β σ ′ − J H 2 ∑ α ≠ β , σ , σ ′ c α σ † c β σ ′ † c β σ c α σ ′ + J H ∑ α ≠ β c α ↑ † c α ↓ † c β ↓ c β ↑ + Δ c ∑ α ∈ e , σ c α σ † c α σ . (3) H_{ee}=U\sum_{\alpha}n_{\alpha\uparrow}n_{\alpha\downarrow}+\frac{U'}{2}\sum_{\substack{\alpha\ne\beta,\sigma,\sigma'}}n_{\alpha\sigma}n_{\beta\sigma'}-\frac{J_H}{2}\sum_{\substack{\alpha\ne\beta,\sigma,\sigma'}}c^\dagger_{\alpha\sigma}c^\dagger_{\beta\sigma'}c_{\beta\sigma}c_{\alpha\sigma'}+J_H\sum_{\alpha\ne\beta}c^\dagger_{\alpha\uparrow}c^\dagger_{\alpha\downarrow}c_{\beta\downarrow}c_{\beta\uparrow}+\Delta_c\sum_{\substack{\alpha\in e,\sigma}}c^\dagger_{\alpha\sigma}c_{\alpha\sigma}. \tag*{(3)} H ee = U α ∑ n α ↑ n α ↓ + 2 U ′ α = β , σ , σ ′ ∑ n α σ n β σ ′ − 2 J H α = β , σ , σ ′ ∑ c α σ † c β σ ′ † c β σ c α σ ′ + J H α = β ∑ c α ↑ † c α ↓ † c β ↓ c β ↑ + Δ c α ∈ e , σ ∑ c α σ † c α σ . (3)
The density operator satisfies the expression n α σ = c α σ † c α σ n_{\alpha\sigma}=c^\dagger_{\alpha\sigma}c_{\alpha\sigma} n α σ = c α σ † c α σ , where c α σ † c^\dagger_{\alpha\sigma} c α σ † and c α σ c_{\alpha\sigma} c α σ represent the creation and annihilation operators for orbital α \alpha α and spin σ \sigma σ on the Fe site, respectively. The terms involving U U U and U ′ U' U ′ correspond to the Hubbard interactions for intra-orbital and inter-orbital processes, respectively. The J H J_H J H terms describe Hund’s coupling, where the third and fourth terms represent spin-exchange interactions between different orbitals and pair-hopping processes, respectively. The Δ c \Delta_c Δ c term denotes the tetrahedral crystal-field splitting acting on the Fe site. Under the tetrahedral crystal field, the Fe d d d orbitals split into a lower-energy double degenerate e e e level and a higher-energy triply degenerate t 2 t_2 t 2 level, with the electronic configuration denoted as e 3 t 2 3 e^3t_2^3 e 3 t 2 3 . The Hund’s coupling selects the high-spin S = 2 S=2 S = 2 ground state for FeTe, and in the hole representation, the t 2 t_2 t 2 level is preferentially occupied, yielding the configuration t 2 3 e 1 t_2^3e^1 t 2 3 e 1 . Consequently, the intermediate states for hole-hopping exhibit multiple possibilities, with energy variations dependent on U U U , U ′ U' U ′ , J H J_H J H , and Δ c \Delta_c Δ c . Under the strong-coupling approximation, the tight-binding hopping integrals are treated as perturbations, as they are much smaller than the excitation energy changes.
In the edge-sharing tetrahedral lattice structure (as shown in Figure 1 ), each Fe atom forms four Fe-Fe bonds with its 1NNs, where each bond plane contains two coordinating Te atoms. Similarly, four Fe-Fe bonds are formed with 2NNs, but each bond plane contains only one coordinating Te atom. The magnetic exchange interactions between Fe atoms can be decomposed into two microscopic mechanisms: (1) direct exchange processes, where electrons undergo virtual hopping via direct Fe-Fe paths, and (2) indirect superexchange processes, where virtual hopping occurs via Fe-Te-Fe pathways, with the SOC of the Te p p p orbitals playing a critical role. Based on this analysis, the tight-binding Hamiltonian describing the interaction between Fei _i i and Fej _j j can be constructed as follows [27 ] :
H T B 1 = ( 0 5 × 5 T M 1 M 2 T M 1 X 1 T M 1 X 2 T M 1 M 2 † 0 5 × 5 T M 2 X 1 T M 2 X 2 T M 1 X 1 † T M 2 X 1 † 0 3 × 3 0 3 × 3 T M 1 X 2 † T M 2 X 2 † 0 3 × 3 0 3 × 3 ) , H T B 2 = ( 0 5 × 5 T M 1 M 2 T M 1 X 1 T M 1 M 2 † 0 5 × 5 T M 2 X 1 T M 1 X 1 † T M 2 X 1 † 0 3 × 3 ) . (4) H_{\mathrm{TB}_1}=
\begin{pmatrix}
\mathbf{0}_{5\times5} & T_{\mathrm{M}_1\mathrm{M}_2} & T_{\mathrm{M}_1\mathrm{X}_1} & T_{\mathrm{M}_1\mathrm{X}_2}\\
T^\dagger_{\mathrm{M}_1\mathrm{M}_2} & \mathbf{0}_{5\times5} & T_{\mathrm{M}_2\mathrm{X}_1} & T_{\mathrm{M}_2\mathrm{X}_2}\\
T^\dagger_{\mathrm{M}_1\mathrm{X}_1} & T^\dagger_{\mathrm{M}_2\mathrm{X}_1} & \mathbf{0}_{3\times3} & \mathbf{0}_{3\times3}\\
T^\dagger_{\mathrm{M}_1\mathrm{X}_2} & T^\dagger_{\mathrm{M}_2\mathrm{X}_2} & \mathbf{0}_{3\times3} & \mathbf{0}_{3\times3}
\end{pmatrix},
\qquad
H_{\mathrm{TB}_2}=
\begin{pmatrix}
\mathbf{0}_{5\times5} & T_{\mathrm{M}_1\mathrm{M}_2} & T_{\mathrm{M}_1\mathrm{X}_1}\\
T^\dagger_{\mathrm{M}_1\mathrm{M}_2} & \mathbf{0}_{5\times5} & T_{\mathrm{M}_2\mathrm{X}_1}\\
T^\dagger_{\mathrm{M}_1\mathrm{X}_1} & T^\dagger_{\mathrm{M}_2\mathrm{X}_1} & \mathbf{0}_{3\times3}
\end{pmatrix}. \tag*{(4)} H TB 1 = 0 5 × 5 T M 1 M 2 † T M 1 X 1 † T M 1 X 2 † T M 1 M 2 0 5 × 5 T M 2 X 1 † T M 2 X 2 † T M 1 X 1 T M 2 X 1 0 3 × 3 0 3 × 3 T M 1 X 2 T M 2 X 2 0 3 × 3 0 3 × 3 , H TB 2 = 0 5 × 5 T M 1 M 2 † T M 1 X 1 † T M 1 M 2 0 5 × 5 T M 2 X 1 † T M 1 X 1 T M 2 X 1 0 3 × 3 . (4)
where 0 n × n \mathbf{0}_{n \times n} 0 n × n denotes an n × n n \times n n × n null matrix, and T † \mathbf{T}^{\dagger} T † , T \mathbf{T} T represent hopping operators between atomic orbitals, with subscripts indicating the involved atoms. Here, M i M_i M i and X m X_m X m (i , m = 1 , 2 i,m=1,2 i , m = 1 , 2 ) denote Fe and Te atoms, respectively. Specifically, T M 1 M 2 \mathbf{T}_{M_1M_2} T M 1 M 2 describes direct hopping from the d d d orbitals of Fe2 _2 2 to Fe1 _1 1 , while T M i X m \mathbf{T}_{M_iX_m} T M i X m represents indirect hopping via Tem _m m p p p orbitals to Fei _i i d d d orbitals. In the global coordinate system shown in Figure 1 , the five d d d -orbital basis states on Fei _i i are defined as: C M i , d † = ( c i , d x y † , c i , d y z † , c i , d x z † , c i , d x 2 − y 2 † , c i , d 3 z 2 − r 2 † ) \mathbf{C}^{\dagger}_{M_i,d} = (c^{\dagger}_{i,d_{xy}}, c^{\dagger}_{i,d_{yz}}, c^{\dagger}_{i,d_{xz}}, c^{\dagger}_{i,d_{x^2-y^2}}, c^{\dagger}_{i,d_{3z^2-r^2}}) C M i , d † = ( c i , d x y † , c i , d y z † , c i , d x z † , c i , d x 2 − y 2 † , c i , d 3 z 2 − r 2 † ) , and the three p p p -orbital basis states on Tem _m m are defined as: C X m , p † = ( c m , p x † , c m , p y † , c m , p z † ) \mathbf{C}^{\dagger}_{X_m,p} = (c^{\dagger}_{m,p_x}, c^{\dagger}_{m,p_y}, c^{\dagger}_{m,p_z}) C X m , p † = ( c m , p x † , c m , p y † , c m , p z † ) . Since our focus is on Kitaev-like interactions, we only consider the d d d –p p p –d d d indirect hopping processes.
Effective d-d Hopping Model and SOC Effects
For the indirect d d d –p p p –d d d hopping, we employ second-order perturbation theory to eliminate the p p p orbitals, yielding an effective d d d –d d d hopping model expression [27 ] , which can be recast into a Green’s function projection form for clarity and computational [41 , 42 ] :
T M j M i e f f = − ∑ a , m T M j X m ∣ a ⟩ ⟨ a ∣ T X m M i Δ E a = ∑ m T M j X m G p T X m M i , (5) \mathbf{T}^{eff}_{M_jM_i}=-\sum_{a,m}\frac{\mathbf{T}_{M_jX_m}|a\rangle\langle a|\mathbf{T}_{X_mM_i}}{\Delta E_a}=\sum_m\mathbf{T}_{M_jX_m}G_p\mathbf{T}_{X_mM_i}, \tag*{(5)} T M j M i e f f = − a , m ∑ Δ E a T M j X m ∣ a ⟩ ⟨ a ∣ T X m M i = m ∑ T M j X m G p T X m M i , (5)
where the summation runs over all intermediate states ∣ a ⟩ |a\rangle ∣ a ⟩ on each ligand atom X m X_m X m . Summing over all intermediate states for a given ligand yields the propagator operator G p G_p G p , where Δ E a = E a − E 0 \Delta E_a=E_a-E_0 Δ E a = E a − E 0 . In the hole representation, the physical process can be described as follows: a hole hops from atom M i M_i M i to atom X m X_m X m , undergoes SOC effects, and then hops to atom M j M_j M j . The hole hopping from M i M_i M i to X m X_m X m is equivalent to T X m M i \mathbf{T}_{X_mM_i} T X m M i , the action on X m X_m X m is equivalent to G p G_p G p , and the subsequent hopping from X m X_m X m to M j M_j M j is equivalent to T M j X m \mathbf{T}_{M_jX_m} T M j X m .
Below, we derive the explicit form of the propagator operator under the SOC effects of the ligand Te atoms. The intermediate state on the Te atom contains only one hole, with orbital angular momentum L = 1 L=1 L = 1 and spin angular momentum S = 1 2 S=\frac{1}{2} S = 2 1 . Under LS coupling with SOC, the states split into a fourfold degenerate manifold with J = L + S = 3 / 2 J=L+S=3/2 J = L + S = 3/2 , M J = ± 3 / 2 , ± 1 / 2 M_J=\pm3/2,\pm1/2 M J = ± 3/2 , ± 1/2 , and a twofold degenerate manifold with J = L − S = 1 / 2 J=L-S=1/2 J = L − S = 1/2 , M J = ± 1 / 2 M_J=\pm1/2 M J = ± 1/2 . Including the energy contribution from SOC, the Hamiltonian is expressed as:
H S O C = λ p ( L ⋅ S ) , (6) H_{\mathrm{SOC}}=\lambda_p(L\cdot S), \tag*{(6)} H SOC = λ p ( L ⋅ S ) , (6)
where λ p \lambda_p λ p represents the SOC strength of the Te p p p orbitals, which takes a negative value in the hole representation. Under different total angular momenta, the corresponding SOC energy corrections are obtained as H J = 3 2 = 1 2 λ p H_{J=\frac{3}{2}}=\frac{1}{2}\lambda_p H J = 2 3 = 2 1 λ p , H J = 1 2 = − λ p H_{J=\frac{1}{2}}=-\lambda_p H J = 2 1 = − λ p . Consequently, the excitation energy of the intermediate state relative to the ground state is given by Δ E a = ϵ p − ϵ d − 1 2 λ p \Delta E_a=\epsilon_p-\epsilon_d-\frac{1}{2}\lambda_p Δ E a = ϵ p − ϵ d − 2 1 λ p or Δ E a = ϵ p − ϵ d + λ p \Delta E_a=\epsilon_p-\epsilon_d+\lambda_p Δ E a = ϵ p − ϵ d + λ p . For subsequent calculations, we define the charge-transfer energy as Δ = ϵ p − ϵ d \Delta=\epsilon_p-\epsilon_d Δ = ϵ p − ϵ d . The projection operator for a specific J J J manifold can be expressed as P J = ∣ a ⟩ ⟨ a ∣ P_J=|a\rangle\langle a| P J = ∣ a ⟩ ⟨ a ∣ , yielding:
P J = 3 2 = 2 3 ( I + L ⋅ S ) , P J = 1 2 = 1 3 ( I − 2 L ⋅ S ) , (7) P_{J=\frac{3}{2}}=\frac{2}{3}(I+L\cdot S),\qquad P_{J=\frac{1}{2}}=\frac{1}{3}(I-2L\cdot S), \tag*{(7)} P J = 2 3 = 3 2 ( I + L ⋅ S ) , P J = 2 1 = 3 1 ( I − 2 L ⋅ S ) , (7)
where I I I denotes the 6 × 6 6\times6 6 × 6 identity matrix.
The propagator operator can then be expressed as:
G p = − P J = 3 2 Δ − 1 2 λ p − P J = 1 2 Δ + λ p = − 1 3 ( 2 Δ − λ p 2 + 1 Δ + λ p ) I − 2 3 ( 1 Δ − λ p 2 − 1 Δ + λ p ) ( L ⋅ S ) , (8) G_p=-\frac{P_{J=\frac{3}{2}}}{\Delta-\frac{1}{2}\lambda_p}-\frac{P_{J=\frac{1}{2}}}{\Delta+\lambda_p}
=-\frac{1}{3}\left(\frac{2}{\Delta-\frac{\lambda_p}{2}}+\frac{1}{\Delta+\lambda_p}\right)I-\frac{2}{3}\left(\frac{1}{\Delta-\frac{\lambda_p}{2}}-\frac{1}{\Delta+\lambda_p}\right)(L\cdot S), \tag*{(8)} G p = − Δ − 2 1 λ p P J = 2 3 − Δ + λ p P J = 2 1 = − 3 1 ( Δ − 2 λ p 2 + Δ + λ p 1 ) I − 3 2 ( Δ − 2 λ p 1 − Δ + λ p 1 ) ( L ⋅ S ) , (8)
where the two terms represent the spin-independent and SOC-dependent contributions, respectively. We define A = − 1 3 ( 2 Δ − λ p / 2 + 1 Δ + λ p ) A=-\frac{1}{3}\left(\frac{2}{\Delta-\lambda_p/2}+\frac{1}{\Delta+\lambda_p}\right) A = − 3 1 ( Δ − λ p /2 2 + Δ + λ p 1 ) as the coefficient for the spin-independent part, and B = − 2 3 ( 1 Δ − λ p / 2 − 1 Δ + λ p ) B=-\frac{2}{3}\left(\frac{1}{\Delta-\lambda_p/2}-\frac{1}{\Delta+\lambda_p}\right) B = − 3 2 ( Δ − λ p /2 1 − Δ + λ p 1 ) as the coefficient for the SOC part.
The contribution of SOC can be rewritten in the following form:
L ⋅ S = L x ⊗ S x + L y ⊗ S y + L z ⊗ S z = ∑ γ L γ ⊗ 1 2 σ γ , (9) L\cdot S=L_x\otimes S_x+L_y\otimes S_y+L_z\otimes S_z=\sum_{\gamma}L_{\gamma}\otimes\frac{1}{2}\sigma_{\gamma}, \tag*{(9)} L ⋅ S = L x ⊗ S x + L y ⊗ S y + L z ⊗ S z = γ ∑ L γ ⊗ 2 1 σ γ , (9)
where L γ L_{\gamma} L γ and σ γ \sigma_{\gamma} σ γ are the orbital angular momentum and Pauli operators for the respective components. The SOC process filters the allowed d d d –p p p and d d d –p p p hopping pathways through the action of the orbital angular momentum operators.
As shown in Figure 1 , the Fe-Te-Fe-Te and Fe-Te-Fe planes formed by the Fe atoms and their neighbors via the coordinating Te atoms are not mutually perpendicular. Consequently, it is difficult to define a single unified coordinate system to describe both the orbital and spin bases in different plane. To address this, we define distinct local coordinate systems ( x ~ , y ~ , z ~ ) (\tilde{x},\tilde{y},\tilde{z}) ( x ~ , y ~ , z ~ ) for the different planes to perform the Slater-Koster transformation [39 ] , as illustrated in Figure S3 . We construct the coordinate system according to the bond directions defined in the model, where the z ~ \tilde{z} z ~ direction
Figure S3. Schematic of the local coordinate configuration. Here, x ~ \tilde{x} x ~ , y ~ \tilde{y} y ~ and z ~ \tilde{z} z ~ denote the local coordinate axes, while α \alpha α and β \beta β represent the directions of the α \alpha α and β \beta β bonds of the plane in the global coordinate system. θ \theta θ is the angle between the coordinate axes and the bond directions. (a) First nearest neighbor. (b) Second nearest neighbor.
corresponds to the w p − p ′ w_{p-p'} w p − p ′ bond direction perpendicular to the plane in Figure 1 , matching the γ \gamma γ direction in the corresponding Hamiltonian. Since the Fe-Te bond angles within the Fe-Te-Fe(-Te) exchange plane are not 90 ∘ 90^\circ 9 0 ∘ , we choose the x ~ \tilde{x} x ~ and y ~ \tilde{y} y ~ directions within the plane at positions corresponding to α \alpha α and β \beta β , as shown in Figure S3 , with z ~ \tilde{z} z ~ pointing out of the page. When applying this to the real structure, we first fix z ~ \tilde{z} z ~ along the w p − p ′ w_{p-p'} w p − p ′ bond direction as the reference, ensuring compliance with the right-hand rule. Under this convention, the resulting local coordinate expressions are identical for all 1NN and 2NN pairs, respectively.
The relationship between the local and global coordinate systems is generally described by a rotation matrix. The local coordinate axes are expressed in the global coordinate system as:
x ~ = ( x 1 , x 2 , x 3 ) , y ~ = ( y 1 , y 2 , y 3 ) , z ~ = ( z 1 , z 2 , z 3 ) , (10) \tilde{x}=(x_1,x_2,x_3),\qquad\tilde{y}=(y_1,y_2,y_3),\qquad\tilde{z}=(z_1,z_2,z_3), \tag*{(10)} x ~ = ( x 1 , x 2 , x 3 ) , y ~ = ( y 1 , y 2 , y 3 ) , z ~ = ( z 1 , z 2 , z 3 ) , (10)
then the transformation from local to global coordinates is given by:
\begin{pmatrix} x_m \\ y_m \\ z_m \end{pmatrix}
=
\begin{pmatrix}
x_1 & y_1 & z_1 \\
x_2 & y_2 & z_2 \\
x_3 & y_3 & z_3
\end{pmatrix}
\begin{pmatrix} \tilde{x}_m \\ \tilde{y}_m \\ \tilde{z}_m \end{pmatrix}
=
R
\begin{pmatrix} \tilde{x}_m \\ \tilde{y}_m \\ \tilde{z}_m \end{pmatrix}, \tag*{(11)}\end{pmatrix}
where R R R is the rotation matrix.
For calculations performed in the local coordinate system, a new orbital basis must be established within that frame, namely C M ~ i , d ′ † C^{\dagger}_{\tilde{M}_i,d'} C M ~ i , d ′ † and C X ~ m , p ′ † C^{\dagger}_{\tilde{X}_m,p'} C X ~ m , p ′ † , where d ′ d' d ′ corresponds to ( d x y ′ , d y z ′ , d x z ′ , d x 2 − y 2 ′ , d 3 z 2 − r 2 ′ ) (d'_{xy},d'_{yz},d'_{xz},d'_{x^2-y^2},d'_{3z^2-r^2}) ( d x y ′ , d y z ′ , d x z ′ , d x 2 − y 2 ′ , d 3 z 2 − r 2 ′ ) and p ′ p' p ′ corresponds to ( p x ′ , p y ′ , p z ′ ) (p'_x,p'_y,p'_z) ( p x ′ , p y ′ , p z ′ ) . Applying the Slater-Koster transformation in the local coordinate system yields the hopping matrix for transitions from the Fe d d d orbitals to the Te p p p orbitals [39 ] :
T X 1 M 1 = − T X 2 M 2 = ( t 1 t 2 0 0 0 t 3 0 0 t 4 t 5 t 6 0 t 7 t 8 0 ) , T X 2 M 1 = − T X 1 M 2 = ( t 2 t 1 0 0 0 t 4 0 0 t 3 − t 6 − t 5 0 t 8 t 7 0 ) . (12) T_{X_1M_1}=-T_{X_2M_2}=
\begin{pmatrix}
t_1 & t_2 & 0 \\
0 & 0 & t_3 \\
0 & 0 & t_4 \\
t_5 & t_6 & 0 \\
t_7 & t_8 & 0
\end{pmatrix},
\qquad
T_{X_2M_1}=-T_{X_1M_2}=
\begin{pmatrix}
t_2 & t_1 & 0 \\
0 & 0 & t_4 \\
0 & 0 & t_3 \\
-t_6 & -t_5 & 0 \\
t_8 & t_7 & 0
\end{pmatrix}. \tag*{(12)} T X 1 M 1 = − T X 2 M 2 = t 1 0 0 t 5 t 7 t 2 0 0 t 6 t 8 0 t 3 t 4 0 0 , T X 2 M 1 = − T X 1 M 2 = t 2 0 0 − t 6 t 8 t 1 0 0 − t 5 t 7 0 t 4 t 3 0 0 . (12)
The corresponding hopping integrals are expressed as:
t 1 = − 2 + 2 12 V p d σ + 2 ( 6 − 3 ) 9 V p d π , t 2 = 2 − 2 12 V p d σ + 2 ( 6 + 3 ) 9 V p d π , t 3 = 6 − 2 3 6 V p d π , t 4 = 6 + 2 3 6 V p d π , t 5 = 1 + 2 3 V p d σ + 2 3 − 6 18 V p d π , t 6 = 1 − 2 3 V p d σ + 2 3 + 6 18 V p d π , t 7 = − 2 3 + 6 12 V p d σ , t 8 = 2 3 − 6 12 V p d σ , (13) \begin{aligned}
t_{1} &= -\frac{2+\sqrt{2}}{12}V_{pd\sigma}+\frac{2(\sqrt{6}-\sqrt{3})}{9}V_{pd\pi},\\
t_{2} &= \frac{2-\sqrt{2}}{12}V_{pd\sigma}+\frac{2(\sqrt{6}+\sqrt{3})}{9}V_{pd\pi},\\
t_{3} &= \frac{\sqrt{6}-2\sqrt{3}}{6}V_{pd\pi},\\
t_{4} &= \frac{\sqrt{6}+2\sqrt{3}}{6}V_{pd\pi},\\
t_{5} &= \frac{1+\sqrt{2}}{3}V_{pd\sigma}+\frac{2\sqrt{3}-\sqrt{6}}{18}V_{pd\pi},\\
t_{6} &= \frac{1-\sqrt{2}}{3}V_{pd\sigma}+\frac{2\sqrt{3}+\sqrt{6}}{18}V_{pd\pi},\\
t_{7} &= -\frac{2\sqrt{3}+\sqrt{6}}{12}V_{pd\sigma},\\
t_{8} &= \frac{2\sqrt{3}-\sqrt{6}}{12}V_{pd\sigma}, \tag*{(13)}
\end{aligned} t 1 t 2 t 3 t 4 t 5 t 6 t 7 t 8 = − 12 2 + 2 V p d σ + 9 2 ( 6 − 3 ) V p d π , = 12 2 − 2 V p d σ + 9 2 ( 6 + 3 ) V p d π , = 6 6 − 2 3 V p d π , = 6 6 + 2 3 V p d π , = 3 1 + 2 V p d σ + 18 2 3 − 6 V p d π , = 3 1 − 2 V p d σ + 18 2 3 + 6 V p d π , = − 12 2 3 + 6 V p d σ , = 12 2 3 − 6 V p d σ , (13)
where V p d σ < 0 V_{pd\sigma}<0 V p d σ < 0 , V p d π > 0 V_{pd\pi}>0 V p d π > 0 and ∣ V p d σ ∣ > ∣ V p d π ∣ \lvert V_{pd\sigma}\rvert>\lvert V_{pd\pi}\rvert ∣ V p d σ ∣ > ∣ V p d π ∣ . The hopping integrals for p → d p\to d p → d and d → p d\to p d → p transitions along the same bond are equal in magnitude.
Superexchange process and effective spin Hamiltonian
The complete superexchange process consists of a hole hopping virtually from Fe1 _1 1 to Fe2 _2 2 mediated by the Te1 _1 1 /Te2 _2 2 atoms, followed by a virtual return hopping from Fe2 _2 2 back to Fe1 _1 1 via Te1 _1 1 /Te2 _2 2 . Accordingly, the Hamiltonian governing the entire virtual hopping process can be expressed as:
H s = − ∑ n T M i M j e f f ∣ n ⟩ ⟨ n ∣ T M j M i e f f Δ E n , (14) H_s=-\sum_n\frac{T^{\mathrm{eff}}_{M_iM_j}\lvert n\rangle\langle n\rvert T^{\mathrm{eff}}_{M_jM_i}}{\Delta E_n}, \tag*{(14)} H s = − n ∑ Δ E n T M i M j eff ∣ n ⟩ ⟨ n ∣ T M j M i eff , (14)
where Δ E n = E n − E 0 \Delta E_n=E_n-E_0 Δ E n = E n − E 0 represents the excitation energy of the virtual intermediate state ∣ n ⟩ \lvert n\rangle ∣ n ⟩ relative to the magnetic ground state.
Although the Te-induced SOC has been analyzed, we must consistently treat the SOC term L ~ ⋅ S ~ \widetilde{L}\cdot\widetilde{S} L ⋅ S in the local coordinate frame, transforming it in the same manner as described above. By combining the local-frame Slater-Koster hopping matrices between the Fe d d d and Te p p p orbitals, we obtain the total superexchange interaction between Fe1 _1 1 and Fe2 _2 2 . Throughout this section, all descriptions of the d d d and p p p orbitals are defined with respect to the local coordinate system established in Figure S3 .
t2–t2 superexchange process
We first consider the superexchange processes involving exclusively the t 2 t_2 t 2 orbitals. Utilizing the Slater-Koster formalism, several representative superexchange pathways in the local coordinate frame are illustrated in Figure S4 . As visually demonstrated in it, several virtual hopping processes are directly visualized. Since the bond directions in the local coordinate system are not aligned along the basis vectors, the inter-orbital interactions exhibit considerable complexity. As a representative example, we present the complete calculation for the Fe1 _1 1 d x y d_{xy} d x y –Fe2 _2 2 d x y d_{xy} d x y process. The total process of a hole hopping from Fe1 _1 1 to Fe2 _2 2 , expressed in terms of operators and eigenstates, is given by:
Figure S4. Schematic of the t 2 t_2 t 2 -t 2 t_2 t 2 virtual hopping processes in the hole representation. Brown and dark-gray spheres denote Fe and Te atoms, respectively. (a)-(d) Spin-conserving superexchange pathways between Fe1 _1 1 and Fe2 _2 2 mediated by the Te atoms. (e)-(g) Spin-flip superexchange pathways between Fe1 _1 1 and Fe2 _2 2 assisted by the SOC of the Te atoms.
⟨ d j , x y ∣ T M 2 M 1 e f f ∣ d i , x y ⟩ = ∑ m , α , β ⟨ d j , x y ∣ c j , d x y † c m , p β ∣ p m , β ⟩ ⟨ p m , β ∣ G p , m ∣ p m , α ⟩ ⟨ p m , α ∣ c m , p α † c i , d x y ∣ d i , x y ⟩ . (15) \langle d_{j,xy}\lvert T^{\mathrm{eff}}_{M_2M_1}\rvert d_{i,xy}\rangle
=\sum_{m,\alpha,\beta}\langle d_{j,xy}\lvert c^\dagger_{j,d_{xy}}c_{m,p\beta}\rvert p_{m,\beta}\rangle\langle p_{m,\beta}\lvert G_{p,m}\rvert p_{m,\alpha}\rangle\langle p_{m,\alpha}\lvert c^\dagger_{m,p\alpha}c_{i,d_{xy}}\rvert d_{i,xy}\rangle. \tag*{(15)} ⟨ d j , x y ∣ T M 2 M 1 eff ∣ d i , x y ⟩ = m , α , β ∑ ⟨ d j , x y ∣ c j , d x y † c m , pβ ∣ p m , β ⟩ ⟨ p m , β ∣ G p , m ∣ p m , α ⟩ ⟨ p m , α ∣ c m , p α † c i , d x y ∣ d i , x y ⟩ . (15)
The term ∣ p m , β ⟩ ⟨ p m , β ∣ G p , m ∣ p m , α ⟩ ⟨ p m , α ∣ \lvert p_{m,\beta}\rangle\langle p_{m,\beta}\lvert G_{p,m}\rvert p_{m,\alpha}\rangle\langle p_{m,\alpha}\rvert ∣ p m , β ⟩ ⟨ p m , β ∣ G p , m ∣ p m , α ⟩ ⟨ p m , α ∣ describes a hole hopping onto the α \alpha α orbital of the Tem _m m site, evolving under G p , m G_{p,m} G p , m , and subsequently transitioning to the β \beta β orbital. For the Fe1 _1 1 d x y d_{xy} d x y –Fe2 _2 2 d x y d_{xy} d x y virtual hopping via Tem _m m , only four pathways exist: d x y d_{xy} d x y -p x p_x p x -d x y d_{xy} d x y (spin-conserving), d x y d_{xy} d x y -p x p_x p x -p y p_y p y -d x y d_{xy} d x y (spin-flip), d x y d_{xy} d x y -p y p_y p y -p x p_x p x -d x y d_{xy} d x y (spin-flip), and d x y d_{xy} d x y -p y p_y p y -d x y d_{xy} d x y (spin-conserving). The corresponding propagator actions on the p p p orbitals can be expressed as:
⟨ p m , x ∣ A I + B ( L ~ ⋅ S ~ ) ∣ p m , x ⟩ = ⟨ p m , x ∣ A I ∣ p m , x ⟩ = A ⟨ p m , x ∣ I ∣ p m , x ⟩ = A = A σ ~ 0 , ⟨ p m , x ∣ A I + B ( L ~ ⋅ S ~ ) ∣ p m , y ⟩ = ⟨ p m , x ∣ B ( L ~ z ⊗ 1 2 σ ~ z ) ∣ p m , y ⟩ = B ⟨ p m , x ∣ L ~ z ∣ p m , y ⟩ ⊗ 1 2 σ ~ z = − i 2 B σ ~ z , ⟨ p m , y ∣ A I + B ( L ~ ⋅ S ~ ) ∣ p m , x ⟩ = ⟨ p m , y ∣ B ( L ~ z ⊗ 1 2 σ ~ z ) ∣ p m , x ⟩ = B ⟨ p m , y ∣ L ~ z ∣ p m , x ⟩ ⊗ 1 2 σ ~ z = i 2 B σ ~ z , ⟨ p m , y ∣ A I + B ( L ~ ⋅ S ~ ) ∣ p m , y ⟩ = ⟨ p m , y ∣ A I ∣ p m , y ⟩ = A ⟨ p m , y ∣ I ∣ p m , y ⟩ = A = A σ ~ 0 . (16) \begin{aligned}
\langle p_{m,x}|AI+B(\widetilde{L}\cdot\widetilde{S})|p_{m,x}\rangle
&=\langle p_{m,x}|AI|p_{m,x}\rangle=A\langle p_{m,x}|I|p_{m,x}\rangle=A=A\widetilde{\sigma}_0,\\
\langle p_{m,x}|AI+B(\widetilde{L}\cdot\widetilde{S})|p_{m,y}\rangle
&=\langle p_{m,x}|B(\widetilde{L}_z\otimes\tfrac{1}{2}\widetilde{\sigma}_z)|p_{m,y}\rangle
=B\langle p_{m,x}|\widetilde{L}_z|p_{m,y}\rangle\otimes\tfrac{1}{2}\widetilde{\sigma}_z=-\tfrac{i}{2}B\widetilde{\sigma}_z,\\
\langle p_{m,y}|AI+B(\widetilde{L}\cdot\widetilde{S})|p_{m,x}\rangle
&=\langle p_{m,y}|B(\widetilde{L}_z\otimes\tfrac{1}{2}\widetilde{\sigma}_z)|p_{m,x}\rangle
=B\langle p_{m,y}|\widetilde{L}_z|p_{m,x}\rangle\otimes\tfrac{1}{2}\widetilde{\sigma}_z=\tfrac{i}{2}B\widetilde{\sigma}_z,\\
\langle p_{m,y}|AI+B(\widetilde{L}\cdot\widetilde{S})|p_{m,y}\rangle
&=\langle p_{m,y}|AI|p_{m,y}\rangle=A\langle p_{m,y}|I|p_{m,y}\rangle=A=A\widetilde{\sigma}_0. \tag*{(16)}
\end{aligned} ⟨ p m , x ∣ A I + B ( L ⋅ S ) ∣ p m , x ⟩ ⟨ p m , x ∣ A I + B ( L ⋅ S ) ∣ p m , y ⟩ ⟨ p m , y ∣ A I + B ( L ⋅ S ) ∣ p m , x ⟩ ⟨ p m , y ∣ A I + B ( L ⋅ S ) ∣ p m , y ⟩ = ⟨ p m , x ∣ A I ∣ p m , x ⟩ = A ⟨ p m , x ∣ I ∣ p m , x ⟩ = A = A σ 0 , = ⟨ p m , x ∣ B ( L z ⊗ 2 1 σ z ) ∣ p m , y ⟩ = B ⟨ p m , x ∣ L z ∣ p m , y ⟩ ⊗ 2 1 σ z = − 2 i B σ z , = ⟨ p m , y ∣ B ( L z ⊗ 2 1 σ z ) ∣ p m , x ⟩ = B ⟨ p m , y ∣ L z ∣ p m , x ⟩ ⊗ 2 1 σ z = 2 i B σ z , = ⟨ p m , y ∣ A I ∣ p m , y ⟩ = A ⟨ p m , y ∣ I ∣ p m , y ⟩ = A = A σ 0 . (16)
where σ ~ 0 \widetilde{\sigma}_0 σ 0 , σ ~ x \widetilde{\sigma}_x σ x , σ ~ y \widetilde{\sigma}_y σ y , and σ ~ z \widetilde{\sigma}_z σ z denote the identity spin operator and the Pauli matrices along the x x x , y y y , and z z z directions in the local coordinate system, respectively. The subscripts x x x and y y y correspond to the p x p_x p x and p y p_y p y orbitals. After evaluating the propagator operator, the p p p -orbital degrees of freedom can be integrated out, yielding:
c m , p β ∣ p m , β ⟩ ( [ A σ ~ 0 ] x x + [ − i 2 B σ ~ z ] y x + [ i 2 B σ ~ z ] x y + [ A σ ~ 0 ] y y ) ⟨ p m , α ∣ c m , p α † = [ A σ ~ 0 ] x x + [ − i 2 B σ ~ z ] y x + [ i 2 B σ ~ z ] x y + [ A σ ~ 0 ] y y . (17) \begin{aligned}
c_{m,p\beta}|p_{m,\beta}\rangle
\left([A\widetilde{\sigma}_0]_{xx}+\left[-\frac{i}{2}B\widetilde{\sigma}_z\right]_{yx}+\left[\frac{i}{2}B\widetilde{\sigma}_z\right]_{xy}+[A\widetilde{\sigma}_0]_{yy}\right)
\langle p_{m,\alpha}|c^\dagger_{m,p\alpha}
&=[A\widetilde{\sigma}_0]_{xx}+\left[-\frac{i}{2}B\widetilde{\sigma}_z\right]_{yx}+\left[\frac{i}{2}B\widetilde{\sigma}_z\right]_{xy}+[A\widetilde{\sigma}_0]_{yy}. \tag*{(17)}
\end{aligned} c m , pβ ∣ p m , β ⟩ ( [ A σ 0 ] xx + [ − 2 i B σ z ] y x + [ 2 i B σ z ] x y + [ A σ 0 ] y y ) ⟨ p m , α ∣ c m , p α † = [ A σ 0 ] xx + [ − 2 i B σ z ] y x + [ 2 i B σ z ] x y + [ A σ 0 ] y y . (17)
This expression is incomplete because the orbital integrals arising from the actions of c m , p α † c i , d x y c^\dagger_{m,p\alpha}c_{i,d_{xy}} c m , p α † c i , d x y and c j , d x y † c m , p β c^\dagger_{j,d_{xy}}c_{m,p\beta} c j , d x y † c m , pβ have not been included; the focus here is on demonstrating the elimination of the p p p -orbital integrals in the empty (filled) state representation. Assuming that the operator c m , p α † c^\dagger_{m,p\alpha} c m , p α † creates an electron with spin σ ~ 1 \widetilde{\sigma}_1 σ 1 that has hopped from the Fe1 _1 1 d d d orbital, and c m , p β c_{m,p\beta} c m , pβ annihilates an electron with spin σ ~ 2 \widetilde{\sigma}_2 σ 2 that hops to the Fe2 _2 2 d d d orbital, the Pauli matrices in Eq. (17) must be projected onto the matrix element [ σ ~ λ ] σ ~ 2 σ ~ 1 [\widetilde{\sigma}_\lambda]_{\widetilde{\sigma}_2\widetilde{\sigma}_1} [ σ λ ] σ 2 σ 1 according to the values of σ ~ 1 \widetilde{\sigma}_1 σ 1 and σ ~ 2 \widetilde{\sigma}_2 σ 2 . Substituting the hopping integrals obtained from the Slater-Koster calculation yields:
⟨ d j , x y ∣ c j , d x y † ( T X 1 M 2 , 11 [ A σ ~ 0 ] x x T X 1 M 1 , 11 + T X 1 M 2 , 11 [ − i 2 B σ ~ z ] y x T X 1 M 1 , 12 + T X 1 M 2 , 12 [ i 2 B σ ~ z ] x y T X 1 M 1 , 11 + T X 1 M 2 , 12 [ A σ ~ 0 ] y y T X 1 M 1 , 12 ) c i , d x y ∣ d i , x y ⟩ = ⟨ d j , x y ∣ c j , d x y † ( − 2 t 1 t 2 A σ ~ 0 − i 2 ( t 1 2 − t 2 2 ) B σ ~ z ) c i , d x y ∣ d i , x y ⟩ . (18) \begin{aligned}
\langle d_{j,xy}|c^\dagger_{j,d_{xy}}\big(&T_{X1M2,11}[A\widetilde{\sigma}_0]_{xx}T_{X1M1,11}
+T_{X1M2,11}\left[-\frac{i}{2}B\widetilde{\sigma}_z\right]_{yx}T_{X1M1,12}\\
&+T_{X1M2,12}\left[\frac{i}{2}B\widetilde{\sigma}_z\right]_{xy}T_{X1M1,11}
+T_{X1M2,12}[A\widetilde{\sigma}_0]_{yy}T_{X1M1,12}\big)c_{i,d_{xy}}|d_{i,xy}\rangle\\
&=\langle d_{j,xy}|c^\dagger_{j,d_{xy}}\left(-2t_1t_2A\widetilde{\sigma}_0-\frac{i}{2}(t_1^2-t_2^2)B\widetilde{\sigma}_z\right)c_{i,d_{xy}}|d_{i,xy}\rangle. \tag*{(18)}
\end{aligned} ⟨ d j , x y ∣ c j , d x y † ( T X 1 M 2 , 11 [ A σ 0 ] xx T X 1 M 1 , 11 + T X 1 M 2 , 11 [ − 2 i B σ z ] y x T X 1 M 1 , 12 + T X 1 M 2 , 12 [ 2 i B σ z ] x y T X 1 M 1 , 11 + T X 1 M 2 , 12 [ A σ 0 ] y y T X 1 M 1 , 12 ) c i , d x y ∣ d i , x y ⟩ = ⟨ d j , x y ∣ c j , d x y † ( − 2 t 1 t 2 A σ 0 − 2 i ( t 1 2 − t 2 2 ) B σ z ) c i , d x y ∣ d i , x y ⟩ . (18)
where − 2 t 1 t 2 A σ ~ 0 − i 2 ( t 2 2 − t 3 2 ) B σ ~ z -2t_{1}t_{2}A\widetilde{\sigma}_{0}-\frac{i}{2}(t_{2}^{2}-t_{3}^{2})B\widetilde{\sigma}_{z} − 2 t 1 t 2 A σ 0 − 2 i ( t 2 2 − t 3 2 ) B σ z represents the effective hopping integral from the Fe1 _1 1 d x y d_{xy} d x y orbital to the Fe2 _2 2 d x y d_{xy} d x y orbital via Te1 _1 1 . For notational simplicity, we omit the hole operators c i , d x y c_{i,d_{xy}} c i , d x y and c j , d x y † c_{j,d_{xy}}^{\dagger} c j , d x y † in this expression and retain only the corresponding coefficients; the hopping operators will be transformed into spin operators when considering the complete superexchange process. However, the contribution via Te2 _2 2 must also be calculated, and summing both contributions yields the total effective hopping integral from Fe1 _1 1 d x y d_{xy} d x y to Fe2 _2 2 d x y d_{xy} d x y as − 4 t 1 t 2 A σ ~ 0 -4t_{1}t_{2}A\widetilde{\sigma}_{0} − 4 t 1 t 2 A σ 0 .
Using the same method, the effective hopping integrals for the 1NN t 2 t_{2} t 2 -t 2 t_{2} t 2 (d x y d_{xy} d x y , d y z d_{yz} d y z , d x z d_{xz} d x z ) exchange can be computed as:
T M 2 M 1 e f f ( t 2 ⊗ t 2 ) = ( − 4 t 1 t 2 A σ ~ 0 T M 2 M 1 , 12 e f f T M 2 M 1 , 13 e f f − i 2 ( t 1 t 4 + t 2 t 3 ) B σ ~ y + i 2 ( t 1 t 3 + t 2 t 4 ) B σ ~ x − 2 t 3 t 4 A σ ~ 0 − ( t 3 2 + t 4 2 ) A σ ~ 0 − i 2 ( t 1 t 3 + t 2 t 4 ) B σ ~ y + i 2 ( t 1 t 4 + t 2 t 3 ) B σ ~ x − ( t 3 2 + t 4 2 ) A σ ~ 0 − 2 t 3 t 4 A σ ~ 0 ) , (19) T_{M_{2}M_{1}}^{\mathrm{eff}}(t_{2}\otimes t_{2})=
\begin{pmatrix}
-4t_{1}t_{2}A\widetilde{\sigma}_{0} & T_{M_{2}M_{1},12}^{\mathrm{eff}} & T_{M_{2}M_{1},13}^{\mathrm{eff}}\\
-\frac{i}{2}(t_{1}t_{4}+t_{2}t_{3})B\widetilde{\sigma}_{y}+\frac{i}{2}(t_{1}t_{3}+t_{2}t_{4})B\widetilde{\sigma}_{x} & -2t_{3}t_{4}A\widetilde{\sigma}_{0} & -(t_{3}^{2}+t_{4}^{2})A\widetilde{\sigma}_{0}\\
-\frac{i}{2}(t_{1}t_{3}+t_{2}t_{4})B\widetilde{\sigma}_{y}+\frac{i}{2}(t_{1}t_{4}+t_{2}t_{3})B\widetilde{\sigma}_{x} & -(t_{3}^{2}+t_{4}^{2})A\widetilde{\sigma}_{0} & -2t_{3}t_{4}A\widetilde{\sigma}_{0}
\end{pmatrix}, \tag*{(19)} T M 2 M 1 eff ( t 2 ⊗ t 2 ) = − 4 t 1 t 2 A σ 0 − 2 i ( t 1 t 4 + t 2 t 3 ) B σ y + 2 i ( t 1 t 3 + t 2 t 4 ) B σ x − 2 i ( t 1 t 3 + t 2 t 4 ) B σ y + 2 i ( t 1 t 4 + t 2 t 3 ) B σ x T M 2 M 1 , 12 eff − 2 t 3 t 4 A σ 0 − ( t 3 2 + t 4 2 ) A σ 0 T M 2 M 1 , 13 eff − ( t 3 2 + t 4 2 ) A σ 0 − 2 t 3 t 4 A σ 0 , (19)
T M 2 M 1 , 12 e f f ( t 2 ⊗ t 2 ) = i 2 ( t 1 t 4 + t 2 t 3 ) B σ ~ y − i 2 ( t 1 t 3 + t 2 t 4 ) B σ ~ x , T_{M_{2}M_{1},12}^{\mathrm{eff}}(t_{2}\otimes t_{2})=\frac{i}{2}(t_{1}t_{4}+t_{2}t_{3})B\widetilde{\sigma}_{y}-\frac{i}{2}(t_{1}t_{3}+t_{2}t_{4})B\widetilde{\sigma}_{x}, T M 2 M 1 , 12 eff ( t 2 ⊗ t 2 ) = 2 i ( t 1 t 4 + t 2 t 3 ) B σ y − 2 i ( t 1 t 3 + t 2 t 4 ) B σ x ,
T M 2 M 1 , 13 e f f ( t 2 ⊗ t 2 ) = i 2 ( t 1 t 3 + t 2 t 4 ) B σ ~ y − i 2 ( t 1 t 4 + t 2 t 3 ) B σ ~ x . T_{M_{2}M_{1},13}^{\mathrm{eff}}(t_{2}\otimes t_{2})=\frac{i}{2}(t_{1}t_{3}+t_{2}t_{4})B\widetilde{\sigma}_{y}-\frac{i}{2}(t_{1}t_{4}+t_{2}t_{3})B\widetilde{\sigma}_{x}. T M 2 M 1 , 13 eff ( t 2 ⊗ t 2 ) = 2 i ( t 1 t 3 + t 2 t 4 ) B σ y − 2 i ( t 1 t 4 + t 2 t 3 ) B σ x .
In the complete superexchange process, as shown in Eq. (14) , the reverse hopping from Fe2 _2 2 back to Fe1 _1 1 must also be considered. Here, the relation T M 1 M 2 e f f = T M 2 M 1 e f f † T_{M_{1}M_{2}}^{\mathrm{eff}}=T_{M_{2}M_{1}}^{\mathrm{eff}\dagger} T M 1 M 2 eff = T M 2 M 1 eff † holds, thereby determining all effective hopping integrals between the Fe1 _1 1 t 2 t_{2} t 2 and Fe2 _2 2 t 2 t_{2} t 2 orbitals.
The effective hopping integrals for the 2NN must also be considered. Their hopping pathways are similar to those of the 1NN, but with only one coordinating atom and different bond directions in the local coordinate system. Denoting the 2NN Fe atom as M 2 ′ M_{2}^{\prime} M 2 ′ , the effective hopping integral can be expressed as:
T M 2 ′ M 1 e f f ( t 2 ⊗ t 2 ) = ( 2 t 1 t 2 A σ ~ 0 + i 2 ( t 2 2 − t 1 2 ) B σ ~ z − i 2 t 2 t 3 B σ ~ y + i 2 t 1 t 3 B σ ~ x − i 2 t 2 t 4 B σ ~ y + i 2 t 1 t 4 B σ ~ x i 2 t 1 t 4 B σ ~ y − i 2 t 2 t 4 B σ ~ x t 3 2 A σ ~ 0 t 4 2 A σ ~ 0 i 2 t 1 t 3 B σ ~ y − i 2 t 2 t 3 B σ ~ x t 3 2 A σ ~ 0 t 3 t 4 A σ ~ 0 ) . (20) T_{M_{2}^{\prime}M_{1}}^{\mathrm{eff}}(t_{2}\otimes t_{2})=
\begin{pmatrix}
2t_{1}t_{2}A\widetilde{\sigma}_{0}+\frac{i}{2}(t_{2}^{2}-t_{1}^{2})B\widetilde{\sigma}_{z} & -\frac{i}{2}t_{2}t_{3}B\widetilde{\sigma}_{y}+\frac{i}{2}t_{1}t_{3}B\widetilde{\sigma}_{x} & -\frac{i}{2}t_{2}t_{4}B\widetilde{\sigma}_{y}+\frac{i}{2}t_{1}t_{4}B\widetilde{\sigma}_{x}\\
\frac{i}{2}t_{1}t_{4}B\widetilde{\sigma}_{y}-\frac{i}{2}t_{2}t_{4}B\widetilde{\sigma}_{x} & t_{3}^{2}A\widetilde{\sigma}_{0} & t_{4}^{2}A\widetilde{\sigma}_{0}\\
\frac{i}{2}t_{1}t_{3}B\widetilde{\sigma}_{y}-\frac{i}{2}t_{2}t_{3}B\widetilde{\sigma}_{x} & t_{3}^{2}A\widetilde{\sigma}_{0} & t_{3}t_{4}A\widetilde{\sigma}_{0}
\end{pmatrix}. \tag*{(20)} T M 2 ′ M 1 eff ( t 2 ⊗ t 2 ) = 2 t 1 t 2 A σ 0 + 2 i ( t 2 2 − t 1 2 ) B σ z 2 i t 1 t 4 B σ y − 2 i t 2 t 4 B σ x 2 i t 1 t 3 B σ y − 2 i t 2 t 3 B σ x − 2 i t 2 t 3 B σ y + 2 i t 1 t 3 B σ x t 3 2 A σ 0 t 3 2 A σ 0 − 2 i t 2 t 4 B σ y + 2 i t 1 t 4 B σ x t 4 2 A σ 0 t 3 t 4 A σ 0 . (20)
This expression also satisfies the relation T M 1 M 2 ′ e f f = T M 2 ′ M 1 e f f † T_{M_{1}M_{2}^{\prime}}^{\mathrm{eff}}=T_{M_{2}^{\prime}M_{1}}^{\mathrm{eff}\dagger} T M 1 M 2 ′ eff = T M 2 ′ M 1 eff † .
The excitation energy change from the d 4 d 4 d^{4}d^{4} d 4 d 4 ground state to the d 3 d 5 d^{3}d^{5} d 3 d 5 intermediate state in the hole representation is calculated using the tight-binding model under the Kanamori interaction in Eq. (3) . For t 2 t_{2} t 2 -t 2 t_{2} t 2 hopping, only one degenerate intermediate-state energy exists, with Δ E n = U + 3 J H \Delta E_{n}=U+3J_{H} Δ E n = U + 3 J H . Substituting this into the complete superexchange process yields:
⟨ L S ∣ H s ∣ G S ⟩ = − 1 U + 3 J H ⟨ L S ∣ c i , d α , σ 4 † T M 2 M 1 e f f † c j , d β , σ 3 ∣ n ⟩ ⟨ n ∣ c j , d β , σ 2 † T M 2 M 1 e f f c i , d α , σ 1 ∣ G S ⟩ , (21) \langle LS|H_{s}|GS\rangle=-\frac{1}{U+3J_{H}}\langle LS|c_{i,d_{\alpha},\sigma_{4}}^{\dagger}T_{M_{2}M_{1}}^{\mathrm{eff}\dagger}c_{j,d_{\beta},\sigma_{3}}|n\rangle\langle n|c_{j,d_{\beta},\sigma_{2}}^{\dagger}T_{M_{2}M_{1}}^{\mathrm{eff}}c_{i,d_{\alpha},\sigma_{1}|GS\rangle,} \tag*{(21)} ⟨ L S ∣ H s ∣ GS ⟩ = − U + 3 J H 1 ⟨ L S ∣ c i , d α , σ 4 † T M 2 M 1 eff † c j , d β , σ 3 ∣ n ⟩ ⟨ n ∣ c j , d β , σ 2 † T M 2 M 1 eff c i , d α , σ 1 ∣ GS ⟩ , (21)
where ⟨ L S ∣ \langle LS| ⟨ L S ∣ and ∣ G S ⟩ |GS\rangle ∣ GS ⟩ denote the final state and the ground state, respectively. The effective hopping integral contains both constant terms and spin operators; we separate these by factoring out the constants, denoting T M 2 M 1 e f f = ∑ γ T M 2 M 1 , γ e f f σ ~ γ T_{M_{2}M_{1}}^{\mathrm{eff}}=\sum_{\gamma}T_{M_{2}M_{1},\gamma}^{\mathrm{eff}}\widetilde{\sigma}_{\gamma} T M 2 M 1 eff = ∑ γ T M 2 M 1 , γ eff σ γ . The effective hopping integral may contain multiple terms; for notational simplicity, we use σ ~ γ \widetilde{\sigma}_{\gamma} σ γ to represent them, where γ = 0 , x , y , z \gamma=0,x,y,z γ = 0 , x , y , z . The expression can then be written as:
⟨ L S ∣ H s ∣ G S ⟩ = − ∑ γ , γ ′ T M 2 M 1 , γ ′ e f f † T M 2 M 1 , γ e f f U + 3 J H ⟨ L S ∣ c i , d α , σ 4 † [ σ ~ γ ′ ] σ 4 σ 3 c j , d β , σ 3 c j , d β , σ 2 † [ σ ~ γ ] σ 2 σ 1 c i , d α , σ 1 ∣ G S ⟩ = ∑ γ , γ ′ T M 2 M 1 , γ ′ e f f † T M 2 M 1 , γ e f f U + 3 J H ⟨ L S ∣ [ σ ~ γ ′ ] σ 4 σ 3 [ σ ~ γ ] σ 2 σ 1 c j , d β , σ 2 † c j , d β , σ 3 c i , d α , σ 4 † c i , d α , σ 1 ∣ G S ⟩ = ∑ γ , γ ′ T M 2 M 1 , γ ′ e f f † T M 2 M 1 , γ e f f U + 3 J H ⟨ L S ∣ ( [ σ ~ γ ′ ] σ 4 σ 3 ⋅ s ~ j , β ) ( [ σ ~ γ ] σ 2 σ 1 ⋅ s ~ i , α ) ∣ G S ⟩ . (22) \begin{aligned}
\langle LS|H_{s}|GS\rangle=-\sum_{\gamma,\gamma'}\frac{T_{M_{2}M_{1},\gamma'}^{\mathrm{eff}\dagger}T_{M_{2}M_{1},\gamma}^{\mathrm{eff}}}{U+3J_{H}}\langle LS|c_{i,d_{\alpha},\sigma_{4}}^{\dagger}[\widetilde{\sigma}_{\gamma'}]_{\sigma_{4}\sigma_{3}}c_{j,d_{\beta},\sigma_{3}}c_{j,d_{\beta},\sigma_{2}}^{\dagger}[\widetilde{\sigma}_{\gamma}]_{\sigma_{2}\sigma_{1}}c_{i,d_{\alpha},\sigma_{1}|GS\rangle} \\
&=\sum_{\gamma,\gamma'}\frac{T_{M_{2}M_{1},\gamma'}^{\mathrm{eff}\dagger}T_{M_{2}M_{1},\gamma}^{\mathrm{eff}}}{U+3J_{H}}\langle LS|[\widetilde{\sigma}_{\gamma'}]_{\sigma_{4}\sigma_{3}}[\widetilde{\sigma}_{\gamma}]_{\sigma_{2}\sigma_{1}}c_{j,d_{\beta},\sigma_{2}}^{\dagger}c_{j,d_{\beta},\sigma_{3}}c_{i,d_{\alpha},\sigma_{4}}^{\dagger}c_{i,d_{\alpha},\sigma_{1}|GS\rangle} \\
&=\sum_{\gamma,\gamma'}\frac{T_{M_{2}M_{1},\gamma'}^{\mathrm{eff}\dagger}T_{M_{2}M_{1},\gamma}^{\mathrm{eff}}}{U+3J_{H}}\langle LS|([\widetilde{\sigma}_{\gamma'}]_{\sigma_{4}\sigma_{3}}\cdot\widetilde{s}_{j,\beta})([\widetilde{\sigma}_{\gamma}]_{\sigma_{2}\sigma_{1}}\cdot\widetilde{s}_{i,\alpha})|GS\rangle. \tag*{(22)}
\end{aligned} ⟨ L S ∣ H s ∣ GS ⟩ = − γ , γ ′ ∑ U + 3 J H T M 2 M 1 , γ ′ eff † T M 2 M 1 , γ eff ⟨ L S ∣ c i , d α , σ 4 † [ σ γ ′ ] σ 4 σ 3 c j , d β , σ 3 c j , d β , σ 2 † [ σ γ ] σ 2 σ 1 c i , d α , σ 1 ∣ GS ⟩ = γ , γ ′ ∑ U + 3 J H T M 2 M 1 , γ ′ eff † T M 2 M 1 , γ eff ⟨ L S ∣ [ σ γ ′ ] σ 4 σ 3 [ σ γ ] σ 2 σ 1 c j , d β , σ 2 † c j , d β , σ 3 c i , d α , σ 4 † c i , d α , σ 1 ∣ GS ⟩ = γ , γ ′ ∑ U + 3 J H T M 2 M 1 , γ ′ eff † T M 2 M 1 , γ eff ⟨ L S ∣ ([ σ γ ′ ] σ 4 σ 3 ⋅ s j , β ) ([ σ γ ] σ 2 σ 1 ⋅ s i , α ) ∣ GS ⟩ . (22)
During this process, spin-independent density operator terms have been omitted. Here, s ~ i , α \widetilde{s}_{i,\alpha} s i , α and s ~ j , β \widetilde{s}_{j,\beta} s j , β are the spin operators for a single hole on the respective orbitals, derived from c i , d α , σ 4 † c i , d α , σ 1 c_{i,d_{\alpha},\sigma_{4}}^{\dagger}c_{i,d_{\alpha},\sigma_{1}} c i , d α , σ 4 † c i , d α , σ 1 and c j , d β , σ 2 † c j , d β , σ 3 c_{j,d_{\beta},\sigma_{2}}^{\dagger}c_{j,d_{\beta},\sigma_{3}} c j , d β , σ 2 † c j , d β , σ 3 , respectively. According to the equivalent operator theorem [41 , 42 ] , in the high-spin S = 2 S=2 S = 2 case, their relation to the total spin can be expressed as:
s ~ i , α = 1 4 S ~ i , s ~ j , β = 1 4 S ~ j . (23) \widetilde{s}_{i,\alpha}=\frac{1}{4}\widetilde{S}_{i},\qquad\widetilde{s}_{j,\beta}=\frac{1}{4}\widetilde{S}_{j}. \tag*{(23)} s i , α = 4 1 S i , s j , β = 4 1 S j . (23)
In the calculation of Eq. (21) , the Pauli matrix combination [ σ ~ γ ′ ] σ ~ 4 σ ~ 3 [ σ ~ γ ] σ ~ 2 σ ~ 1 [\tilde{\sigma}_{\gamma'}]_{\tilde{\sigma}_4\tilde{\sigma}_3}[\tilde{\sigma}_{\gamma}]_{\tilde{\sigma}_2\tilde{\sigma}_1} [ σ ~ γ ′ ] σ ~ 4 σ ~ 3 [ σ ~ γ ] σ ~ 2 σ ~ 1 inherently selects specific spin configurations; therefore, the spin operator calculations can be determined based on these Pauli matrices. We illustrate this with [ σ ~ 0 ] σ ~ 4 σ ~ 3 [ σ ~ 0 ] σ ~ 2 σ ~ 1 [\tilde{\sigma}_{0}]_{\tilde{\sigma}_4\tilde{\sigma}_3}[\tilde{\sigma}_{0}]_{\tilde{\sigma}_2\tilde{\sigma}_1} [ σ ~ 0 ] σ ~ 4 σ ~ 3 [ σ ~ 0 ] σ ~ 2 σ ~ 1 as a concrete example:
[ σ ~ 0 ] σ ~ 4 σ ~ 3 [ σ ~ 0 ] σ ~ 2 σ ~ 1 ( [ σ ~ ] σ ~ 1 σ ~ 4 ⋅ s ~ i , α ) ( [ σ ~ ] σ ~ 3 σ ~ 2 ⋅ s ~ j , β ) = [ σ ~ 0 ] ↑ ↑ [ σ ~ 0 ] ↑ ↑ ( [ σ ~ ] ↑ ↑ ⋅ s ~ i , α ) ( [ σ ~ ] ↑ ↑ ⋅ s ~ j , β ) + [ σ ~ 0 ] ↓ ↓ [ σ ~ 0 ] ↓ ↓ ( [ σ ~ ] ↓ ↓ ⋅ s ~ i , α ) ( [ σ ~ ] ↓ ↓ ⋅ s ~ j , β ) + [ σ ~ 0 ] ↑ ↑ [ σ ~ 0 ] ↓ ↓ ( [ σ ~ ] ↓ ↑ ⋅ s ~ i , α ) ( [ σ ~ ] ↑ ↓ ⋅ s ~ j , β ) + [ σ ~ 0 ] ↓ ↓ [ σ ~ 0 ] ↑ ↑ ( [ σ ~ ] ↑ ↓ ⋅ s ~ i , α ) ( [ σ ~ ] ↓ ↑ ⋅ s ~ j , β ) = 2 s ~ i , α ⋅ s ~ j , β = 1 8 S ~ i ⋅ S ~ j = 1 8 S i ⋅ S j , (24) \begin{aligned}
[\tilde{\sigma}_{0}]_{\tilde{\sigma}_4\tilde{\sigma}_3}[\tilde{\sigma}_{0}]_{\tilde{\sigma}_2\tilde{\sigma}_1}
\left([\tilde{\sigma}]_{\tilde{\sigma}_1\tilde{\sigma}_4}\cdot\tilde{s}_{i,\alpha}\right)
\left([\tilde{\sigma}]_{\tilde{\sigma}_3\tilde{\sigma}_2}\cdot\tilde{s}_{j,\beta}\right)
={}&[\tilde{\sigma}_{0}]_{\uparrow\uparrow}[\tilde{\sigma}_{0}]_{\uparrow\uparrow}
\left([\tilde{\sigma}]_{\uparrow\uparrow}\cdot\tilde{s}_{i,\alpha}\right)
\left([\tilde{\sigma}]_{\uparrow\uparrow}\cdot\tilde{s}_{j,\beta}\right)\\
&+[\tilde{\sigma}_{0}]_{\downarrow\downarrow}[\tilde{\sigma}_{0}]_{\downarrow\downarrow}
\left([\tilde{\sigma}]_{\downarrow\downarrow}\cdot\tilde{s}_{i,\alpha}\right)
\left([\tilde{\sigma}]_{\downarrow\downarrow}\cdot\tilde{s}_{j,\beta}\right)\\
&+[\tilde{\sigma}_{0}]_{\uparrow\uparrow}[\tilde{\sigma}_{0}]_{\downarrow\downarrow}
\left([\tilde{\sigma}]_{\downarrow\uparrow}\cdot\tilde{s}_{i,\alpha}\right)
\left([\tilde{\sigma}]_{\uparrow\downarrow}\cdot\tilde{s}_{j,\beta}\right)\\
&+[\tilde{\sigma}_{0}]_{\downarrow\downarrow}[\tilde{\sigma}_{0}]_{\uparrow\uparrow}
\left([\tilde{\sigma}]_{\uparrow\downarrow}\cdot\tilde{s}_{i,\alpha}\right)
\left([\tilde{\sigma}]_{\downarrow\uparrow}\cdot\tilde{s}_{j,\beta}\right)\\
={}&2\tilde{s}_{i,\alpha}\cdot\tilde{s}_{j,\beta}
=\frac{1}{8}\tilde{S}_{i}\cdot\tilde{S}_{j}
=\frac{1}{8}S_{i}\cdot S_{j}, \tag*{(24)}
\end{aligned} [ σ ~ 0 ] σ ~ 4 σ ~ 3 [ σ ~ 0 ] σ ~ 2 σ ~ 1 ( [ σ ~ ] σ ~ 1 σ ~ 4 ⋅ s ~ i , α ) ( [ σ ~ ] σ ~ 3 σ ~ 2 ⋅ s ~ j , β ) = = [ σ ~ 0 ] ↑↑ [ σ ~ 0 ] ↑↑ ( [ σ ~ ] ↑↑ ⋅ s ~ i , α ) ( [ σ ~ ] ↑↑ ⋅ s ~ j , β ) + [ σ ~ 0 ] ↓↓ [ σ ~ 0 ] ↓↓ ( [ σ ~ ] ↓↓ ⋅ s ~ i , α ) ( [ σ ~ ] ↓↓ ⋅ s ~ j , β ) + [ σ ~ 0 ] ↑↑ [ σ ~ 0 ] ↓↓ ( [ σ ~ ] ↓↑ ⋅ s ~ i , α ) ( [ σ ~ ] ↑↓ ⋅ s ~ j , β ) + [ σ ~ 0 ] ↓↓ [ σ ~ 0 ] ↑↑ ( [ σ ~ ] ↑↓ ⋅ s ~ i , α ) ( [ σ ~ ] ↓↑ ⋅ s ~ j , β ) 2 s ~ i , α ⋅ s ~ j , β = 8 1 S ~ i ⋅ S ~ j = 8 1 S i ⋅ S j , (24)
where the magnetic order selected by [ σ ~ 0 ] σ ~ 4 σ ~ 3 [ σ ~ 0 ] σ ~ 2 σ ~ 1 [\tilde{\sigma}_{0}]_{\tilde{\sigma}_4\tilde{\sigma}_3}[\tilde{\sigma}_{0}]_{\tilde{\sigma}_2\tilde{\sigma}_1} [ σ ~ 0 ] σ ~ 4 σ ~ 3 [ σ ~ 0 ] σ ~ 2 σ ~ 1 is solved and summed. The product of spin and Pauli vector operators is computed by expanding the dot product as σ ~ x s ~ x + σ ~ y s ~ y + σ ~ z s ~ z \tilde{\sigma}^{x}\tilde{s}^{x}+\tilde{\sigma}^{y}\tilde{s}^{y}+\tilde{\sigma}^{z}\tilde{s}^{z} σ ~ x s ~ x + σ ~ y s ~ y + σ ~ z s ~ z and substituting the Pauli matrices according to the spin configuration. As shown, [ σ ~ 0 ] σ ~ 4 σ ~ 3 [ σ ~ 0 ] σ ~ 2 σ ~ 1 [\tilde{\sigma}_{0}]_{\tilde{\sigma}_4\tilde{\sigma}_3}[\tilde{\sigma}_{0}]_{\tilde{\sigma}_2\tilde{\sigma}_1} [ σ ~ 0 ] σ ~ 4 σ ~ 3 [ σ ~ 0 ] σ ~ 2 σ ~ 1 indicates that, in the absence of spin flips, the superexchange contributes to the Heisenberg interaction. Here, the dot product in the local coordinate system can be directly transformed into the global coordinate system. The results for different possible Pauli matrix combinations are presented below:
[ σ ~ x ] σ ~ 4 σ ~ 3 [ σ ~ x ] σ ~ 2 σ ~ 1 ( [ σ ~ ] σ ~ 1 σ ~ 4 ⋅ s ~ i , α ) ( [ σ ~ ] σ ~ 3 σ ~ 2 ⋅ s ~ j , β ) = 1 8 ( S ~ i x S ~ j x − S ~ i y S ~ j y − S ~ i z S ~ j z ) , [\tilde{\sigma}_{x}]_{\tilde{\sigma}_4\tilde{\sigma}_3}[\tilde{\sigma}_{x}]_{\tilde{\sigma}_2\tilde{\sigma}_1}
\left([\tilde{\sigma}]_{\tilde{\sigma}_1\tilde{\sigma}_4}\cdot\tilde{s}_{i,\alpha}\right)
\left([\tilde{\sigma}]_{\tilde{\sigma}_3\tilde{\sigma}_2}\cdot\tilde{s}_{j,\beta}\right)
=\frac{1}{8}\left(\tilde{S}_{i}^{x}\tilde{S}_{j}^{x}-\tilde{S}_{i}^{y}\tilde{S}_{j}^{y}-\tilde{S}_{i}^{z}\tilde{S}_{j}^{z}\right), [ σ ~ x ] σ ~ 4 σ ~ 3 [ σ ~ x ] σ ~ 2 σ ~ 1 ( [ σ ~ ] σ ~ 1 σ ~ 4 ⋅ s ~ i , α ) ( [ σ ~ ] σ ~ 3 σ ~ 2 ⋅ s ~ j , β ) = 8 1 ( S ~ i x S ~ j x − S ~ i y S ~ j y − S ~ i z S ~ j z ) ,
[ σ ~ x ] σ ~ 4 σ ~ 3 [ σ ~ y ] σ ~ 2 σ ~ 1 ( [ σ ~ ] σ ~ 1 σ ~ 4 ⋅ s ~ i , α ) ( [ σ ~ ] σ ~ 3 σ ~ 2 ⋅ s ~ j , β ) = 1 8 ( S ~ i x S ~ j y + S ~ i y S ~ j x ) , [\tilde{\sigma}_{x}]_{\tilde{\sigma}_4\tilde{\sigma}_3}[\tilde{\sigma}_{y}]_{\tilde{\sigma}_2\tilde{\sigma}_1}
\left([\tilde{\sigma}]_{\tilde{\sigma}_1\tilde{\sigma}_4}\cdot\tilde{s}_{i,\alpha}\right)
\left([\tilde{\sigma}]_{\tilde{\sigma}_3\tilde{\sigma}_2}\cdot\tilde{s}_{j,\beta}\right)
=\frac{1}{8}\left(\tilde{S}_{i}^{x}\tilde{S}_{j}^{y}+\tilde{S}_{i}^{y}\tilde{S}_{j}^{x}\right), [ σ ~ x ] σ ~ 4 σ ~ 3 [ σ ~ y ] σ ~ 2 σ ~ 1 ( [ σ ~ ] σ ~ 1 σ ~ 4 ⋅ s ~ i , α ) ( [ σ ~ ] σ ~ 3 σ ~ 2 ⋅ s ~ j , β ) = 8 1 ( S ~ i x S ~ j y + S ~ i y S ~ j x ) ,
[ σ ~ y ] σ ~ 4 σ ~ 3 [ σ ~ y ] σ ~ 2 σ ~ 1 ( [ σ ~ ] σ ~ 1 σ ~ 4 ⋅ s ~ i , α ) ( [ σ ~ ] σ ~ 3 σ ~ 2 ⋅ s ~ j , β ) = 1 8 ( − S ~ i x S ~ j x + S ~ i y S ~ j y − S ~ i z S ~ j z ) , [\tilde{\sigma}_{y}]_{\tilde{\sigma}_4\tilde{\sigma}_3}[\tilde{\sigma}_{y}]_{\tilde{\sigma}_2\tilde{\sigma}_1}
\left([\tilde{\sigma}]_{\tilde{\sigma}_1\tilde{\sigma}_4}\cdot\tilde{s}_{i,\alpha}\right)
\left([\tilde{\sigma}]_{\tilde{\sigma}_3\tilde{\sigma}_2}\cdot\tilde{s}_{j,\beta}\right)
=\frac{1}{8}\left(-\tilde{S}_{i}^{x}\tilde{S}_{j}^{x}+\tilde{S}_{i}^{y}\tilde{S}_{j}^{y}-\tilde{S}_{i}^{z}\tilde{S}_{j}^{z}\right), [ σ ~ y ] σ ~ 4 σ ~ 3 [ σ ~ y ] σ ~ 2 σ ~ 1 ( [ σ ~ ] σ ~ 1 σ ~ 4 ⋅ s ~ i , α ) ( [ σ ~ ] σ ~ 3 σ ~ 2 ⋅ s ~ j , β ) = 8 1 ( − S ~ i x S ~ j x + S ~ i y S ~ j y − S ~ i z S ~ j z ) ,
[ σ ~ x ] σ ~ 4 σ ~ 3 [ σ ~ z ] σ ~ 2 σ ~ 1 ( [ σ ~ ] σ ~ 1 σ ~ 4 ⋅ s ~ i , α ) ( [ σ ~ ] σ ~ 3 σ ~ 2 ⋅ s ~ j , β ) = 1 8 ( S ~ i x S ~ j z + S ~ i z S ~ j x ) , [\tilde{\sigma}_{x}]_{\tilde{\sigma}_4\tilde{\sigma}_3}[\tilde{\sigma}_{z}]_{\tilde{\sigma}_2\tilde{\sigma}_1}
\left([\tilde{\sigma}]_{\tilde{\sigma}_1\tilde{\sigma}_4}\cdot\tilde{s}_{i,\alpha}\right)
\left([\tilde{\sigma}]_{\tilde{\sigma}_3\tilde{\sigma}_2}\cdot\tilde{s}_{j,\beta}\right)
=\frac{1}{8}\left(\tilde{S}_{i}^{x}\tilde{S}_{j}^{z}+\tilde{S}_{i}^{z}\tilde{S}_{j}^{x}\right), [ σ ~ x ] σ ~ 4 σ ~ 3 [ σ ~ z ] σ ~ 2 σ ~ 1 ( [ σ ~ ] σ ~ 1 σ ~ 4 ⋅ s ~ i , α ) ( [ σ ~ ] σ ~ 3 σ ~ 2 ⋅ s ~ j , β ) = 8 1 ( S ~ i x S ~ j z + S ~ i z S ~ j x ) ,
[ σ ~ y ] σ ~ 4 σ ~ 3 [ σ ~ z ] σ ~ 2 σ ~ 1 ( [ σ ~ ] σ ~ 1 σ ~ 4 ⋅ s ~ i , α ) ( [ σ ~ ] σ ~ 3 σ ~ 2 ⋅ s ~ j , β ) = 1 8 ( S ~ i y S ~ j z + S ~ i z S ~ j y ) , [\tilde{\sigma}_{y}]_{\tilde{\sigma}_4\tilde{\sigma}_3}[\tilde{\sigma}_{z}]_{\tilde{\sigma}_2\tilde{\sigma}_1}
\left([\tilde{\sigma}]_{\tilde{\sigma}_1\tilde{\sigma}_4}\cdot\tilde{s}_{i,\alpha}\right)
\left([\tilde{\sigma}]_{\tilde{\sigma}_3\tilde{\sigma}_2}\cdot\tilde{s}_{j,\beta}\right)
=\frac{1}{8}\left(\tilde{S}_{i}^{y}\tilde{S}_{j}^{z}+\tilde{S}_{i}^{z}\tilde{S}_{j}^{y}\right), [ σ ~ y ] σ ~ 4 σ ~ 3 [ σ ~ z ] σ ~ 2 σ ~ 1 ( [ σ ~ ] σ ~ 1 σ ~ 4 ⋅ s ~ i , α ) ( [ σ ~ ] σ ~ 3 σ ~ 2 ⋅ s ~ j , β ) = 8 1 ( S ~ i y S ~ j z + S ~ i z S ~ j y ) ,
[ σ ~ z ] σ ~ 4 σ ~ 3 [ σ ~ z ] σ ~ 2 σ ~ 1 ( [ σ ~ ] σ ~ 1 σ ~ 4 ⋅ s ~ i , α ) ( [ σ ~ ] σ ~ 3 σ ~ 2 ⋅ s ~ j , β ) = 1 8 ( − S ~ i x S ~ j x − S ~ i y S ~ j y + S ~ i z S ~ j z ) . (25) [\tilde{\sigma}_{z}]_{\tilde{\sigma}_4\tilde{\sigma}_3}[\tilde{\sigma}_{z}]_{\tilde{\sigma}_2\tilde{\sigma}_1}
\left([\tilde{\sigma}]_{\tilde{\sigma}_1\tilde{\sigma}_4}\cdot\tilde{s}_{i,\alpha}\right)
\left([\tilde{\sigma}]_{\tilde{\sigma}_3\tilde{\sigma}_2}\cdot\tilde{s}_{j,\beta}\right)
=\frac{1}{8}\left(-\tilde{S}_{i}^{x}\tilde{S}_{j}^{x}-\tilde{S}_{i}^{y}\tilde{S}_{j}^{y}+\tilde{S}_{i}^{z}\tilde{S}_{j}^{z}\right). \tag*{(25)} [ σ ~ z ] σ ~ 4 σ ~ 3 [ σ ~ z ] σ ~ 2 σ ~ 1 ( [ σ ~ ] σ ~ 1 σ ~ 4 ⋅ s ~ i , α ) ( [ σ ~ ] σ ~ 3 σ ~ 2 ⋅ s ~ j , β ) = 8 1 ( − S ~ i x S ~ j x − S ~ i y S ~ j y + S ~ i z S ~ j z ) . (25)
swapping the positions of the two Pauli matrices of [ σ ~ γ ′ ] σ ~ 4 σ ~ 3 [ σ ~ γ ] σ ~ 2 σ ~ 1 [\tilde{\sigma}_{\gamma'}]_{\tilde{\sigma}_4\tilde{\sigma}_3}[\tilde{\sigma}_{\gamma}]_{\tilde{\sigma}_2\tilde{\sigma}_1} [ σ ~ γ ′ ] σ ~ 4 σ ~ 3 [ σ ~ γ ] σ ~ 2 σ ~ 1 does not affect the result in Eq. (25) . Substituting the spin operator results and effective hopping integrals into the calculation yields, for the 1NN:
H s 1 t 2 − t 2 = 1 4 ( U + 3 J H ) [ 8 t 1 2 t 2 2 + 4 t 2 2 t 3 2 + ( t 3 2 + t 4 2 ) 2 ] A 2 ( S i ⋅ S j ) − 1 8 ( U + 3 J H ) [ ( t 1 t 3 + t 2 t 4 ) 2 + ( t 1 t 4 + t 2 t 3 ) 2 ] B 2 ( S ~ i z S ~ j z ) − 1 4 ( U + 3 J H ) ( t 1 t 3 + t 2 t 4 ) ( t 1 t 4 + t 2 t 3 ) B 2 ( S ~ i x S ~ j y + S ~ i y S ~ j x ) , (26) \begin{aligned}
H_{s_1}^{t_2-t_2}={}&\frac{1}{4(U+3J_{\mathrm{H}})}
\left[8t_1^2t_2^2+4t_2^2t_3^2+(t_3^2+t_4^2)^2\right]A^2\left(S_i\cdot S_j\right)\\
&-\frac{1}{8(U+3J_{\mathrm{H}})}
\left[(t_1t_3+t_2t_4)^2+(t_1t_4+t_2t_3)^2\right]B^2\left(\tilde{S}_{i}^{z}\tilde{S}_{j}^{z}\right)\\
&-\frac{1}{4(U+3J_{\mathrm{H}})}
(t_1t_3+t_2t_4)(t_1t_4+t_2t_3)B^2
\left(\tilde{S}_{i}^{x}\tilde{S}_{j}^{y}+\tilde{S}_{i}^{y}\tilde{S}_{j}^{x}\right), \tag*{(26)}
\end{aligned} H s 1 t 2 − t 2 = 4 ( U + 3 J H ) 1 [ 8 t 1 2 t 2 2 + 4 t 2 2 t 3 2 + ( t 3 2 + t 4 2 ) 2 ] A 2 ( S i ⋅ S j ) − 8 ( U + 3 J H ) 1 [ ( t 1 t 3 + t 2 t 4 ) 2 + ( t 1 t 4 + t 2 t 3 ) 2 ] B 2 ( S ~ i z S ~ j z ) − 4 ( U + 3 J H ) 1 ( t 1 t 3 + t 2 t 4 ) ( t 1 t 4 + t 2 t 3 ) B 2 ( S ~ i x S ~ j y + S ~ i y S ~ j x ) , (26)
and for the 2NN:
H s 2 t 2 − t 2 = 1 16 ( U + 3 J H ) { 4 [ t 3 2 t 4 2 + 2 t 1 2 t 2 2 + 1 2 t 4 4 + 1 2 t 3 4 ] A 2 − 1 2 ( t 2 2 − t 1 2 ) 2 B 2 } ( S i ⋅ S j ) − 1 16 ( U + 3 J H ) [ ( t 1 2 + t 2 2 ) ( t 3 2 + t 4 2 ) − ( t 2 2 − t 1 2 ) 2 ] B 2 ( S ~ i z S ~ j z ) − 1 8 ( U + 3 J H ) t 1 t 2 ( t 3 2 + t 4 2 ) B 2 ( S ~ i x S ~ j y + S ~ i y S ~ j x ) + 1 16 ( U + 3 J H ) ( t 1 2 − t 2 2 ) ( t 3 2 + t 4 2 ) B 2 ( S ~ i x S ~ j x − S ~ i y S ~ j y ) . (27) \begin{aligned}
H_{s_2}^{t_2-t_2}={}&\frac{1}{16(U+3J_{\mathrm{H}})}
\left\{4\left[t_3^2t_4^2+2t_1^2t_2^2+\frac{1}{2}t_4^4+\frac{1}{2}t_3^4\right]A^2-\frac{1}{2}(t_2^2-t_1^2)^2B^2\right\}
\left(S_i\cdot S_j\right)\\
&-\frac{1}{16(U+3J_{\mathrm{H}})}
\left[(t_1^2+t_2^2)(t_3^2+t_4^2)-(t_2^2-t_1^2)^2\right]B^2
\left(\tilde{S}_{i}^{z}\tilde{S}_{j}^{z}\right)\\
&-\frac{1}{8(U+3J_{\mathrm{H}})}
t_1t_2(t_3^2+t_4^2)B^2
\left(\tilde{S}_{i}^{x}\tilde{S}_{j}^{y}+\tilde{S}_{i}^{y}\tilde{S}_{j}^{x}\right)\\
&+\frac{1}{16(U+3J_{\mathrm{H}})}
(t_1^2-t_2^2)(t_3^2+t_4^2)B^2
\left(\tilde{S}_{i}^{x}\tilde{S}_{j}^{x}-\tilde{S}_{i}^{y}\tilde{S}_{j}^{y}\right). \tag*{(27)}
\end{aligned} H s 2 t 2 − t 2 = 16 ( U + 3 J H ) 1 { 4 [ t 3 2 t 4 2 + 2 t 1 2 t 2 2 + 2 1 t 4 4 + 2 1 t 3 4 ] A 2 − 2 1 ( t 2 2 − t 1 2 ) 2 B 2 } ( S i ⋅ S j ) − 16 ( U + 3 J H ) 1 [ ( t 1 2 + t 2 2 ) ( t 3 2 + t 4 2 ) − ( t 2 2 − t 1 2 ) 2 ] B 2 ( S ~ i z S ~ j z ) − 8 ( U + 3 J H ) 1 t 1 t 2 ( t 3 2 + t 4 2 ) B 2 ( S ~ i x S ~ j y + S ~ i y S ~ j x ) + 16 ( U + 3 J H ) 1 ( t 1 2 − t 2 2 ) ( t 3 2 + t 4 2 ) B 2 ( S ~ i x S ~ j x − S ~ i y S ~ j y ) . (27)
However, the above results are expressed in the local coordinate system; the spin operators must now be rotated back to the global coordinate system. Based on the previously defined rotation matrix and the bond-direction definitions in the Kitaev-like bond-dependent model, the following relations are obtained:
S α = cos θ S ~ x − sin θ S ~ y , S β = − sin θ S ~ x + cos θ S ~ y , S γ = S ~ z . (28) \begin{aligned}
S^{\alpha}&=\cos\theta\,\tilde{S}^{x}-\sin\theta\,\tilde{S}^{y},\\
S^{\beta}&=-\sin\theta\,\tilde{S}^{x}+\cos\theta\,\tilde{S}^{y},\\
S^{\gamma}&=\tilde{S}^{z}. \tag*{(28)}
\end{aligned} S α S β S γ = cos θ S ~ x − sin θ S ~ y , = − sin θ S ~ x + cos θ S ~ y , = S ~ z . (28)
where S α S^{\alpha} S α , S β S^{\beta} S β , and S γ S^{\gamma} S γ correspond to the bond directions on the Fe-Te-Fe(-Te) plane in the Kitaev-like model, as shown in Figure S3 . Through this transformation, the correspondence between the spin components in the local coordinate system and those in the global coordinate system is given by:
S ~ i z S ~ j z = S i γ S j γ , S ~ i x S ~ j y + S ~ i y S ~ j x = ( S i α S j β + S i β S j α ) + 1 3 S i ⋅ S j − 1 3 S i γ S j γ , (29) \widetilde{S}_{i}^{z}\widetilde{S}_{j}^{z}=S_{i}^{\gamma}S_{j}^{\gamma},\qquad\widetilde{S}_{i}^{x}\widetilde{S}_{j}^{y}+\widetilde{S}_{i}^{y}\widetilde{S}_{j}^{x}=\left(S_{i}^{\alpha}S_{j}^{\beta}+S_{i}^{\beta}S_{j}^{\alpha}\right)+\frac{1}{3}S_{i}\cdot S_{j}-\frac{1}{3}S_{i}^{\gamma}S_{j}^{\gamma}, \tag*{(29)} S i z S j z = S i γ S j γ , S i x S j y + S i y S j x = ( S i α S j β + S i β S j α ) + 3 1 S i ⋅ S j − 3 1 S i γ S j γ , (29)
where the Kitaev-like interaction term is extracted from S ~ i z S ~ j z \widetilde{S}_{i}^{z}\widetilde{S}_{j}^{z} S i z S j z , the off-diagonal interaction term is extracted from S ~ i x S ~ j y + S ~ i y S ~ j x \widetilde{S}_{i}^{x}\widetilde{S}_{j}^{y}+\widetilde{S}_{i}^{y}\widetilde{S}_{j}^{x} S i x S j y + S i y S j x , and these also contain partial Heisenberg and Kitaev-like interaction contributions. This demonstrates that the Heisenberg interaction in our Hamiltonian originates not only from spin-conserving processes but also receives contributions from spin-flip processes. This is also evident from the expression for the second-nearest neighbors in Eq. (27) , where the Heisenberg interaction arising from spin flips is not solely from the S ~ i x S ~ j y + S ~ i y S ~ j x \widetilde{S}_{i}^{x}\widetilde{S}_{j}^{y}+\widetilde{S}_{i}^{y}\widetilde{S}_{j}^{x} S i x S j y + S i y S j x term. For different planes, the results should be identical under the local coordinate system defined in Figure S3 , because the correspondence between x ~ , y ~ , z ~ \widetilde{x},\widetilde{y},\widetilde{z} x , y , z and α , β , γ \alpha,\beta,\gamma α , β , γ is the same under our coordinate selection. For the additional ( S ~ i x S ~ j x − S ~ i y S ~ j y ) (\widetilde{S}_{i}^{x}\widetilde{S}_{j}^{x}-\widetilde{S}_{i}^{y}\widetilde{S}_{j}^{y}) ( S i x S j x − S i y S j y ) term in the 2NN case, which is independent of the bond direction, this term vanishes upon transformation to the global coordinate system and summation over different coordinate selections, and thus does not appear in the total Hamiltonian.
Substituting Eq. (29) into Eqs. (26) and (S26) yields the Hamiltonian expressions in the global coordinate system. For the 1NN:
H s 1 t 2 − t 2 = 1 4 ( U + 3 J H ) { [ 8 t 1 2 t 2 2 + 4 t 3 2 t 4 2 + ( t 3 2 + t 4 2 ) 2 ] A 2 − 1 3 ( t 1 t 3 + t 2 t 4 ) ( t 1 t 4 + t 2 t 3 ) B 2 } ( S i ⋅ S j ) H_{s1}^{t_{2}-t_{2}}=\frac{1}{4(U+3J_{H})}\left\{\left[8t_{1}^{2}t_{2}^{2}+4t_{3}^{2}t_{4}^{2}+(t_{3}^{2}+t_{4}^{2})^{2}\right]A^{2}-\frac{1}{3}(t_{1}t_{3}+t_{2}t_{4})(t_{1}t_{4}+t_{2}t_{3})B^{2}\right\}(S_{i}\cdot S_{j}) H s 1 t 2 − t 2 = 4 ( U + 3 J H ) 1 { [ 8 t 1 2 t 2 2 + 4 t 3 2 t 4 2 + ( t 3 2 + t 4 2 ) 2 ] A 2 − 3 1 ( t 1 t 3 + t 2 t 4 ) ( t 1 t 4 + t 2 t 3 ) B 2 } ( S i ⋅ S j )
− 1 8 ( U + 3 J H ) [ ( t 1 t 3 + t 2 t 4 ) 2 + ( t 1 t 4 + t 2 t 3 ) 2 − 2 3 ( t 1 t 3 + t 2 t 4 ) ( t 1 t 4 + t 2 t 3 ) ] B 2 ( S i γ S j γ ) -\frac{1}{8(U+3J_{H})}\left[(t_{1}t_{3}+t_{2}t_{4})^{2}+(t_{1}t_{4}+t_{2}t_{3})^{2}-\frac{2}{3}(t_{1}t_{3}+t_{2}t_{4})(t_{1}t_{4}+t_{2}t_{3})\right]B^{2}(S_{i}^{\gamma}S_{j}^{\gamma}) − 8 ( U + 3 J H ) 1 [ ( t 1 t 3 + t 2 t 4 ) 2 + ( t 1 t 4 + t 2 t 3 ) 2 − 3 2 ( t 1 t 3 + t 2 t 4 ) ( t 1 t 4 + t 2 t 3 ) ] B 2 ( S i γ S j γ )
− 1 4 ( U + 3 J H ) ( t 1 t 3 + t 2 t 4 ) ( t 1 t 4 + t 2 t 3 ) B 2 ( S i α S j β + S i β S j α ) , (30) -\frac{1}{4(U+3J_{H})}(t_{1}t_{3}+t_{2}t_{4})(t_{1}t_{4}+t_{2}t_{3})B^{2}(S_{i}^{\alpha}S_{j}^{\beta}+S_{i}^{\beta}S_{j}^{\alpha}), \tag*{(30)} − 4 ( U + 3 J H ) 1 ( t 1 t 3 + t 2 t 4 ) ( t 1 t 4 + t 2 t 3 ) B 2 ( S i α S j β + S i β S j α ) , (30)
and for the second-nearest neighbors:
H s 2 t 2 − t 2 = 1 16 ( U + 3 J H ) { 2 [ 4 t 1 2 t 2 2 + ( t 3 2 + t 4 2 ) 2 ] A 2 − [ 1 2 ( t 2 2 − t 1 2 ) 2 + 2 3 t 1 t 2 ( t 3 2 + t 4 2 ) ] B 2 } ( S i ⋅ S j ) H_{s2}^{t_{2}-t_{2}}=\frac{1}{16(U+3J_{H})}\left\{2\left[4t_{1}^{2}t_{2}^{2}+(t_{3}^{2}+t_{4}^{2})^{2}\right]A^{2}-\left[\frac{1}{2}(t_{2}^{2}-t_{1}^{2})^{2}+\frac{2}{3}t_{1}t_{2}(t_{3}^{2}+t_{4}^{2})\right]B^{2}\right\}(S_{i}\cdot S_{j}) H s 2 t 2 − t 2 = 16 ( U + 3 J H ) 1 { 2 [ 4 t 1 2 t 2 2 + ( t 3 2 + t 4 2 ) 2 ] A 2 − [ 2 1 ( t 2 2 − t 1 2 ) 2 + 3 2 t 1 t 2 ( t 3 2 + t 4 2 ) ] B 2 } ( S i ⋅ S j )
− 1 16 ( U + 3 J H ) [ ( t 1 2 + t 2 2 ) ( t 3 2 + t 4 2 ) − ( t 2 2 − t 1 2 ) 2 − 2 3 t 1 t 2 ( t 3 2 + t 4 2 ) ] B 2 ( S i γ S j γ ) -\frac{1}{16(U+3J_{H})}\left[(t_{1}^{2}+t_{2}^{2})(t_{3}^{2}+t_{4}^{2})-(t_{2}^{2}-t_{1}^{2})^{2}-\frac{2}{3}t_{1}t_{2}(t_{3}^{2}+t_{4}^{2})\right]B^{2}(S_{i}^{\gamma}S_{j}^{\gamma}) − 16 ( U + 3 J H ) 1 [ ( t 1 2 + t 2 2 ) ( t 3 2 + t 4 2 ) − ( t 2 2 − t 1 2 ) 2 − 3 2 t 1 t 2 ( t 3 2 + t 4 2 ) ] B 2 ( S i γ S j γ )
− 1 8 ( U + 3 J H ) t 1 t 2 ( t 3 2 + t 4 2 ) 2 B 2 ( S i α S j β + S i β S j α ) . (31) -\frac{1}{8(U+3J_{H})}t_{1}t_{2}(t_{3}^{2}+t_{4}^{2})^{2}B^{2}(S_{i}^{\alpha}S_{j}^{\beta}+S_{i}^{\beta}S_{j}^{\alpha}). \tag*{(31)} − 8 ( U + 3 J H ) 1 t 1 t 2 ( t 3 2 + t 4 2 ) 2 B 2 ( S i α S j β + S i β S j α ) . (31)
Here, the Heisenberg interaction coefficient corresponds to the coefficient of the S i ⋅ S j S_{i}\cdot S_{j} S i ⋅ S j term, the Kitaev-like interaction coefficient corresponds to the coefficient of the S i γ S j γ S_{i}^{\gamma}S_{j}^{\gamma} S i γ S j γ term, and the off-diagonal interaction coefficient corresponds to the coefficient of the S i α S j β + S i β S j α S_{i}^{\alpha}S_{j}^{\beta}+S_{i}^{\beta}S_{j}^{\alpha} S i α S j β + S i β S j α term, clearly verifying the Kitaev-like bond-dependent model.
For the 2NN, inversion symmetry is broken, leading to the emergence of Dzyaloshinskii-Moriya (DM) interactions under SOC. In the t 2 t_{2} t 2 -t 2 t_{2} t 2 exchange, the DM interaction originates from the d x y d_{xy} d x y -d x y d_{xy} d x y coupling. Due to the symmetry breaking between Fe1 _1 1 and Fe2 _2 2 , the Pauli matrix combinations [ σ ~ 0 ] σ ~ 4 σ ~ 3 [ σ ~ γ ] σ ~ 2 σ ~ 1 [\widetilde{\sigma}_{0}]_{\widetilde{\sigma}_{4}\widetilde{\sigma}_{3}}[\widetilde{\sigma}_{\gamma}]_{\widetilde{\sigma}_{2}\widetilde{\sigma}_{1}} [ σ 0 ] σ 4 σ 3 [ σ γ ] σ 2 σ 1 and [ σ ~ γ ] σ ~ 4 σ ~ 3 [ σ ~ 0 ] σ ~ 2 σ ~ 1 [\widetilde{\sigma}_{\gamma}]_{\widetilde{\sigma}_{4}\widetilde{\sigma}_{3}}[\widetilde{\sigma}_{0}]_{\widetilde{\sigma}_{2}\widetilde{\sigma}_{1}} [ σ γ ] σ 4 σ 3 [ σ 0 ] σ 2 σ 1 do not cancel across different pathways but are instead retained. These combinations correspond to the DM interaction on different planes, namely:
[ σ ~ 0 ] σ ~ 4 σ ~ 3 [ σ ~ x ] σ ~ 2 σ ~ 1 ( [ σ ~ ] σ ~ 1 σ ~ 4 ⋅ s ~ i , α ) ( [ σ ~ ] σ ~ 3 σ ~ 2 ⋅ s ~ j , β ) = i 8 ( S ~ i y S ~ j z − S ~ i z S ~ j y ) , [\widetilde{\sigma}_{0}]_{\widetilde{\sigma}_{4}\widetilde{\sigma}_{3}}[\widetilde{\sigma}_{x}]_{\widetilde{\sigma}_{2}\widetilde{\sigma}_{1}}([\widetilde{\sigma}]_{\widetilde{\sigma}_{1}\widetilde{\sigma}_{4}}\cdot\widetilde{s}_{i,\alpha})([\widetilde{\sigma}]_{\widetilde{\sigma}_{3}\widetilde{\sigma}_{2}}\cdot\widetilde{s}_{j,\beta})=\frac{i}{8}(\widetilde{S}_{i}^{y}\widetilde{S}_{j}^{z}-\widetilde{S}_{i}^{z}\widetilde{S}_{j}^{y}), [ σ 0 ] σ 4 σ 3 [ σ x ] σ 2 σ 1 ([ σ ] σ 1 σ 4 ⋅ s i , α ) ([ σ ] σ 3 σ 2 ⋅ s j , β ) = 8 i ( S i y S j z − S i z S j y ) ,
[ σ ~ 0 ] σ ~ 4 σ ~ 3 [ σ ~ y ] σ ~ 2 σ ~ 1 ( [ σ ~ ] σ ~ 1 σ ~ 4 ⋅ s ~ i , α ) ( [ σ ~ ] σ ~ 3 σ ~ 2 ⋅ s ~ j , β ) = i 8 ( S ~ i z S ~ j x − S ~ i x S ~ j z ) , [\widetilde{\sigma}_{0}]_{\widetilde{\sigma}_{4}\widetilde{\sigma}_{3}}[\widetilde{\sigma}_{y}]_{\widetilde{\sigma}_{2}\widetilde{\sigma}_{1}}([\widetilde{\sigma}]_{\widetilde{\sigma}_{1}\widetilde{\sigma}_{4}}\cdot\widetilde{s}_{i,\alpha})([\widetilde{\sigma}]_{\widetilde{\sigma}_{3}\widetilde{\sigma}_{2}}\cdot\widetilde{s}_{j,\beta})=\frac{i}{8}(\widetilde{S}_{i}^{z}\widetilde{S}_{j}^{x}-\widetilde{S}_{i}^{x}\widetilde{S}_{j}^{z}), [ σ 0 ] σ 4 σ 3 [ σ y ] σ 2 σ 1 ([ σ ] σ 1 σ 4 ⋅ s i , α ) ([ σ ] σ 3 σ 2 ⋅ s j , β ) = 8 i ( S i z S j x − S i x S j z ) ,
[ σ ~ 0 ] σ ~ 4 σ ~ 3 [ σ ~ z ] σ ~ 2 σ ~ 1 ( [ σ ~ ] σ ~ 1 σ ~ 4 ⋅ s ~ i , α ) ( [ σ ~ ] σ ~ 3 σ ~ 2 ⋅ s ~ j , β ) = i 8 ( S ~ i x S ~ j y − S ~ i y S ~ j x ) , (32) [\widetilde{\sigma}_{0}]_{\widetilde{\sigma}_{4}\widetilde{\sigma}_{3}}[\widetilde{\sigma}_{z}]_{\widetilde{\sigma}_{2}\widetilde{\sigma}_{1}}([\widetilde{\sigma}]_{\widetilde{\sigma}_{1}\widetilde{\sigma}_{4}}\cdot\widetilde{s}_{i,\alpha})([\widetilde{\sigma}]_{\widetilde{\sigma}_{3}\widetilde{\sigma}_{2}}\cdot\widetilde{s}_{j,\beta})=\frac{i}{8}(\widetilde{S}_{i}^{x}\widetilde{S}_{j}^{y}-\widetilde{S}_{i}^{y}\widetilde{S}_{j}^{x}), \tag*{(32)} [ σ 0 ] σ 4 σ 3 [ σ z ] σ 2 σ 1 ([ σ ] σ 1 σ 4 ⋅ s i , α ) ([ σ ] σ 3 σ 2 ⋅ s j , β ) = 8 i ( S i x S j y − S i y S j x ) , (32)
where [ σ ~ 0 ] σ ~ 4 σ ~ 3 [ σ ~ y ] σ ~ 2 σ ~ 1 [\widetilde{\sigma}_{0}]_{\widetilde{\sigma}_{4}\widetilde{\sigma}_{3}}[\widetilde{\sigma}_{y}]_{\widetilde{\sigma}_{2}\widetilde{\sigma}_{1}} [ σ 0 ] σ 4 σ 3 [ σ y ] σ 2 σ 1 and [ σ ~ y ] σ ~ 4 σ ~ 3 [ σ ~ 0 ] σ ~ 2 σ ~ 1 [\widetilde{\sigma}_{y}]_{\widetilde{\sigma}_{4}\widetilde{\sigma}_{3}}[\widetilde{\sigma}_{0}]_{\widetilde{\sigma}_{2}\widetilde{\sigma}_{1}} [ σ y ] σ 4 σ 3 [ σ 0 ] σ 2 σ 1 yield opposite results. The result in the local coordinate system is:
H D = − 1 4 ( U + 3 J H ) t 1 t 2 ( t 2 2 − t 1 2 ) A B ( S ~ i x S ~ j y − S ~ i y S ~ j x ) , (33) H_{D}=-\frac{1}{4(U+3J_{H})}t_{1}t_{2}(t_{2}^{2}-t_{1}^{2})AB(\widetilde{S}_{i}^{x}\widetilde{S}_{j}^{y}-\widetilde{S}_{i}^{y}\widetilde{S}_{j}^{x}), \tag*{(33)} H D = − 4 ( U + 3 J H ) 1 t 1 t 2 ( t 2 2 − t 1 2 ) A B ( S i x S j y − S i y S j x ) , (33)
which corresponds to the following result in the global coordinate system:
H D = − 1 4 ( U + 3 J H ) t 1 t 2 ( t 2 2 − t 1 2 ) [ ( z × d i j ) ⋅ ( S i × S j ) ] . (34) H_{D}=-\frac{1}{4(U+3J_{H})}t_{1}t_{2}(t_{2}^{2}-t_{1}^{2})[(\mathbf{z}\times\mathbf{d}_{ij})\cdot(S_{i}\times S_{j})]. \tag*{(34)} H D = − 4 ( U + 3 J H ) 1 t 1 t 2 ( t 2 2 − t 1 2 ) [( z × d ij ) ⋅ ( S i × S j )] . (34)
In the global Hamiltonian, this is expressed as D i j = D 2 ( z × d i j ) D_{ij}=D_{2}(\mathbf{z}\times\mathbf{d}_{ij}) D ij = D 2 ( z × d ij ) , where D 2 D_{2} D 2 denotes the magnitude of the DM interaction parameter for the 2NN, z \mathbf{z} z is the unit vector pointing from the Te atom toward the Fe layer, and d i j \mathbf{d}_{ij} d ij is the unit vector pointing from Fei _i i toward Fej _j j .
t2–e superexchange process
We next consider the hopping process in which a hole moves from the t 2 t_{2} t 2 orbital of Fe1 _1 1 to the e e e orbital of Fe2 _2 2 . In this case, however, an orbital degeneracy arises. In the ideal tetrahedral lattice considered here, the d z 2 d_{z^{2}} d z 2 (d 3 z 2 − r 2 d_{3z^{2}-r^{2}} d 3 z 2 − r 2 ) and d x 2 − y 2 d_{x^{2}-y^{2}} d x 2 − y 2 orbitals in the e e e manifold are degenerate. Therefore, in the hole representation d 4 ( t 2 g 3 e 1 ) d^{4}(t_{2g}^{3}e^{1}) d 4 ( t 2 g 3 e 1 ) , the single hole occupying the e e e level retains an orbital degree of freedom. To account for this degree of freedom, we introduce an independent orbital pseudospin τ \tau τ . The occupations of the d z 2 d_{z^{2}} d z 2 and d x 2 − y 2 d_{x^{2}-y^{2}} d x 2 − y 2 orbitals can be written as
∣ d z 2 ⟩ = ∣ τ z = 1 2 ⟩ , ∣ d x 2 − y 2 ⟩ = ∣ τ z = − 1 2 ⟩ , τ z = 1 2 ( n z − n x ) , (35) |d_{z^{2}}\rangle=|\tau^{z}=\tfrac{1}{2}\rangle,\qquad|d_{x^{2}-y^{2}}\rangle=|\tau^{z}=-\tfrac{1}{2}\rangle,\qquad\tau^{z}=\tfrac{1}{2}(n_{z}-n_{x}), \tag*{(35)} ∣ d z 2 ⟩ = ∣ τ z = 2 1 ⟩ , ∣ d x 2 − y 2 ⟩ = ∣ τ z = − 2 1 ⟩ , τ z = 2 1 ( n z − n x ) , (35)
where τ z \tau^{z} τ z denotes the z z z component of the orbital pseudospin, and n z n_{z} n z and n x n_{x} n x are the occupation numbers of the d z 2 d_{z^{2}} d z 2 and d x 2 − y 2 d_{x^{2}-y^{2}} d x 2 − y 2 orbitals, respectively. The orbital occupation can then be identified from the following pseudospin states:
∣ d z 2 ⟩ ⟨ d z 2 ∣ = 1 2 + τ z , |d_{z^{2}}\rangle\langle d_{z^{2}}|=\tfrac{1}{2}+\tau^{z}, ∣ d z 2 ⟩ ⟨ d z 2 ∣ = 2 1 + τ z ,
∣ d x 2 − y 2 ⟩ ⟨ d x 2 − y 2 ∣ = 1 2 − τ z . (36) |d_{x^{2}-y^{2}}\rangle\langle d_{x^{2}-y^{2}}|=\tfrac{1}{2}-\tau^{z}. \tag*{(36)} ∣ d x 2 − y 2 ⟩ ⟨ d x 2 − y 2 ∣ = 2 1 − τ z . (36)
In the t 2 t_{2} t 2 -e e e superexchange process, several intermediate states appear. We briefly discuss them within the Kanamori Hamiltonian. There are three relevant classes of intermediate states. In the first case, a hole hops from the t 2 t_{2} t 2 orbital of Fe1 _1 1 to an empty e e e orbital of Fe2 _2 2 , with its spin parallel to the spin already present on Fe2 _2 2 . The corresponding energy difference between the intermediate state and the ground state is Δ E t 2 − e 1 = U ′ − J H + Δ c \Delta E^{1}_{t_{2}-e}=U^{\prime}-J_{\mathrm{H}}+\Delta_{c} Δ E t 2 − e 1 = U ′ − J H + Δ c . In the second case, the hole hops from the t 2 t_{2} t 2 orbital of Fe1 _1 1 to an already occupied e e e orbital of Fe2 _2 2 . The e e e manifold then contains one doubly occupied orbital and one empty orbital, so that the pair-hopping term in the Kanamori interaction becomes relevant. The corresponding intermediate-state energies are Δ E t 2 − e 2 = U + 4 J H + Δ c \Delta E^{2}_{t_{2}-e}=U+4J_{\mathrm{H}}+\Delta_{c} Δ E t 2 − e 2 = U + 4 J H + Δ c , Δ E t 2 − e 2 ′ = U + 2 J H + Δ c \Delta E^{2\prime}_{t_{2}-e}=U+2J_{\mathrm{H}}+\Delta_{c} Δ E t 2 − e 2′ = U + 2 J H + Δ c , which correspond to the antisymmetric and symmetric orbital states, respectively, each carrying a weight of 1 / 2 1/2 1/2 . In the third case, the hole hops from the t 2 t_{2} t 2 orbital of Fe1 _1 1 to an empty e e e orbital of Fe2 _2 2 , but with spin antiparallel to the original spin. This gives rise to high-spin and low-spin intermediate multiplets with energies Δ E t 2 − e 3 = U ′ − J H + Δ c \Delta E^{3}_{t_{2}-e}=U^{\prime}-J_{\mathrm{H}}+\Delta_{c} Δ E t 2 − e 3 = U ′ − J H + Δ c , Δ E t 2 − e 3 ′ = U ′ + 4 J H + Δ c \Delta E^{3\prime}_{t_{2}-e}=U^{\prime}+4J_{\mathrm{H}}+\Delta_{c} Δ E t 2 − e 3′ = U ′ + 4 J H + Δ c , with weights 1 / 5 1/5 1/5 and 4 / 5 4/5 4/5 , respectively.
Since the evaluation of the t 2 t_{2} t 2 -e e e superexchange processes proceeds in close analogy to the t 2 t_{2} t 2 -t 2 t_{2} t 2 case, apart from the inclusion of the orbital pseudospin and the modified intermediate-state energies, we do not repeat the detailed derivation here. Representative hopping paths are shown in Figure S5 . Below we present only the resulting effective hopping integrals and the corresponding effective spin Hamiltonians.
Figure S5. Schematic of the t 2 t_{2} t 2 -e e e virtual hopping processes in the hole representation. Brown and dark-gray spheres denote Fe and Te atoms, respectively. Only representative virtual hopping paths that contribute to the 1NN t 2 t_{2} t 2 -e e e effective hopping integrals are shown.
The 1NN effective hopping integrals for the t 2 − e t_{2}-e t 2 − e channel are
T M 2 M 1 e f f ( t 2 ⊗ e ) = ( i ( t 1 t 5 − t 2 t 6 ) B σ ~ z − 2 ( t 1 t 8 + t 2 t 7 ) A σ ~ 0 i 1 2 ( t 3 t 6 − t 4 t 5 ) B σ ~ y + i 1 2 ( t 4 t 6 − t 3 t 5 ) B σ ~ x − i 1 2 ( t 3 t 8 + t 4 t 7 ) B σ ~ y + i 1 2 ( t 3 t 7 + t 4 t 8 ) B σ ~ x i 1 2 ( t 4 t 6 − t 3 t 5 ) B σ ~ y + i 1 2 ( t 3 t 6 − t 4 t 5 ) B σ ~ x i 1 2 ( t 3 t 8 + t 4 t 7 ) B σ ~ x − i 1 2 ( t 3 t 7 + t 4 t 8 ) B σ ~ y ) . (37) T^{\mathrm{eff}}_{M_{2}M_{1}}(t_{2}\otimes e)=
\begin{pmatrix}
i(t_{1}t_{5}-t_{2}t_{6})B\widetilde{\sigma}_{z} & -2(t_{1}t_{8}+t_{2}t_{7})A\widetilde{\sigma}_{0}\\
i\frac{1}{2}(t_{3}t_{6}-t_{4}t_{5})B\widetilde{\sigma}_{y}+i\frac{1}{2}(t_{4}t_{6}-t_{3}t_{5})B\widetilde{\sigma}_{x} & -i\frac{1}{2}(t_{3}t_{8}+t_{4}t_{7})B\widetilde{\sigma}_{y}+i\frac{1}{2}(t_{3}t_{7}+t_{4}t_{8})B\widetilde{\sigma}_{x}\\
i\frac{1}{2}(t_{4}t_{6}-t_{3}t_{5})B\widetilde{\sigma}_{y}+i\frac{1}{2}(t_{3}t_{6}-t_{4}t_{5})B\widetilde{\sigma}_{x} & i\frac{1}{2}(t_{3}t_{8}+t_{4}t_{7})B\widetilde{\sigma}_{x}-i\frac{1}{2}(t_{3}t_{7}+t_{4}t_{8})B\widetilde{\sigma}_{y}
\end{pmatrix}. \tag*{(37)} T M 2 M 1 eff ( t 2 ⊗ e ) = i ( t 1 t 5 − t 2 t 6 ) B σ z i 2 1 ( t 3 t 6 − t 4 t 5 ) B σ y + i 2 1 ( t 4 t 6 − t 3 t 5 ) B σ x i 2 1 ( t 4 t 6 − t 3 t 5 ) B σ y + i 2 1 ( t 3 t 6 − t 4 t 5 ) B σ x − 2 ( t 1 t 8 + t 2 t 7 ) A σ 0 − i 2 1 ( t 3 t 8 + t 4 t 7 ) B σ y + i 2 1 ( t 3 t 7 + t 4 t 8 ) B σ x i 2 1 ( t 3 t 8 + t 4 t 7 ) B σ x − i 2 1 ( t 3 t 7 + t 4 t 8 ) B σ y . (37)
The 2NN effective hopping integrals for the t 2 t_{2} t 2 -e e e channel are
T M 2 ′ M 1 e f f ( t 2 ⊗ e ) = ( ( t 2 t 5 + t 1 t 6 ) A σ ~ 0 + i 1 2 ( t 2 t 6 − t 1 t 5 ) B σ ~ z ( t 2 t 7 + t 1 t 8 ) A σ ~ 0 + i 1 2 ( t 2 t 8 − t 1 t 7 ) B σ ~ z i 1 2 t 4 t 5 B σ ~ y − i 1 2 t 4 t 6 B σ ~ x i 1 2 t 4 t 7 B σ ~ y − i 1 2 t 4 t 8 B σ ~ x i 1 2 t 3 t 5 B σ ~ y − i 1 2 t 3 t 6 B σ ~ x i 1 2 t 3 t 7 B σ ~ y − i 1 2 t 3 t 8 B σ ~ x ) . (38) T^{\mathrm{eff}}_{M_{2}^{\prime}M_{1}}(t_{2}\otimes e)=
\begin{pmatrix}
(t_{2}t_{5}+t_{1}t_{6})A\widetilde{\sigma}_{0}+i\frac{1}{2}(t_{2}t_{6}-t_{1}t_{5})B\widetilde{\sigma}_{z} & (t_{2}t_{7}+t_{1}t_{8})A\widetilde{\sigma}_{0}+i\frac{1}{2}(t_{2}t_{8}-t_{1}t_{7})B\widetilde{\sigma}_{z}\\
i\frac{1}{2}t_{4}t_{5}B\widetilde{\sigma}_{y}-i\frac{1}{2}t_{4}t_{6}B\widetilde{\sigma}_{x} & i\frac{1}{2}t_{4}t_{7}B\widetilde{\sigma}_{y}-i\frac{1}{2}t_{4}t_{8}B\widetilde{\sigma}_{x}\\
i\frac{1}{2}t_{3}t_{5}B\widetilde{\sigma}_{y}-i\frac{1}{2}t_{3}t_{6}B\widetilde{\sigma}_{x} & i\frac{1}{2}t_{3}t_{7}B\widetilde{\sigma}_{y}-i\frac{1}{2}t_{3}t_{8}B\widetilde{\sigma}_{x}
\end{pmatrix}. \tag*{(38)} T M 2 ′ M 1 eff ( t 2 ⊗ e ) = ( t 2 t 5 + t 1 t 6 ) A σ 0 + i 2 1 ( t 2 t 6 − t 1 t 5 ) B σ z i 2 1 t 4 t 5 B σ y − i 2 1 t 4 t 6 B σ x i 2 1 t 3 t 5 B σ y − i 2 1 t 3 t 6 B σ x ( t 2 t 7 + t 1 t 8 ) A σ 0 + i 2 1 ( t 2 t 8 − t 1 t 7 ) B σ z i 2 1 t 4 t 7 B σ y − i 2 1 t 4 t 8 B σ x i 2 1 t 3 t 7 B σ y − i 2 1 t 3 t 8 B σ x . (38)
Substituting these effective hopping integrals into the superexchange expression gives the effective spin Hamiltonian in the local coordinate frame. After rotating back to the global coordinate frame, the 1NN effective spin Hamiltonian becomes
H s 1 t 2 − e = 1 8 { [ 1 3 ( t 3 t 6 − t 4 t 5 ) ( t 4 t 6 − t 3 t 5 ) − ( t 1 t 5 − t 2 t 6 ) ] B 2 Δ t 2 − e 2 + [ 4 ( t 1 t 8 + t 2 t 7 ) A 2 − 1 3 ( t 3 t 8 + t 4 t 7 ) ( t 3 t 7 + t 4 t 8 ) B 2 ] Δ t 2 − e 2 } ( S i ⋅ S j ) + 1 16 { [ 2 ( t 1 t 5 − t 2 t 6 ) 2 − ( t 3 t 6 − t 4 t 5 ) 2 − ( t 4 t 6 − t 3 t 5 ) 2 + 2 3 ( t 3 t 6 − t 4 t 5 ) ( t 4 t 6 − t 3 t 5 ) ] Δ t 2 − e 1 − [ ( t 3 t 8 + t 4 t 7 ) 2 + ( t 3 t 7 + t 4 t 8 ) 2 − 2 3 ( t 3 t 8 + t 4 t 7 ) ( t 3 t 7 + t 4 t 8 ) ] Δ t 2 − e 2 } B 2 S i γ S j γ + 1 8 [ ( t 3 t 6 − t 4 t 5 ) ( t 4 t 6 − t 3 t 5 ) Δ t 2 − e 1 − ( t 3 t 8 + t 4 t 7 ) ( t 3 t 7 + t 4 t 8 ) Δ t 2 − e 2 ] B 2 ( S i α S j β + S i β S j α ) . (39) \begin{aligned}
H_{s1}^{t_{2}-e} = \frac{1}{8}\left\{\left[\frac{1}{3}(t_{3}t_{6}-t_{4}t_{5})(t_{4}t_{6}-t_{3}t_{5})-(t_{1}t_{5}-t_{2}t_{6})\right]B^{2}\Delta_{t_{2}-e}^{2}
+\left[4(t_{1}t_{8}+t_{2}t_{7})A^{2}-\frac{1}{3}(t_{3}t_{8}+t_{4}t_{7})(t_{3}t_{7}+t_{4}t_{8})B^{2}\right]\Delta_{t_{2}-e}^{2}\right\}(\mathbf{S}_{i}\cdot\mathbf{S}_{j}) \\
&+\frac{1}{16}\left\{\left[2(t_{1}t_{5}-t_{2}t_{6})^{2}-(t_{3}t_{6}-t_{4}t_{5})^{2}-(t_{4}t_{6}-t_{3}t_{5})^{2}+\frac{2}{3}(t_{3}t_{6}-t_{4}t_{5})(t_{4}t_{6}-t_{3}t_{5})\right]\Delta_{t_{2}-e}^{1}
-\left[(t_{3}t_{8}+t_{4}t_{7})^{2}+(t_{3}t_{7}+t_{4}t_{8})^{2}-\frac{2}{3}(t_{3}t_{8}+t_{4}t_{7})(t_{3}t_{7}+t_{4}t_{8})\right]\Delta_{t_{2}-e}^{2}\right\}B^{2}S_{i}^{\gamma}S_{j}^{\gamma} \\
&+\frac{1}{8}\left[(t_{3}t_{6}-t_{4}t_{5})(t_{4}t_{6}-t_{3}t_{5})\Delta_{t_{2}-e}^{1}-(t_{3}t_{8}+t_{4}t_{7})(t_{3}t_{7}+t_{4}t_{8})\Delta_{t_{2}-e}^{2}\right]B^{2}(S_{i}^{\alpha}S_{j}^{\beta}+S_{i}^{\beta}S_{j}^{\alpha}). \tag*{(39)}
\end{aligned} H s 1 t 2 − e = 8 1 { [ 3 1 ( t 3 t 6 − t 4 t 5 ) ( t 4 t 6 − t 3 t 5 ) − ( t 1 t 5 − t 2 t 6 ) ] B 2 Δ t 2 − e 2 + [ 4 ( t 1 t 8 + t 2 t 7 ) A 2 − 3 1 ( t 3 t 8 + t 4 t 7 ) ( t 3 t 7 + t 4 t 8 ) B 2 ] Δ t 2 − e 2 } ( S i ⋅ S j ) + 16 1 { [ 2 ( t 1 t 5 − t 2 t 6 ) 2 − ( t 3 t 6 − t 4 t 5 ) 2 − ( t 4 t 6 − t 3 t 5 ) 2 + 3 2 ( t 3 t 6 − t 4 t 5 ) ( t 4 t 6 − t 3 t 5 ) ] Δ t 2 − e 1 − [ ( t 3 t 8 + t 4 t 7 ) 2 + ( t 3 t 7 + t 4 t 8 ) 2 − 3 2 ( t 3 t 8 + t 4 t 7 ) ( t 3 t 7 + t 4 t 8 ) ] Δ t 2 − e 2 } B 2 S i γ S j γ + 8 1 [ ( t 3 t 6 − t 4 t 5 ) ( t 4 t 6 − t 3 t 5 ) Δ t 2 − e 1 − ( t 3 t 8 + t 4 t 7 ) ( t 3 t 7 + t 4 t 8 ) Δ t 2 − e 2 ] B 2 ( S i α S j β + S i β S j α ) . (39)
The corresponding 2NN effective spin Hamiltonian is
H s 2 t 2 − e = 1 16 { [ 2 ( t 2 t 5 + t 1 t 6 ) 2 A 2 − 1 2 ( t 2 t 6 − t 1 t 5 ) B 2 − 1 3 ( t 3 2 + t 4 2 ) t 5 t 6 B 2 ] Δ t 2 − e 1 + [ 2 ( t 2 t 7 + t 1 t 8 ) 2 A 2 − 1 2 ( t 2 t 8 − t 1 t 7 ) B 2 − 1 3 ( t 3 2 + t 4 2 ) t 7 t 8 B 2 ] Δ t 2 − e 2 } ( S i ⋅ S j ) + 1 16 { [ ( t 2 t 6 − t 1 t 5 ) 2 − 1 2 ( t 3 2 + t 4 2 ) ( t 5 2 + t 6 2 ) + 1 3 ( t 3 2 + t 4 2 ) t 5 t 6 ] B 2 Δ t 2 − e 1 + [ ( t 2 t 8 − t 1 t 7 ) 2 − 1 2 ( t 3 2 + t 4 2 ) ( t 7 2 + t 8 2 ) + 1 3 ( t 3 2 + t 4 2 ) t 7 t 8 ] B 2 Δ t 2 − e 2 } S i γ S j γ (40) \begin{aligned}
H_{s2}^{t_{2}-e}=\frac{1}{16}\left\{\left[2(t_{2}t_{5}+t_{1}t_{6})^{2}A^{2}-\frac{1}{2}(t_{2}t_{6}-t_{1}t_{5})B^{2}-\frac{1}{3}(t_{3}^{2}+t_{4}^{2})t_{5}t_{6}B^{2}\right]\Delta_{t_{2}-e}^{1}
+\left[2(t_{2}t_{7}+t_{1}t_{8})^{2}A^{2}-\frac{1}{2}(t_{2}t_{8}-t_{1}t_{7})B^{2}-\frac{1}{3}(t_{3}^{2}+t_{4}^{2})t_{7}t_{8}B^{2}\right]\Delta_{t_{2}-e}^{2}\right\}(\mathbf{S}_{i}\cdot\mathbf{S}_{j}) \\
&+\frac{1}{16}\left\{\left[(t_{2}t_{6}-t_{1}t_{5})^{2}-\frac{1}{2}(t_{3}^{2}+t_{4}^{2})(t_{5}^{2}+t_{6}^{2})+\frac{1}{3}(t_{3}^{2}+t_{4}^{2})t_{5}t_{6}\right]B^{2}\Delta_{t_{2}-e}^{1}
+\left[(t_{2}t_{8}-t_{1}t_{7})^{2}-\frac{1}{2}(t_{3}^{2}+t_{4}^{2})(t_{7}^{2}+t_{8}^{2})+\frac{1}{3}(t_{3}^{2}+t_{4}^{2})t_{7}t_{8}\right]B^{2}\Delta_{t_{2}-e}^{2}\right\}S_{i}^{\gamma}S_{j}^{\gamma} \tag*{(40)}
\end{aligned} H s 2 t 2 − e = 16 1 { [ 2 ( t 2 t 5 + t 1 t 6 ) 2 A 2 − 2 1 ( t 2 t 6 − t 1 t 5 ) B 2 − 3 1 ( t 3 2 + t 4 2 ) t 5 t 6 B 2 ] Δ t 2 − e 1 + [ 2 ( t 2 t 7 + t 1 t 8 ) 2 A 2 − 2 1 ( t 2 t 8 − t 1 t 7 ) B 2 − 3 1 ( t 3 2 + t 4 2 ) t 7 t 8 B 2 ] Δ t 2 − e 2 } ( S i ⋅ S j ) + 16 1 { [ ( t 2 t 6 − t 1 t 5 ) 2 − 2 1 ( t 3 2 + t 4 2 ) ( t 5 2 + t 6 2 ) + 3 1 ( t 3 2 + t 4 2 ) t 5 t 6 ] B 2 Δ t 2 − e 1 + [ ( t 2 t 8 − t 1 t 7 ) 2 − 2 1 ( t 3 2 + t 4 2 ) ( t 7 2 + t 8 2 ) + 3 1 ( t 3 2 + t 4 2 ) t 7 t 8 ] B 2 Δ t 2 − e 2 } S i γ S j γ (40)
− 1 16 ( t 3 2 + t 4 2 ) ( t 5 t 6 Δ t 2 − e 1 + t 7 t 8 Δ t 2 − e 2 ) ( S i α S j β + S i β S j α ) -\frac{1}{16}(t_{3}^{2}+t_{4}^{2})(t_{5}t_{6}\Delta_{t_{2}-e}^{1}+t_{7}t_{8}\Delta_{t_{2}-e}^{2})(S_{i}^{\alpha}S_{j}^{\beta}+S_{i}^{\beta}S_{j}^{\alpha}) − 16 1 ( t 3 2 + t 4 2 ) ( t 5 t 6 Δ t 2 − e 1 + t 7 t 8 Δ t 2 − e 2 ) ( S i α S j β + S i β S j α )
− 1 8 [ ( t 2 t 5 + t 1 t 6 ) ( t 2 t 6 − t 1 t 5 ) Δ t 2 − e 1 + ( t 2 t 7 + t 1 t 8 ) ( t 2 t 8 − t 1 t 7 ) Δ t 2 − e 2 ] A B [ ( z × d i j ) ⋅ ( S i × S j ) ] . -\frac{1}{8}\left[(t_{2}t_{5}+t_{1}t_{6})(t_{2}t_{6}-t_{1}t_{5})\Delta_{t_{2}-e}^{1}+(t_{2}t_{7}+t_{1}t_{8})(t_{2}t_{8}-t_{1}t_{7})\Delta_{t_{2}-e}^{2}\right]AB\left[(\mathbf{z}\times\mathbf{d}_{ij})\cdot(\mathbf{S}_{i}\times\mathbf{S}_{j})\right]. − 8 1 [ ( t 2 t 5 + t 1 t 6 ) ( t 2 t 6 − t 1 t 5 ) Δ t 2 − e 1 + ( t 2 t 7 + t 1 t 8 ) ( t 2 t 8 − t 1 t 7 ) Δ t 2 − e 2 ] A B [ ( z × d ij ) ⋅ ( S i × S j ) ] .
In Eqs. (39) and (S39), we have incorporated the orbital degree of freedom and the multiple intermediate states into Δ t 2 − e 1 \Delta^{1}_{t_{2}-e} Δ t 2 − e 1 and Δ t 2 − e 2 \Delta^{2}_{t_{2}-e} Δ t 2 − e 2 . Their explicit forms are
Δ t 2 − e 1 = 1 2 + τ j z U ′ − J H + Δ c + ( 1 2 − τ j z ) ( U + 3 J H + Δ c ) ( U + 4 J H + Δ c ) ( U + 2 J H + Δ c ) + ( 1 2 − τ j z ) ( U ′ + Δ c ) ( U ′ + 4 J H + Δ c ) ( U ′ − J H + Δ c ) , Δ t 2 − e 2 = 1 2 − τ j z U ′ − J H + Δ c + ( 1 2 + τ j z ) ( U + 3 J H + Δ c ) ( U + 4 J H + Δ c ) ( U + 2 J H + Δ c ) + ( 1 2 − τ j z ) ( U ′ + Δ c ) ( U ′ + 4 J H + Δ c ) ( U ′ − J H + Δ c ) . (41) \begin{aligned}
\Delta^{1}_{t_{2}-e}={}&\frac{\frac{1}{2}+\tau^{z}_{j}}{U'-J_{H}+\Delta_{c}}+\frac{\left(\frac{1}{2}-\tau^{z}_{j}\right)(U+3J_{H}+\Delta_{c})}{(U+4J_{H}+\Delta_{c})(U+2J_{H}+\Delta_{c})}+\frac{\left(\frac{1}{2}-\tau^{z}_{j}\right)(U'+\Delta_{c})}{(U'+4J_{H}+\Delta_{c})(U'-J_{H}+\Delta_{c})},\\
\Delta^{2}_{t_{2}-e}={}&\frac{\frac{1}{2}-\tau^{z}_{j}}{U'-J_{H}+\Delta_{c}}+\frac{\left(\frac{1}{2}+\tau^{z}_{j}\right)(U+3J_{H}+\Delta_{c})}{(U+4J_{H}+\Delta_{c})(U+2J_{H}+\Delta_{c})}+\frac{\left(\frac{1}{2}-\tau^{z}_{j}\right)(U'+\Delta_{c})}{(U'+4J_{H}+\Delta_{c})(U'-J_{H}+\Delta_{c})}. \tag*{(41)}
\end{aligned} Δ t 2 − e 1 = Δ t 2 − e 2 = U ′ − J H + Δ c 2 1 + τ j z + ( U + 4 J H + Δ c ) ( U + 2 J H + Δ c ) ( 2 1 − τ j z ) ( U + 3 J H + Δ c ) + ( U ′ + 4 J H + Δ c ) ( U ′ − J H + Δ c ) ( 2 1 − τ j z ) ( U ′ + Δ c ) , U ′ − J H + Δ c 2 1 − τ j z + ( U + 4 J H + Δ c ) ( U + 2 J H + Δ c ) ( 2 1 + τ j z ) ( U + 3 J H + Δ c ) + ( U ′ + 4 J H + Δ c ) ( U ′ − J H + Δ c ) ( 2 1 − τ j z ) ( U ′ + Δ c ) . (41)
The expressions above give the effective hopping integrals and effective spin Hamiltonians arising from the t 2 t_{2} t 2 -e e e superexchange processes in the hole representation. They show that the t 2 t_{2} t 2 -e e e channel contributes not only to the Heisenberg interaction, but also to the Kitaev-like interaction, the off-diagonal Γ \Gamma Γ interaction, and the DM interaction.
e-t2 superexchange process
We next consider the hole hopping process from the e e e orbital of Fe1 _1 1 to the t 2 t_{2} t 2 orbital of Fe2 _2 2 . The orbital degree of freedom must again be taken into account. In this case, however, only a single intermediate state exists: the hole in the e e e orbital of Fe1 _1 1 hops to the t 2 t_{2} t 2 orbital of Fe2 _2 2 . The energy difference between this intermediate state and the ground state is Δ E e − t 2 = U + 3 J H − Δ c \Delta E_{e-t_{2}}=U+3J_{H}-\Delta_{c} Δ E e − t 2 = U + 3 J H − Δ c . We again omit the detailed derivation; representative virtual hopping paths are shown in Figure S5 , and only the resulting effective hopping integrals and effective spin Hamiltonians are presented below. The 1NN effective hopping integral for the e e e -t 2 t_{2} t 2 channel is
T M 2 M 1 e f f ( e ⊗ t 2 ) = ( i ( t 2 t 6 − t 1 t 5 ) B σ ~ z i 2 ( t 4 t 5 − t 3 t 6 ) B σ ~ y + i 2 ( t 3 t 5 − t 4 t 6 ) B σ ~ x − 2 ( t 1 t 8 + t 2 t 7 ) A σ ~ 0 i 2 ( t 3 t 8 + t 4 t 7 ) B σ ~ y − i 2 ( t 3 t 7 + t 4 t 8 ) B σ ~ x ) , T M 2 M 1 , 13 e f f ( e ⊗ t 2 ) = i 2 ( t 3 t 5 − t 4 t 6 ) B σ ~ y + i 2 ( t 4 t 5 − t 3 t 6 ) B σ ~ x , T M 2 M 1 , 23 e f f ( e ⊗ t 2 ) = − i 2 ( t 3 t 8 + t 4 t 7 ) B σ ~ x + i 2 ( t 3 t 7 + t 4 t 8 ) B σ ~ y . (42) \begin{aligned}
T^{\mathrm{eff}}_{M_{2}M_{1}}(e\otimes t_{2})={}&\left(\begin{array}{cc}
i(t_{2}t_{6}-t_{1}t_{5})B\widetilde{\sigma}_{z} & \frac{i}{2}(t_{4}t_{5}-t_{3}t_{6})B\widetilde{\sigma}_{y}+\frac{i}{2}(t_{3}t_{5}-t_{4}t_{6})B\widetilde{\sigma}_{x}\\
-2(t_{1}t_{8}+t_{2}t_{7})A\widetilde{\sigma}_{0} & \frac{i}{2}(t_{3}t_{8}+t_{4}t_{7})B\widetilde{\sigma}_{y}-\frac{i}{2}(t_{3}t_{7}+t_{4}t_{8})B\widetilde{\sigma}_{x}
\end{array}\right),\\
T^{\mathrm{eff}}_{M_{2}M_{1},13}(e\otimes t_{2})={}&\frac{i}{2}(t_{3}t_{5}-t_{4}t_{6})B\widetilde{\sigma}_{y}+\frac{i}{2}(t_{4}t_{5}-t_{3}t_{6})B\widetilde{\sigma}_{x},\\
T^{\mathrm{eff}}_{M_{2}M_{1},23}(e\otimes t_{2})={}&-\frac{i}{2}(t_{3}t_{8}+t_{4}t_{7})B\widetilde{\sigma}_{x}+\frac{i}{2}(t_{3}t_{7}+t_{4}t_{8})B\widetilde{\sigma}_{y}. \tag*{(42)}
\end{aligned} T M 2 M 1 eff ( e ⊗ t 2 ) = T M 2 M 1 , 13 eff ( e ⊗ t 2 ) = T M 2 M 1 , 23 eff ( e ⊗ t 2 ) = ( i ( t 2 t 6 − t 1 t 5 ) B σ z − 2 ( t 1 t 8 + t 2 t 7 ) A σ 0 2 i ( t 4 t 5 − t 3 t 6 ) B σ y + 2 i ( t 3 t 5 − t 4 t 6 ) B σ x 2 i ( t 3 t 8 + t 4 t 7 ) B σ y − 2 i ( t 3 t 7 + t 4 t 8 ) B σ x ) , 2 i ( t 3 t 5 − t 4 t 6 ) B σ y + 2 i ( t 4 t 5 − t 3 t 6 ) B σ x , − 2 i ( t 3 t 8 + t 4 t 7 ) B σ x + 2 i ( t 3 t 7 + t 4 t 8 ) B σ y . (42)
The 2NN effective hopping integral for the e e e -t 2 t_{2} t 2 channel is
T M 2 ′ M 1 e f f ( e ⊗ t 2 ) = ( − ( t 1 t 5 + t 2 t 6 ) A σ ~ 0 − i 2 ( t 1 t 6 − t 2 t 5 ) B σ ~ z i 2 t 3 t 6 B σ ~ y − i 2 t 3 t 5 B σ ~ x ( t 1 t 8 + t 2 t 7 ) A σ ~ 0 + i 2 ( t 2 t 8 − t 1 t 7 ) B σ ~ z − i 2 t 3 t 8 B σ ~ y + i 2 t 3 t 7 B σ ~ x ) , T M 2 ′ M 1 , 13 e f f ( e ⊗ t 2 ) = i 2 t 4 t 6 B σ ~ y − i 2 t 4 t 5 B σ ~ x , T M 2 ′ M 1 , 23 e f f ( e ⊗ t 2 ) = − i 2 t 4 t 8 B σ ~ y + i 2 t 4 t 7 B σ ~ x . (43) \begin{aligned}
T^{\mathrm{eff}}_{M'_{2}M_{1}}(e\otimes t_{2})={}&\left(\begin{array}{cc}
-(t_{1}t_{5}+t_{2}t_{6})A\widetilde{\sigma}_{0}-\frac{i}{2}(t_{1}t_{6}-t_{2}t_{5})B\widetilde{\sigma}_{z} & \frac{i}{2}t_{3}t_{6}B\widetilde{\sigma}_{y}-\frac{i}{2}t_{3}t_{5}B\widetilde{\sigma}_{x}\\
(t_{1}t_{8}+t_{2}t_{7})A\widetilde{\sigma}_{0}+\frac{i}{2}(t_{2}t_{8}-t_{1}t_{7})B\widetilde{\sigma}_{z} & -\frac{i}{2}t_{3}t_{8}B\widetilde{\sigma}_{y}+\frac{i}{2}t_{3}t_{7}B\widetilde{\sigma}_{x}
\end{array}\right),\\
T^{\mathrm{eff}}_{M'_{2}M_{1},13}(e\otimes t_{2})={}&\frac{i}{2}t_{4}t_{6}B\widetilde{\sigma}_{y}-\frac{i}{2}t_{4}t_{5}B\widetilde{\sigma}_{x},\\
T^{\mathrm{eff}}_{M'_{2}M_{1},23}(e\otimes t_{2})={}&-\frac{i}{2}t_{4}t_{8}B\widetilde{\sigma}_{y}+\frac{i}{2}t_{4}t_{7}B\widetilde{\sigma}_{x}. \tag*{(43)}
\end{aligned} T M 2 ′ M 1 eff ( e ⊗ t 2 ) = T M 2 ′ M 1 , 13 eff ( e ⊗ t 2 ) = T M 2 ′ M 1 , 23 eff ( e ⊗ t 2 ) = ( − ( t 1 t 5 + t 2 t 6 ) A σ 0 − 2 i ( t 1 t 6 − t 2 t 5 ) B σ z ( t 1 t 8 + t 2 t 7 ) A σ 0 + 2 i ( t 2 t 8 − t 1 t 7 ) B σ z 2 i t 3 t 6 B σ y − 2 i t 3 t 5 B σ x − 2 i t 3 t 8 B σ y + 2 i t 3 t 7 B σ x ) , 2 i t 4 t 6 B σ y − 2 i t 4 t 5 B σ x , − 2 i t 4 t 8 B σ y + 2 i t 4 t 7 B σ x . (43)
Substituting these effective hopping integrals into the superexchange expression yields the effective spin Hamiltonian for the e e e -t 2 t_{2} t 2 channel. The 1NN effective spin Hamiltonian is
H s 1 e − t 2 = { [ 1 3 ( t 3 t 6 − t 4 t 5 ) ( t 4 t 6 − t 3 t 5 ) − ( t 1 t 5 − t 2 t 6 ) 2 ] B 2 ( 1 − τ i z ) + [ 4 ( t 1 t 8 + t 2 t 7 ) 2 A 2 − 1 3 ( t 3 t 8 + t 4 t 7 ) ( t 3 t 7 + t 4 t 8 ) B 2 ] ( 1 2 + τ i z ) } ( S i ⋅ S j ) 8 ( U + 3 J H − Δ c ) + { [ 2 ( t 1 t 5 − t 2 t 6 ) 2 − ( t 3 t 6 − t 4 t 5 ) 2 − ( t 4 t 6 − t 3 t 5 ) 2 − 2 3 ( t 3 t 6 − t 4 t 5 ) ( t 4 t 6 − t 3 t 5 ) ] ( 1 2 − τ i z ) − [ ( t 3 t 8 + t 4 t 7 ) 2 + ( t 3 t 7 + t 4 t 8 ) 2 − 2 3 ( t 3 t 8 + t 4 t 7 ) ( t 3 t 7 + t 4 t 8 ) ] ( 1 2 + τ i z ) } B 2 S i z S j z 16 ( U + 3 J H − Δ c ) + [ ( t 3 t 6 − t 4 t 5 ) ( t 4 t 6 − t 3 t 5 ) ( 1 2 − τ i z ) − ( t 3 t 8 + t 4 t 7 ) ( t 3 t 7 + t 4 t 8 ) ( 1 2 + τ i z ) ] B 2 ( S i x S j y + S i y S j x ) 8 ( U + 3 J H − Δ c ) . (44) \begin{aligned}
H^{e-t_{2}}_{s1}={}&\left\{\left[\frac{1}{3}(t_{3}t_{6}-t_{4}t_{5})(t_{4}t_{6}-t_{3}t_{5})-(t_{1}t_{5}-t_{2}t_{6})^{2}\right]B^{2}(1-\tau^{z}_{i})\right.\\
&+\left.\left[4(t_{1}t_{8}+t_{2}t_{7})^{2}A^{2}-\frac{1}{3}(t_{3}t_{8}+t_{4}t_{7})(t_{3}t_{7}+t_{4}t_{8})B^{2}\right]\left(\frac{1}{2}+\tau^{z}_{i}\right)\right\}\frac{\left(\mathbf{S}_{i}\cdot\mathbf{S}_{j}\right)}{8(U+3J_{H}-\Delta_{c})}\\
&+\left\{\left[2(t_{1}t_{5}-t_{2}t_{6})^{2}-(t_{3}t_{6}-t_{4}t_{5})^{2}-(t_{4}t_{6}-t_{3}t_{5})^{2}-\frac{2}{3}(t_{3}t_{6}-t_{4}t_{5})(t_{4}t_{6}-t_{3}t_{5})\right]\left(\frac{1}{2}-\tau^{z}_{i}\right)\right.\\
&\left.-\left[(t_{3}t_{8}+t_{4}t_{7})^{2}+(t_{3}t_{7}+t_{4}t_{8})^{2}-\frac{2}{3}(t_{3}t_{8}+t_{4}t_{7})(t_{3}t_{7}+t_{4}t_{8})\right]\left(\frac{1}{2}+\tau^{z}_{i}\right)\right\}\frac{B^{2}S^{z}_{i}S^{z}_{j}}{16(U+3J_{H}-\Delta_{c})}\\
&+\left[(t_{3}t_{6}-t_{4}t_{5})(t_{4}t_{6}-t_{3}t_{5})\left(\frac{1}{2}-\tau^{z}_{i}\right)-(t_{3}t_{8}+t_{4}t_{7})(t_{3}t_{7}+t_{4}t_{8})\left(\frac{1}{2}+\tau^{z}_{i}\right)\right]\frac{B^{2}(S^{x}_{i}S^{y}_{j}+S^{y}_{i}S^{x}_{j})}{8(U+3J_{H}-\Delta_{c})}. \tag*{(44)}
\end{aligned} H s 1 e − t 2 = { [ 3 1 ( t 3 t 6 − t 4 t 5 ) ( t 4 t 6 − t 3 t 5 ) − ( t 1 t 5 − t 2 t 6 ) 2 ] B 2 ( 1 − τ i z ) + [ 4 ( t 1 t 8 + t 2 t 7 ) 2 A 2 − 3 1 ( t 3 t 8 + t 4 t 7 ) ( t 3 t 7 + t 4 t 8 ) B 2 ] ( 2 1 + τ i z ) } 8 ( U + 3 J H − Δ c ) ( S i ⋅ S j ) + { [ 2 ( t 1 t 5 − t 2 t 6 ) 2 − ( t 3 t 6 − t 4 t 5 ) 2 − ( t 4 t 6 − t 3 t 5 ) 2 − 3 2 ( t 3 t 6 − t 4 t 5 ) ( t 4 t 6 − t 3 t 5 ) ] ( 2 1 − τ i z ) − [ ( t 3 t 8 + t 4 t 7 ) 2 + ( t 3 t 7 + t 4 t 8 ) 2 − 3 2 ( t 3 t 8 + t 4 t 7 ) ( t 3 t 7 + t 4 t 8 ) ] ( 2 1 + τ i z ) } 16 ( U + 3 J H − Δ c ) B 2 S i z S j z + [ ( t 3 t 6 − t 4 t 5 ) ( t 4 t 6 − t 3 t 5 ) ( 2 1 − τ i z ) − ( t 3 t 8 + t 4 t 7 ) ( t 3 t 7 + t 4 t 8 ) ( 2 1 + τ i z ) ] 8 ( U + 3 J H − Δ c ) B 2 ( S i x S j y + S i y S j x ) . (44)
The corresponding 2NN effective spin Hamiltonian is
H s 2 e − t 2 = { [ 2 ( t 1 t 5 + t 2 t 6 ) 2 A 2 − 1 2 ( t 1 t 6 − t 2 t 5 ) 2 B 2 − 1 3 ( t 3 2 + t 4 2 ) t 5 t 6 B 2 ] ( 1 2 − τ i z ) + [ 2 ( t 1 t 8 + t 2 t 7 ) 2 A 2 − 1 2 ( t 2 t 8 − t 1 t 7 ) 2 B 2 − 1 3 ( t 3 2 + t 4 2 ) t 7 t 8 B 2 ] ( 1 2 + τ i z ) } S i ⋅ S j 16 ( U + 3 J H − Δ c ) + { [ ( t 1 t 6 − t 2 t 5 ) 2 − 1 2 ( t 3 2 + t 4 2 ) ( t 5 2 + t 6 2 ) + 1 3 ( t 3 2 + t 4 2 ) t 5 t 6 ] ( 1 2 − τ i z ) + [ ( t 2 t 8 − t 1 t 7 ) 2 − 1 2 ( t 3 2 + t 4 2 ) ( t 7 2 + t 8 2 ) + 1 3 ( t 3 2 + t 4 2 ) t 7 t 8 ] ( 1 2 + τ i z ) } B 2 S i γ S j γ 16 ( U + 3 J H − Δ c ) (45) \begin{aligned}
H_{s2}^{e-t_2}=\left\{\left[2(t_1t_5+t_2t_6)^2A^2-\frac{1}{2}(t_1t_6-t_2t_5)^2B^2-\frac{1}{3}(t_3^2+t_4^2)t_5t_6B^2\right]\left(\frac{1}{2}-\tau_i^z\right)+\left[2(t_1t_8+t_2t_7)^2A^2-\frac{1}{2}(t_2t_8-t_1t_7)^2B^2-\frac{1}{3}(t_3^2+t_4^2)t_7t_8B^2\right]\left(\frac{1}{2}+\tau_i^z\right)\right\}\frac{\mathbf{S}_i\cdot\mathbf{S}_j}{16(U+3J_H-\Delta_c)} \\
&+\left\{\left[(t_1t_6-t_2t_5)^2-\frac{1}{2}(t_3^2+t_4^2)(t_5^2+t_6^2)+\frac{1}{3}(t_3^2+t_4^2)t_5t_6\right]\left(\frac{1}{2}-\tau_i^z\right)+\left[(t_2t_8-t_1t_7)^2-\frac{1}{2}(t_3^2+t_4^2)(t_7^2+t_8^2)+\frac{1}{3}(t_3^2+t_4^2)t_7t_8\right]\left(\frac{1}{2}+\tau_i^z\right)\right\}\frac{B^2S_i^\gamma S_j^\gamma}{16(U+3J_H-\Delta_c)} \tag*{(45)}
\end{aligned} H s 2 e − t 2 = { [ 2 ( t 1 t 5 + t 2 t 6 ) 2 A 2 − 2 1 ( t 1 t 6 − t 2 t 5 ) 2 B 2 − 3 1 ( t 3 2 + t 4 2 ) t 5 t 6 B 2 ] ( 2 1 − τ i z ) + [ 2 ( t 1 t 8 + t 2 t 7 ) 2 A 2 − 2 1 ( t 2 t 8 − t 1 t 7 ) 2 B 2 − 3 1 ( t 3 2 + t 4 2 ) t 7 t 8 B 2 ] ( 2 1 + τ i z ) } 16 ( U + 3 J H − Δ c ) S i ⋅ S j + { [ ( t 1 t 6 − t 2 t 5 ) 2 − 2 1 ( t 3 2 + t 4 2 ) ( t 5 2 + t 6 2 ) + 3 1 ( t 3 2 + t 4 2 ) t 5 t 6 ] ( 2 1 − τ i z ) + [ ( t 2 t 8 − t 1 t 7 ) 2 − 2 1 ( t 3 2 + t 4 2 ) ( t 7 2 + t 8 2 ) + 3 1 ( t 3 2 + t 4 2 ) t 7 t 8 ] ( 2 1 + τ i z ) } 16 ( U + 3 J H − Δ c ) B 2 S i γ S j γ (45)
− ( t 3 2 + t 4 2 ) [ t 5 t 6 ( 1 2 − τ i z ) + t 7 t 8 ( 1 2 + τ i z ) ] B 2 ( S i α S j β + S i β S j α ) 16 ( U + 3 J H − Δ c ) -(t_3^2+t_4^2)\left[t_5t_6\left(\frac{1}{2}-\tau_i^z\right)+t_7t_8\left(\frac{1}{2}+\tau_i^z\right)\right]\frac{B^2(S_i^\alpha S_j^\beta+S_i^\beta S_j^\alpha)}{16(U+3J_H-\Delta_c)} − ( t 3 2 + t 4 2 ) [ t 5 t 6 ( 2 1 − τ i z ) + t 7 t 8 ( 2 1 + τ i z ) ] 16 ( U + 3 J H − Δ c ) B 2 ( S i α S j β + S i β S j α )
− [ ( t 1 t 5 + t 2 t 6 ) ( t 1 t 6 − t 2 t 5 ) ( 1 2 − τ i z ) + ( t 1 t 8 + t 2 t 7 ) ( t 2 t 8 − t 1 t 7 ) ( 1 2 + τ i z ) ] A B [ ( z × d i j ) ⋅ ( S i × S j ) ] 8 ( U + 3 J H − Δ c ) . -\left[(t_1t_5+t_2t_6)(t_1t_6-t_2t_5)\left(\frac{1}{2}-\tau_i^z\right)+(t_1t_8+t_2t_7)(t_2t_8-t_1t_7)\left(\frac{1}{2}+\tau_i^z\right)\right]\frac{AB[(\mathbf{z}\times\mathbf{d}_{ij})\cdot(\mathbf{S}_i\times\mathbf{S}_j)]}{8(U+3J_H-\Delta_c)}. − [ ( t 1 t 5 + t 2 t 6 ) ( t 1 t 6 − t 2 t 5 ) ( 2 1 − τ i z ) + ( t 1 t 8 + t 2 t 7 ) ( t 2 t 8 − t 1 t 7 ) ( 2 1 + τ i z ) ] 8 ( U + 3 J H − Δ c ) A B [( z × d ij ) ⋅ ( S i × S j )] .
The expressions above give the effective hopping integrals and effective spin Hamiltonians arising from the e e e -t 2 t_2 t 2 superexchange processes in the hole representation. As the result, the e e e -t 2 t_2 t 2 channel contributes to the Heisenberg interaction, the Kitaev-like interaction, the off-diagonal Γ \Gamma Γ interaction, and the DM interaction.
e-e superexchange process
The final contribution arises from the superexchange process between the e e e orbitals of Fe1 _1 1 and Fe2 _2 2 . Since the orbital degrees of freedom on both Fe sites must be retained and several intermediate states are involved, the resulting parameters of effective spin Hamiltonian is more intricate than those obtained for other channels.
We first specify the relevant intermediate states. Again, three classes of intermediate states occur. In the first case, a hole hops from an e e e orbital of Fe1 _1 1 to an empty e e e orbital of Fe2 _2 2 , with its spin parallel to the spin of the hole initially occupying the other e e e orbital on Fe2 _2 2 . The corresponding excitation energy is Δ E e − e 1 = U ′ − J H \Delta E_{e-e}^{1}=U^\prime-J_H Δ E e − e 1 = U ′ − J H . In the second case, a hole hops from an e e e orbital of Fe1 _1 1 to an already occupied e e e orbital of Fe2 _2 2 . This process generates a doubly occupied orbital and an empty orbital within the e e e manifold. The pair-hopping term in the Kanamori interaction then mixes these configurations, producing antisymmetric and symmetric orbital states with excitation energies Δ E e − e 2 = U + 4 J H \Delta E_{e-e}^{2}=U+4J_H Δ E e − e 2 = U + 4 J H , Δ E e − e 2 ′ = U + 2 J H \Delta E_{e-e}^{2\prime}=U+2J_H Δ E e − e 2′ = U + 2 J H , respectively, each with weight 1 / 2 1/2 1/2 . In the third case, a hole hops from an e e e orbital of Fe1 _1 1 to an empty e e e orbital of Fe2 _2 2 with spin antiparallel to the original spin. This process gives rise to high-spin and low-spin intermediate multiplets, with excitation energies Δ E e − e 3 = U ′ − J H \Delta E_{e-e}^{3}=U^\prime-J_H Δ E e − e 3 = U ′ − J H , Δ E e − e 3 ′ = U ′ + 4 J H \Delta E_{e-e}^{3\prime}=U^\prime+4J_H Δ E e − e 3′ = U ′ + 4 J H and weights 1 / 5 1/5 1/5 and 4 / 5 4/5 4/5 , respectively. The corresponding e e e -e e e virtual hopping processes are illustrated in Figure S6 . From the e e e -e e e virtual hopping paths, the effective hopping integrals can be obtained. For 1NN pairs, the result is
Figure S6. Schematic of the e e e -e e e virtual hopping processes in the hole representation. Brown and dark-gray spheres denote Fe and Te atoms, respectively. Only representative virtual hopping paths that contribute to the 1NN e e e -e e e effective hopping integrals are shown.
T M 2 M 1 e f f ( e ⊗ e ) = ( 4 t 5 t 6 A σ 0 − i ( t 5 t 7 − t 6 t 8 ) B σ z i ( t 5 t 7 − t 6 t 8 ) B σ z − 4 t 7 t 8 A σ 0 ) . (46) T_{M_2M_1}^{\mathrm{eff}}(e\otimes e)=\begin{pmatrix}4t_5t_6A\sigma_0&-i(t_5t_7-t_6t_8)B\sigma_z\\i(t_5t_7-t_6t_8)B\sigma_z&-4t_7t_8A\sigma_0\end{pmatrix}. \tag*{(46)} T M 2 M 1 eff ( e ⊗ e ) = ( 4 t 5 t 6 A σ 0 i ( t 5 t 7 − t 6 t 8 ) B σ z − i ( t 5 t 7 − t 6 t 8 ) B σ z − 4 t 7 t 8 A σ 0 ) . (46)
For 2NN pairs, the effective hopping integral is
T M 2 ′ M 1 ′ e f f ( e ⊗ e ) = ( − 2 t 5 t 6 A σ 0 + i 1 2 ( t 5 2 − t 6 2 ) B σ z − ( t 5 t 8 + t 6 t 7 ) A σ 0 + i 1 2 ( t 5 t 7 − t 6 t 8 ) B σ z ( t 5 t 8 + t 6 t 7 ) A σ 0 + i 1 2 ( t 6 t 8 − t 5 t 7 ) B σ z 2 t 7 t 8 A σ 0 + i 1 2 ( t 8 2 − t 7 2 ) B σ z ) . (47) T_{M'_2M'_1}^{\mathrm{eff}}(e\otimes e)=\begin{pmatrix}-2t_5t_6A\sigma_0+i\frac{1}{2}(t_5^2-t_6^2)B\sigma_z&-(t_5t_8+t_6t_7)A\sigma_0+i\frac{1}{2}(t_5t_7-t_6t_8)B\sigma_z\\(t_5t_8+t_6t_7)A\sigma_0+i\frac{1}{2}(t_6t_8-t_5t_7)B\sigma_z&2t_7t_8A\sigma_0+i\frac{1}{2}(t_8^2-t_7^2)B\sigma_z\end{pmatrix}. \tag*{(47)} T M 2 ′ M 1 ′ eff ( e ⊗ e ) = ( − 2 t 5 t 6 A σ 0 + i 2 1 ( t 5 2 − t 6 2 ) B σ z ( t 5 t 8 + t 6 t 7 ) A σ 0 + i 2 1 ( t 6 t 8 − t 5 t 7 ) B σ z − ( t 5 t 8 + t 6 t 7 ) A σ 0 + i 2 1 ( t 5 t 7 − t 6 t 8 ) B σ z 2 t 7 t 8 A σ 0 + i 2 1 ( t 8 2 − t 7 2 ) B σ z ) . (47)
Substituting the effective hopping integrals into the superexchange expression yields the 1NN effective spin Hamiltonian,
H s 1 e − e = 1 8 [ 16 ( t 5 2 t 6 2 Δ e − e 1 + t 7 2 t 8 2 Δ e − e 1 ) A 2 + ( t 5 t 7 − t 6 t 8 ) 2 ( Δ e − e 2 + Δ e − e 3 ) B 2 ] ( S i ⋅ S j ) − 1 4 ( t 5 t 7 − t 6 t 8 ) 2 ( Δ e − e 2 + Δ e − e 3 ) B 2 S i γ S j γ . (48) H_{s1}^{e-e}=\frac{1}{8}\left[16(t_5^2t_6^2\Delta_{e-e}^{1}+t_7^2t_8^2\Delta_{e-e}^{1})A^2+(t_5t_7-t_6t_8)^2(\Delta_{e-e}^{2}+\Delta_{e-e}^{3})B^2\right](\mathbf{S}_i\cdot\mathbf{S}_j)-\frac{1}{4}(t_5t_7-t_6t_8)^2(\Delta_{e-e}^{2}+\Delta_{e-e}^{3})B^2S_i^\gamma S_j^\gamma. \tag*{(48)} H s 1 e − e = 8 1 [ 16 ( t 5 2 t 6 2 Δ e − e 1 + t 7 2 t 8 2 Δ e − e 1 ) A 2 + ( t 5 t 7 − t 6 t 8 ) 2 ( Δ e − e 2 + Δ e − e 3 ) B 2 ] ( S i ⋅ S j ) − 4 1 ( t 5 t 7 − t 6 t 8 ) 2 ( Δ e − e 2 + Δ e − e 3 ) B 2 S i γ S j γ . (48)
Similarly, the 2NN effective spin Hamiltonian is
H s 2 e − e = 1 16 { [ 8 t 5 2 t 6 2 A 2 + 1 2 ( t 5 2 − t 6 2 ) 2 B 2 ] Δ e − e 1 + [ 2 ( t 5 t 8 + t 6 t 7 ) 2 A 2 + 1 2 ( t 5 t 7 − t 6 t 8 ) 2 B 2 ] Δ e − e 2 + [ 2 ( t 5 t 8 + t 6 t 7 ) 2 A 2 + 1 2 ( t 5 t 7 − t 6 t 8 ) 2 B 2 ] Δ e − e 3 + [ 8 t 7 2 t 8 2 A 2 + 1 2 ( t 8 2 − t 7 2 ) 2 B 2 ] Δ e − e 4 } ( S i ⋅ S j ) − 1 16 [ ( t 5 2 − t 6 2 ) 2 Δ e − e 1 + ( t 5 t 7 − t 6 t 8 ) 2 ( Δ e − e 2 + Δ e − e 3 ) + ( t 8 2 − t 7 2 ) 2 Δ e − e 4 ] B 2 S i y S j y + 1 8 [ 2 ( t 5 2 − t 6 2 ) t 5 t 6 Δ e − e 1 + ( t 5 t 8 + t 6 t 7 ) ( t 5 t 7 − t 6 t 8 ) ( Δ e − e 2 + Δ e − e 3 ) − 2 ( t 8 2 − t 7 2 ) t 7 t 8 Δ e − e 4 ] A B [ ( z × d i j ) ⋅ ( S i × S j ) ] . (49) \begin{aligned}
H_{s2}^{e-e}={}&\frac{1}{16}\Bigl\{\left[8t_{5}^{2}t_{6}^{2}A^{2}+\frac{1}{2}(t_{5}^{2}-t_{6}^{2})^{2}B^{2}\right]\Delta_{e-e}^{1}
+\left[2(t_{5}t_{8}+t_{6}t_{7})^{2}A^{2}+\frac{1}{2}(t_{5}t_{7}-t_{6}t_{8})^{2}B^{2}\right]\Delta_{e-e}^{2}\\
&+\left[2(t_{5}t_{8}+t_{6}t_{7})^{2}A^{2}+\frac{1}{2}(t_{5}t_{7}-t_{6}t_{8})^{2}B^{2}\right]\Delta_{e-e}^{3}\\
&+\left[8t_{7}^{2}t_{8}^{2}A^{2}+\frac{1}{2}(t_{8}^{2}-t_{7}^{2})^{2}B^{2}\right]\Delta_{e-e}^{4}\Bigr\}(\mathbf{S}_{i}\cdot\mathbf{S}_{j})\\
&-\frac{1}{16}\left[(t_{5}^{2}-t_{6}^{2})^{2}\Delta_{e-e}^{1}+(t_{5}t_{7}-t_{6}t_{8})^{2}(\Delta_{e-e}^{2}+\Delta_{e-e}^{3})+(t_{8}^{2}-t_{7}^{2})^{2}\Delta_{e-e}^{4}\right]B^{2}S_{i}^{y}S_{j}^{y}\\
&+\frac{1}{8}\left[2(t_{5}^{2}-t_{6}^{2})t_{5}t_{6}\Delta_{e-e}^{1}+(t_{5}t_{8}+t_{6}t_{7})(t_{5}t_{7}-t_{6}t_{8})(\Delta_{e-e}^{2}+\Delta_{e-e}^{3})\right.\\
&\left.\qquad-2(t_{8}^{2}-t_{7}^{2})t_{7}t_{8}\Delta_{e-e}^{4}\right]AB\left[(\mathbf{z}\times\mathbf{d}_{ij})\cdot(\mathbf{S}_{i}\times\mathbf{S}_{j})\right]. \tag*{(49)}
\end{aligned} H s 2 e − e = 16 1 { [ 8 t 5 2 t 6 2 A 2 + 2 1 ( t 5 2 − t 6 2 ) 2 B 2 ] Δ e − e 1 + [ 2 ( t 5 t 8 + t 6 t 7 ) 2 A 2 + 2 1 ( t 5 t 7 − t 6 t 8 ) 2 B 2 ] Δ e − e 2 + [ 2 ( t 5 t 8 + t 6 t 7 ) 2 A 2 + 2 1 ( t 5 t 7 − t 6 t 8 ) 2 B 2 ] Δ e − e 3 + [ 8 t 7 2 t 8 2 A 2 + 2 1 ( t 8 2 − t 7 2 ) 2 B 2 ] Δ e − e 4 } ( S i ⋅ S j ) − 16 1 [ ( t 5 2 − t 6 2 ) 2 Δ e − e 1 + ( t 5 t 7 − t 6 t 8 ) 2 ( Δ e − e 2 + Δ e − e 3 ) + ( t 8 2 − t 7 2 ) 2 Δ e − e 4 ] B 2 S i y S j y + 8 1 [ 2 ( t 5 2 − t 6 2 ) t 5 t 6 Δ e − e 1 + ( t 5 t 8 + t 6 t 7 ) ( t 5 t 7 − t 6 t 8 ) ( Δ e − e 2 + Δ e − e 3 ) − 2 ( t 8 2 − t 7 2 ) t 7 t 8 Δ e − e 4 ] A B [ ( z × d ij ) ⋅ ( S i × S j ) ] . (49)
In Eqs. (48) and (S48), the orbital degrees of freedom of the e e e orbitals on both Fe sites and the multiple intermediate states in the virtual hopping processes have been incorporated into Δ e − e 1 \Delta_{e-e}^{1} Δ e − e 1 , Δ e − e 2 \Delta_{e-e}^{2} Δ e − e 2 , Δ e − e 3 \Delta_{e-e}^{3} Δ e − e 3 , and Δ e − e 4 \Delta_{e-e}^{4} Δ e − e 4 . Their explicit forms are
Δ e − e 1 = ( 1 2 − τ i z ) ( 1 2 + τ j z ) U ′ − J H + ( 1 2 − τ i z ) ( 1 2 − τ j z ) ( U + 3 J H ) ( U + 2 J H ) ( U + 4 J H ) + ( 1 2 − τ i z ) ( 1 2 + τ j z ) U ′ ( U ′ − J H ) ( U ′ + 4 J H ) , (50) \Delta_{e-e}^{1}=\frac{\left(\frac{1}{2}-\tau_{i}^{z}\right)\left(\frac{1}{2}+\tau_{j}^{z}\right)}{U'-J_{\mathrm{H}}}+\frac{\left(\frac{1}{2}-\tau_{i}^{z}\right)\left(\frac{1}{2}-\tau_{j}^{z}\right)(U+3J_{\mathrm{H}})}{(U+2J_{\mathrm{H}})(U+4J_{\mathrm{H}})}+\frac{\left(\frac{1}{2}-\tau_{i}^{z}\right)\left(\frac{1}{2}+\tau_{j}^{z}\right)U'}{(U'-J_{\mathrm{H}})(U'+4J_{\mathrm{H}})}, \tag*{(50)} Δ e − e 1 = U ′ − J H ( 2 1 − τ i z ) ( 2 1 + τ j z ) + ( U + 2 J H ) ( U + 4 J H ) ( 2 1 − τ i z ) ( 2 1 − τ j z ) ( U + 3 J H ) + ( U ′ − J H ) ( U ′ + 4 J H ) ( 2 1 − τ i z ) ( 2 1 + τ j z ) U ′ , (50)
Δ e − e 2 = ( 1 2 − τ i z ) ( 1 2 − τ j z ) U ′ − J H + ( 1 2 − τ i z ) ( 1 2 + τ j z ) ( U + 3 J H ) ( U + 2 J H ) ( U + 4 J H ) + ( 1 2 − τ i z ) ( 1 2 − τ j z ) U ′ ( U ′ − J H ) ( U ′ + 4 J H ) , \Delta_{e-e}^{2}=\frac{\left(\frac{1}{2}-\tau_{i}^{z}\right)\left(\frac{1}{2}-\tau_{j}^{z}\right)}{U'-J_{\mathrm{H}}}+\frac{\left(\frac{1}{2}-\tau_{i}^{z}\right)\left(\frac{1}{2}+\tau_{j}^{z}\right)(U+3J_{\mathrm{H}})}{(U+2J_{\mathrm{H}})(U+4J_{\mathrm{H}})}+\frac{\left(\frac{1}{2}-\tau_{i}^{z}\right)\left(\frac{1}{2}-\tau_{j}^{z}\right)U'}{(U'-J_{\mathrm{H}})(U'+4J_{\mathrm{H}})}, Δ e − e 2 = U ′ − J H ( 2 1 − τ i z ) ( 2 1 − τ j z ) + ( U + 2 J H ) ( U + 4 J H ) ( 2 1 − τ i z ) ( 2 1 + τ j z ) ( U + 3 J H ) + ( U ′ − J H ) ( U ′ + 4 J H ) ( 2 1 − τ i z ) ( 2 1 − τ j z ) U ′ ,
Δ e − e 3 = ( 1 2 + τ i z ) ( 1 2 + τ j z ) U ′ − J H + ( 1 2 + τ i z ) ( 1 2 − τ j z ) ( U + 3 J H ) ( U + 2 J H ) ( U + 4 J H ) + ( 1 2 + τ i z ) ( 1 2 + τ j z ) U ′ ( U ′ − J H ) ( U ′ + 4 J H ) , \Delta_{e-e}^{3}=\frac{\left(\frac{1}{2}+\tau_{i}^{z}\right)\left(\frac{1}{2}+\tau_{j}^{z}\right)}{U'-J_{\mathrm{H}}}+\frac{\left(\frac{1}{2}+\tau_{i}^{z}\right)\left(\frac{1}{2}-\tau_{j}^{z}\right)(U+3J_{\mathrm{H}})}{(U+2J_{\mathrm{H}})(U+4J_{\mathrm{H}})}+\frac{\left(\frac{1}{2}+\tau_{i}^{z}\right)\left(\frac{1}{2}+\tau_{j}^{z}\right)U'}{(U'-J_{\mathrm{H}})(U'+4J_{\mathrm{H}})}, Δ e − e 3 = U ′ − J H ( 2 1 + τ i z ) ( 2 1 + τ j z ) + ( U + 2 J H ) ( U + 4 J H ) ( 2 1 + τ i z ) ( 2 1 − τ j z ) ( U + 3 J H ) + ( U ′ − J H ) ( U ′ + 4 J H ) ( 2 1 + τ i z ) ( 2 1 + τ j z ) U ′ ,
Δ e − e 4 = ( 1 2 + τ i z ) ( 1 2 − τ j z ) U ′ − J H + ( 1 2 + τ i z ) ( 1 2 + τ j z ) ( U + 3 J H ) ( U + 2 J H ) ( U + 4 J H ) + ( 1 2 + τ i z ) ( 1 2 − τ j z ) U ′ ( U ′ − J H ) ( U ′ + 4 J H ) . \Delta_{e-e}^{4}=\frac{\left(\frac{1}{2}+\tau_{i}^{z}\right)\left(\frac{1}{2}-\tau_{j}^{z}\right)}{U'-J_{\mathrm{H}}}+\frac{\left(\frac{1}{2}+\tau_{i}^{z}\right)\left(\frac{1}{2}+\tau_{j}^{z}\right)(U+3J_{\mathrm{H}})}{(U+2J_{\mathrm{H}})(U+4J_{\mathrm{H}})}+\frac{\left(\frac{1}{2}+\tau_{i}^{z}\right)\left(\frac{1}{2}-\tau_{j}^{z}\right)U'}{(U'-J_{\mathrm{H}})(U'+4J_{\mathrm{H}})}. Δ e − e 4 = U ′ − J H ( 2 1 + τ i z ) ( 2 1 − τ j z ) + ( U + 2 J H ) ( U + 4 J H ) ( 2 1 + τ i z ) ( 2 1 + τ j z ) ( U + 3 J H ) + ( U ′ − J H ) ( U ′ + 4 J H ) ( 2 1 + τ i z ) ( 2 1 − τ j z ) U ′ .
The above results constitute the effective spin Hamiltonian generated by the e e e -e e e superexchange channel. This channel contributes to the Heisenberg interaction, the Kitaev-like interaction, and the DM interaction, while it does not generate an off-diagonal interaction Γ \Gamma Γ .
Effective spin Hamiltonian of FeX
In § 4.4 , we presented the superexchange processes between Fe and its 1NN and 2NN Fe sites mediated by the coordinating ligands X (X=Te, Se), together with the resulting effective spin Hamiltonians. As shown there, the 1NN interactions contain three types of magnetic couplings, namely, the Heisenberg, Kitaev-like, and off-diagonal exchange interactions, whereas the 2NN interactions additionally host a Dzyaloshinskii–Moriya (DM) term. The effective Hamiltonian for monolayer FeX can therefore be written as
H = ∑ ⟨ i , j ⟩ 1 , 2 1 2 [ J i j ( S i ⋅ S j ) + K i j S i γ S j γ + Γ i j ( S i α S j β + S i β S j α ) ] + ∑ ⟨ i , j ⟩ 2 1 2 D i j [ ( z × d i j ) ⋅ ( S i × S j ) ] , (51) H = \sum_{\langle i,j\rangle_{1,2}} \frac{1}{2}\left[J_{ij}(\mathbf{S}_i \cdot\mathbf{S}_j) + K_{ij}S_i^\gamma S_j^\gamma+ \Gamma_{ij}(S_i^\alpha S_j^\beta+ S_i^\beta S_j^\alpha)\right] + \sum_{\langle i,j\rangle_2} \frac{1}{2}D_{ij}\left[(\mathbf{z} \times\mathbf{d}_{ij}) \cdot(\mathbf{S}_i \times\mathbf{S}_j)\right], \tag*{(51)} H = ⟨ i , j ⟩ 1 , 2 ∑ 2 1 [ J ij ( S i ⋅ S j ) + K ij S i γ S j γ + Γ ij ( S i α S j β + S i β S j α ) ] + ⟨ i , j ⟩ 2 ∑ 2 1 D ij [ ( z × d ij ) ⋅ ( S i × S j ) ] , (51)
where the terms represent the Heisenberg interaction J i j J_{ij} J ij , the Kitaev-like interaction K i j K_{ij} K ij , the off-diagonal interaction Γ i j \Gamma_{ij} Γ ij , and the DM interaction D i j D_{ij} D ij with their respective coupling coefficients, incorporating a factor of 1 / 2 1/2 1/2 to avoid double counting. The ⟨ i , j ⟩ \langle i,j\rangle ⟨ i , j ⟩ represent Fe atom neighbor pairs, and S \mathbf{S} S is the normalized spin operator. The local spin quantization axes { α , β , γ } \{\alpha,\beta,\gamma\} { α , β , γ } are the spin component directions of the planar basis in Figure 1 (b),(d) and (e). Unlike the edge-sharing octahedral Kitaev interaction, α \alpha α and β \beta β form an angle of 109.47 ∘ 109.47^\circ 109.4 7 ∘ and are not perpendicular. i , j , k \mathbf{i},\mathbf{j},\mathbf{k} i , j , k are the basis vectors of the Cartesian coordinate system.
The Kitaev-like and off-diagonal interactions originate from spin-flip virtual hopping processes [27 ] in which the SOC of the ligand atoms participates in both the Fei → _i \to i → Fej _j j and Fej → _j \to j → Fei _i i hopping pathways. In contrast, the DM interaction arises from superexchange processes where SOC is active in only one of the two virtual hopping steps. The Heisenberg interaction receives contributions from both spin-conserving processes (without SOC) and spin-flip processes.
Furthermore, the contributions from different orbital channels vary. In the local coordinate frame, the t 2 t_2 t 2 -t 2 t_2 t 2 , t 2 t_2 t 2 -e e e , and e e e -t 2 t_2 t 2 superexchange channels all contribute to the Heisenberg, Kitaev-like, off-diagonal, and DM interactions, whereas the e e e -e e e channel does not contribute to the off-diagonal interaction.
When discussing magnetic anisotropy, studies often consider single-ion anisotropy (SIA). Previous investigations of FeX systems have largely been restricted to SIA as the sole source of magnetic anisotropy [27 , 37 ] . To provide a complete description, we incorporate SIA into the Hamiltonian, which is expressed in its full form as
H = ∑ ⟨ i , j ⟩ 1 , 2 1 2 [ J i j ( S i ⋅ S j ) + K i j S i γ S j γ + Γ i j ( S i α S j β + S i β S j α ) ] + ∑ ⟨ i , j ⟩ 2 1 2 D i j [ ( z × d i j ) ⋅ ( S i × S j ) ] + ∑ i A k [ 1 − ( S i ⋅ k ) 2 ] , (52) \begin{aligned}
H ={}& \sum_{\langle i,j\rangle_{1,2}} \frac{1}{2}\left[J_{ij}(\mathbf{S}_i \cdot\mathbf{S}_j) + K_{ij}S_i^\gamma S_j^\gamma+ \Gamma_{ij}(S_i^\alpha S_j^\beta+ S_i^\beta S_j^\alpha)\right] \\
&+ \sum_{\langle i,j\rangle_2} \frac{1}{2}D_{ij}\left[(\mathbf{z} \times\mathbf{d}_{ij}) \cdot(\mathbf{S}_i \times\mathbf{S}_j)\right] + \sum_i A_k\left[1 - (\mathbf{S}_i \cdot\mathbf{k})^2\right], \tag*{(52)}
\end{aligned} H = ⟨ i , j ⟩ 1 , 2 ∑ 2 1 [ J ij ( S i ⋅ S j ) + K ij S i γ S j γ + Γ ij ( S i α S j β + S i β S j α ) ] + ⟨ i , j ⟩ 2 ∑ 2 1 D ij [ ( z × d ij ) ⋅ ( S i × S j ) ] + i ∑ A k [ 1 − ( S i ⋅ k ) 2 ] , (52)
where the SIA term is denoted by A k A_k A k . In the following, the anisotropy parameters in the Hamiltonian will be extracted via density functional theory (DFT) calculations.
In the effective spin Hamiltonian, since the various anisotropic interactions and the Heisenberg interaction fundamentally arise from combinations of different spin components, perturbation theory allows us to cast the Hamiltonian in the general tensor form:
H s ( i , j ) = ( S ~ i μ ) T M i j μ ν S ~ j ν , (53) H_s(i,j) = (\widetilde{\mathbf{S}}_i^\mu)^T M_{ij}^{\mu\nu} \widetilde{\mathbf{S}}_j^\nu, \tag*{(53)} H s ( i , j ) = ( S i μ ) T M ij μν S j ν , (53)
where M i j μ ν M_{ij}^{\mu\nu} M ij μν represents the exchange tensor of the spin Hamiltonian between magnetic atoms i i i and j j j . However, for bond-dependent Kitaev-like interactions in the 2D edge-sharing tetrahedral structure, the bond directions are not mutually orthogonal, preventing the use of a single unified Cartesian coordinate system. Having derived the spin-exchange expressions in the local coordinate frame via perturbation theory, we can explicitly write the exchange tensor for the 1NN Fe pairs as
M ~ 1 = ( J 1 − 1 3 Γ 1 Γ 1 6 2 Γ 1 ′ Γ 1 J 1 − 1 3 Γ 1 6 2 Γ 1 ′ 6 2 Γ 1 ′ 6 2 Γ 1 ′ J 1 + K 1 ) , (54) \widetilde{M}_1 =
\begin{pmatrix}
J_1-\frac{1}{3}\Gamma_1 & \Gamma_1 & \frac{\sqrt{6}}{2}\Gamma_1' \\
\Gamma_1 & J_1-\frac{1}{3}\Gamma_1 & \frac{\sqrt{6}}{2}\Gamma_1' \\
\frac{\sqrt{6}}{2}\Gamma_1' & \frac{\sqrt{6}}{2}\Gamma_1' & J_1+K_1
\end{pmatrix}, \tag*{(54)} M 1 = J 1 − 3 1 Γ 1 Γ 1 2 6 Γ 1 ′ Γ 1 J 1 − 3 1 Γ 1 2 6 Γ 1 ′ 2 6 Γ 1 ′ 2 6 Γ 1 ′ J 1 + K 1 , (54)
The appearance of Γ 1 \Gamma_1 Γ 1 in the diagonal components stems from the fact that the Heisenberg parameters receive contributions not only from S ~ i x S ~ j x + S ~ i y S ~ j y + S ~ i z S ~ j z \widetilde{S}_i^x\widetilde{S}_j^x+\widetilde{S}_i^y\widetilde{S}_j^y+\widetilde{S}_i^z\widetilde{S}_j^z S i x S j x + S i y S j y + S i z S j z in the local frame, but also from off-diagonal terms introduced through our bond-dependent coordinate transformation, as shown in Eq. (29) . The off-diagonal component Γ ′ \Gamma^{\prime} Γ ′ vanishes, as proved in § 5.3.4 (i.e., its corresponding coefficient is zero).
For the 2NNs, the exchange tensor can be expressed as
M ~ 2 = ( J 2 − 1 3 Γ 2 Γ 2 + D 2 6 2 Γ 2 ′ Γ 2 − D 2 J 2 − 1 3 Γ 2 6 2 Γ 2 ′ 6 2 Γ 2 ′ 6 2 Γ 2 ′ J 2 + K 2 ) , (55) \widetilde{M}_{2} =
\begin{pmatrix}
J_{2}-\frac{1}{3}\Gamma_{2} & \Gamma_{2}+D_{2} & \frac{\sqrt{6}}{2}\Gamma^{\prime}_{2} \\
\Gamma_{2}-D_{2} & J_{2}-\frac{1}{3}\Gamma_{2} & \frac{\sqrt{6}}{2}\Gamma^{\prime}_{2} \\
\frac{\sqrt{6}}{2}\Gamma^{\prime}_{2} & \frac{\sqrt{6}}{2}\Gamma^{\prime}_{2} & J_{2}+K_{2}
\end{pmatrix}, \tag*{(55)} M 2 = J 2 − 3 1 Γ 2 Γ 2 − D 2 2 6 Γ 2 ′ Γ 2 + D 2 J 2 − 3 1 Γ 2 2 6 Γ 2 ′ 2 6 Γ 2 ′ 2 6 Γ 2 ′ J 2 + K 2 , (55)
where D 2 D_{2} D 2 denotes the DM interaction parameter, and its sign depends on the convention adopted in matrices under global coordinates. Furthermore, since no secondary off-diagonal interactions are present for either 1NN or 2NN pairs, Γ 1 ′ = Γ 2 ′ = 0 \Gamma^{\prime}_{1}=\Gamma^{\prime}_{2}=0 Γ 1 ′ = Γ 2 ′ = 0 .
Since these exchange tensors are defined in the local coordinate system, the exchange tensors for equivalent neighbor bonds are identical in the designated local frames. Upon mapping to the global coordinate frame, they transform according to the specific bond directions to yield the corresponding global Hamiltonians.
DFT calculation scheme and validation
A major obstacle to generalizing the Kitaev interaction to broader lattice structures is the lack of a concise and universal density functional theory (DFT) scheme capable of disentangling complex magnetic interactions within a lattice. To address this, our group employs an “Energy Mapping + MAE” approach [21 , 22 , 23 ] . By calculating the angle-dependent MAE for various collinear magnetic orders, we obtain MAE curves that isolate contributions solely from Kitaev(-like) interactions, off-diagonal interactions, and single-ion anisotropy (SIA). Since the functional forms of the MAE curves and the corresponding interaction expressions differ across magnetic configurations, considering a sufficient number of collinear magnetic orders allows for the separation of distinct interaction parameters. Below, we present the detailed formalism of this method applied to the two-dimensional edge-sharing tetrahedral structure, along with the MAE results for FeX (X = T e , S e X = \mathrm{Te}, \mathrm{Se} X = Te , Se ).
Energy mapping + MAE method
In this method, the first step is to select appropriate collinear magnetic orders based on the specific lattice structure. For the two-dimensional monolayer FeX system, we adopt three commonly studied magnetic configurations, as illustrated in Figure S7 : ferromagnetic (FM) order, Néel-type antiferromagnetic order (commonly abbreviated as GAFM in square lattices), and bicollinear antiferromagnetic (BAFM) order. The computed MAE includes both in-plane and out-of-plane components, expressed as:
Figure S7. Magnetic orders and corresponding anisotropy landscapes in monolayer FeX Schematic of Possible Magnetic Orders in Monolayer FeX and S I A ( θ ) \mathrm{SIA}(\theta) SIA ( θ ) , M A E ( θ / φ ) \mathrm{MAE}(\theta/\varphi) MAE ( θ / φ ) . (a) Ferromagnetic order (FM), (b) Neel antiferromagnetic order (Néel AFM), (c) Bilinear antiferromagnetic order (BAFM). Red and blue arrows indicate opposite magnetic moment directions. The right side shows the polar plots of S I A ( θ ) \mathrm{SIA}(\theta) SIA ( θ ) and M A E ( θ / φ ) \mathrm{MAE}(\theta/\varphi) MAE ( θ / φ ) for each magnetic order (all parameters set to 1), divided into components within the Fe plane (gray plane) and perpendicular to the Fe layer (out-of-plane). The in-plane component uses i i i as the polar axis, and the out-of-plane component uses k k k as the polar axis; distance from the origin represents the magnitude of the effect in that direction. Blue solid lines represent SIA (positive values), while purple and red dashed lines represent positive and negative values of M A E ( θ / φ ) \mathrm{MAE}(\theta/\varphi) MAE ( θ / φ ) , respectively.
H M A E i n = H M A E θ = 90 ∘ , φ − H M A E θ = 90 ∘ , φ = 0 ∘ , H M A E o u t = H M A E θ , φ = 0 − H M A E θ = 90 ∘ , φ = 0 ∘ , (56) H_{MAE}^{in}=H_{MAE}^{\theta=90^\circ,\varphi}-H_{MAE}^{\theta=90^\circ,\varphi=0^\circ},\qquad H_{MAE}^{out}=H_{MAE}^{\theta,\varphi=0}-H_{MAE}^{\theta=90^\circ,\varphi=0^\circ}, \tag*{(56)} H M A E in = H M A E θ = 9 0 ∘ , φ − H M A E θ = 9 0 ∘ , φ = 0 ∘ , H M A E o u t = H M A E θ , φ = 0 − H M A E θ = 9 0 ∘ , φ = 0 ∘ , (56)
where the in-plane term corresponds to the MAE relative to the [100] direction as a function of the in-plane angle φ \varphi φ , and the out-of-plane term corresponds to the MAE relative to the [100] direction as a function of the out-of-plane angle θ \theta θ . The definitions of θ \theta θ and φ \varphi φ are provided in Figure S7 .
Consequently, in our angle-dependent MAE calculations, the contribution from the isotropic Heisenberg interaction remains constant and thus does not appear explicitly. Furthermore, since the magnetic orders considered are strictly collinear (parallel or antiparallel), the Dzyaloshinskii–Moriya (DM) interaction vanishes identically. Therefore, the MAE for different magnetic orders contains only the pure contributions from Kitaev-like interactions, off-diagonal interactions, and SIA. This can be expressed as:
H M A E = H K + H Γ + H S I A , (57) H_{MAE}=H_K+H_\Gamma+H_{SIA}, \tag*{(57)} H M A E = H K + H Γ + H S I A , (57)
where the bond-dependent Kitaev-like interaction is collectively denoted as H K i t = H K + H Γ H_{Kit}=H_K+H_\Gamma H K i t = H K + H Γ in subsequent discussions. To derive the MAE expressions for different magnetic configurations, we represent the spin directions in spherical coordinates as:
S = S 0 ( sin θ cos φ i + sin θ sin φ j + cos θ k ) , (58) \mathbf{S}=S_0(\sin\theta\cos\varphi\mathbf{i}+\sin\theta\sin\varphi\mathbf{j}+\cos\theta\mathbf{k}), \tag*{(58)} S = S 0 ( sin θ cos φ i + sin θ sin φ j + cos θ k ) , (58)
where the opposite magnetic moment direction undergoes the following transformation: θ → ( π − θ ) \theta\to(\pi-\theta) θ → ( π − θ ) , φ → ( φ + π ) \varphi\to(\varphi+\pi) φ → ( φ + π ) . According to the Hamiltonian, the SIA energy acts solely along the direction perpendicular to the plane and is independent of the spin arrangement; thus, its contribution to the MAE is identical across all magnetic orders and can be written as:
H S I A F M = H S I A G A F M = H S I A B A F M = A k cos 2 θ . (59) H_{SIA}^{FM}=H_{SIA}^{GAFM}=H_{SIA}^{BAFM}=A_k\cos^2\theta. \tag*{(59)} H S I A F M = H S I A G A F M = H S I A B A F M = A k cos 2 θ . (59)
In contrast, the bond-dependent Kitaev-like interaction varies with the magnetic configuration and exhibits angular dependence in both the in-plane and out-of-plane directions. For the FM order, this contribution can be written as (with analogous expressions for other magnetic orders):
H K i t F M ( θ , φ ) = H K F M ( θ , φ ) + H Γ F M ( θ , φ ) , (60) H_{Kit}^{FM}(\theta,\varphi)=H_K^{FM}(\theta,\varphi)+H_\Gamma^{FM}(\theta,\varphi), \tag*{(60)} H K i t F M ( θ , φ ) = H K F M ( θ , φ ) + H Γ F M ( θ , φ ) , (60)
where the expression above represents the FM case; analogous expressions apply to other magnetic configurations. Consequently, the final expression for the total MAE (taking FM as an example) becomes:
H M A E F M ( θ , φ ) = H K i t F M ( θ , φ ) + H S I A ( θ ) , (61) H_{MAE}^{FM}(\theta,\varphi)=H_{Kit}^{FM}(\theta,\varphi)+H_{SIA}(\theta), \tag*{(61)} H M A E F M ( θ , φ ) = H K i t F M ( θ , φ ) + H S I A ( θ ) , (61)
where the forms for other magnetic configurations are similar. Thus, by analyzing the MAE results across different magnetic orders, we can first extract the SIA contribution and subsequently determine the parameters for the bond-dependent Kitaev-like interactions by incorporating multiple magnetic configurations.
Therefore, a simple criterion to determine whether anisotropic magnetic interactions beyond SIA exist in the system is to calculate the out-of-plane MAE for both FM and G-AFM orders. If the trends of the MAE curves differ between these two orders, interactions beyond SIA must be present. Conversely, if the trends are identical, further calculations involving additional magnetic orders are required for confirmation.
Contributions of Kitaev-like and Off-diagonal Interactions to the MAE
The effective spin Hamiltonian for the Kitaev-like and off-diagonal interactions in two-dimensional monolayer FeX is given by Eq. (S51), while the bond directions between different atoms are listed in Table S1 . We can therefore expand the bond-dependent Hamiltonian into a form expressed in terms of the distinct bonds:
H K i t = ∑ ∣ m ∣ = ∣ n ∣ = 1 1 2 [ K 1 S 00 w ( p + 2 ) − ( p + 3 ) S m n w ( p + 2 ) − ( p + 3 ) + Γ 1 ( S 00 w ~ p + 1 S m n w p + 1 + S 00 w ~ p + 1 S m n w ~ p + 1 ) ] + ∑ ∣ m ∣ o r ∣ n ∣ = 2 1 2 [ K 2 S 00 w ( p + 1 ) − ( p + 3 ) S m n w ( p + 1 ) − ( p + 3 ) + Γ 2 S 00 w ~ p + 2 S m n w ~ p + 2 ] , (62) \begin{aligned}
H_{\mathrm{Kit}}={}&\sum_{\lvert m\rvert=\lvert n\rvert=1}\frac{1}{2}\left[K_{1}S_{00}^{w(p+2)-(p+3)}S_{mn}^{w(p+2)-(p+3)}+\Gamma_{1}\left(S_{00}^{\widetilde{w}_{p+1}}S_{mn}^{w_{p+1}}+S_{00}^{\widetilde{w}_{p+1}}S_{mn}^{\widetilde{w}_{p+1}}\right)\right]\\
&+\sum_{\lvert m\rvert\ \mathrm{or}\ \lvert n\rvert=2}\frac{1}{2}\left[K_{2}S_{00}^{w(p+1)-(p+3)}S_{mn}^{w(p+1)-(p+3)}+\Gamma_{2}S_{00}^{\widetilde{w}_{p+2}}S_{mn}^{\widetilde{w}_{p+2}}\right], \tag*{(62)}
\end{aligned} H Kit = ∣ m ∣ = ∣ n ∣ = 1 ∑ 2 1 [ K 1 S 00 w ( p + 2 ) − ( p + 3 ) S mn w ( p + 2 ) − ( p + 3 ) + Γ 1 ( S 00 w p + 1 S mn w p + 1 + S 00 w p + 1 S mn w p + 1 ) ] + ∣ m ∣ or ∣ n ∣ = 2 ∑ 2 1 [ K 2 S 00 w ( p + 1 ) − ( p + 3 ) S mn w ( p + 1 ) − ( p + 3 ) + Γ 2 S 00 w p + 2 S mn w p + 2 ] , (62)
where the spin subscript 00 00 00 denotes the spin of the central atom under consideration, and the subscript m n mn mn denotes the spin of the atom located at R = m i + n j \mathbf{R}=m\mathbf{i}+n\mathbf{j} R = m i + n j relative to the central atom. Different values of m n mn mn correspond to different bond labels p p p , with the specific correspondence detailed in Table S1 .
Based on the above bond-dependent Hamiltonian, we can derive the respective contributions of the Kitaev-like and off-diagonal interactions to the MAE under different magnetic orders, as presented below.
For the 1NN, the angular dependence of the Kitaev-like interaction under different magnetic orders is:
H K 1 F M = 1 2 K 1 S 0 2 ( 1 + cos 2 θ ) , H K 1 G A F M = − 1 2 K 1 S 0 2 ( 1 + cos 2 θ ) , H K 1 B A F M = 2 K 1 S 0 2 sin θ cos θ cos φ . (63) \begin{aligned}
H_{K_{1}}^{FM}&=\frac{1}{2}K_{1}S_{0}^{2}(1+\cos^{2}\theta),\\
H_{K_{1}}^{GAFM}&=-\frac{1}{2}K_{1}S_{0}^{2}(1+\cos^{2}\theta),\\
H_{K_{1}}^{BAFM}&=\sqrt{2}K_{1}S_{0}^{2}\sin\theta\cos\theta\cos\varphi. \tag*{(63)}
\end{aligned} H K 1 F M H K 1 G A F M H K 1 B A F M = 2 1 K 1 S 0 2 ( 1 + cos 2 θ ) , = − 2 1 K 1 S 0 2 ( 1 + cos 2 θ ) , = 2 K 1 S 0 2 sin θ cos θ cos φ . (63)
The corresponding result for the 1NN off-diagonal interaction is:
H Γ 1 F M = 4 3 Γ 1 S 0 2 cos 2 θ , H Γ 1 G A F M = − 4 3 Γ 1 S 0 2 cos 2 θ , H Γ 1 B A F M = − 4 2 3 Γ 1 S 0 2 sin θ cos θ cos φ . (64) \begin{aligned}
H_{\Gamma_{1}}^{FM}&=\frac{4}{3}\Gamma_{1}S_{0}^{2}\cos^{2}\theta,\\
H_{\Gamma_{1}}^{GAFM}&=-\frac{4}{3}\Gamma_{1}S_{0}^{2}\cos^{2}\theta,\\
H_{\Gamma_{1}}^{BAFM}&=-\frac{4\sqrt{2}}{3}\Gamma_{1}S_{0}^{2}\sin\theta\cos\theta\cos\varphi. \tag*{(64)}
\end{aligned} H Γ 1 F M H Γ 1 G A F M H Γ 1 B A F M = 3 4 Γ 1 S 0 2 cos 2 θ , = − 3 4 Γ 1 S 0 2 cos 2 θ , = − 3 4 2 Γ 1 S 0 2 sin θ cos θ cos φ . (64)
For the 2NN, the Kitaev-like interaction result is:
H K 2 F M = K 2 S 0 2 ( 1 − cos 2 θ ) , H K 2 G A F M = K 2 S 0 2 ( 1 − cos 2 θ ) , H K 2 B A F M = K 2 S 0 2 ( 1 − cos 2 θ ) ( 1 − 2 sin 2 θ ) . (65) \begin{aligned}
H_{K_{2}}^{FM}&=K_{2}S_{0}^{2}(1-\cos^{2}\theta),\\
H_{K_{2}}^{GAFM}&=K_{2}S_{0}^{2}(1-\cos^{2}\theta),\\
H_{K_{2}}^{BAFM}&=K_{2}S_{0}^{2}(1-\cos^{2}\theta)(1-2\sin^{2}\theta). \tag*{(65)}
\end{aligned} H K 2 F M H K 2 G A F M H K 2 B A F M = K 2 S 0 2 ( 1 − cos 2 θ ) , = K 2 S 0 2 ( 1 − cos 2 θ ) , = K 2 S 0 2 ( 1 − cos 2 θ ) ( 1 − 2 sin 2 θ ) . (65)
The 2NN off-diagonal interaction result is:
H Γ 2 F M = 4 3 Γ 2 S 0 2 ( 2 cos 2 θ − 1 ) , H Γ 2 G A F M = 4 3 Γ 2 S 0 2 ( 2 cos 2 θ − 1 ) , H Γ 2 B A F M = 2 3 Γ 2 S 0 2 ( 1 − cos 2 θ ) ( 2 cos 2 φ − 1 ) . (66) \begin{aligned}
H_{\Gamma_{2}}^{FM}&=\frac{4}{3}\Gamma_{2}S_{0}^{2}(2\cos^{2}\theta-1),\\
H_{\Gamma_{2}}^{GAFM}&=\frac{4}{3}\Gamma_{2}S_{0}^{2}(2\cos^{2}\theta-1),\\
H_{\Gamma_{2}}^{BAFM}&=\frac{2}{3}\Gamma_{2}S_{0}^{2}(1-\cos^{2}\theta)(2\cos^{2}\varphi-1). \tag*{(66)}
\end{aligned} H Γ 2 F M H Γ 2 G A F M H Γ 2 B A F M = 3 4 Γ 2 S 0 2 ( 2 cos 2 θ − 1 ) , = 3 4 Γ 2 S 0 2 ( 2 cos 2 θ − 1 ) , = 3 2 Γ 2 S 0 2 ( 1 − cos 2 θ ) ( 2 cos 2 φ − 1 ) . (66)
The above expressions describe the bond-dependent Kitaev-like interactions for spins oriented at arbitrary angles in space. However, our goal is to obtain the energy variation relative to the [100] axis. We therefore process these results separately to obtain the out-of-plane and in-plane components. For the out-of-plane component, we set φ = 0 ∘ \varphi=0^{\circ} φ = 0 ∘ and subtract the result at θ = 90 ∘ \theta=90^{\circ} θ = 9 0 ∘ . For the in-plane component, we set θ = 90 ∘ \theta=90^{\circ} θ = 9 0 ∘ and subtract the result at φ = 0 ∘ \varphi=0^{\circ} φ = 0 ∘ .
After applying these operations and summing the in-plane and out-of-plane expressions of the individual interaction terms, we obtain the total in-plane and out-of-plane MAE expressions.
The out-of-plane MAE is given by:
H M A E o u t F M = ( 1 2 K 1 + 4 3 Γ 1 − K 2 + 8 3 Γ 2 + A k ) S 0 2 cos 2 θ , H_{\mathrm{MAEout}}^{FM} = \left(\frac{1}{2}K_1+\frac{4}{3}\Gamma_1-K_2+\frac{8}{3}\Gamma_2+A_k\right)S_0^2\cos^2\theta, H MAEout F M = ( 2 1 K 1 + 3 4 Γ 1 − K 2 + 3 8 Γ 2 + A k ) S 0 2 cos 2 θ ,
H M A E o u t G A F M = ( − 1 2 K 1 − 4 3 Γ 1 − K 2 + 8 3 Γ 2 + A k ) S 0 2 cos 2 θ , H_{\mathrm{MAEout}}^{GAFM} = \left(-\frac{1}{2}K_1-\frac{4}{3}\Gamma_1-K_2+\frac{8}{3}\Gamma_2+A_k\right)S_0^2\cos^2\theta, H MAEout G A F M = ( − 2 1 K 1 − 3 4 Γ 1 − K 2 + 3 8 Γ 2 + A k ) S 0 2 cos 2 θ ,
H M A E o u t B A F M = ( 2 K 1 − 4 2 3 Γ 1 ) S 0 2 sin θ cos θ + ( − K 2 − 4 3 Γ 2 + A k ) S 0 2 cos 2 θ . (67) H_{\mathrm{MAEout}}^{BAFM} = \left(\sqrt{2}K_1-\frac{4\sqrt{2}}{3}\Gamma_1\right)S_0^2\sin\theta\cos\theta+\left(-K_2-\frac{4}{3}\Gamma_2+A_k\right)S_0^2\cos^2\theta. \tag*{(67)} H MAEout B A F M = ( 2 K 1 − 3 4 2 Γ 1 ) S 0 2 sin θ cos θ + ( − K 2 − 3 4 Γ 2 + A k ) S 0 2 cos 2 θ . (67)
The in-plane MAE is given by:
H M A E i n F M = 0 , H_{\mathrm{MAEin}}^{FM}=0, H MAEin F M = 0 ,
H M A E i n G A F M = 0 , H_{\mathrm{MAEin}}^{GAFM}=0, H MAEin G A F M = 0 ,
H M A E i n B A F M = ( − 2 K 2 − 8 3 Γ 2 ) S 0 2 sin 2 φ . (68) H_{\mathrm{MAEin}}^{BAFM}=\left(-2K_2-\frac{8}{3}\Gamma_2\right)S_0^2\sin^2\varphi. \tag*{(68)} H MAEin B A F M = ( − 2 K 2 − 3 8 Γ 2 ) S 0 2 sin 2 φ . (68)
Based on the above in-plane and out-of-plane MAE expressions, we first set all interaction parameters to unity for a preliminary analysis, with the results shown in Figure S7 .
Our analytical derivation reveals a critical decoupling mechanism: while the SIA energy is determined solely by the local ligand field and thus remains robust against magnetic ordering, the bond-dependent Kitaev-like energy fluctuates significantly with the magnetic structure. We visualize this contrast in Figure S7 by plotting the theoretical angular dependence for FM, Néel AFM, and BAFM orders (with all parameters set to 1). As shown by the blue curves, SIA is identical across different magnetic orders and only has an out-of-plane component. However, the total MAE exhibits drastic variations driven by bond-dependent terms: they amplify the anisotropy in the FM state [Figure S7 ], effectively cancel the SIA in the GAFM state [Figure S7 ], and induce a complex in-plane anisotropy in the BAFM [Figure S7 ]. The differences in MAE among these three magnetic configurations intuitively and concisely express the existence of Kitaev-like interactions beyond SIA in the model.
In our Kitaev-like bond-dependent model, the MAE consists of the 1NN Kitaev-like term and off-diagonal term, the 2NN Kitaev-like term and off-diagonal term, and SIA. For specific FeX systems, we can obtain the parameters for Kitaev-like and SIA interactions by fitting the theoretically derived formulas to DFT calculation results using the least squares method. Substituting these parameters into the formulas to generate MAE curves, self-consistency with calculation results would verify the correctness of the model.
DFT calculation results and analysis
After theoretically verifying that our Kitaev-like bond-dependent model, combined with the Energy Mapping + MAE scheme, can successfully disentangle the individual interaction parameters, we performed the corresponding DFT calculations. These include the electronic structure of FeX and the MAE under different magnetic orders. Some of these results have already been presented in Figure 3 of the main text and will not be repeated here. In this section, we describe our computational methods, analyze the electronic structure, and present the results obtained when the secondary off-diagonal interaction Γ ′ \Gamma' Γ ′ is taken into account.
DFT Computational Methods
We employed first-principles calculations under the generalized Bloch condition using the Vienna Ab initio Simulation Package (VASP) [36 ] based on DFT pseudopotential plane-wave method. The Perdew-Burke-Ernzerhof (PBE) functional [37 ] was used. VASP solves the Kohn-Sham equations self-consistently using plane-wave basis functions and calculates forces and tensors via wavefunctions [38 ] . To study monolayer FeTe and FeSe properties effectively, we included a vacuum layer of 20 Å to avoid periodic interactions. The force convergence criterion was set to 5 × 10 − 3 5 \times10^{-3} 5 × 1 0 − 3 eV/Å, and the plane-wave cutoff energy was chosen as 400 eV. The self-consistent electronic step convergence criterion was 1.0 × 10 − 6 1.0 \times10^{-6} 1.0 × 1 0 − 6 eV. Calculations used the GGA + U U U method [39 ] with an effective on-site Coulomb interaction parameter U = 0.5 U = 0.5 U = 0.5 eV. To ensure convergence, k k k -point meshes in the reciprocal space of the primitive cell were set to 18 × 18 × 1 18 \times18 \times1 18 × 18 × 1 for FeTe and FeSe. For 2 × 1 2 \times1 2 × 1 supercells, k k k -points were 9 × 18 × 1 9 \times18 \times1 9 × 18 × 1 , and for 2 × 2 \sqrt{2} \times\sqrt{2} 2 × 2 supercells, they were 12 × 12 × 1 12 \times12 \times1 12 × 12 × 1 . Our calculations also included SOC as implemented by Hobbs et al. in VASP [40 ] .
Electronic Structure of FeTe
The significance of SOC is primarily manifested in the electronic structure. As illustrated in Figure S8 , in the ferromagnetic ground state, the Fermi level crosses electronic states in both cases with and without SOC, indicating metallic behavior. Upon inclusion of SOC, band crossings that appear in the absence of SOC are observed to open gaps (see red ellipses in Figure S8 and (Figure S8 )), demonstrating that SOC plays a crucial role in this system. This observation provides the physical foundation for the emergence of Kitaev-like interactions.
Figure S8. Electronic structure of monolayer FeTe with and without SOC, along with schematic illustrations of orbital interactions. (a) and (b) show the band structures without and with SOC, respectively. The red ellipses indicate the lifting of band degeneracies induced by SOC. (c)-(e) display the projected density of states (PDOS) onto Fe 3 d 3d 3 d orbitals and Te 5 p 5p 5 p orbitals in the absence of SOC. (f)-(h) present the corresponding PDOS when SOC is included. (i) Schematic of the five-fold degenerate 3 d 3d 3 d orbitals of the Fe atom.
In the 2D tetrahedral coordination environment, the Fe atom is situated at the center of the crystal field formed by the surrounding Te atoms. The five degenerate d d d orbitals split into a higher-energy triply degenerate t 2 t_{2} t 2 manifold and a lower-energy doubly degenerate e e e manifold (see Figure S8 ). The density of states shown in Figure S8 -(Figure S8 ) reveals that, overall, the e e e orbitals exhibit relatively localized character, whereas the t 2 t_{2} t 2 orbitals are more spatially extended, indicating that the t 2 t_{2} t 2 states predominantly mediate the interatomic bonding interactions. However, owing to the two-dimensional nature of FeTe, the t 2 g t_{2g} t 2 g and e e e states are not perfectly degenerate.
The main peak of Fe d x y d_{xy} d x y is located well below the Fermi level, reflecting partial direct interactions between first nearest-neighbor Fe atoms. The SOC-induced effects, however, are primarily mediated through indirect exchange interactions via the Te atoms. The Fe d x z d_{xz} d x z and d y z d_{yz} d y z orbitals, together with the Te p x p_{x} p x and p y p_{y} p y states, exhibit relatively extended distributions near and below the Fermi level, characterizing the indirect Fe-Te-Fe superexchange pathways. Specifically, the d x z d_{xz} d x z orbitals of Fe0 _0 0 and the d y z d_{yz} d y z orbitals of Fe1 _1 1 mediate the first nearest-neighbor Kitaev-like interactions. through the Te p p p orbitals, while the d x z d_{xz} d x z orbitals of Fe0 _0 0 and the d x z d_{xz} d x z orbitals of Fe2 _2 2 mediate the second nearest-neighbor Kitaev-like interactions via the same pathway (see Figure 2 (a)). The presence of these bond-dependent Kitaev-like interactions is further corroborated by first-principles DFT calculations.
Electronic Structure of FeSe
The electronic structure of monolayer FeSe exhibits similar characteristics to that of monolayer FeTe. Upon inclusion of SOC, band gaps emerge at the crossing points that appear in the band structure calculated without SOC. The Fe atom is situated at the center of the tetrahedral crystal field formed by the surrounding Se atoms. Under the influence of this coordination field, the Kitaev-like interactions in FeSe are also mediated through the p p p orbitals of the Se atoms.
Figure S9. Electronic structure of monolayer FeSe with and without SOC, along with schematic illustrations of orbital interactions. (a) and (b) show the band structures without and with SOC, respectively. The red ellipses indicate the lifting of band degeneracies induced by SOC. (c)-(e) display the PDOS onto Fe 3 d 3d 3 d orbitals and Se 4 p 4p 4 p orbitals in the absence of SOC. (f)-(h) present the corresponding PDOS when SOC is included.
Discussion of the Off-diagonal Interaction '
In studies of bond-dependent interactions, an additional off-diagonal interaction, commonly denoted by Γ ′ \Gamma' Γ ′ , often appears. Similar to Γ \Gamma Γ , it generally arises from lattice distortions in octahedrally coordinated systems [11 , 23 , 27 ] . The distinction between Γ \Gamma Γ and Γ ′ \Gamma' Γ ′ lies in the bond components involved: Γ \Gamma Γ describes the coupling between the α \alpha α and β \beta β components, whereas Γ ′ \Gamma' Γ ′ describes the coupling between the α \alpha α and γ \gamma γ components or between the β \beta β and γ \gamma γ components. If the relevant symmetry is absent, a further parameter Γ ′ ′ \Gamma'' Γ ′′ may be introduced to distinguish these contributions. In the two-dimensional edge-sharing tetrahedral system, the α \alpha α and β \beta β bonds are symmetry-equivalent in the ideal structure. We therefore introduce a single parameter Γ ′ \Gamma' Γ ′ to describe these interactions. The corresponding Hamiltonian can be written as
H Γ ′ = ∑ ⟨ i , j ⟩ 1 2 Γ i j ′ ( S i α S j γ + S i γ S j α + S i β S j γ + S i γ S j β ) , (69) H_{\Gamma'} = \sum_{\langle i,j\rangle} \frac{1}{2}\Gamma'_{ij}\left(S_i^\alpha S_j^\gamma+ S_i^\gamma S_j^\alpha+ S_i^\beta S_j^\gamma+ S_i^\gamma S_j^\beta\right), \tag*{(69)} H Γ ′ = ⟨ i , j ⟩ ∑ 2 1 Γ ij ′ ( S i α S j γ + S i γ S j α + S i β S j γ + S i γ S j β ) , (69)
We next examine whether the off-diagonal interaction Γ ′ \Gamma' Γ ′ exists in FeX in the absence of lattice distortions, using both perturbation theory and DFT calculations. According to Eq. (28) , the effective spin interaction associated with Γ ′ \Gamma' Γ ′ in the local coordinate frame takes the form
S i α S j γ + S i γ S j α = cos θ ( S ~ i x S ~ j z + S ~ i z S ~ j x ) − sin θ ( S ~ i y S ~ j z + S ~ i z S ~ j y ) , S_i^\alpha S_j^\gamma+ S_i^\gamma S_j^\alpha= \cos\theta\left(\widetilde{S}_i^x\widetilde{S}_j^z + \widetilde{S}_i^z\widetilde{S}_j^x\right) - \sin\theta\left(\widetilde{S}_i^y\widetilde{S}_j^z + \widetilde{S}_i^z\widetilde{S}_j^y\right), S i α S j γ + S i γ S j α = cos θ ( S i x S j z + S i z S j x ) − sin θ ( S i y S j z + S i z S j y ) ,
S i β S j γ + S i γ S j β = − sin θ ( S ~ i x S ~ j z + S ~ i z S ~ j x ) + cos θ ( S ~ i y S ~ j z + S ~ i z S ~ j y ) , S_i^\beta S_j^\gamma+ S_i^\gamma S_j^\beta= -\sin\theta\left(\widetilde{S}_i^x\widetilde{S}_j^z + \widetilde{S}_i^z\widetilde{S}_j^x\right) + \cos\theta\left(\widetilde{S}_i^y\widetilde{S}_j^z + \widetilde{S}_i^z\widetilde{S}_j^y\right), S i β S j γ + S i γ S j β = − sin θ ( S i x S j z + S i z S j x ) + cos θ ( S i y S j z + S i z S j y ) ,
S i α S j γ + S i γ S j α + S i β S j γ + S i γ S j β = 6 3 ( S ~ i x S ~ j z + S ~ i z S ~ j x + S ~ i y S ~ j z + S ~ i z S ~ j y ) . (70) S_i^\alpha S_j^\gamma+ S_i^\gamma S_j^\alpha+ S_i^\beta S_j^\gamma+ S_i^\gamma S_j^\beta= \frac{\sqrt{6}}{3}\left(\widetilde{S}_i^x\widetilde{S}_j^z + \widetilde{S}_i^z\widetilde{S}_j^x + \widetilde{S}_i^y\widetilde{S}_j^z + \widetilde{S}_i^z\widetilde{S}_j^y\right). \tag*{(70)} S i α S j γ + S i γ S j α + S i β S j γ + S i γ S j β = 3 6 ( S i x S j z + S i z S j x + S i y S j z + S i z S j y ) . (70)
For this off-diagonal interaction, the α , γ \alpha, \gamma α , γ and β , γ \beta, \gamma β , γ components cannot be separated and have identical coefficients. We therefore describe them using a single parameter Γ ′ \Gamma' Γ ′ . The corresponding cross terms between spin components do not arise in the theoretical derivation presented in Sec. S2. We therefore conclude that, in the absence of lattice distortions, the Γ ′ \Gamma' Γ ′ interaction is absent from the model considered here.
We further verify this conclusion using DFT calculations. Starting from the spin Hamiltonian in Eq. (69) , one can derive the MAE resulting from a finite Γ ′ \Gamma' Γ ′ for different magnetic configurations:
H Γ 1 ′ F M = H Γ 1 ′ G A F M = 0 , H Γ 1 ′ B A F M = − 2 3 3 Γ 1 ′ S 0 2 sin θ cos θ sin φ , H Γ 2 ′ F M = H Γ 2 ′ G A F M = 0 , H Γ 2 ′ B A F M = 4 6 3 Γ 2 ′ S 0 2 sin 2 θ sin φ cos φ . (71) \begin{aligned}
H_{\Gamma'_1}^{FM} &= H_{\Gamma'_1}^{GAFM} = 0, \qquad H_{\Gamma'_1}^{BAFM} = -\frac{2\sqrt{3}}{3}\Gamma'_1 S_0^2 \sin\theta\cos\theta\sin\varphi,\\
H_{\Gamma'_2}^{FM} &= H_{\Gamma'_2}^{GAFM} = 0, \qquad H_{\Gamma'_2}^{BAFM} = \frac{4\sqrt{6}}{3}\Gamma'_2 S_0^2 \sin^2\theta\sin\varphi\cos\varphi. \tag*{(71)}
\end{aligned} H Γ 1 ′ F M H Γ 2 ′ F M = H Γ 1 ′ G A F M = 0 , H Γ 1 ′ B A F M = − 3 2 3 Γ 1 ′ S 0 2 sin θ cos θ sin φ , = H Γ 2 ′ G A F M = 0 , H Γ 2 ′ B A F M = 3 4 6 Γ 2 ′ S 0 2 sin 2 θ sin φ cos φ . (71)
The expression shows that the contribution of Γ ′ \Gamma' Γ ′ vanishes for both the 1NN and 2NN interactions in the FM and GAFM configurations. This cancellation results from symmetry: the contributions of the Γ ′ \Gamma' Γ ′ interactions on different bonds exactly cancel in the total MAE. In the BAFM configuration, however, Γ ′ \Gamma' Γ ′ gives a finite contribution to the MAE because the magnetic order lowers the corresponding symmetry.
The extraction of the 1NN parameter Γ 1 ′ \Gamma'_1 Γ 1 ′ from the MAE therefore requires an additional calculation. As seen from Eq. (71) , for the originally defined in-plane rotation, [ 100 ] → [ 010 ] [100] \to[010] [ 100 ] → [ 010 ] with θ = 90 ∘ \theta= 90^\circ θ = 9 0 ∘ , and the out-of-plane rotation, [1 ] → \rightarrow → [10 ] with φ = 90 ∘ \varphi= 90^\circ φ = 9 0 ∘ , the BAFM MAE remains angle-independent and is therefore insensitive to Γ 1 ′ \Gamma'_1 Γ 1 ′ . To determine its contribution, we consider an additional out-of-plane rotation, [1 ] → \rightarrow → [10 ] , corresponding to φ = 90 ∘ \varphi= 90^\circ φ = 9 0 ∘ . In this geometry, the out-of-plane MAE contains a contribution from Γ 1 ′ \Gamma'_1 Γ 1 ′ and is given by
E M A E p e r 2 B A F M = − 2 3 Γ 1 ′ S 0 2 sin θ cos θ + ( K 2 + 2 3 Γ 2 + A k ) S 0 2 cos 2 θ (72) E_{\mathrm{MAEper2}}^{\mathrm{BAFM}}=-\frac{2}{\sqrt{3}}\Gamma'_1 S_0^2\sin\theta\cos\theta+\left(K_2+\frac{2}{3}\Gamma_2+A_k\right)S_0^2\cos^2\theta \tag*{(72)} E MAEper2 BAFM = − 3 2 Γ 1 ′ S 0 2 sin θ cos θ + ( K 2 + 3 2 Γ 2 + A k ) S 0 2 cos 2 θ (72)
Similarly, the in-plane MAE contains a contribution from Γ 2 ′ \Gamma'_2 Γ 2 ′ and should be modified as
E M A E i n B A F M = ( − 2 K 2 − 4 3 Γ 2 ) S 0 2 sin 2 φ + 4 6 3 Γ 2 ′ S 0 2 sin φ cos φ . (73) E_{\mathrm{MAEin}}^{\mathrm{BAFM}}=\left(-2K_2-\frac{4}{3}\Gamma_2\right)S_0^2\sin^2\varphi+\frac{4\sqrt{6}}{3}\Gamma'_2S_0^2\sin\varphi\cos\varphi. \tag*{(73)} E MAEin BAFM = ( − 2 K 2 − 3 4 Γ 2 ) S 0 2 sin 2 φ + 3 4 6 Γ 2 ′ S 0 2 sin φ cos φ . (73)
Refitting the calculated MAE data gives the interaction parameters for FeTe and FeSe listed in Table S2 . All values are given in meV/Fe, and the results are compared with those obtained previously.
Table S2. Interaction parameters of FeTe and FeSe obtained after including the off-diagonal interaction Γ ′ \Gamma' Γ ′ .
Here, “Previous” denotes the results obtained from the original fitting procedure without including Γ ′ \Gamma' Γ ′ . The comparison shows that, even after incorporating the out-of-plane zy-plane MAE of the BAFM configuration, the extracted interaction parameters remain nearly unchanged, with relative variations below 1%. Furthermore, both Γ 1 ′ \Gamma'_1 Γ 1 ′ and Γ 2 ′ \Gamma'_2 Γ 2 ′ are nearly zero in FeTe and FeSe and are much smaller than the other interaction parameters. We therefore conclude that the contribution of the Γ ′ \Gamma' Γ ′ interaction in monolayer FeTe and FeSe is negligible and may be safely omitted.
To further substantiate this conclusion, we compare the calculated MAE with the analytical expressions evaluated using the fitted parameters, as shown in Figure S10 .
Figure S10. Calculated and fitted MAE curves for monolayer FeTe and FeSe based on the Kitaev-like model. The discrete points denote the calculated values, with one data point obtained every 5 ∘ 5^\circ 5 ∘ . The solid curves are obtained by substituting the fitted interaction parameters into the analytical expressions. The blue, red, and green curves correspond to the out-of-plane rotation in the z x zx z x plane (φ = 0 ∘ \varphi= 0^\circ φ = 0 ∘ ), the in-plane rotation in the x y xy x y plane (θ = 90 ∘ \theta= 90^\circ θ = 9 0 ∘ ), and the out-of-plane rotation in the z y zy z y plane (φ = 90 ∘ \varphi= 90^\circ φ = 9 0 ∘ ), respectively, for the different magnetic orders. Panels (a)-(c) show the results for FeTe, while panels (d)-(e) show the results for FeSe.
The fitted curves agree very well with the DFT results for all interaction parameters. We also performed fits using the parameters obtained previously without including Γ ′ \Gamma' Γ ′ . The resulting curves are nearly indistinguishable from those obtained after including Γ ′ \Gamma' Γ ′ , and are therefore not shown. This indicates that including Γ ′ \Gamma' Γ ′ leads to overfitting and further supports the validity of our model, as well as the conclusion that any Γ ′ \Gamma' Γ ′ interaction in this structure is extremely weak. Combined with the perturbative derivation, which shows that the spin-component terms associated with Γ ′ \Gamma' Γ ′ do not arise in the two-dimensional edge-sharing tetrahedral structure, these results demonstrate that the ideal FeX structure does not support a Γ ′ \Gamma' Γ ′ off-diagonal interaction in the absence of lattice distortions.
Studies of the Dzyaloshinskii-Moriya interaction
In Sec. S2, perturbation theory shows that the Dzyaloshinskii–Moriya (DM) interaction arises between 2NN atoms due to broken inversion symmetry. However, because the Energy Mapping + MAE scheme employs only collinear magnetic configurations, the DM interaction parameter cannot be extracted directly within that framework. To address this limitation, we designed two specific noncollinear magnetic configurations and performed DFT calculations to determine the DM interaction parameters in two-dimensional monolayer FeX (X = T e , S e X = \mathrm{Te}, \mathrm{Se} X = Te , Se ). We also discuss the influence of the DM interaction on the ground-state magnetic ordering.
Multiple magnetic interactions coexist in FeX, including the Heisenberg interaction, biquadratic Heisenberg exchange, single-ion anisotropy, and Kitaev-like and off-diagonal interactions. To extract the pure DM interaction, the optimal approach is to construct two magnetic configurations that alter exclusively the DM interaction while keeping the contributions of all other magnetic interactions identical between the two. The energy difference between these configurations thus isolates the pure DM contribution. The magnetic configurations designed for this purpose are illustrated in Figure S11 .
Figure S11. Magnetic configurations designed for verifying the DM interaction. (a) and (b) show the two respective configurations. The arrows indicate the orientations of the magnetic moments at each site: red arrows denote the ( 1 , 0 , 0 ) (1,0,0) ( 1 , 0 , 0 ) direction, blue arrows denote the ( − 1 , 0 , 0 ) (-1,0,0) ( − 1 , 0 , 0 ) direction, and green arrows denote the ( 0 , 0 , 1 ) (0,0,1) ( 0 , 0 , 1 ) and ( 0 , 0 , − 1 ) (0,0,-1) ( 0 , 0 , − 1 ) directions, respectively.
The two configurations shown in Figure S11 involve noncollinear spin arrangements. Through analytical derivation, we verify that the Heisenberg interaction vanishes in both configurations, while the biquadratic exchange and the SIA contribute equally per atomic site in the two structures. Furthermore, the total energies of the Kitaev-like and off-diagonal interactions are identical in both configurations, and the contributions from atoms with spins along ( 0 , 0 , 1 ) (0,0,1) ( 0 , 0 , 1 ) and ( 0 , 0 , − 1 ) (0,0,-1) ( 0 , 0 , − 1 ) cancel each other exactly between the two configurations. Consequently, the energy difference contains only the pure DM contribution, which can be written as
E b − a = 8 D 2 , (74) E_{b-a} = 8D_{2}, \tag*{(74)} E b − a = 8 D 2 , (74)
From our DFT calculations, the energy difference between configurations (b) and (a) in Figure S11 is found to be E b − a = − 60.493 m e V E_{b-a}=-60.493\,\mathrm{meV} E b − a = − 60.493 meV for FeTe, yielding a DM interaction coefficient of D 2 = − 7.56 m e V / F e D_{2}=-7.56\,\mathrm{meV/Fe} D 2 = − 7.56 meV/Fe . For FeSe, the energy difference is E b − a = − 39.833 m e V E_{b-a}=-39.833\,\mathrm{meV} E b − a = − 39.833 meV , corresponding to a DM interaction coefficient of D 2 = − 4.98 m e V / F e D_{2}=-4.98\,\mathrm{meV/Fe} D 2 = − 4.98 meV/Fe . These values indicate that the DM interaction is relatively robust. Below, we discuss in detail its effects on the spin configuration.
Ground-state spin configuration under the DM interaction
In the perturbative derivation of Sec. S2, the effective spin Hamiltonian for the 2NN DM interaction is given by
H D M I = 1 2 D i j ⋅ ( S i × S j ) , (75) H_{DMI} = \frac{1}{2}\mathbf{D}_{ij} \cdot(\mathbf{S}_i \times\mathbf{S}_j), \tag*{(75)} H D M I = 2 1 D ij ⋅ ( S i × S j ) , (75)
where D i j = D 2 ( z × d i j ) \mathbf{D}_{ij} = D_2(\mathbf{z} \times\mathbf{d}_{ij}) D ij = D 2 ( z × d ij ) , D 2 D_2 D 2 denotes the magnitude of the 2NN DM interaction parameter, z \mathbf{z} z is the unit vector pointing from the Te ligand toward the Fe layer, and d i j \mathbf{d}_{ij} d ij is the unit vector from F e i \mathrm{Fe}_i Fe i to F e j \mathrm{Fe}_j Fe j . Thus, the DM interaction depends on the ligand position relative to the Fe plane, the relative Fe-Fe geometry, and the spin configuration. In monolayer FeTe the DM interaction exists only between 2NNs and therefore acts independently on the two sublattices (the 1NNs within each sublattice coincide with the 2NNs of the full lattice), as illustrated in Figure S12 .
Figure S12. DM interaction representations within different sublattices. Purple and brown spheres with arrows denote the two sublattices of Fe atoms and their corresponding parameters, respectively; gray and black spheres represent the upper and lower Te atoms, respectively. Red and blue arrows indicate the spin directions and the signs of the DM interaction for different sublattices. (a) When both sublattices adopt the same spin arrangement (both clockwise or both counterclockwise), the local DM interaction contributions are opposite, causing cancellation globally, so that only local DM interaction survives. (b) When the two sublattices adopt opposite spin arrangements, the DM interaction contributions add constructively both locally and globally.
To define the spin-rotation direction, we adopt the following convention: if atoms are arranged along the i i i -axis, the perspective is taken along the positive j j j -axis direction; if arranged along the j j j -axis, the perspective is along the positive i i i -axis direction. The physics of Figure S12 can then be interpreted as follows. When both sublattices follow the same spin sequence (both clockwise or both counterclockwise), the DM interaction appears locally within each sublattice but cancels globally, yielding no net DM interaction, as shown in Figure S12 (a). Conversely, when they follow opposite sequences (one clockwise, one counterclockwise), the DM interaction is present both locally and globally, as shown in Figure S12 (b). The above describes the DM interaction rule for Fe-Fe chains along the i i i direction. Assuming the arrangement in Figure S12 (b), i.e., the purple sublattice is arranged clockwise and the brown sublattice counterclockwise, then along the j j j direction the spins should be arranged counterclockwise and clockwise, respectively, to manifest global DM interaction, as depicted in Figure S13 .
Figure S13. Single Fe-atom chain arrangements along the i i i and j j j axes under pure DM interaction (D < 0 D < 0 D < 0 ). Red and blue arrows correspond to the spin arrangements of the purple and brown sublattices, respectively. Along the j j j direction, the spins are oriented along the k k k directions; atoms without spin labels represent sites where any spin configuration yields zero DM interaction energy.
In the two-dimensional configuration of Figure S13 , we identify the lowest-energy arrangement for a single-atom cross-chain along the i i i and j j j axes under pure DM interaction (with D < 0 D < 0 D < 0 ; the DFT extraction scheme and results are presented below). For the purple sublattice, the spin arrangements along the positive i i i and j j j axes are both clockwise; for the brown sublattice, both are counterclockwise. We now evaluate the total energy. In this configuration, the spins along the k k k -axis contribute 1 / 4 1/4 1/4 of the total, with a per-site DM interaction energy of 2 D 2D 2 D ; the spins along the i i i or j j j axes contribute 1 / 2 1/2 1/2 , with a per-site contribution of D D D ; and the remaining 1 / 4 1/4 1/4 of sites yield zero DM interaction regardless of spin arrangement. Thus, the average DM interaction energy per atom is D D D , which is not the globally lowest-energy configuration.
Next, we determine the ground state by employing the spin-spiral method. We assume the spin at site ( m , n ) (m,n) ( m , n ) forms a spiral wave and represent it as a unit vector:
S m n = ( A sin ( q x m + q y n ) , B sin ( q x m + q y n ) , C cos ( q x m + q y n ) ) , (76) \mathbf{S}_{mn} = \left(A\sin(q_xm+q_yn), B\sin(q_xm+q_yn), C\cos(q_xm+q_yn)\right), \tag*{(76)} S mn = ( A sin ( q x m + q y n ) , B sin ( q x m + q y n ) , C cos ( q x m + q y n ) ) , (76)
where q = ( q x , q y ) \mathbf{q}=(q_x,q_y) q = ( q x , q y ) denotes the wavevector components, R = m i + n j \mathbf{R}=m\mathbf{i}+n\mathbf{j} R = m i + n j is the lattice position, and the parameters satisfy A 2 + B 2 = C = 1 A^2+B^2=C=1 A 2 + B 2 = C = 1 . In this framework, we represent the interaction between neighboring spins in a bond-dependent form. Only two types of 2NN bonds exist: those oriented along the i i i direction are designated as x-bonds, and those along the j j j direction as y-bonds. The energies of the x and y bonds can therefore be expressed as
E x = D { C cos ( q x m + q y n ) A sin [ q x ( m + 1 ) + q y n ] − A sin ( q x m + q y n ) C cos [ q x ( m + 1 ) + q y n ] } = D A sin ( q x ) , E y = D { B sin ( q x m + q y n ) C cos [ q x m + q y ( n + 1 ) ] − C cos ( q x m + q y n ) B sin [ q x m + q y ( n + 1 ) ] } = D B sin ( − q y ) , (77) \begin{aligned}
E_x &=D\{C\cos(q_xm+q_yn)A\sin[q_x(m+1)+q_yn]\\
&\quad-A\sin(q_xm+q_yn)C\cos[q_x(m+1)+q_yn]\}=DA\sin(q_x),\\
E_y &=D\{B\sin(q_xm+q_yn)C\cos[q_xm+q_y(n+1)]\\
&\quad-C\cos(q_xm+q_yn)B\sin[q_xm+q_y(n+1)]\}=DB\sin(-q_y), \tag*{(77)}
\end{aligned} E x E y = D { C cos ( q x m + q y n ) A sin [ q x ( m + 1 ) + q y n ] − A sin ( q x m + q y n ) C cos [ q x ( m + 1 ) + q y n ]} = D A sin ( q x ) , = D { B sin ( q x m + q y n ) C cos [ q x m + q y ( n + 1 )] − C cos ( q x m + q y n ) B sin [ q x m + q y ( n + 1 )]} = D B sin ( − q y ) , (77)
where E x E_x E x and E y E_y E y depend only on q x q_x q x and q y q_y q y , not on the absolute atomic positions. Summing over all bonds yields the total energy
E B o n d = E x + E y = D [ A sin ( q x ) − B sin ( q y ) ] , (78) E_{\mathrm{Bond}}=E_x+E_y=D[A\sin(q_x)-B\sin(q_y)], \tag*{(78)} E Bond = E x + E y = D [ A sin ( q x ) − B sin ( q y )] , (78)
Minimization of E E E with respect to q x q_x q x and q y q_y q y gives the lowest-energy state when A = B = 1 2 A=B=\frac{1}{\sqrt{2}} A = B = 2 1 , q x = π 2 q_x=\frac{\pi}{2} q x = 2 π , q y = − π 2 q_y=-\frac{\pi}{2} q y = − 2 π , which is one of several degenerate solutions. The corresponding spin configuration for the purple sublattice is
S m n = ( 1 2 sin π 2 ( m − n ) , 1 2 sin π 2 ( m − n ) , cos π 2 ( m − n ) ) , (79) \mathbf{S}_{mn}=\left(\frac{1}{\sqrt{2}}\sin\frac{\pi}{2}(m-n),\frac{1}{\sqrt{2}}\sin\frac{\pi}{2}(m-n),\cos\frac{\pi}{2}(m-n)\right), \tag*{(79)} S mn = ( 2 1 sin 2 π ( m − n ) , 2 1 sin 2 π ( m − n ) , cos 2 π ( m − n ) ) , (79)
For the brown sublattice, the spin configuration is obtained by reversing the chirality (e.g., changing the sign of the A A A or B B B component), yielding the ground-state arrangement shown in Figure S14 .
Figure S14. Ground-state spin arrangement of monolayer FeX (X = T e , S e X = \mathrm{Te}, \mathrm{Se} X = Te , Se ) under pure DMI. The spin directions are restricted to ( 0 , 0 , 1 ) (0, 0, 1) ( 0 , 0 , 1 ) , ( 0 , 0 , − 1 ) (0, 0, -1) ( 0 , 0 , − 1 ) , ( 1 , 1 , 0 ) (1, 1, 0) ( 1 , 1 , 0 ) and ( 1 , − 1 , 0 ) (1, -1, 0) ( 1 , − 1 , 0 ) .
As shown in Figure S14 , the spin arrangement along the Fe-Fe chains in the i i i and j j j directions corresponds to a rotation within the ( 1 1 ˉ 0 ) (1\bar{1}0) ( 1 1 ˉ 0 ) plane. At this configuration, the average DMI energy per site is 2 D \sqrt{2}D 2 D , which is the global minimum for pure DM interaction. Conversely, if the spiral propagates along the ( 1 1 ˉ 0 ) (1\bar{1}0) ( 1 1 ˉ 0 ) direction, the energy is equivalent and also minimal. So it selects the chirality and propagation direction of the spin rotation.
In quadratically symmetric lattices, multiple symmetry-equivalent spiral wavevectors exist. Under the combined effects of external magnetic fields, magnetic anisotropy, strain, or higher-order exchange interactions, these single-q q q spiral states may compete or superpose, potentially forming multi-q q q noncollinear magnetic structures such as square skyrmions, antiskyrmions, or bimerons [54 , 55 ] . However, the stability of such structures and their corresponding phase diagrams require further verification through Monte Carlo simulations, spin dynamics, or energy-barrier calculations based on the complete spin Hamiltonian. [56 , 57 ] Consequently, monolayer FeX is expected to serve as a promising platform for studying DM-interaction-driven two-dimensional noncollinear magnetism, as well as the external-field or strain-controlled topological spin textures.
Orbital superexchange paths in the global coordinate frame
In the preceding analysis, the orbital components were defined in the local coordinate frame. To elucidate the overall nature of the bond-dependent Kitaev-like interaction, we now analyze the corresponding superexchange pathways in the global coordinate frame. For this purpose, we also provide the Slater–Koster expressions in the global frame to facilitate the analysis of the hopping integrals between different orbitals. Since the symbol t t t was already used for the hopping integrals in the local coordinate frame, we denote by S S S the hopping integrals containing both V p d σ V_{pd\sigma} V p d σ and V p d π V_{pd\pi} V p d π contributions, and by P P P those containing only the V p d π V_{pd\pi} V p d π contribution. The hopping matrix from the Fe d d d orbitals to the Te p p p orbitals is then written as
T X 1 M 0 = ( 0 P 3 0 0 − P 4 0 S 3 0 S 1 S 1 0 S 2 P 1 0 − P 2 ) , T X 2 M 0 = ( P 3 0 0 0 − S 3 S 1 P 4 0 0 0 − S 1 S 2 0 P 1 P 2 ) , T X 1 M 2 = ( 0 − P 3 0 0 − P 4 0 S 3 0 − S 1 − S 1 0 S 2 − P 1 0 − P 2 ) . (80) \mathbf{T}_{X_{1}M_{0}} =
\begin{pmatrix}
0 & P_{3} & 0 \\
0 & -P_{4} & 0 \\
S_{3} & 0 & S_{1} \\
S_{1} & 0 & S_{2} \\
P_{1} & 0 & -P_{2}
\end{pmatrix}, \qquad
\mathbf{T}_{X_{2}M_{0}} =
\begin{pmatrix}
P_{3} & 0 & 0 \\
0 & -S_{3} & S_{1} \\
P_{4} & 0 & 0 \\
0 & -S_{1} & S_{2} \\
0 & P_{1} & P_{2}
\end{pmatrix}, \qquad
\mathbf{T}_{X_{1}M_{2}} =
\begin{pmatrix}
0 & -P_{3} & 0 \\
0 & -P_{4} & 0 \\
S_{3} & 0 & -S_{1} \\
-S_{1} & 0 & S_{2} \\
-P_{1} & 0 & -P_{2}
\end{pmatrix}. \tag*{(80)} T X 1 M 0 = 0 0 S 3 S 1 P 1 P 3 − P 4 0 0 0 0 0 S 1 S 2 − P 2 , T X 2 M 0 = P 3 0 P 4 0 0 0 − S 3 0 − S 1 P 1 0 S 1 0 S 2 P 2 , T X 1 M 2 = 0 0 S 3 − S 1 − P 1 − P 3 − P 4 0 0 0 0 0 − S 1 S 2 − P 2 . (80)
The individual hopping components are given by
S 1 = 2 3 V p d σ + 6 9 V p d π , S 2 = − 1 3 V p d σ + 2 3 9 V p d π , S 3 = − 2 3 V p d σ + 3 9 V p d π , P 1 = − 2 3 V p d π , P 2 = + 2 3 V p d π , P 3 = + 6 3 V p d π , P 4 = + 3 3 V p d π . (81) \begin{aligned}
S_{1} &= \frac{\sqrt{2}}{3}V_{pd\sigma}+\frac{\sqrt{6}}{9}V_{pd\pi},\\
S_{2} &= -\frac{1}{3}V_{pd\sigma}+\frac{2\sqrt{3}}{9}V_{pd\pi},\\
S_{3} &= -\frac{2}{3}V_{pd\sigma}+\frac{\sqrt{3}}{9}V_{pd\pi},\\
P_{1} &= -\frac{\sqrt{2}}{3}V_{pd\pi},\\
P_{2} &= +\frac{2}{3}V_{pd\pi},\\
P_{3} &= +\frac{\sqrt{6}}{3}V_{pd\pi},\\
P_{4} &= +\frac{\sqrt{3}}{3}V_{pd\pi}. \tag*{(81)}
\end{aligned} S 1 S 2 S 3 P 1 P 2 P 3 P 4 = 3 2 V p d σ + 9 6 V p d π , = − 3 1 V p d σ + 9 2 3 V p d π , = − 3 2 V p d σ + 9 3 V p d π , = − 3 2 V p d π , = + 3 2 V p d π , = + 3 6 V p d π , = + 3 3 V p d π . (81)
where V p d σ V_{pd\sigma} V p d σ and V p d π V_{pd\pi} V p d π have the same meanings as in the local coordinate frame. The orbital bases in the global coordinate frame are defined as ( d x y , d y z , d x z , d x 2 − y 2 , d 3 z 2 − r 2 ) (d_{xy},d_{yz},d_{xz},d_{x^{2}-y^{2}},d_{3z^{2}-r^{2}}) ( d x y , d y z , d x z , d x 2 − y 2 , d 3 z 2 − r 2 ) and ( p x , p y , p z ) (p_{x},p_{y},p_{z}) ( p x , p y , p z ) .
The exchange pathways of the d x z d_{xz} d x z and d y z d_{yz} d y z orbitals along the x x x and y y y directions are symmetry-related. Likewise, the exchange pathways involving the d x 2 − y 2 d_{x^{2}-y^{2}} d x 2 − y 2 and d z 2 d_{z^{2}} d z 2 orbitals have the same structure and differ only in their hopping amplitudes. We therefore present only representative superexchange processes below. The global-frame Slater–Koster hopping matrix further shows that the σ \sigma σ -bond channel appears only in the hopping pathways involving the d x z d_{xz} d x z , d y z d_{yz} d y z , and d z 2 d_{z^{2}} d z 2 orbitals. Accordingly, Figure 2 of the main text focuses on the superexchange processes involving these three orbitals. We next briefly discuss several representative processes in the complete superexchange pathways. The σ \sigma σ - and π \pi π -bond channels will not be distinguished separately here; the corresponding hopping integrals can be obtained from Eq. (81) .
As shown in Figure S15 , the d x z d_{xz} d x z -d x z d_{xz} d x z channel contains spin-flip processes in the superexchange pathway. For the 1NN pair, a hole hops from the Fe0 d x z d_{xz} d x z orbital to the hybridized Te1 p x / p z p_{x}/p_{z} p x / p z orbital, is projected onto the Te1 p x p_{x} p x orbital through SOC, and subsequently hops to the Fe d x z d_{xz} d x z orbital of the neighboring Fe atom. Among these pathways, only the d x z d_{xz} d x z -p z p_{z} p z -p x p_{x} p x -d x z d_{xz} d x z path contributes to spin-flip hopping. The corresponding process through Te2 is analogous, giving the spin-flip path d x z d_{xz} d x z -p x p_{x} p x -p z p_{z} p z -d x z d_{xz} d x z . Because the pathways through Te1 and Te2 are symmetry-related, their hopping integrals are identical. However, the SOC matrix elements associated with the two spin-flip processes are ⟨ p z ∣ H S O C ∣ p x ⟩ \langle p_{z}|H_{\mathrm{SOC}}|p_{x}\rangle ⟨ p z ∣ H SOC ∣ p x ⟩ and ⟨ p x ∣ H S O C ∣ p z ⟩ \langle p_{x}|H_{\mathrm{SOC}}|p_{z}\rangle ⟨ p x ∣ H SOC ∣ p z ⟩ , respectively, and their contributions cancel each other. Thus, although spin-flip pathways are present for the 1NN pair, they do not produce a net bond-dependent Kitaev-like interaction. For the 2NN pair, this cancellation does not occur because the relevant symmetry is broken and the virtual hopping proceeds through only one coordinating atom. The corresponding pathways are d x z d_{xz} d x z -p x p_{x} p x -p z p_{z} p z -d x z d_{xz} d x z and d x z d_{xz} d x z -p z p_{z} p z -p x p_{x} p x -d x z d_{xz} d x z . Although these pathways appear analogous to those discussed above, their hopping integrals are different, as specified in Eq. (81) . Consequently, the two pathways yield the same net contribution and contribute equally to the spin-flip exchange.
Figure S15. Representative spin-flip superexchange pathways between t 2 t_2 t 2 orbitals. The labels 0, 1 and 2 denote the central Fe atom, 1NN Fe atom, and 2NN Fe atom, respectively. The paths connecting different atoms represent hopping processes, whereas the paths involving the same Te atom represent transformations between Te p p p orbitals induced by SOC. In the schematic representation of the Te orbitals, the blue regions denote hybridized p x / p z p_x/p_z p x / p z or p y / p z p_y/p_z p y / p z orbitals, while the brown regions denote p z p_z p z orbitals. (a) d x z d_{xz} d x z -d x z d_{xz} d x z . (b) d x z d_{xz} d x z -d y z d_{yz} d y z . (c) d x y d_{xy} d x y -d x y d_{xy} d x y . (d) d x z d_{xz} d x z -d x y d_{xy} d x y .
The d x z d_{xz} d x z -d y z d_{yz} d y z and d x z d_{xz} d x z -d y z d_{yz} d y z pathways shown in Figure S15 and (Figure S15 ), respectively, correspond to conventional spin-flip processes and will not be discussed further here. The Slater–Koster hopping integrals in Eq. (81) can be used to identify the specific spin-flip superexchange pathways.
For the d x y d_{xy} d x y -d x y d_{xy} d x y channel shown in Figure S15 , the 1NN pathway is analogous to the d x z d_{xz} d x z -d x z d_{xz} d x z case: spin-flip pathways exist, but their contributions cancel in the total exchange process. For the 2NN pair, no spin-flip hopping pathway exists, and this channel therefore does not contribute to the Kitaev-like interaction.
Figure S16 shows representative superexchange pathways involving the e e e orbitals. Similar to the t 2 t_2 t 2 -t 2 t_2 t 2 channels, spin-flip pathways exist between the same orbitals for 1NN pairs, but their contributions cancel in the total exchange, as illustrated in Figure S16 . By contrast, interorbital exchange pathways generally contain nonvanishing spin-flip contributions.
Figure S16. Representative spin-flip superexchange pathways involving the e e e orbitals. The labels 0, 1, and 2 denote the central Fe atom, 1NN Fe atom, and 2NN Fe atom, respectively. The paths connecting different atoms represent hopping processes, whereas the paths involving the same Te atom represent transformations between Te p p p orbitals induced by SOC. In the schematic representation of the Te orbitals, the blue regions denote hybridized p x / p z p_x/p_z p x / p z or p y / p z p_y/p_z p y / p z orbitals, while the brown regions denote p z p_z p z orbitals. (a) d y z − d z 2 d_{yz}-d_{z^2} d y z − d z 2 . (b) d x y − d x 2 − y 2 d_{xy}-d_{x^2-y^2} d x y − d x 2 − y 2 . (c) d z 2 − d z 2 d_{z^2}-d_{z^2} d z 2 − d z 2 . (d) d z 2 − d x 2 − y 2 d_{z^2}-d_{x^2-y^2} d z 2 − d x 2 − y 2 .