WorksPhysics and Astronomy
Unveiling Excitonic Insulator Signatures in TaNiSe
Abstract
The high-temperature phase of TaNiSe, a near-zero-gap semiconductor ( = 0), is a promising candidate for an excitonic insulator. Given the dome-like evolution expected for an excitonic insulator around , we investigated TaNiSe, the more semi-metallic Ta(Ni,Co)Se, and semiconducting TaNiS using high-resolution single-crystal x-ray diffraction and near-edge x-ray absorption fine structure (NEXAFS). Our findings reveal a second-order structural phase transition from orthorhombic (space group: ) to monoclinic (space group: ) in TaNiSe and Ta(Ni,Co)Se, but no transition in TaNiS down to 2 K. This transition breaks two mirror symmetries, enabling and enhancing the hybridization of Ta, Ni, and Se atoms, shortening bond lengths, and strengthening orbital interactions. NEXAFS data confirm stronger hybridization, significant changes in excitonic binding energies, and a key alteration in orbital character, suggesting an excitonic insulating state in TaNiSe and emphasizing the crucial electronic role of orbitals in the formation of the excitonic insulator state.
Introduction
Quantum materials harbor a plethora of very intriguing exotic many-body phases among which one of the most fascinating phenomena is the emergence of an excitonic insulator (EI). The idea of an EI was theoretically already proposed about half a century ago. It is well known that poorly screened Coulomb interactions between conduction-band electrons and valence-band holes can lead to the formation of electron-hole pairs called excitons [1, 2]. Especially, if these electron-hole pairs are formed in the presence of an almost vanishing band gap such charge neutral quasi-particles have the potential to condense in an unconventional insulating ground state, the EI [3, 4]. Consequently, the EI was predicted to be realized in narrow band gap semiconductors (semimetals) with a small positive (negative) band gap. Theoretically, it has been established that the BCS theory can be used to describe the EI phase in semimetals, while for semiconductors it can be better described in terms of Bose-Einstein condensation (BEC) [5, 6, 7].
Candidate materials for possible realizations of an EI include TmSeTe, where it has been claimed, based on transport properties, that the pressure induced transition in this compound points to an EI state [8, 9, 10]. An alternative candidate is the semi-metallic compound 1T-TiSe, where a chiral charge density wave (CDW) was observed below K [11] and interpreted to be of the EI type [12, 13, 14]. However, in contrast to the pure EI scenario where lattice distortions play no or merely a secondary role, the transitions in both compounds are strongly influenced by lattice degrees of freedom, thereby, obscuring a definitive assessment and leaving room for questioning the existence of an EI phase in these materials [15, 16, 17].
The transition-metal chalcogenide TaNiSe, which was first reported by Sunshine et al. [18] and Di Salvo et al. [19], is considered a further promising candidate for the realization of the EI state. This compound has a layered structure for which planes composed of one Ni-Se and two Ta-Se chains along the crystallographic axis are stacked by van der Waals interactions along the direction, as illustrated in Figure 6. Within the chains, Ni (Ta) is tetrahedrally (octahedrally) coordinated with Se ions. The compound undergoes a second order structural phase transition from orthorhombic, space group (SG) , to monoclinic, SG , at K [18, 19]. A semiconductor- or semimetal-to-insulator (SI) transition is observed at and has been assumed to be driven by an EI origin [17, 20, 21, 22].

Figure 6. Orbital model for TaNiSe: (a) In the orthorhombic phase at 350 K, and (b) in the monoclinic phase at 80 K. In both sketches the Ni and orbitals symbolize the states relevant for the hybridization in the NiSe tetrahedron. Similarly, the Ta orbitals (coordinates were defined with respect to the unit cell) pointing between the Se neighbors and towards the Ni atoms symbolize the states relevant for the hybridization in the TaSe octahedron and for the direct Ta-Ni bond. Among the Ta states, these orbitals are the only ones that can effectively hybridize simultaneously with both the Ni and Se orbitals. In the orthorhombic phase, hybridization between Ni and Ta states are forbidden due to their opposite mirror parity, while in the monoclinic phase, two mirror symmetries are broken, permitting hybridization between the relevant Ta, Ni, and Se orbitals. Note, the perspective view and the projections along the main directions change between (a) and (b), as indicated by the red and orange rectangles in the figures.
First indications for an EI ground state were found in TaNiSe using optical conductivity measurements where an opening of a gap of eV, comparable to the expected excitonic binding energy consistent with the enhanced activation energy observed in the transport data and the rapid increase in the resistivity below [17]. Moreover, angle resolved photoemission spectroscopy (ARPES) shows an M-shaped flattening of the valence band top below , which was interpreted as the formation of an additional gap induced by the EI transition [23]. This result was also supported by band structure calculations where a very small direct gap at the point of the Brillouin zone is reported above . No superlattice reflections are observed below , which excludes a CDW as the origin for the structural transition in TaNiSe [24]. Furthermore, the leading role of electronic correlations in driving the EI transition is supported by several theoretical studies [24, 25, 26, 27] and by the observation of critical charge fluctuations in Raman spectra [22, 28]. Nevertheless, there is an ongoing debate about whether the transition is primarily of excitonic origin [17, 20, 21, 22, 23, 24, 25, 26, 27, 28, 29] or driven by lattice distortion and electron-phonon coupling [30, 31, 32].
Previous investigations on TaNi(Se,S), where Se is replaced by S, revealed that increasing S substitution weakens the hybridization between Ni and Se (S ) orbitals, consequently enhancing the band gap energy, . Specifically, the critical temperature, , decreases with increasing and disappears around , while exhibits a linear increase as a function of [17, 33]. For the quasi-1D end member of the series, TaNiS, no phase transition was observed in the temperature-dependent resistivity down to 2 K. Furthermore, both transport and optical measurements indicate that exceeds the binding energy of the exciton, , suggesting the absence of an excitonic transition in TaNiS [17, 34]. Additionally, ARPES revealed no evidence for a flattening at the top of the valence band, indicating that the ground state of an EI cannot be realized in this material [35]. In contrast, substitution of Co for Ni in Ta(Ni,Co)Se increases the negative band gap and reduces the transition temperature, reinforcing the semimetallic character. As shown in Ref. [36], the transition temperature significantly decreases to 250 K for a Co content around 10 %, and disappears at around [37].
Hence, through appropriate S and Co substitution in TaNiSe, is suppressed by directing the system towards either a semiconducting (positive band gap) or a more semimetallic state (negative band gap), resulting in a substitution-dependent dome-shaped behavior centered at , as anticipated for an excitonic insulator (EI) [17]. Along these lines, the primary motivation for our present investigation lies in probing the spatial and electronic structure of Ta(Ni,Co)(Se,S) employing temperature-dependent high-resolution single-crystal x-ray diffraction (SC-XRD) as well as near-edge x-ray absorption fine structure (NEXAFS), to scrutinize the existence of an EI state in TaNiSe and, if the EI state indeed exists, to identify the relevant Ta, Ni, and Se orbitals responsible for its formation.
We will demonstrate that not only TaNiSe but also TaNiCoSe undergo a second-order structural phase transition from orthorhombic () to monoclinic () symmetry. This transition is characterized by mirror symmetry breaking, which facilitates the hybridization of Ta, Ni, and Se atoms, leading to shortened bond lengths and enhanced orbital interactions. In contrast, TaNiS—in which numerous studies have been conducted, leading to various interpretations and controversies about the existence of a phase transition [17, 18, 19, 33, 38, 39, 40]—shows no evidence of symmetry breaking or changes in bond lengths. The structural findings are corroborated by NEXAFS data, which reveal changes in orbital symmetries and hybridization, unequivocally indicating the formation of an EI state. This effect is most pronounced in TaNiSe, significantly reduced in TaNiCoSe, and, in principle, absent in TaNiS.
Experimental Methods
Single crystals of TaNiSe, Ta(Ni,Co)(Se,S), and TaNiS were synthesized using the chemical vapor transport method, as described in the Supplementary Material (SM) and shown in Fig. S1 [41]. The composition of the obtained samples was verified using energy-dispersive x-ray spectroscopy (EDX)—as illustrated in Fig. S2 in the SM—and SC-XRD. The Co content of the Ta(Ni,Co)Se sample was determined to 7 %.
Temperature-dependent SC-XRD data on TaNiSe, TaNiS, and TaNiCoSe were collected between 360 and 80 K using our in-house high-flux, high-resolution, rotating anode Rigaku Synergy-DW (Mo/Ag) diffractometer with Mo radiation. The system is equipped with a background-less Hypix-Arc150° detector, which guarantees minimal reflection profile distortion and ensures uniform detection conditions for all reflections. All samples were measured to a resolution better than 0.5 Å. The samples exhibited no mosaic spread and no additional reflections from secondary phases, highlighting their high quality and allowing for excellent evaluation using the latest version of the CrysAlisPro software package [42]. The crystal structures of the compounds were refined using JANA2006 [43], including all averaged symmetry-independent reflections () for the refinements in their respective space groups. For each measured temperature, the unit cell and space group were determined, atoms were localized within the unit cell using random phases and the structure was completed and solved using difference Fourier analysis. The structural refinements converged well for all investigated temperatures, exhibiting excellent reliability factors (see Tables S1 and S2 in the SM [41] for residuals , , and goodness of fit, GOF, values). For TaNiS we complemented the data by an SC-XRD experiment at ID15B of the European Synchrotron Radiation Facility (ESRF) [44] down to a temperature of 2 K: Samples were mounted with Apiezon N grease on a 300 m thick diamond plate and were cooled with a Helium flow cryostat. Monochromatic x-rays with a wavelength of 0.41 Å were used, with a spot size of 4 × 4 m. Data were collected through continuous -rotation over a range of 70°, with shutterless readout at intervals of 0.5°. For the data acquisition an EIGER2 X CdTe 9M detector (Dectris, Switzerland) was employed. The detector distance of 179.60 mm was determined utilizing CeO powder and the geometry was calibrated with a natural vanadinite single crystal.
Before conducting our NEXAFS studies at the Institute for Quantum Materials and Technologies beamline WERA at the KIT light source KARA, the orientation of the individual samples was precisely determined using a completely motorized Photonic Science Laue diffraction system (details of our Laue measurements can be found in Fig. S7 in the SM [41]). For TaNiSe, TaNiS, and TaNiCoSe, temperature-dependent NEXAFS data were collected at the edges of Ni. To ensure measurements on freshly prepared and shiny surfaces, the samples were cleaved shortly before the experiment in a preparation chamber with a base pressure of mbar and then transferred in ultrahigh vacuum to the NEXAFS chamber with a pressure of mbar. The surface cleanliness of the samples before and during data collection was monitored with x-ray photoemission spectroscopy. Using linearly polarized light, the spectra were taken in total electron yield (TEY) mode. Photon energy calibration was ensured by adjusting the Ni peak position measured on a NiO standard before and after each NEXAFS scan to the established peak position. While the in-plane spectra with the polarization parallel to the and direction ( and ) were directly obtained using the corresponding normal incidence alignment, out-of-plane spectra () were determined by measuring in a grazing incidence setup (with a grazing angle of ) and subsequently extrapolating the grazing spectra to with the aid of the relevant in-plane spectrum. With regard to an optimal presentation, only the important Ni edges recorded at different temperatures in the TEY mode along the , and directions will be presented in the main text. For completeness a full NEXAFS spectrum measured over the whole energy range is shown as an example in Fig. S8 in the SM [41].
Results and Discussion
XRD: Structure determination and refinement
The structural properties of TaNiSe, TaNiS, and TaNiCoSe, have been studied in detail as a function of temperature. For TaNiSe, the high-temperature structure was refined in the orthorhombic SG as illustrated on the left side of Figure 6, while the low-temperature phase was refined in the monoclinic SG . TaNiCoSe and TaNiS share the same orthorhombic high-temperature structure as TaNiSe with slightly smaller lattice parameters—more structural details on the lattice parameters and atomic positions at the highest and lowest measured temperature are given in the SM [41].
Figure 1 shows the -dependent behavior of the monoclinic order parameter for (a) TaNiSe and (b) TaNiCoSe. Upon cooling, a continuously increasing difference from coupled to the second order structural phase transition is evident below and finally reaches values of for TaNiSe and

Figure 1. Monoclinic order parameter of (a) TaNiSe and (b) TaNiCoSe as a function of temperature. Errors shown are statistical errors from the refinements. Statistical error bars are smaller than the symbol sizes.
for TaNiCoSe at 80 K. The transition temperatures can be determined to K for TaNiSe and K for TaNiCoSe, which is in good agreement with values reported in literature [18, 19, 33, 36]. As representatives, panels (b) and (e) in Figure 2 illustrate a series of reflections taken from precession images for the reciprocal plane reconstructed from single-crystal XRD data collected at 360 and 80 K for TaNiSe and at 280 and 80 K for TaNiCoSe and demonstrate the splitting of the reflections at and below the orthorhombic-to-monoclinic phase transition.

Figure 2. (a) Part of the crystal structure in the monoclinic phase at 80 K. The yellow (brown) arrows represent the bonds that become shorter (longer) below the transition. Interatomic Ta-Ni and Ta-Se3/S3 distances in TaNiSe (c and d), TaNiCoSe (f and g) and TaNiS (i and j) as a function of temperature. Error bars reflect the statistical errors from the refinement and are smaller than the symbol sizes. A clear splitting of the Ta-Ni and Ta-Se3 bond lengths can be observed for TaNiSe and TaNiCoSe which is totally absent for TaNiS where only a small reduction with decreasing temperature is found, simply reflecting the standard thermal expansion behavior. Panels (b) and (e) are parts of the x-ray precession images for the reciprocal (5 3 ) series ( is changing from 4 to -3) reconstructed from single-crystal XRD data collected at 360 and 80 K for TaNiSe, and at 280 and 80 K for TaNiCoSe show the splitting of the reflections below the orthorhombic-to-monoclinic phase transition. Panel (h) shows the precession images for TaNiS again for the reciprocal (5 3 ) series ( is from 4 to -3) at 360 and 80 K from our XRD data, and for the reciprocal (5 1 ) series ( is from 4 to 11) at 2 K from Synchrotron XRD data, where no splitting of the reflections is seen in the whole temperature range. More details and complete precession plots are given in the SM.
For TaNiS, across the entire measured temperature range from 360 to 80 K, the lattice parameters (and consequently the unit cell volume) change smoothly, showing a gradual decrease with decreasing temperature. However, throughout the entire reciprocal lattice, and particularly in the layers with no evidence of reflection splitting is observed. This clearly indicates that the compound does not undergo any phase transition down to 80 K. To ensure that no phase transition occurs below 80 K, we conducted additional synchrotron XRD measurements down to 2 K. None of the accessible reflections in the planes with showed any signs of a splitting or a broadening expected in the case of a twinned crystal below the phase transition. This is exemplified, for instance, by the series of reflections in panel (h) of Figure 2 which were taken from the reciprocal plane reconstructed from single-crystal XRD data collected at 360 and 80 K, as well as from synchrotron XRD data collected at 2 K (full precession figures are shown in Figs. S3, S4, S5, and S6 in the SM [41]). Thus, we can rule out any structural phase transition in our samples down to 2 K. These findings are in line with the resistivity measurements of Ref. [17], which also reported no phase transition in TaNiS down to 2 K. The conflicting reports of a structural instability [39] and even a structural phase transition [38] from Raman experiments, supported by first principle calculations [40], may arise from the fact that XRD and Raman probe different sample volumes with varying coherence lengths. Furthermore, the Raman results might suggest a tendency towards a phase transition that is never fully realized and long-range ordered, and which, therefore, remains undetected in diffraction experiments. With the transition from the orthorhombic () to the monoclinic () phase observed in TaNiSe and TaNiCoSe, the two mirror symmetries shown in Figure 6 (left) are broken, resulting in the splitting of the Ni and Se atomic positions (see Figure 2 and Table S2 in the SM [41]): Above , the Ta-Ni bond length remains almost unchanged. However, below , this bond splits, and one bond, Ta-Nib, increases significantly, while the other, Ta-Nia, decreases markedly [Figure 2 (c) and (f)]. A similar pattern is observed for the splitting of the Ta-Se3 distances, as shown in Figure 2 (d) and (g). From this we conclude that the breaking of mirror symmetries and the subsequent shortening of the Ta-Nia and Ta-Se3a bond lengths substantially enhance the hybridization between the relevant Ta, Nia, and Se3a orbital states. Conversely, the lengthening of the Ta-Nib and Ta-Se3b bonds weakens the hybridization between the Ta, Nib, and Se3b orbital states. This interplay of decreased and increased bond lengths, together with the observed changes in the angle, aligns with the shearing of the system already reported in the literature [30]. It is worth mentioning that a minor lattice contraction can be seen with Co substitution at the Ni site (see Table S1 in the SM [41]), due to the fact that Co is the immediate neighbor of Ni in the periodic table and therefore has a slightly smaller atomic radius and one less electron in the shell. Due to the minimal magnitude of this effect, along with the negligible changes in the relevant Ta-Ni/Co and Ni/Co-Se bond distances, a change in the valence or spin state can be ruled out. Furthermore, it can be seen from Figure 2 (c), (d), (f), and (g), that the bond distances split, but the average distance—which determines the volume around the Ni atom—remains the same above and below the transition. This strongly suggests a change in hybridization rather than a change in the valence or spin state.
In TaNiS, on the other hand, the dependence of the Ta-Ni and Ta-S3 bond lengths displays a small reduction with decreasing temperature, thereby, simply reflecting the standard thermal expansion behavior. In the whole temperature range, however, no changes in the bond distances signaling a symmetry breaking are found [see Figure 2 (i) and (j)].
NEXAFS: Electronic properties of TaNiSe, TaNiCoSe, and TaNiS
To investigate the relationship between the observed electronic and structural effects, particularly in relation to the orbital physics at the electronically dominant Ni sites, NEXAFS experiments were conducted in TEY mode at the Ni edge. In all the compounds studied here, the -type orbitals ( and ) located at lower energies are fully occupied and, therefore, spectroscopically silent. In contrast, the orbitals () are active at the Fermi level and play a crucial role in the material’s electronic behavior [45].
Figure 3 shows TaNiSe NEXAFS spectra recorded in the monoclinic phase at 30 K along the crystallographic , , and directions at the Ni edge. The distinct behavior of the three spectra enables the identification of the orbitals involved and clearly demonstrates that the in-plane orbitals are dominating in the lower energy range below 854 eV with their spectral contributions along and , while the orbitals with out-of-plane character are primarily probed at slightly higher energies with spectral contributions along and for and along and for orbitals.

Figure 3. Comparison of Ni NEXAFS spectra of TaNiSe recorded at 30 K along , , and . The in-plane orbitals are observed in the lower energy range, below 854 eV, with spectral contributions along and . In contrast, out-of-plane orbitals show contributions along and , whereas the orbitals exhibit contributions along and , and are found at slightly higher energies.
In order to emphasize the orbital occupation redistribution across the investigated temperature range, we show in Figure 4 the high-temperature NEXAFS spectra measured along , , and directions at the Ni edge for TaNiSe (left), TaNiCoSe (middle), and TaNiS (right) together with the corresponding temperature-dependent difference spectra . The insets in the upper left corners of panels (a), (b), and (c) illustrate the orientation of the beam relative to the sample.

Figure 4. NEXAFS spectra of TaNiSe (left), TaNiCoSe (middle) and TaNiS (right) with the beam parallel to , , and directions. Shown is the spectrum taken at 390 K and the difference (390 K) - (T) between the spectra taken at 390 K and at the respective temperatures given in the graph. Note that is multiplied by a factor of -6 and that a negative area means loss of spectral weight relative to 390 K. The green arrows point to the isosbestic point. The insets in the upper left corner of figures (a), (b), and (c) illustrate the polarization of light with respect to the sample’s orientation. A significant charge transfer from orbitals with exclusively in-plane character, i.e., , to orbitals with in- and out-of-plane character, i.e., , is found below for TaNiSe and TaNiCoSe, while only a minor charge transfer of this type is found between in-plane and out-of-plane orbitals for TaNiS. For TaNiS, temperature effects seem to play the dominant role.
For TaNiSe, spectra were recorded in the orthorhombic phase at 390 K and in the monoclinic phase at 290, 150, and 30 K. In this compound, the spectra exhibit strong anisotropy with respect to the crystallographic directions. Additionally, an isosbestic point (ISP) where all temperature-dependent spectra coincide and which is indicated by an arrow, is observed around 854 eV for all three directions, with a notable temperature-dependent transfer of spectral weight from the low-energy to the high-energy side of the ISP as the temperature decreases below T. This transfer of spectral weight signals that hole carriers are being shifted away from (i.e., from the region below the ISP) towards the first unoccupied states above (i.e., to the region above the ISP).
Comparing the directional dependence in Figure 4 (a), (Figure 4b), and (Figure 4c), it is evident that the holes are removed from the in-plane orbitals at , which are predominantly probed below the ISP for and [Fig. 4 (a) and (b)]. These holes are transferred to orbitals with out-of-plane character, which are primarily probed above the ISP for and [Fig. 4 (b) and (c)], as illustrated in Figure 3. In effect, this indicates a substantial charge transfer from in-plane to out-of-plane orbitals below the transition temperature of TaNiSe. It should be noted that the small change in the order parameter , as shown in Figure 1, which is associated with the mirror symmetry breaking, is solely in-plane and too minor to be detected in NEXAFS experiments. However, what is clearly observed is a transfer of spectral weight between different directions, which is only possible if the orbital character changes from in-plane to out-of-plane as previously described (see also discussion of Fig. 9 in the SM [41]). Furthermore, the spectral shape indicates a Ni state, and—consistent with our structural data—a temperature-dependent change in the valence or spin state can be ruled out due to the minimal modifications observed in the multiplet structures (see discussion in the SM [41] and Refs. [46, 47, 48]).
Ni spectra of TaNiCoSe, measured at 390 and 290 K (above the transition) and at 150 K and 30 K (below the transition), exhibit a similar but less pronounced behavior compared to TaNiSe, as can be seen in Figure 4 and Figure 5. With Co doping, the material becomes a more distinct semimetal, characterized by a larger overlap between the conduction and valence bands, as depicted in the band structure sketch in panel (e) of Figure 5. As a result, the Ni spectra of the high- phase of TaNiCoSe show in the low-energy region below the ISP a higher spectral weight compared to TaNiSe (see the yellow shaded area in panel (b) of Figure 5). Below the transition, both compounds behave as EIs. Yet, TaNiCoSe has a larger negative energy gap, leading to a stronger screening and, thus, to a reduced transition temperature and a smaller number of excitons, and as a consequence results in less spectral weight transfer. Therefore, fewer hole states move from the low-energy to the high-energy side of the ISP at the onset of the EI. A sketch of the corresponding band structures derived from our NEXAFS data is illustrated for all three systems in (d), (e), and (f) of Figure 5.

Figure 5. Comparison of Ni L NEXAFS spectra of (a) TaNiSe, (b) TaNiCoSe, and (c) TaNiS recorded at 390 and 30 K along the direction. In the plot for TaNiCoSe, the spectra of TaNiSe are additionally depicted in gray. The right panel of the figure represents sketches for the band structure of the three compounds before and after the formation of the excitonic insulator state as derived from our NEXAFS data.
For TaNiSe, the shift between the 390 and 30 K spectra can be determined from Figure 5 and amounts to meV. This value is roughly consistent with the optical excitation gap eV below and the estimated exciton binding energy eV from a previous ARPES study [17]. In the case of TaNiCoSe, the shift and with it its exciton binding energy is strongly reduced and can be estimated to be meV. Notably, the respective spectra above the transition (at 290 K and 390 K) are nearly on top of each other, as do
the spectra below the transition (at 150 K and 30 K) and we can conclude that the observed gap opens at the transition to the EI state. The slight difference between the spectra above (and below) can be attributed to phononic broadening, which increases at higher temperatures: In other words, as the temperature rises, the spectral width slightly broadens while its maximum intensity decreases. This trend is consistent across all individual crystallographic directions and is observed in the spectra of TaNiSe and TaNiS (see below) as well.
With S substitution, the semiconducting band gap widens, as shown in the sketch (f) of Figure 5. The Ni spectra of TaNiS, measured at 390 K, 290 K, and 30 K along the three crystallographic directions are almost isotropic (see panel (c) of Figure 5) and from the directional-dependent spectra in Figure 4 (g), (h), and (i), a much smaller spectral weight transfer between in-plane and out-of-plane spectra is observed: In contrast to the other two compounds where a clear charge transfer between and orbitals is found, the spectral weight is for TaNiS predominantly transferred from the low-energy to the high-energy region within the spectra of the same directional character, i. e., from the low-energy to the high-energy side of the respective , , and spectra but for example not between and or . This indicates that the observed kind of spectral weight transfer likely originates from temperature effects
rather than from any intrinsic EI mechanism. Furthermore, no shift is observed between the 390 and 30 K spectra in Figure 5 (c), providing no evidence for an EI binding energy and strongly suggests the absence of a stable long-range EI phase in TaNiS. However, a small, potentially fluctuating EI component cannot be entirely ruled out based on our NEXAFS data, and might even be consistent with the EI instability recently reported in Raman experiments [38, 39].
In the following, we will combine our structural and electronic findings to construct a comprehensive orbital picture: In the orthorhombic phase above , Ni and Ta states exhibit different mirror symmetries, as discussed above and in [30], which prevents any hybridization between them. Our NEXAFS measurements at the Ni edge indicate that in the high-temperature phase orbitals dominate at . Consequently, hybridization is mainly possible along the crystallographic direction, specifically between the orbitals of Ni and the orbitals of Se atoms, as illustrated in the left panel of Figure 6. In contrast, below the transition to the monoclinic phase, the mirror symmetries both parallel and perpendicular to the chain direction are broken as exemplified in our structural data. This symmetry breaking enables hybridization between the relevant Ta, Ni, and Se orbitals in the low-temperature phase and the shortening of the Ta-Ni and Ta-Se3a bond lengths (see Figure 2) clearly manifests the significantly enhanced hybridization. Our NEXAFS data, along with the structural findings, reveal that upon cooling, holes move from in-plane orbitals () to orbitals with out-of-plane components ( and ), with the orbitals becoming the dominant states just above . The right panel of Figure 6 summarizes this hybridization scenario where Ni orbitals hybridize directly with lower-energy Ta orbitals as well as indirectly mediated through hybridization with orbitals of Se atoms (as explained in the caption of Figure 6), and it seems to be precisely this hybridization among these three elements that stabilizes the EI state in this system. According to our XRD and NEXAFS results, this trend is most pronounced for TaNiSe and significantly reduced for TaNiCoSe and almost absent in the case of TaNiS.
This is a key feature of the EI, where we do not observe a simple or compensated shift of the valence and conduction bands relative to the Fermi level to open a band gap, as one might expect in conventional scenarios. Nor is the gap driven by strong correlation effects of charge carriers of the identical type residing in the same orbital states, as described by the Hubbard model. Instead, the band gap emerges due to symmetry breaking and the hybridization of Ta, Ni, and Se states that are forbidden by the symmetry of the high-temperature phase.
Our data not only confirm the occurrence of symmetry breaking and orbital state hybridization, but also allow us to identify the orbitals involved in this process. Furthermore, our results are completely in line with the expected dome-shaped trend characteristic of an EI: The formation of the EI is most pronounced in TaNiSe, but as Co is substituted, shifting the material towards the semi-metallic side, the emergence of the EI is rapidly suppressed due to the stronger screening of the excitons (TaNiCoSe). On the contrary, with increasing S content the semiconducting band gap widens and the EI state is strongly suppressed. However, a certain resuming tendency to an EI state, likely driven by short-range fluctuations, cannot be entirely ruled out. Nonetheless, there is no evidence in our data for symmetry breaking, for an EI binding energy, or changes in the hybridization properties in TaNiS.
While our experiments alone, similar to all previous investigations, do not offer definitive smoking-gun proof for an EI state in TaNiSe, the likelihood of such a state is very high, especially when our findings are considered alongside the amount of results gathered from numerous other experiments. Yet, one of the most fundamental issues for the EI in TaNiSe, but in general as well, is that symmetry breaking is absolutely essential for allowing the electronic hybridization effect. At the same time, however, this unavoidably also leads to a structural distortion that is intrinsically coupled to the electronic one, resulting in a nearly inseparable chicken-and-egg problem, where it is difficult to ultimately disentangle whether the electronic or structural degrees of freedom are the driving force behind the phase transition. However, the data presented here unambiguously show that this is not merely a simple symmetry breaking, but that the low- phase is also accompanied by a change in the orbital symmetries of the Ta, Ni, and Se states at .
This finding is crucial: it suggests that, in addition to enabling the hybridization, the structural transition is accompanied by significant changes in the orbital character, which play a vital role in the formation of the EI. This points to a strong, and potentially dominant, electronic component that is essential for the phase transition to an EI.
Acknowledgments
We gratefully acknowledge the KIT Light Source KARA, Karlsruhe and the Karlsruhe Nano Micro Facility (KNMF) for the provision of beamtime as well as the European Synchrotron Radiation Facility (ESRF) for provision of synchrotron radiation facilities. We are indebted to Siegmar Roth and Andre Beck at the Institute for Quantum Materials and Technologies, Karlsruhe Institute of Technology for their great technical assistance.
Author Contributions
M.M. conceived and managed the project. A.-A.H. and S.P. grew the single crystals. A.-A.H. performed EDX experiments. N.M., B.W., F.A., G.G., M.Y., M.L.T., and M.M. performed XRD experiments. N.M. and M.M. refined the XRD data and interpreted the results. P.N., F.G., A.G., S.S., N.M., and M.M. performed NEXAFS measurements. P.N., S.S., and M.M. evaluated the NEXAFS spectra and M.M. interpreted the data. N.M. and M.M. wrote the manuscript with inputs from all authors.
DATA AND MATERIALS AVAILABILITY
All data is available in the manuscript or the supplementary information. Data taken at the ESFR, ID15B beamline are available under https://doi.esrf.fr/10.15151/ESRF-DC-2227830403.
Competing interests
The authors declare no competing interests.
ADDITIONAL INFORMATION
Supplementary Information accompanies this paper.
References
- [1]N. Mott, The transition to the metallic state, Phil. Mag. 6, 287 (1961).DOI
- [2]L. V. Keldysh and Y. V. Kopaev, Possible instability of the semimetallic state toward coulomb interaction, Sov. Phys. Solid State 6, 2219 (1965).
- [3]D. Jérôme, T. M. Rice, and W. Kohn, Excitonic insulator, Phys. Rev. 158, 462 (1967).DOI
- [4]W. Kohn, Excitonic phases, Phys. Rev. Lett. 19, 439 (1967).DOI
- [5]F. X. Bronold and H. Fehske, Possibility of an excitonic insulator at the semiconductor-semimetal transition, Phys. Rev. B 74, 165107 (2006).DOI
- [6]D. Ihle, M. Pfafferott, E. Burovskiand, F. X. Bronold, and H. Fehske, Bound state formation and the nature of the excitonic insulator phase in the extended falicov-kimball model, Phys. Rev. B 78, 193103 (2008).DOI
- [7]V. N. Phan, K. W. Becker, and H. Fehske, Spectral signatures of the bcs-bec crossover in the excitonic insulator phase of the extended falicov-kimball model, Phys. Rev. B 81, 205117 (2010).DOI
- [8]J. Neuenschwander and P. Wachter, Pressure-driven semiconductor-metal transition in intermediate-valence TmSe1−xTex and the concept of an excitonic insulator, Phys. Rev. B Condens Matter 41, 12693 (1990).pubmed.ncbi.nlm.nih.gov/9993746
- [9]B. Bucher, P. Steiner, and P. Wachter, Excitonic insulator phase in TmSe0.45Te0.55, Phys. Rev. Lett. 67, 2717 (1991).pubmed.ncbi.nlm.nih.gov/10044499
- [10]P. Wachter, Exciton condensation in an intermediate valence compound: TmSe0.45Te0.55, Solid State Commun 118, 645 (2001).DOI
- [11]K. Kim, H. W. J. Kim, S. Ha, H. Kim, J. K. Kim, J. Kwon, J. Seol, S. Jung, C. Kim, D. Ishikawa, T. Manjo, H. Fukui, A. Q. R. Baron, A. Alatas, A. Said, M. Merz, M. L. Tacon, J. M. Bok, K. S. Kim, and B. J. Kim, Origin of the chiral charge density wave in transition-metal dichalcogenide, Nature Physics 10.1038/s41567-024-02668-w (2024).DOI
- [12]H. Cercier, C. Monney, F. Clerc, C. Battaglia, L. Despont, M. G. Garnier, H. Beck, P. Aebi, L. Patthey, H. Berger, and L. Forró, Evidence for an excitonic insulator phase in 1T-TiSe2, Phys. Rev. Lett. 99, 164603 (2007).
- [13]J. A. Wilson, Concerning the semimetallic characters of TiS2 and TiSe2, Solid State Commun. 22, 551 (1977).DOI
- [14]M. Maschek, S. Rosenkranz, R. Hott, R. Heid, M. Merz, D. A. Zocco, A. H. Said, A. Alatas, G. Karapetrov, S. Zhu, J. van Wezel, and F. Weber, Superconductivity and hybrid soft modes in TiSe2, Phys. Rev. B 94, 214507 (2016).DOI
- [15]B. Bucher, T. Park, J. D. Thompson, and P. Wachter, Thermodynamical signatures of an excitonic insulator (2008), arXiv:0802.3354 [cond-mat.str-el].arxiv.org/abs/0802.3354
- [16]R. A. Craven, F. D. Salvo, and F. Hsu, Mechanisms for the 200 k transition in TiSe2: A measurement of the specific heat, Solid State Communications 25, 39 (1978).DOI
- [17]Y. F. Lu, H. Kono, T. I. Larkin, A. W. Rost, T. Takayama, A. V. Boris, B. Keimer, and H. Takagi, Zero-gap semiconductor to excitonic insulator transition in Ta2NiSe5, Nat. Commun. 8, 14408 (2017).DOI
- [18]S. A. Sunshine and J. A. Ibers, Structure and physical properties of the new layered ternary chalcogenides tantalum nickel sulfide (Ta2NiS5) and tantalum nickel selenide (Ta2NiSe5), Inorg. Chem. 24, 3611 (1985).DOI
- [19]F. D. Salvo, C. Chen, R. Fleming, J. Waszczak, R. Dunn, S. Sunshine, and J. A. Ibers, Physical and structural properties of the new layered compounds Ta2NiS5 and Ta2NiSe5, Journal of the Less Common Metals 116, 51 (1986).
- [20]J. Lee, C.-J. Kang, M. J. Eom, J. S. Kim, B. I. Min, and H. W. Yeom, Strong interband interaction in the excitonic insulator phase of Ta2NiSe5, Phys. Rev. B 99, 075408 (2019).DOI
- [21]K. Fukutani, R. Stania, J. Jung, E. F. Schwier, K. Shimada, C. I. Kwon, J. S. Kim, and H. W. Yeom, Electrical tuning of the excitonic insulator ground state of Ta2NiSe5, Phys. Rev. Lett. 123, 206401 (2019).DOI
- [22]K. Kim, H. Kim, J. Kim, C. Kwon, J. S. Kim, and B. J. Kim, Direct observation of excitonic instability in Ta2NiSe5, Nat. Commun. 12, 1969 (2021).
- [23]Y. Wakisaka, T. Sudayama, K. Takubo, T. Mizokawa, M. Arita, H. Namatame, M. Taniguchi, N. Katayama, M. Nohara, and H. Takagi, Excitonic insulator state in Ta2NiSe5 probed by photoemission spectroscopy, Phys. Rev. Lett. 103, 026402 (2009).pubmed.ncbi.nlm.nih.gov/19659224
- [24]T. Kaneko, T. Toritama, T. Konishi, and Y. Ohta, Orthorhombic-to-monoclinic phase transition of Ta2NiSe5 induced by the bose-einstein condensation of excitons, Phys. Rev. B 87, 035121 (2013).DOI
- [25]K. Sugimoto, T. Kaneko, and Y. Ohta, Microscopic quantum interference in excitonic condensation of Ta2NiSe5, Phys. Rev. B 93, 041105 (2016).DOI
- [26]K. Sugimoto, S. Nishimoto, T. Kaneko, and Y. Ohta, Strong coupling nature of the excitonic insulator state in Ta2NiSe5, Phys. Rev. Lett. 120, 247602 (2018).DOI
- [27]T. Yamada, K. Domon, and Y. Ono, Flo excitonic state in the three-chain hubbard model for Ta2NiSe5, J. Phys. Soc. Jpn. 85, 053703 (2016).DOI
- [28]K. Katsumi, A. Alekhinand, S. M. Souliou, M. Merz, A. A. Haghighirad, M. L. Tacon, S. Houver, M. Cazayous, A. Sacuto, and Y. Gallais, Disentangling lattice and electronic instabilities in the excitonic insulator candidate Ta2NiSe5 by nonequilibrium spectroscopy, Phys. Rev. Lett. 130, 106904 (2023).DOI
- [29]G. Mazza, M. Rösner, L. Windgätter, S. Latini, H. Hübener, A. J. Millis, A. Rubio, and A. Georges, Nature of symmetry breaking at the excitonic insulator transition: Ta2NiSe5, Phys. Rev. Lett. 124, 197601 (2020).DOI
- [30]M. D. Watson, I. Marković, E. A. Morales, P. L. Fèvre, M. Merz, A. A. Haghighirad, and P. D. C.King, Band hybridization at the semimetal-semiconductor transition of Ta2NiSe5 enabled by mirror-symmetry breaking, Phys. Rev. Res. 2, 013236 (2020).
- [31]E. Baldini, A. Zon, D. Choi, C. Lee, M. H. Michael, L. Windgaetter, I. I. Mazin, S. Latini, D. Azoury, B. Lv, A. Kogar, Y. Su, Y. Wang, Y. Lu, T. Takayama, H. Takagi, A. J. Millis, A. Rubio, E. Demler, and N. Gedik, The spontaneous symmetry breaking in Ta2NiSe5 is structural in nature, Proc. Natl. Acad. Sci. 120, e2221688120 (2023).
- [32]C. Chen, X. Chen, W. Tang, Z. Li, S. Wang, S. Ding, Z. Kang, C. Jozwiak, A. Bostwick, E. Rotenberg, M. Hashimoto, D. Lu, J. P. C. Ruff, S. G. Louie, R. J. Birgeneau, Y. Chen, Y.Wang, and Y. He, Role of electron-phonon coupling in excitonic insulator candidate Ta2NiSe5, Phys. Rev. Res. 5, 043089 (2023).DOI
- [33]S. Sano, Y. Hirose, T. Hirahara, Y. Uwatoko, J. Gouchi, T. Takeuchi, and R. Settai, Single crystal growth of ta and ni site substituted Ta₂NiS₅, JPS Conf. Proc. 30 (2020).
- [34]T. I. Larkin, Excitonic Fano resonances in Ta₂NiSe₅ and Ta₂NiS₅, Ph.D. thesis (2016).DOI
- [35]K. Mu, H. Chen, Y. Li, Y. Zhang, P. Wang, B. Zhang, Y. Liu, G. Zhang, L. Song, and Z. Sun, Electronic structures of layered Ta₂NiS₅ single crystals revealed by high-resolution angle-resolved photoemission spectroscopy, J. Mater. Chem. C 6, 3976 (2018).DOI
- [36]T. Mitsuoka, T. Suzuki, H. Takagi, N. Katayama, H. Sawa, M. Nohara, M. Watanabe, J. Xu, Q. Ren, M. Fujisawa, T. Kanai, J. Itatani, S. Shin, K. Okazaki, and T. Mizokawa, Photoinduced phase transition from excitonic insulator to semimetal-like state in Ta₂Ni₁−xCoₓSe₅ (x = 0.10), J. Phys. Soc. Jpn 89, 124703 (2020).
- [37]Y. Hirose, S. Sano, T. Hirahara, Y. Uwatoko, J. Gouchi, T. Takeuchi, and R. Settai, Destabilization of excitonic phase by elemental substitution in (Ta₁−xMₓ)₂NiSe₅ (M = V, Nb) and Ta₂(Ni₁−yTᵧ)Se₅ (T = Fe, Co), J. Phys. Soc. Jpn. 92, 084705 (2023).
- [38]M. Ye, P. A. Volkov, H. Lohani, I. Feldman, M. Kim, A. Kanigel, and G. Blumberg, Lattice dynamics of the excitonic insulator Ta₂Ni(Se₁−xSₓ), Phys. Rev. B 104, 045102 (2021).
- [39]M. Ye and T. Lacmann and M. Frachet and I. Vinograd and G. Garbarino and N. Marayatta and M. Merz and R. Heid and A.-A. Haghighirad and M. Le Tacon, Anomalous phonon Grüneisen parameters in the semiconductor Ta₂NiS₅, Phys. Rev. B 110, 035120 (2024).
- [40]L. Windgätter, M. Rösner, G. Mazza, H. Hübener, A. Georges, A. J. Millis, S. Latini, and A. Rubio, Common microscopic origin of the phase transitions in Ta₂NiS₅ and the excitonic insulator candidate Ta₂NiSe₅, npj Comput Mater 7, 210 (2021).arxiv.org/abs/2105.13924
- [41]See Supplementary Material at [URL will be inserted by publisher] for the experimental details including single crystal growth, structural characterization, x-ray photoemission spectroscopy, near edge x-ray absorption spectroscopy and angle-resolved photoemission spectroscopy. (2025).
- [42]Rigaku Oxford Diffraction Ltd, CrysAlisPro software system, version 1.171.44, Rigaku Corporation, Wroclaw, Poland, Rigaku Oxford Diffraction, Yantón, Oxfordshire, E 2015 CrysAlisPro.
- [43]V. Petříček and M. Dušek and L. Palatinus, Crystallographic computing system jana2006: General features, Zeitschrift für Kristallographie - Crystalline Materials 229, 345 (2014).
- [44]G. Garbarino, M. E. Hanfland, S. Gallego-Parra, A. Rosa, M. Mezouar, D. Duran, K. Martel, E. Papillon, T. Roth, P. Got, and J. Jacobs, Extreme conditions x-ray diffraction and imaging beamline id15b on the esrf extremely brilliant source, High Press. Res. 44, 199 (2024).DOI
- [45]M. Merz, P. Schweiss, P. Nagel, M.-J. Huang, R. Eder, T. Wolf, H. V. Loehneysen, and S. Schuppler, Of substitution and doping: Spatial and electronic structure in fe pnictides, J. Phys. Soc. Jpn. 85, 044707 (2016).DOI
- [46]M. Merz, B. Ying, P. Nagel, S. Schuppler, and K. Kleiner, Reversible and irreversible redox processes in li-rich layered oxides, Chem. Mater. 33, 9534 (2021).DOI
- [47]K. Kleiner, J. Melke, M. Merz, P. Jakes, P. Nagel, S. Schuppler, V. Liebau, and H. Ehrenberg, Unraveling the degradation process of LiNi₀.₈Co₀.₁₅Al₀.₀₅O₂ electrodes in commercial lithium ion batteries by electronic structure investigations, ACS Appl. Mater. Interfaces 7 (2015).DOI
- [48]K. Kleiner, C. A. Murray, C. Grosu, B. Ying, M. Winter, P. Nagel, S. Schuppler, and M. Merz, On the origin of reversible and irreversible reactions in LiNiₓCo₁−x/₂Mn₁−x/₂O₂, J. Electrochem. Soc. 168 (2021).