betaFeedback

WorksComputer Science

RoES: Rotational Equivariant Selective-frequency Fusion for Multimodal Images

Abstract

Infrared-visible image fusion facilitates robust multimodal perception by integrating complementary textural nuances from visible sensors with thermal signatures from infrared systems. Due to the task's inherently ill-posed nature, existing methods heavily rely on structural priors but typically enforce rotation equivariance uniformly across all features. Such a holistic approach overlooks a critical distinction where low-frequency shared structures strictly adhere to equivariant constraints while high-frequency modality-specific details require greater flexibility to preserve unique information. To bridge this gap, we propose RoES, a Rotational Equivariant Selective-frequency fusion network. Instead of employing static decomposition, we introduce a trainable rotation-enhanced updater/predictor module to dynamically decouple low- and high-frequency components. The resulting representations are then processed through a dual-branch fusion module tailored for spectral consistency. Specifically, a rotation-equivariant Mamba is employed to capture long-range structural dependencies in the low-frequency domain, while a polar spectral attention-based Dual-Fourier block refines high-frequency details under explicit low-frequency guidance. Extensive experiments demonstrate that RoES consistently achieves state-of-the-art performance in both fusion quality and downstream object detection, establishing a robust solution for multimodal fusion by reconciling frequency-selective features with equivariant constraints. The source code is available at https://github.com/BryceLosky/RoES-Fusion.

CCS Concepts • Computing methodologies → Computer vision; Image representations.

Keywords Multimodal image fusion, Rotational equivariance, Frequency decomposition, Polar spectral attention

ACM Reference Format: Jiabao Wang, Wenjian Liu, Yaoming Cai, Gengyu Zhang, Boyan Zhao, Zijia Zhang, Yao Ding, and Xiaobo Liu. 2026. RoES: Rotational Equivariant Selective-frequency Fusion for Multimodal Images. In Proceedings of the 34th ACM International Conference on Multimedia (MM ’26), November 10–14, 2026, Rio de Janeiro, Brazil. ACM, New York, NY, USA, 10 pages. https://doi.org/10.1145/3767308.3835904

Introduction

Robust visual perception in safety-critical applications such as autonomous driving and surveillance demands reliable scene understanding across diverse and adverse environmental conditions [18, 19, 20, 25]. Infrared-visible image fusion is a prominent approach to this end, combining the complementary characteristics of two widely deployed sensor modalities. Visible cameras provide rich spatial textures and fine structural detail, yet degrade severely under low illumination or glare. Infrared sensors offer illumination-invariant thermal detection, yet sacrifice texture resolution and are susceptible to thermal crossover artifacts [69]. Through cross-modal complementarity, fusion produces imagery that is simultaneously detail-preserving and environmentally robust. This unified representation directly benefits downstream tasks such as object detection [22] and semantic segmentation [24], which require both properties to perform reliably in real-world conditions.

This work is licensed under a Creative Commons Attribution 4.0 International License.

MM ’26, Rio de Janeiro, Brazil

© 2026 Copyright held by the owner/author(s).

ACM ISBN 979-8-4007-2213-4/2026/11

https://doi.org/10.1145/3767308.3835904

However, infrared-visible image fusion is fundamentally ill-posed. For any registered source pair, no ground-truth image simultaneously preserves all complementary content from both modalities [25]. Consequently, fusion models cannot be trained against a definitive target and must instead rely on structural priors to constrain the solution space [62]. This raises the question of what structural prior to impose.

Among possible priors, geometric consistency—particularly rotation equivariance—has recently emerged as a promising choice [4]. It reflects a simple geometric principle: when the input rotates, the representation should transform accordingly in a predictable manner. This inductive bias has shown clear benefits in visual recognition and dense prediction [48, 54]. More recently, Zhao et al. [65] demonstrated that geometric consistency directly benefits multimodal fusion by introducing rotation-equivariant modeling into the fusion pipeline.

However, when rotation-equivariant modeling is introduced into fusion, it is typically enforced uniformly across the representation [65], overlooking the fact that low- and high-frequency components are not equally compatible with the same geometric prior. As shown in Figure 1, a fusion model exhibits evident operational preferences across frequency, where standard convolutions are prioritized for high-frequency textures while rotation-equivariant convolutions are selectively invoked for low-frequency structural components. Applying the same rigid equivariant constraint to both regimes may therefore stabilize shared structure while over-regularizing detail modeling, suppressing the complementary information that fusion is meant to preserve. This concern is also echoed in recent low-level vision studies. Xie et al. [53, 54] note that practical equivariant modules can incur approximation errors on complex local patterns. Bai et al. [1] further show that strict equivariance constraints can limit representation accuracy when the assumed symmetry does not hold precisely. In Sec. 3.1, we formalize this observation as a frequency-selective equivariance constraint and further validate it through a controlled frequency-threshold experiment.

Figure 1. Motivation and performance of RoES. (a) An attention-based router reveals frequency-dependent operator selection in a preliminary U-Net fusion task: rotation-equivariant convolutions for low-frequency structures (green) and standard convolutions for high-frequency textures (red). (b) RoES achieves state-of-the-art performance on the M3M^{3}FD benchmark.

A further challenge is that realizing frequency-selective fusion first requires a meaningful low/high-frequency decomposition, yet many frequency-aware fusion methods still rely on fixed, hand-crafted transforms [33, 64, 70] that predefine the boundary between shared structure and modality-specific detail rather than adapting it to the fusion objective.

Therefore, we propose Rotational Equivariant Selective-frequency fusion (RoES) to address both the mismatch of uniform equivariance and the rigidity of fixed frequency decomposition. RoES first employs a learnable lifting-based module, RoUP, to adaptively separate multimodal features into low-frequency structure and high-frequency detail. It then selectively imposes rotation equivariance only on the low-frequency branch, where a rotation-equivariant Mamba-style encoder [5] produces a geometry-consistent representation with stable global structure. Rather than constraining high-frequency features with the same rigid prior, RoES refines them in the frequency domain through a Frequency-guided Dual-Fourier (FDF) block, where the stable low-frequency representation guides high-frequency spectral optimization via Polar Spectral Attention (PSA). This selective design achieves a better balance between structural robustness and detail preservation, as validated by extensive experiments.

Our contributions are four-fold:

  • Frequency-selective equivariance. We formulate a frequency-selective prior for multimodal fusion, showing that rotation equivariance is more suitable for low-frequency structure than for high-frequency details.

  • Learnable decomposition. We introduce RoUP, a learnable lifting module with an asymmetric CNN/F-Conv predictor/updater and a direction-aware loss for task-adaptive low/high-frequency decomposition.

  • RoES framework. We propose a selective-frequency fusion architecture that uses rotation-equivariant Mamba for low-frequency global structure and a low-frequency-guided Dual-Fourier block for high-frequency refinement.

  • Polar spectral refinement. We design PSA to inject rotation-consistent low-frequency cues into the high-frequency amplitude spectrum in polar coordinates, enabling low-frequency-guided high-frequency refinement in the frequency domain while preserving phase-sensitive localization.

Infrared-visible Image Fusion

Infrared-visible image fusion aims to extract and integrate complementary multimodal cues [3, 12, 30, 60, 63]. Early methods mainly relied on multi-scale transforms [15], whereas later deep models shifted toward data-driven decomposition [23]. Autoencoder- and CNN-based frameworks commonly decompose inputs into base/detail or shared/specific representations [11, 14, 16, 46], while more recent dual-branch or disentanglement-based designs use correlation-aware objectives to align shared representations and preserve modality-specific textures [7, 55, 56, 64, 66]. Although these strategies improve complementary representation learning, they mostly disentangle information by functional role and rarely consider the different geometric properties of structural low-frequency and detail-dominant high-frequency components during fusion.

Recent studies have also explored Fourier- or spectral-domain interaction for frequency-specific fusion [47, 51, 67], but many still rely on fixed transforms, predefined bands, or whole-spectrum processing, limiting adaptive low/high-frequency partitioning and low-to-high structural guidance. Among learnable decomposition frameworks, the lifting scheme, a second-generation wavelet framework [38], is particularly relevant because it supports task-adaptive decomposition with an explicit inverse path. By factorizing wavelet transforms into split, predict, and update steps [6], it replaces linear filters with learnable nonlinear operators [68] while supporting exact or explicit reconstruction under invertible designs [10, 35]. Its effectiveness in image compression, denoising, and time-series modeling [29] makes it well suited to adaptive and reversible frequency decoupling in multimodal fusion.

For low-level dense prediction, however, strict equivariance is usually realized through finite angular sampling or basis approximations [37, 54], which can limit accuracy on complex local patterns [1, 53]. This limitation is particularly relevant to multimodal fusion, where low-frequency components mainly encode stable global structure, whereas high-frequency components are more heterogeneous, modality-dependent, and sensitive to local reliability [1, 21]. Imposing the same rigid equivariant constraint on all frequency bands may therefore be unnecessarily restrictive for detail modeling. Attention-based fusion offers a flexible mechanism for complementary feature selection [26, 28, 34, 42, 64], but such geometry-agnostic interaction does not explain how rotation-consistent structure should guide locally variable details. Prior infrared-visible fusion studies have paid limited attention to whether low- and high-frequency branches differ in their compatibility with strict rotation-equivariant modeling, motivating selective rotation priors: explicit equivariant modeling for structurally stable representations and greater flexibility for high-frequency detail fusion.

Rotation Equivariance Constraint

For a rotation group GG and g∈Gg \in G, a mapping ff is rotation-equivariant if f(g∘X)=g∘f(X)f(g \circ X) = g \circ f(X), where g∘(⋅)g \circ(\cdot) denotes the action of gg on a feature map. Rotation equivariance thus provides a geometric prior for predictable feature transformations under input rotations [48]. Typical realizations include group convolutions and steerable filters [49, 50], which are widely used in rotation-sensitive tasks such as oriented object detection and visual matching [2, 9, 13, 36, 43].

Methodology

RoES is built on a simple principle: rotation equivariance ought to be imposed selectively instead of uniformly across low- and high-frequency components. As illustrated in Figure 2, we realize this design philosophy through a frequency decomposition and selectivity framework. The detailed architecture and components are elaborated in the following subsections.

Figure 2. Overview of the proposed RoES model. Paired infrared-visible images are encoded by a shared Restormer encoder and decomposed into low- and high-frequency components via the trainable RoUP module. Our frequency-selective fusion module comprises a rotation-equivariant Mamba block for low-frequency structure modeling, and a frequency-guided Dual-Fourier block for high-frequency detail refinement under low-frequency guidance.

Frequency-Selective Equivariance Prior

We treat rotation equivariance as a branch-selective prior at the fusion stage rather than a uniform hard constraint across low- and high-frequency branches, because it is better aligned with low-frequency structural fusion than with high-frequency detail fusion. Let G=CNG = C_N be the discrete rotation group, and let HG⊂H\mathcal{H}^{G} \subset\mathcal{H} denote the corresponding equivariant subset of the base function class H\mathcal{H}. Functionally, low-frequency fusion mainly models shared spatial layout, whose response should rotate consistently with the input, making rotation-equivariant modeling suitable. High-frequency fusion instead learns to select reliable modality-specific details, suppress redundant or noisy responses, and enhance complementary edges, textures, or thermal cues under low-frequency guidance. These operations depend on local context and modality reliability rather than rotational symmetry alone, so enforcing the same strict equivariant constraint is unnecessary and can limit detail recovery. RoES therefore applies explicit equivariant modeling only to the low-frequency branch and uses the resulting rotation-consistent low-frequency representation to guide high-frequency fusion. Formally, this branch-selective design is summarized by the following definition.

Definition 3.1 (Frequency-selective equivariance constraint). Let fLf_L and fHf_H denote the low- and high-frequency fusion mappings, respectively. A fusion model satisfies the frequency-selective equivariance constraint if

fL∈HG,fH∈H.(1)f_L \in\mathcal{H}^{G}, \qquad f_H \in\mathcal{H}. \tag*{(1)}

That is, the low-frequency branch is constrained to be rotation-equivariant, whereas no strict rotation-equivariant constraint is imposed on the high-frequency branch.

Controlled validation. To examine the scope of Definition 3.1 independently of RoES-specific modules, we conduct a controlled experiment with a basic two-branch U-Net. Following FFT decomposition at a frequency threshold τ\tau, components below τ\tau are processed by a strictly C4C_4-equivariant branch, E(x)=14∑g∈C4g−1B(gx)E(x) = \frac{1}{4} \sum_{g \in C_4} g^{-1}B(gx), where BB denotes a standard branch, while components above τ\tau are processed by the standard branch before recombination. Thus, τ=0\tau= 0 applies no equivariance, whereas τ=1\tau= 1 applies it to the full spectrum. The model is trained on MSRS and evaluated on 300 M3FD image pairs.

As shown in Figure 3, VIF peaks and NabfN_{abf} reaches its minimum when equivariance is restricted to low frequencies, while extending the constraint toward the full spectrum reverses both gains. VIF measures transferred visual information, whereas NabfN_{abf} quantifies fusion-induced noise and artifacts, jointly covering fidelity preservation and artifact suppression. These aligned optima support the design principle formalized in Definition 3.1, grounding the frequency-selective equivariance constraint in controlled experimental evidence.

Figure 3. Controlled validation of the frequency-selective equivariance constraint. The threshold τ\tau spans no equivariance (τ=0\tau= 0) to full-frequency equivariance (τ=1\tau= 1), with both VIF and NabfN_{abf} attaining their optima in the low-frequency regime.

In our implementation, N=4N = 4, i.e., G=C4G = C_4. Guided by the above frequency-selective equivariance constraint, RoES first builds interaction-enhanced features with a front-end encoder before RoUP. The encoder contains modality-specific 3×33 \times3 convolution stems for initial embedding and shallow feature extraction, a parameter-shared Restormer backbone [59] for aligned structural representation learning and long-range context modeling, and a bidirectional cross-attention module for complementary cross-modal information exchange. The resulting encoder features are then directly fed into RoUP for trainable low/high-frequency decomposition.

Trainable Frequency Decoupling via RoUP

Using the interaction-enhanced encoder features, we employ Rotation-enhanced Updater/Predict networks (RoUP), an asymmetric lifting module with a CNN predictor and an F-Conv updater [54], to learn a task-adaptive low/high-frequency decomposition for subsequent fusion. Two RoUP modules with the same architecture but independent parameters are used for the infrared and visible streams. For each modality, a wavelet initializer produces initial low/high states, which are then refined by the lifting predictor–updater process.

Consistent with the frequency-selective equivariance constraint (Definition 3.1), RoUP adopts an asymmetric predictor–updater design: a CNN predictor for flexible high-frequency residual estimation and an F-Conv updater for rotation-consistent low-frequency correction. We instantiate the updater with F-Conv [54], whose Fourier-series filter parametrization improves equivariant filtering accuracy and reduces rotation-induced aliasing. The low- and high-frequency states are then refined through MM lifting steps. Specifically, the predictor and updater are shallow residual subnetworks, each comprising an input layer, two residual blocks, and an output layer; the predictor uses standard convolutions, whereas the updater uses F-Conv, with ReLU activations after the intermediate layers. Compared with a fixed wavelet split, this learned refinement is more adaptive while retaining an exact inverse path before fusion.

Lifting-based decomposition

For modality q∈{I,V}q \in\{\mathrm{I}, \mathrm{V}\}, let Lq,(m)L^{q,(m)} and Hq,(m)H^{q,(m)} denote the low- and high-frequency states at lifting step mm, respectively. Because the two modalities are processed by independent RoUP instances, their predictors and updaters are modality- and step-specific, yielding the high-frequency residual ΔHq,(m)=Pred⁡q,m(Lq,(m−1))\Delta H^{q,(m)}=\operatorname{Pred}_{q,m}(L^{q,(m-1)}) and the low-frequency residual ΔLq,(m)=Upd⁡q,m(Hq,(m))\Delta L^{q,(m)}=\operatorname{Upd}_{q,m}(H^{q,(m)}). At step mm, the lifting update is formulated as

Hq,(m)=Hq,(m−1)−ΔHq,(m),Lq,(m)=Lq,(m−1)+ΔLq,(m).(2)H^{q,(m)}=H^{q,(m-1)}-\Delta H^{q,(m)}, \qquad L^{q,(m)}=L^{q,(m-1)}+\Delta L^{q,(m)}. \tag*{(2)}

Under the lifting interpretation, the CNN-based predictor estimates detail from the coarse state with greater flexibility, so that the high-frequency residual can better preserve modality-specific components, whereas the F-Conv-based updater feeds rotation-consistent structural corrections back to the low-frequency state to progressively emphasize shared structure. After MM steps, this yields the modality-specific outputs {LI,LV,HI,HV}\{L^{\mathrm{I}},L^{\mathrm{V}},H^{\mathrm{I}},H^{\mathrm{V}}\} for subsequent fusion. Because each lifting step admits an explicit inverse, the decomposition remains exactly reversible before fusion, which helps preserve sparse high-frequency information that is otherwise prone to attenuation during forward propagation.

3.2.2 Cross-modal decomposition supervision. Following the multimodal prior that low-frequency components are more correlated across modalities while high-frequency components are more modality-specific and complementary, we supervise the two parts differently. For the low-frequency part, we compare corresponding structures within matched directional subspaces before aggregation; although the two RoUP modules use independent parameters, subspaces with the same θ\theta remain geometrically matched because θ\theta indexes the same F-Conv orientation axis. Compared with global supervision on aggregated features, this subspace-level comparison focuses more directly on within-subspace correspondence and is less sensitive to residual modality-specific differences. For the high-frequency part, redundant cross-modal correlation is suppressed on the final states to promote complementary detail decomposition. Let Lq,θL^{q,\theta} denote the low-frequency response of modality qq in the θ\theta-th directional subspace produced by the F-Conv-based updater before aggregation, where θ∈{1,…,Nθ}\theta\in\{1,\ldots,N_\theta\}, and let HqH^q denote the final high-frequency state after the lifting steps. Using the Pearson correlation coefficient CC⁡(⋅,⋅)\operatorname{CC}(\cdot,\cdot) computed on vectorized feature responses, we define

Llow=1Nθ∑θ=1Nθ(1−CC⁡(LI,θ,LV,θ)),(3)\mathcal{L}_{\mathrm{low}}=\frac{1}{N_\theta}\sum_{\theta=1}^{N_\theta}\left(1-\operatorname{CC}\left(L^{\mathrm{I},\theta},L^{\mathrm{V},\theta}\right)\right), \tag*{(3)}
Lhigh=CC⁡(HI,HV)2.\mathcal{L}_{\mathrm{high}}=\operatorname{CC}\left(H^{\mathrm{I}},H^{\mathrm{V}}\right)^2.

This supervision encourages agreement of low-frequency shared structure within matched directional subspaces while suppressing redundant high-frequency correlation, thereby promoting complementary detail decomposition in a better-aligned comparison space.

Frequency-Selective Fusion

After RoUP, the low- and high-frequency components of the two modalities are concatenated as FL=[LI∥LV]F^{\mathrm{L}}=[L^{\mathrm{I}}\parallel L^{\mathrm{V}}] and FH=[HI∥HV]F^{\mathrm{H}}=[H^{\mathrm{I}}\parallel H^{\mathrm{V}}], respectively, where ∥\parallel denotes channel-wise concatenation. RoES then performs branch-specific fusion on these two components. The low-frequency branch extracts stable shared structure, while the high-frequency branch enhances complementary details under its guidance.

Equivariant Mamba (EM) Block. Low-frequency components mainly preserve dominant structure, global layout, and stable spectral energy, and are therefore better suited to global modeling than sparse high-frequency details [70]. State space models provide global receptive fields with linear complexity, making them effective for large-scale structural modeling. We therefore adopt the spectral state space modeling strategy of Dastani et al. [5] as the low-frequency encoder. In implementation, FLF^{\mathrm{L}} is first passed through a rotate–Patchification–inverse–max-aggregate pipeline, then through the spectral traversal scan, S6 blocks, and spectral traversal merge for global structural modeling. For the four quarter-turn rotations r∈C4r\in C_4, we rotate FLF^{\mathrm{L}}, apply a shared CNN patchification stem, rotate the responses back by r−1r^{-1}, and perform pointwise max aggregation before the spectral SSM stack:

FEqL=fSSM(Max⁡r∈C4(r−1∘ρ(r∘FL))),(4)F_{\mathrm{Eq}}^{\mathrm{L}}=f_{\mathrm{SSM}}\left(\operatorname{Max}_{r\in C_4}\left(r^{-1}\circ\rho\left(r\circ F^{\mathrm{L}}\right)\right)\right), \tag*{(4)}

where ρ(⋅)\rho(\cdot) denotes the shared CNN patchification stem, fSSM(⋅)f_{\mathrm{SSM}}(\cdot) denotes the subsequent spectral SSM stack, and Max⁡(⋅)\operatorname{Max}(\cdot) denotes pointwise max aggregation. This design injects rotation-consistent structural cues before global spectral modeling and yields the fused low-frequency feature FEqLF_{\mathrm{Eq}}^{\mathrm{L}}.

Frequency-guided Dual-Fourier (FDF) Block. High-frequency details are sparse and more vulnerable to degradation, whereas low-frequency components preserve dominant structure and more stable global statistics [70]. We therefore use the fused low-frequency feature FEqLF_{\mathrm{Eq}}^{\mathrm{L}} to guide high-frequency fusion. We term the proposed high-frequency fusion module the Frequency-guided Dual-Fourier block. Since the amplitude spectrum (A)(A) encodes modality-specific features while the phase spectrum (Φ)(\Phi) preserves shared structures [69], our main guidance focuses on amplitude, with only light refinement on phase. Because the preceding wavelet/lifting decomposition yields a 384-channel concatenated high-frequency representation, we first apply lightweight feature encoding, dimensionality reduction, and layer normalization to obtain F~H\widetilde{F}^{\mathrm{H}}. We then extract the high-frequency amplitude AHA_{\mathrm{H}} and phase ΦH\Phi_{\mathrm{H}}, and the low-frequency amplitude ALA_{\mathrm{L}}, via FFT:

FFT⁡(F~H)=AH⊙eiΦH,AL=∣FFT⁡(FEqL)∣.(5)\operatorname{FFT}(\widetilde{F}^{\mathrm{H}})=A_{\mathrm{H}}\mathbin{\odot}e^{i\Phi_{\mathrm{H}}}, \qquad A_{\mathrm{L}}=\left|\operatorname{FFT}(F_{\mathrm{Eq}}^{\mathrm{L}})\right|. \tag*{(5)}

To better exploit low-frequency rotation-consistent structure, the amplitude spectra are mapped to polar coordinates, where rotation-related variations are easier to align along the angular dimension. In this geometry-aligned space, PSA injects low-frequency guidance into the high-frequency amplitude responses; details are given in Sec. 3.4. Specifically, the first PSA stage produces a low-frequency-guided query for the subsequent multi-head self-attention-based channel recalibration, and the same PSA formulation is reused after channel mixing to further correct the updated amplitude. The corrected amplitude is recombined with the original phase ΦH\Phi_{\mathrm{H}} through iFFT to produce a spatial residual. In parallel, the phase path lightly refines ΦH\Phi_{\mathrm{H}} in an Euler-encoded representation and recombines it with the original amplitude AHA_{\mathrm{H}} through iFFT; the two spatial features are then concatenated, fused by a 1×11\times1 convolution, and added to F~H\widetilde{F}^{\mathrm{H}} to obtain FRefinedHF_{\mathrm{Refined}}^{\mathrm{H}}. Finally, the fused low- and high-frequency features are concatenated and fed into a decoder of four cascaded Restormer blocks [59] to reconstruct the final fused image. This symmetric decoder helps preserve both global structural consistency and fine local details.

Polar Spectral Attention (PSA)

Next, we elaborate on the polar spectral attention module used in the amplitude path of the frequency-guided Dual-Fourier block, depicted in Figure 4. PSA does not rigidly constrain high-frequency features; instead, it injects low-frequency rotation-consistent structure into the high-frequency amplitude spectrum in a geometry-aligned space. This design is motivated by the observation that enhanced low-frequency representations can be used to match and correct high-frequency responses, thereby providing a more stable guidance signal for detail restoration [70]. Both PSA stages share the same formulation but use independent parameters.

Figure 4. Illustration of polar spectral attention.

3.4.1 Polar Representation. Let RαR_{\alpha} denote rotation by angle α\alpha, let Ax=∣FFT⁡(x)∣A_x = \lvert\operatorname{FFT}(x)\rvert be the amplitude spectrum of a feature map xx, and let Π(⋅)\Pi(\cdot) denote differentiable Cartesian-to-polar resampling. Since the Fourier transform is rotation-equivariant up to coordinate rotation, rotating xx by α\alpha rotates its amplitude spectrum by the same angle; in polar coordinates, this becomes a shift along the angular axis:

Π(ARαx)(ρ,ϕ)=Π(RαAx)(ρ,ϕ)=Π(Ax)(ρ,ϕ−α),(6)\Pi(A_{R_{\alpha}x})(\rho,\phi) = \Pi(R_{\alpha}A_x)(\rho,\phi) = \Pi(A_x)(\rho,\phi-\alpha), \tag*{(6)}

where ρ\rho and ϕ\phi denote radial frequency and angle, respectively. This converts a geometrically complex rotation in Cartesian spectra into a simpler translation along the angular axis in polar spectra. This conversion is naturally handled by convolution along the angular axis and is also well aligned with similarity computation and softmax-based cross-frequency attention. We therefore project the high- and low-frequency amplitude spectra into polar coordinates:

PH=Π(AH),PL=Π(AL),(7)P_H = \Pi(A_H), \qquad P_L = \Pi(A_L), \tag*{(7)}

where PH,PL∈RC×S×NϕP_H, P_L \in\mathbb{R}^{C \times S \times N_{\phi}}, CC is the number of channels, SS is the number of radial bins, and NϕN_{\phi} is the number of angular bins. Let PH(u),PL(u)∈RC×NϕP_H^{(u)}, P_L^{(u)} \in\mathbb{R}^{C \times N_{\phi}} denote the uu-th radial ring. This representation provides a geometry-aligned space for PSA to perform low-to-high structural matching.

3.4.2 Cross-Frequency Matching and Angular Gating. We first aggregate the low-frequency polar spectrum into a global descriptor,

PˉL=Norm⁡ϕ(∑u=1SPL(u)),(8)\bar{P}_L = \operatorname{Norm}_{\phi}\left(\sum_{u=1}^{S} P_L^{(u)}\right), \tag*{(8)}

where Norm⁡ϕ\operatorname{Norm}_{\phi} denotes L2L_2 normalization along the ϕ\phi axis. For each high-frequency ring u∈{1,…,S}u \in\{1,\ldots,S\}, we treat the normalized high-frequency response as the query Q(u)=Norm⁡ϕ(PH(u))Q^{(u)} = \operatorname{Norm}_{\phi}(P_H^{(u)}), and the global low-frequency descriptor as both the key and value K=V=PˉLK = V = \bar{P}_L. The cross-frequency guidance G(u)G^{(u)} for this specific ring is then computed via a scaled dot-product attention:

G(u)=Softmax⁡row⁡(Q(u)KTNϕ)V.(9)G^{(u)} = \operatorname{Softmax}_{\operatorname{row}}\left(\frac{Q^{(u)}K^{\mathsf{T}}}{\sqrt{N_{\phi}}}\right)V. \tag*{(9)}

Stacking {G(u)}u=1S\{G^{(u)}\}_{u=1}^{S} yields the guidance tensor G∈RC×S×NϕG \in\mathbb{R}^{C \times S \times N_{\phi}}. Essentially, each high-frequency radial ring is matched to the global low-frequency descriptor. Because a radial ring corresponds to one spatial frequency across different orientations, the resulting guidance acts as a frequency-aware cross-frequency attention that lets each ring query low-frequency structural cues for detail recovery at its own scale.

The concatenation of PHP_H and GG is processed by a 1×11 \times1 convolution and a circularly padded 1×31 \times3 angular depthwise convolution to preserve continuity across the 00–2π2\pi boundary; the resulting gate 2σ(⋅)2\sigma(\cdot), where σ(⋅)\sigma(\cdot) denotes the sigmoid function, rescales PHP_H to produce P~H\widetilde{P}_H. Finally, after reverting to Cartesian coordinates, the PSA block outputs the refined high-frequency amplitude:

A~H=PSA⁡(AH,AL).(10)\widetilde{A}_H = \operatorname{PSA}(A_H,A_L). \tag*{(10)}

In this way, PSA acts as a geometry-aware cross-frequency attention module that aligns low- and high-frequency spectra in polar coordinates and extracts structurally reliable low-frequency cues to guide high-frequency detail recovery.

Learning Objectives

The overall training objective of RoES is formulated as:

L=Lfuse+λRoUPLRoUP,(11)\mathcal{L} = \mathcal{L}_{\mathrm{fuse}} + \lambda_{\mathrm{RoUP}}\mathcal{L}_{\mathrm{RoUP}}, \tag*{(11)}

where Lfuse\mathcal{L}_{\mathrm{fuse}} is the primary image fusion loss, and LRoUP\mathcal{L}_{\mathrm{RoUP}} is the auxiliary decomposition loss. The λ\lambda coefficients denote the empirical trade-off weights.

3.5.1 Fusion objective. The fusion loss ensures that the final fused image IFI_F faithfully integrates complementary information from the infrared (III_I) and visible (IVI_V) inputs:

Lfuse=Lint+λgradLgrad+λssimLssim+λtvLtv+λvggLvgg,(12)\mathcal{L}_{\mathrm{fuse}} = \mathcal{L}_{\mathrm{int}} + \lambda_{\mathrm{grad}}\mathcal{L}_{\mathrm{grad}} + \lambda_{\mathrm{ssim}}\mathcal{L}_{\mathrm{ssim}} + \lambda_{\mathrm{tv}}\mathcal{L}_{\mathrm{tv}} + \lambda_{\mathrm{vgg}}\mathcal{L}_{\mathrm{vgg}}, \tag*{(12)}

where Lint\mathcal{L}_{\mathrm{int}} and Lgrad\mathcal{L}_{\mathrm{grad}} denote the L1L_1 penalty on pixel intensities and spatial gradients, respectively. Lssim\mathcal{L}_{\mathrm{ssim}} enforces structural similarity, Ltv\mathcal{L}_{\mathrm{tv}} suppresses local artifacts via total variation, and Lvgg\mathcal{L}_{\mathrm{vgg}} improves perceptual fidelity.

3.5.2 RoUP objective. The RoUP loss ensures accurate frequency decoupling and faithful feature reconstruction. Let XqX_q denote the interaction-enhanced encoder feature for modality q∈{I,V}q \in\{I,V\}, and let X^q=InvRoUP⁡(Lq,Hq)\widehat{X}_q = \operatorname{InvRoUP}(L^q,H^q) denote the feature reconstructed by the exact inverse RoUP transform. We first define the reconstruction loss Lrec\mathcal{L}_{\mathrm{rec}} to preserve feature fidelity:

Lrec=∑q∈{I,V}(∥X^q−Xq∥1+λstr(1−SSIM⁡(X^q,Xq))).(13)\mathcal{L}_{\mathrm{rec}} = \sum_{q \in\{I,V\}}\left(\lVert\widehat{X}_q-X_q\rVert_1 + \lambda_{\mathrm{str}}\left(1-\operatorname{SSIM}(\widehat{X}_q,X_q)\right)\right). \tag*{(13)}

where λstr\lambda_{\mathrm{str}} controls the weight of the structural similarity penalty. The overall RoUP objective is then formulated by combining Lrec\mathcal{L}_{\mathrm{rec}} with the cross-modal decomposition losses:

LRoUP=Lrec+λlowLlow+λhighLhigh,(14)\mathcal{L}_{\mathrm{RoUP}} = \mathcal{L}_{\mathrm{rec}} + \lambda_{\mathrm{low}}\mathcal{L}_{\mathrm{low}} + \lambda_{\mathrm{high}}\mathcal{L}_{\mathrm{high}}, \tag*{(14)}

where Llow\mathcal{L}_{\mathrm{low}} and Lhigh\mathcal{L}_{\mathrm{high}} (defined in Sec. 3.2) supervise the shared low-frequency and modality-specific high-frequency components, respectively.

Experiments

Experimental Setup

Datasets

We train RoES on 1,602 infrared-visible pairs from MSRS [40] and evaluate it directly on M³FD [22], TNO [44], and RoadScene [57] without fine-tuning. The differences in scenes, imaging conditions, and data distributions provide a rigorous test of cross-dataset robustness and generalization.

Comparison Methods

We compare RoES with the 17 CNN-, GAN-, transformer-, and task- or semantic-aware fusion methods listed in the result tables. We use official implementations and released weights with identical inputs, preprocessing, test splits, and evaluation protocols unless stated otherwise.

Implementation Details

RoES is implemented in PyTorch and trained end-to-end on MSRS using one NVIDIA RTX 5090 GPU. Inputs are resized to 128×128128 \times128 for training, and AdamW uses an initial learning rate of 5.05×10−55.05 \times10^{-5} and weight decay of 1×10−41 \times10^{-4}. The loss weights are set to λgrad=26.7\lambda_{\mathrm{grad}} = 26.7, λssim=25.6\lambda_{\mathrm{ssim}} = 25.6, λtv=16.0\lambda_{\mathrm{tv}} = 16.0, λvgg=3.8\lambda_{\mathrm{vgg}} = 3.8, λRoUP=33.0\lambda_{\mathrm{RoUP}} = 33.0, λstr=5.6×104\lambda_{\mathrm{str}} = 5.6 \times10^{4}, λlow=0.003\lambda_{\mathrm{low}} = 0.003, and λhigh=0.03\lambda_{\mathrm{high}} = 0.03. We use four RoUP lifting steps, a width-64 four-layer spectral SSM, eight attention heads, and two PSA stages. Polar resampling uses (S,Nϕ)=(512,2048)(S, N_{\phi}) = (512, 2048) during training and (3072,3072)(3072, 3072) during inference; the random seed is fixed to 42, and rotation augmentation is disabled.

Metrics

We report nine standard metrics: entropy (EN), spatial frequency (SF), average gradient (AG), visual information fidelity (VIF), QabfQ_{abf}, mutual information (MI), feature mutual information (FMI), QcbQ_{cb}, and peak signal-to-noise ratio (PSNR). EN, MI, and FMI measure retained information; SF and AG measure detail; and the remaining metrics assess fidelity and structural preservation. Higher is better for all metrics. We also compute TOPSIS rankings from normalized scores to summarize overall performance.

Main Results

Quantitative Comparison

Table 1 reports cross-dataset results on M³FD, RoadScene, and TNO. RoES obtains the best TOPSIS score on all three datasets, with clear margins over the second-ranked methods. It leads on SF, VIF, QabfQ_{abf}, MI, FMI, and PSNR on M³FD; ranks first on seven of nine M³FD metrics; and leads TNO on VIF, QabfQ_{abf}, MI, FMI, QcbQ_{cb}, and TOPSIS. These results show strong and balanced generalization across adverse-weather, road, and conventional fusion scenes.

ModelEN↑SF↑AG↑VIF↑QabfQ_{abf}↑MI↑FMI↑QcbQ_{cb}↑PSNR↑TOPSIS↑
M³FD Dataset
DenseFuse [14]6.4269.2063.2000.6530.4192.7770.2510.43115.7620.280
FusionGAN [32]6.6368.0012.6850.4790.2662.9650.2170.35115.8270.671
PMGI [61]6.3049.1993.2280.5880.3683.0110.2260.42516.1170.219
U2Fusion [56]6.66112.7624.6960.6640.5082.6680.2640.50415.9450.495
RFN-Nest [16]6.8647.8272.8690.6440.3622.8120.2470.43615.5640.221
SeAFusion [39]6.84814.4764.9230.7370.5493.3400.2750.44915.6310.600
SwinFusion [31]6.80714.2464.7770.7660.5503.7180.2820.46315.8910.630
TarDAL [22]7.15212.5394.1160.6430.3773.0910.2390.43215.2160.374
YDTR [41]6.54712.0013.9080.6780.4573.0070.2550.41615.5090.406
CDDFuse [64]6.90514.7824.8440.7890.5493.4470.2800.45515.9140.629
IRFS [45]6.74410.8003.5030.6860.4582.7470.2580.41415.7080.356
LRRNet [17]6.43510.7343.5840.6390.4572.7430.2430.42916.5820.339
SegMiF [24]6.98514.3474.8280.7870.5872.8970.2880.44815.9260.619
EMMA [65]6.92215.3275.3250.7990.5483.5790.2830.46216.7930.668
Text-IF [58]6.93115.7555.2840.8520.6113.3510.3010.50517.1450.721
SAGE [52]6.85013.8674.6360.7740.5853.0830.3450.44915.1720.656
DAFusion [8]6.70611.3473.9510.7360.5643.0210.3440.45615.5280.557
RoES (Ours)6.85815.9325.0980.9410.6424.1350.4210.49518.1270.957
RoadScene Dataset
DenseFuse [14]6.83211.6874.5360.6760.4822.8680.3660.50814.4700.496
FusionGAN [32]7.0658.4683.3100.4570.2522.7300.2450.45012.6450.076
PMGI [61]6.31911.4654.6710.5970.4053.0250.2850.52014.3160.387
U2Fusion [56]6.88914.5235.8540.6510.5032.7190.3440.52815.7400.617
RFN-Nest [16]7.3387.8113.3090.5870.2972.7240.2730.47213.9690.188
SeAFusion [39]7.34018.1557.0550.6590.4883.0120.3290.48415.0430.744
SwinFusion [31]6.99017.3146.3570.6960.4383.2540.3270.49414.2990.700
TarDAL [22]7.32013.5604.7680.6040.4153.2700.3160.44315.0410.481
YDTR [41]6.90913.9195.2090.6810.4772.9840.3360.48015.3480.573
CDDFuse [64]7.48218.9537.6860.6860.4693.0160.3350.49114.8630.752
IRFS [45]7.00711.6834.4650.6510.4462.7350.3420.44115.8920.450
LRRNet [17]7.13312.5134.6740.5520.3412.7730.2520.50812.1070.323
SegMiF [24]7.34115.7016.0270.6930.5122.7050.3440.48215.4360.661
EMMA [65]7.48216.1596.1880.6910.4363.2190.3140.50414.8050.664
Text-IF [58]7.38616.7986.6380.7030.5232.8500.3400.50215.4840.732
SAGE [52]7.00915.2755.6790.6400.4373.0040.3270.44814.1930.577
DAFusion [8]7.08113.8405.4090.6860.5122.8900.3570.50015.0920.605
RoES (Ours)7.46120.0397.0910.7760.5613.8110.3670.53215.5700.986
TNO Dataset
DenseFuse [14]6.4688.8243.3950.6390.3662.2540.2500.46315.3920.350
FusionGAN [32]6.5666.5022.4470.4530.2132.3660.2120.40913.9640.086
PMGI [61]6.0878.7263.4170.5890.3322.3490.2230.45614.8190.292
U2Fusion [56]6.56211.7874.8630.6190.3822.0090.2550.51315.7260.480
RFN-Nest [16]7.0715.9442.7040.6120.2892.2750.2420.46015.0620.214
SeAFusion [39]7.18612.5734.9640.6960.4252.8360.2630.47614.9170.595
SwinFusion [31]6.98512.6304.8760.7300.4423.1140.2920.48914.8850.661
TarDAL [22]7.18912.3194.2020.6060.3572.7590.2420.43814.5790.472
YDTR [41]6.59710.1243.6460.6760.3822.7060.2640.43416.2290.451
CDDFuse [64]7.15612.7764.6360.7370.4422.9240.2860.48915.0030.640
IRFS [45]6.7279.5603.4400.6230.3552.2500.2500.44315.3500.353
LRRNet [17]7.0689.7523.8280.5810.3212.6000.2290.48415.6730.365
SegMiF [24]7.15412.1514.6050.7590.4842.8260.3070.52615.3440.678
EMMA [65]7.24912.1844.9210.7130.4102.9110.2670.50215.2390.599
Text-IF [58]7.25513.2765.1080.7280.4542.9230.2760.51315.4900.654
SAGE [52]7.09411.6764.3960.7320.4522.7460.3270.47014.4790.635
DAFusion [8]6.82910.2523.9460.7820.4932.6400.3450.48815.2740.636
RoES (Ours)7.20712.9554.6700.9280.5553.7410.4010.53016.0580.950

Table 1. Quantitative evaluation of fusion models on the M³FD, RoadScene, and TNO datasets.

RoES ranks first on MI, FMI, VIF, and QabfQ_{abf} throughout. The MI/FMI gains support RoUP’s task-adaptive separation of shared low-frequency structure and complementary high-frequency information. The VIF/QabfQ_{ab}^{f} gains support the selective design: equivariant low-frequency modeling stabilizes global structure, while low-frequency-guided spectral refinement preserves details without rigidly constraining them. Together, these results demonstrate a favorable balance of information retention, structural fidelity, and detail enhancement.

Qualitative Comparison

Figure 5 compares fusion results on M3^{3}FD. Although all methods highlight thermal targets, their detail restoration and color fidelity differ markedly under severe smoke occlusion.

Qualitative comparison of infrared-visible fusion results on the M3FD dataset

Figure 5. Qualitative comparison of infrared-visible fusion results on the M³FD dataset.

RoES preserves thermal targets, background consistency, and fine textures, whereas FusionGAN [32] and TarDAL [22] show color distortion or structural blur and SwinFusion [31] loses fine details. The results support frequency-selective equivariance for balancing thermal saliency and visible-detail recovery.

Efficiency

At 1024 × 768 resolution, RoES has 2.28M parameters, requires 1,989G FLOPs, and uses approximately 6.3GB of peak memory. Although RoES is not the lightest in FLOPs, its computational cost remains acceptable for practical deployment, and its peak memory usage fits within a single consumer GPU.

Ablation Studies

Ablation Setup. Table 2 evaluates three components on M3^{3}FD under the Sec. 4.1 protocol. w/o Low-Eq, w/o Low-Eq with RotAug, and w/o Low-Eq with C4C_{4}-TTA compare built-in equivariance with augmentation and test-time averaging. w/ Fixed Wavelet and w/o Dec. Sup. test adaptive decomposition, while w/o PSA removes low-frequency-guided spectral refinement. The w/ Cartesian Attn. variant replaces PSA’s polar projection with Cartesian cross-attention under otherwise identical settings.

ExperimentEN↑SF↑AG↑VIF↑QabfQ_{ab}^{f}↑MI↑FMI↑QcbQ_{cb}↑PSNR↑
w/o Low-Eq6.85217.1615.4220.9080.6153.5780.3900.48117.427
w/o Low-Eq with RotAug6.83915.9215.0240.8660.6173.3340.3770.47616.653
w/o Low-Eq with C4C_{4}-TTA6.85117.1715.6210.9170.6193.6250.3990.48217.433
w/ Fixed Wavelet6.81716.4515.0800.8640.6133.5000.3750.48117.174
w/o Dec. Sup.6.84816.7345.1680.9270.6133.6470.3880.48416.048
w/o PSA6.83015.8184.9330.9150.6273.8740.4010.49017.749
w/ Cartesian Attn.6.97715.9745.3650.8190.6003.0830.3670.47716.610
Baseline (RoES)6.85815.9325.0980.9410.6424.1350.4210.49518.127

Table 2. Ablation study on key components of RoES.

Ablation Analysis. Removing low-frequency equivariance increases SF/AG by 1.229/0.324 but reduces VIF, MI, QabfQ_{ab}^{f}, and PSNR by 0.033, 0.557, 0.027, and 0.700, showing that equivariance improves fidelity rather than sharpness alone. Rotation augmentation does not recover the overall loss, whereas C4C_{4}-TTA partly recovers it. Fixed wavelets and removal of decomposition supervision also degrade performance; the latter decreases MI, FMI, and PSNR by 0.488, 0.033, and 2.079. Removing PSA reduces VIF, MI, FMI, and PSNR by 0.026, 0.261, 0.020, and 0.378. The Cartesian variant trades minor EN/SF/AG gains for substantial fidelity loss, confirming the necessity of PSA’s polar projection for accurate cross-frequency mapping. Thus, adaptive decomposition, low-frequency equivariance, and PSA make complementary contributions.

Downstream Object Detection Task

Following [25], we evaluate YOLOv5 v7.0 on M3^{3}FD images fused by each method using the same detector architecture, with a separate detector trained for each fusion method under the same protocol. Table 3 shows that RoES achieves 0.679 [email protected], with strong results for Lamp (0.632), Motor (0.637), Car (0.849), and People (0.733). The fusion model is trained only on MSRS and transferred to M3^{3}FD without fine-tuning, demonstrating cross-domain generalization across object scales.

MethodLampCarBusMotorTruckPeople[email protected]↑
DenseFuse [14]0.4640.8200.7100.3100.4830.6680.576
FusionGAN [32]0.3990.7080.7080.3010.4860.6930.549
U2Fusion [56]0.4330.8350.7100.3480.4610.7040.582
RFN-Nest [16]0.4970.8300.7020.3850.4380.6710.587
SeAFusion [39]0.3980.8260.7190.3400.4710.6940.575
SwinFusion [31]0.4090.8260.7170.3080.5160.6860.577
TarDAL [22]0.4510.8240.7400.3690.4770.7270.598
YDTIR [41]0.3380.8200.6820.2630.4150.6040.520
CDDFuse [64]0.4270.8210.7040.3290.4530.6650.566
IRFS [45]0.3700.8220.6870.2780.4440.6150.536
LRRNet [17]0.4610.8280.6960.3310.4540.6640.572
SegMiF [24]0.4680.8310.7270.3650.4380.6840.585
EMMA [65]0.4620.8270.6470.3920.4730.6820.581
Text-IF [58]0.3980.8140.6810.3940.4530.6540.566
RoES (Ours)0.6320.8490.6590.6370.5650.7330.679

Table 3. Quantitative evaluation of the downstream object detection task on the M3^{3}FD dataset.

Figure 6 further shows sharp contours, salient thermal targets, and precise boxes for small or distant objects. Most competing methods except LRRNet [17] miss or misalign pedestrians, supporting the benefit of stable low-frequency structure and preserved high-frequency texture.

Object detection results on the M3FD dataset

Figure 6. Object detection results on the M3^{3}FD dataset.

Conclusion

We presented RoES, an infrared-visible fusion method that applies rotation equivariance selectively rather than uniformly. RoUP learns adaptive frequency decomposition; an equivariant Mamba branch models low-frequency structure; and a frequency-guided Dual-Fourier branch with polar spectral attention refines high-frequency details. Results across fusion benchmarks and downstream detection demonstrate state-of-the-art performance and support frequency-selective equivariance as an effective principle for robust multimodal fusion.

Acknowledgments

This work is partially supported by the National Natural Science Foundation of China (No. 62401632), the Hubei Provincial Natural Science Foundation (Nos. 2024AFB484 and 2026AFB758), the Natural Science Foundation of Wuhan (No. 2026040301020058), the Hubei Provincial Department of Education Key Projects (No. D20251003), the Open Fund of the Research Platforms, School of Computer Science, China University of Geosciences, Wuhan (No. PTLH2024-B-06), and the Fundamental Research Funds for the Central Universities of Zhongnan University of Economics and Law (No. 2722025BJ037).

References

References

  1. [1]Yulu Bai, Jiahong Fu, Qi Xie, and Deyu Meng. 2025. A regularization-guided equivariant approach for image restoration. In 2025 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). IEEE, 2300–2310.arxiv.org/abs/2505.19799
  2. [2]Georg Böhm, Johan Edstedt, Michael Felsberg, and Fredrik Kahl. 2024. Affine steerers for structured keypoint description. In European Conference on Computer Vision. Springer, 449–468.
  3. [3]Yaoming Cai, Zijia Zhang, Xiaobo Liu, Yao Ding, Fei Li, and Jinhua Tan. 2025. Learning Unified Anchor Graphs for Joint Clustering of Hyperspectral and LiDAR Data. IEEE Transactions on Neural Networks and Learning Systems 36, 4 (2025), 6341–6354. doi:10.1109/TNNLS.2024.3392484DOI
  4. [4]Taco S. Cohen and Max Welling. 2016. Group Equivariant Convolutional Networks. In Proceedings of the 33rd International Conference on Machine Learning (ICML). 2990–2999.
  5. [5]Sahar Dastani, Ali Bahri, Moslem Yazdanpanah, Mehrdad Noori, David Osowiechi, Gustavo Adolfo Vargas Hakim, Farzad Beizaee, Milad Cheraghalkhani, Arnab Kumar Mondal, Herve Lombart, et al. 2025. Spectral state space model for rotation-invariant visual representation learning. In Proceedings of the Computer Vision and Pattern Recognition Conference. 23881–23890.
  6. [6]Ingrid Daubechies and Wim Sweldens. 1998. Factoring wavelet transforms into lifting steps. Journal of Fourier analysis and applications 4, 3 (1998), 247–269.
  7. [7]Jun Gao, Shiwei Ma, and Jingjing Liu. 2022. DCDCR-GAN: A densely connected disentangled representation generative adversarial network for infrared and visible image fusion. IEEE Transactions on Circuits and Systems for Video Technology 33, 2 (2022), 549–561.
  8. [8]Tianwei Guan, Haozhen Wei, Yuhan Zhou, Jun Ma, Zecheng Xu, Zhiying Jiang, Jinyuan Liu, and Xingyuan Li. 2026. Domain adaptation guided infrared and visible image fusion. In Proceedings of the AAAI Conference on Artificial Intelligence, Vol. 40. 4376–4384.
  9. [9]Jiaqiang Han, Jian Ding, Nan Xue, and Gui-Song Xia. 2021. Redet: A rotation-equivariant detector for aerial object detection. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition. 2786–2795.
  10. [10]Jun-Jie Huang and Pier Luigi Dragotti. 2022. WINNet: Wavelet-inspired invertible network for image denoising. IEEE Transactions on Image Processing 31 (2022), 4377–4392.
  11. [11]Lihua Jian, Xiaomin Yang, Zheng Liu, Gwanggil Jeon, Mingliang Gao, and David Chisholm. 2020. SEDRFuse: A symmetric encoder–decoder with residual block network for infrared and visible image fusion. IEEE Transactions on Instrumentation and Measurement 70 (2020), 1–15.
  12. [12]Shahid Karim, Geng Tong, Jinyang Li, Akeel Qadir, Umar Farooq, and Yiting Yu. 2023. Current advances and future perspectives of image fusion: A comprehensive review. Information Fusion 90 (2023), 185–217.
  13. [13]Jongmin Lee, Byungjin Kim, Seungwook Kim, and Minsu Cho. 2023. Learning rotation-equivariant features for visual correspondence. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 21887–21897.arxiv.org/abs/2303.15472
  14. [14]Hui Li and Xiao-Jun Wu. 2019. DenseFuse: A Fusion Approach to Infrared and Visible Images. IEEE Transactions on Image Processing 28, 5 (2019), 2614–2623.
  15. [15]Hui Li, Xiao-Jun Wu, and Josef Kittler. 2020. MDLatLRR: A novel decomposition method for infrared and visible image fusion. IEEE Transactions on Image Processing 29 (2020), 4733–4746.
  16. [16]Hui Li, Xiao-Jun Wu, and Josef Kittler. 2021. RFN-Nest: An end-to-end residual fusion network for infrared and visible images. Information Fusion 73 (2021), 72–86.
  17. [17]Hui Li, Tianyang Xu, Xiao-Jun Wu, Jiwen Lu, and Josef Kittler. 2023. Lrrnet: A novel representation learning guided fusion network for infrared and visible images. IEEE transactions on pattern analysis and machine intelligence 45, 9 (2023), 11040–11052.
  18. [18]Xilai Li, Xiaosong Li, Tianshu Tan, Huafeng Li, and Tao Ye. 2025. UMCFuse: A unified multi-scale complex scenes infrared and visible image fusion framework. IEEE Transactions on Image Processing (2025), 6231–6245.
  19. [19]Xilai Li, Huichun Liu, Xiaosong Li, Yao Ye, Zhenyu Kuang, and Huafeng Li. 2026. Awm-fuse: Multi-modality image fusion for adverse weather via global and local text perception. IEEE Transactions on Image Processing (2026), 5151–5164.
  20. [20]Xilai Li, Yuanygu Liu, Xiaosong Li, Fuqiang Zhou, Huafeng Li, and Feiping Nie. 2026. All-weather multi-modality image fusion: Unified framework and 100k benchmark. Information Fusion (2026), 104130.
  21. [21]Hanze Liu, Jiahong Fu, Qi Xie, and Deyu Meng. 2025. Rotation-equivariant self-supervised method in image denoising. In Proceedings of the Computer Vision and Pattern Recognition Conference. 12720–12730.
  22. [22]Jinyuan Liu, Xin Fan, Zhanbo Huang, Guanyao Wu, Risheng Liu, Wei Zhong, and Zhongxuan Luo. 2022. Target-Aware Dual Adversarial Learning and a Multi-Scenario Multi-Modality Benchmark to Fuse Infrared and Visible for Object Detection. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). 5802–5811.
  23. [23]Jinyuan Liu, Xin Fan, Ji Jiang, Risheng Liu, and Zhongxuan Luo. 2021. Learning a deep multi-scale feature ensemble and an edge-attention guidance for image fusion. IEEE Transactions on Circuits and Systems for Video Technology 32, 1 (2021), 105–119.
  24. [24]Jinyuan Liu, Zhu Liu, Guanyao Wu, Long Ma, Risheng Liu, Wei Zhong, Zhongxuan Luo, and Xin Fan. 2023. Multi-Interactive Feature Learning and a Full-Time Multi-Modality Benchmark for Image Fusion and Segmentation. In Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV). 8115–8124.
  25. [25]Jinyuan Liu, Guanyao Wu, Zhu Liu, Di Wang, Zhiying Jiang, Long Ma, Wei Zhong, Xin Fan, and Risheng Liu. 2025. Infrared and Visible Image Fusion: From Data Compatibility to Task Adaptation. IEEE Transactions on Pattern Analysis and Machine Intelligence 47, 4 (2025), 2349–2369. doi:10.1109/TPAMI.2024.3521416DOI
  26. [26]Yuanye Liu, Renwei Dian, and Shitao Li. 2025. Low-rank transformer for high-resolution hyperspectral computational imaging. International Journal of Computer Vision 133, 2 (2025), 809–824.
  27. [28]Yuanye Liu, Jinyang Liu, Renwei Dian, and Shitao Li. 2025. A selective re-learning mechanism for hyperspectral fusion imaging. In 2025 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). IEEE, 7437–7446.
  28. [29]Haichuan Ma, Dong Liu, Ruiqin Xiong, and Feng Wu. 2019. iWave: CNN-based wavelet-like transform for image compression. IEEE Transactions on Multimedia 22, 7 (2019), 1667–1679.
  29. [30]Jiayi Ma, Yeng Ma, and Chang Li. 2019. Infrared and visible image fusion methods and applications: A survey. Information fusion 45 (2019), 153–178.
  30. [31]Jiayi Ma, Lienfeng Tang, Fan Fan, Jun Huang, Xiaoguang Mei, and Yong Ma. 2022. SwimFusion: Cross-domain long-range learning for general image fusion via swin transformer. IEEE/CAA Journal of Automatica Sinica 9, 7 (2022), 1200–1217.
  31. [32]Jiayi Ma, Wei Yu, Pengwei Liang, Chang Li, and Junjun Jiang. 2019. FusionGAN: A generative adversarial network for infrared and visible image fusion. Information fusion 48 (2019), 11–26.
  32. [33]Gonzalo Pajares and Jesus Manuel de la Cruz. 2004. A Wavelet-Based Image Fusion Tutorial. Pattern Recognition 37, 9 (2004), 1855–1872. doi:10.1016/j.patcog.2004.03.010DOI
  33. [34]Seonghyun Park, An Gia Vien, and Chul Lee. 2023. Cross-modal transformers for infrared and visible image fusion. IEEE Transactions on Circuits and Systems for Video Technology 34, 2 (2023), 770–785.
  34. [35]Maria Ximena Bastidas Rodriguez, Adrien Gruson, Luisa Polania, Shin Fujieda, Flavio Prieto, Kohei Takayama, and Toshiya Hachisuka. 2020. Deep adaptive wavelet network. In Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision. 3111–3119.
  35. [36]Pourya Shamsolmali, Masoumeh Zareapoor, Jocelyn Chanussot, Huiyu Zhou, and Jie Yang. 2021. Rotation equivariant feature image pyramid network for object detection in optical remote sensing imagery. IEEE Transactions on Geoscience and Remote Sensing 60 (2021), 1–14.
  36. [37]Zhengyang Shen, Lingshen He, Zhouchen Lin, and Jinwen Ma. 2020. Pdo-econvs: Partial differential operator based equivariant convolutions. In International Conference on Machine Learning. PMLR, 8697–8706.
  37. [38]Wim Sweldens. 1998. The lifting scheme: A construction of second generation wavelets. SIAM Journal on mathematical analysis 29, 2 (1998), 511–546.
  38. [39]Linfeng Tang, Jinying Yuan, and Jiayi Ma. 2022. Image fusion in the loop of high-level vision tasks: A semantic-aware real-time infrared and visible image fusion network. Information Fusion 82 (2022), 28–42.
  39. [40]Linfeng Tang, Jieting Yuan, Hao Zhang, Xingyu Jiang, and Jiayi Ma. 2022. PI-AFusion: A progressive infrared and visible image fusion network based on illumination aware. Information Fusion 83 (2022), 79–92.
  40. [41]Wei Tang, Fazhi He, and Yu Liu. 2022. YDTR: Infrared and visible image fusion via Y-shape dynamic transformer. IEEE Transactions on Multimedia 25 (2022), 5413–5428.
  41. [42]Wei Tang, Fazhi He, Yu Liu, Yansong Duan, and Tongzhen Si. 2023. DATFuse: Infrared and visible image fusion via dual attention transformer. IEEE Transactions on Circuits and Systems for Video Technology 33, 7 (2023), 3159–3172.
  42. [43]Yang Tian, Mengmeng Zhang, Jinyu Li, Yangfan Li, Hong Yang, and Wei Li. 2024. PPFormer: Rethink the method of processing the rotation-invariance and rotation-equivariance on arbitrary-oriented object detection. IEEE Transactions on Geoscience and Remote Sensing 62 (2024), 1–10.
  43. [44]Alexander Toet. 2017. The TNO Multiband Image Data Collection. Data in Brief 15 (2017), 249–251. doi:10.1016/j.dib.2017.09.038DOI
  44. [45]Di Wang, Jinyuan Liu, Risheng Liu, and Xin Fan. 2023. An interactively reinforced paradigm for joint infrared-visible image fusion and saliency object detection. Information Fusion 98 (2023), 101828.
  45. [46]Zhishhe Wang, Yuanyuan Wu, Junyuan Wang, Jiawei Xu, and Wenyu Shao. 2022. Res2Fusion: Infrared and visible image fusion based on dense Res2net and double nonlocal attention models. IEEE Transactions on Instrumentation and Measurement 71 (2022), 1–12.
  46. [47]Zhishhe Wang, Zhuoqun Zhang, Wuqiang Qi, Fengbao Yang, and Jiawei Xu. 2024. FreqGAN: Infrared and visible image fusion via unified frequency adversarial learning. IEEE Transactions on Circuits and Systems for Video Technology 35, 1 (2024), 728–740.
  47. [48]Maurice Weiler and Gabriele Cesa. 2019. General E(2)-Equivariant Steerable CNNs. In Advances in Neural Information Processing Systems (NeurIPS), Vol. 32.
  48. [49]Maurice Weiler, Fred A Hamprecht, and Martin Storath. 2018. Learning steerable filters for rotation equivariant cnns. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 849–858.
  49. [50]Daniel E Worrall, Stephan J Garbin, Daniyar Turmukhambetov, and Gabriel J Brostow. 2017. Harmonic networks: Deep translation and rotation equivariance. In Proceedings of the IEEE conference on computer vision and pattern recognition. 5028–5037.
  50. [51]Dan Wu, Mina Han, Yang Yang, Shan Zhao, Yujing Rao, Hao Li, Xing Lin, Chengjiang Zhou, and Haicheng Bai. 2023. DCFUsion: A Dual-Frequency Cross-Enhanced Fusion Network for Infrared and Visible Image Fusion. IEEE Transactions on Instrumentation and Measurement 72 (2023), 1–15. doi:10.1109/TIM.2023.3267380DOI
  51. [52]Guanyao Wu, Haoyu Liu, Hongming Fu, Yichuan Peng, Jinyuan Liu, Xin Fan, and Risheng Liu. 2025. Every SAM drop counts: Embracing semantic priors for multi-modality image fusion and beyond. In 2025 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). IEEE, 17882–17891.
  52. [53]Qi Xie, Jiahong Fu, Zongben Xu, and Deyu Meng. 2025. Rotation equivariant arbitrary-scale image super-resolution. IEEE Transactions on Pattern Analysis and Machine Intelligence (2025).
  53. [54]Qi Xie, Qian Zhao, Zongben Xu, and Deyu Meng. 2023. Fourier Series Expansion Based Filter Parametrization for Equivariant Convolutions. IEEE Transactions on Pattern Analysis and Machine Intelligence 45, 4 (2023), 4537–4551. doi:10.1109/TPAMI.2022.3196652
  54. [55]Han Xu, Meiqi Gong, Xin Tian, Jun Huang, and Jiayi Ma. 2022. CUFD: An encoder-decoder network for visible and infrared image fusion based on common and unique feature decomposition. Computer Vision and Image Understanding 218 (2022), 103407.
  55. [56]Han Xu, Jiayi Ma, Junjun Jiang, Xiaojie Guo, and Haibin Ling. 2020. U2Fusion: A unified unsupervised image fusion network. IEEE transactions on pattern analysis and machine intelligence 44, 1 (2020), 502–518.
  56. [57]Han Xu, Jiayi Ma, Zhuliang Le, Junjun Jiang, and Xiaojie Guo. 2020. Fusiondn: A unified densely connected network for image fusion. In Proceedings of the AAAI conference on artificial intelligence, Vol. 34. 12484–12491.
  57. [58]Xupeng Yu, Han Xu, Hao Zhang, Linfeng Tang, and Jiayi Ma. 2024. Text-if: Leveraging semantic text guidance for degradation-aware and interactive image fusion. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 27026–27035.
  58. [59]Syed Waqas Zamir, Aditya Arora, Salman Khan, Munawar Hayat, Fahad Shahbaz Khan, and Ming-Hsuan Yang. 2022. Restormer: Efficient Transformer for High-Resolution Image Restoration. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). 5728–5739.
  59. [60]Hao Zhang, Han Xu, Xin Tian, Junjun Jiang, and Jiayi Ma. 2021. Image fusion meets deep learning: A survey and perspective. Information Fusion 76 (2021), 323–336.
  60. [61]Hao Zhang, Han Xu, Yang Xiao, Xiaojie Guo, and Jiayi Ma. 2020. Rethinking the image fusion: A fast unified image fusion network based on proportional maintenance of gradient and intensity. In Proceedings of the AAAI conference on artificial intelligence, Vol. 34. 12797–12804.
  61. [62]Xingchen Zhang and Yiannis Demiris. 2023. Visible and infrared image fusion using deep learning. IEEE Transactions on Pattern Analysis and Machine Intelligence 45, 8 (2023), 10535–10554.
  62. [63]Zijia Zhang, Yaoming Cai, Wenyin Gong, Xiaobo Liu, Cheng Zeng, and Gan Yu. 2025. MMAGL: Multiobjective Multiview Attributed Graph Learning for Joint Clustering of Hyperspectral and LiDAR Data. IEEE Transactions on Geoscience and Remote Sensing 63 (2025), 1–14. doi:10.1109/TGRS.2025.3540269DOI
  63. [64]Zixiang Zhao, Haowen Bai, Jiangshe Zhang, Yulun Zhang, Shuangxu Xu, Zudi Lin, Radu Timofte, and Luc Van Gool. 2023. CDDFuse: Correlation-Driven Dual-Branch Feature Decomposition for Multi-Modality Image Fusion. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). 5906–5916.
  64. [65]Zixiang Zhao, Haowen Bai, Jiangshe Zhang, Yulun Zhang, Kai Zhang, Shuang Xu, Dongdong Chen, Radu Timofte, and Luc Van Gool. 2024. Equivariant Multi-Modality Image Fusion. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). 25912–25921.
  65. [66]Zixiang Zhao, Shuang Xu, Chunming Zhang, Junjun Liu, Pengfei Li, and Jiangshe Zhang. 2020. DDIFuse: Deep image decomposition for infrared and visible image fusion. arXiv preprint arXiv:2003.09210 (2020).DOI
  66. [67]Naishan Zheng, Man Zhou, Jie Huang, Junming Hou, Haoying Li, Yuan Xu, and Feng Zhao. 2024. Probign synergistic high-order interaction in infrared and visible image fusion. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition. 26384–26395.
  67. [68]Yi Zheng, Ruijin Wang, and Jianping Li. 2010. Nonlinear wavelets and BP neural networks adaptive lifting scheme. In The 2010 International Conference on Appereicing Computing and Intelligence Analysis Proceeding. IEEE, 316–319.
  68. [69]Man Zhou, Jie Huang, Keyu Yan, Danfeng Hong, Xuping Jia, Jocelyn Chansussot, and Chongyi Li. 2025. A General Spatial-Frequency Learning Framework for Multimodal Image Fusion. IEEE Transactions on Pattern Analysis and Machine Intelligence 47, 7 (2025), 5281–5298. doi:10.1109/TPAMI.2024.3368112DOI
  69. [70]Wenbin Zou, Hongxia Gao, Weipeng Yang, and Tongtong Liu. 2024. Wave-Mamba: Wavelet State Space Model for Ultra-High-Definition Low-Light Image Enhancement. In Proceedings of the 32nd ACM International Conference on Multimedia (Melbourne VIC, Australia) (MM ’24). Association for Computing Machinery, New York, NY, USA, 1534–1543. doi:10.1145/3664647.3681580DOI

Paper details

vlnc:W_jvqd7gbw
Version
v1 · latest
Published
Added
License
CC BY 4.0
Usage

views · 0

downloads · 0

A view counts once per browser per day, a download each time the PDF is requested.

  • Computer Science

Not recorded for this paper

View statement
Looking for related papers…
Contents