betaFeedback

WorksComputer Science

MADSNet: A Multi-Scale Axial-Attentive Dense Spiking Network for Spike Camera Image Reconstruction

Abstract

Spike cameras encode dynamic scenes into high-temporal-resolution binary spike streams for efficient high-speed vision. Although spiking neural networks (SNNs) can process such signals with sparse and accumulate-only computation, existing SNN-based reconstruction methods are typically shallow and still lag behind ANN-based models in reconstruction fidelity. This paper proposes MADSNet, a Multi-scale Axial-attentive Dense Spiking Network for spike camera image reconstruction. MADSNet adopts a hierarchical spiking encoder–decoder, where a multi-scale spike stem extracts early features, axial spike attention captures long-range dependencies, a dense spiking bottleneck enhances feature reuse, and a spike-to-continuous decoder reconstructs continuous intensity images from spike-domain representations. Experiments on gray-scale and color spike reconstruction show that MADSNet outperforms SSIR by 4.3 dB PSNR and Spk2ImgNet by 1.94 dB on SREDS. Compared with Spk2ImgNet, it achieves about 15×\times lower latency and uses only about 1/40 of the estimated energy, demonstrating that carefully designed SNN architectures can improve reconstruction quality while preserving computational efficiency.

Introduction

CONVENTIONAL frame-based cameras capture visual scenes by integrating photons within a fixed exposure interval. This imaging mechanism works well for ordinary scenes, but it inevitably suffers from motion blur, temporal discretization, and missing inter-frame information when recording high-speed dynamic processes. Spike cameras provide a neuromorphic alternative by representing visual signals as continuous binary spike streams [1, 2]. In a spike camera, each pixel independently accumulates incoming light intensity and asynchronously fires a binary spike once the accumulated signal reaches a predefined threshold, after which the accumulator is reset for the next integration cycle. This integrate-and-fire sensing mechanism enables ultra-high temporal resolution, e.g., up to tens of thousands of spike frames per second, while preserving dense temporal information for dynamic scene analysis [1, 2, 3].

The development of spike cameras has stimulated a broad range of visual applications, including high-speed image reconstruction [4, 5, 6, 7, 8], motion deblurring [9, 10], optical flow estimation [11, 12, 13, 14], tracking [15, 16], joint visual compression and perception [17], and color spike imaging [18, 19, 20, 21]. In parallel, the SpikeCV platform has been developed to support the spike vision ecosystem by providing spike datasets, hardware interfaces, data loaders, algorithm libraries, and unified processing pipelines for different spike-based visual tasks [2]. These efforts indicate that spike cameras are evolving from a specialized high-speed imaging sensor into a broader computational vision platform.

Within this ecosystem, spike-to-image reconstruction occupies a foundational position: it converts raw spike streams into the intensity representations that human visualization and most downstream vision tasks still depend on. Yet recovering high-quality intensity remains challenging, as the task is inherently ill-posed: the network must infer continuous scene radiance from observations that are sparse, binary, temporally accumulated, and corrupted by photon-arrival noise and motion. Early reconstruction methods, such as TFP [3], TFI [3], TFSTP [22], and MAHTF [4], estimate image intensity from hand-crafted spike statistics or temporal filtering. Although these methods are computationally simple, they are limited in recovering high-fidelity textures and complex spatiotemporal structures. Learning-based methods, including Spk2ImgNet and recent self-supervised reconstruction frameworks, significantly improve reconstruction quality by exploiting data-driven feature learning and motion-aware supervision [5, 7, 8, 9, 23]. However, most of these methods are based on artificial neural networks (ANNs), whose dense multiply-accumulate operations lead to high latency and energy consumption.

Spiking neural networks (SNNs) [24] provide a natural computational counterpart to spike cameras. By processing spike-domain activations with event-driven accumulate-only operations, SNNs have the potential to realize low-power and low-latency for spike cameras [6, 25, 26, 27]. SSIR [6] is a representative SNN-based reconstruction method that combines Spike-Element-Wise (SEW) residual blocks with a reparameterizable Progressive Filtering Module (PFM), reporting substantially lower energy consumption than ANN-based reconstruction. However, there remains a clear reconstruction-quality gap between existing SNN-based methods and high-capacity ANN models. We argue that this gap is not solely caused by the spiking neuron model, but also by insufficient architecture design. In particular, existing SNN reconstruction networks typically use single-scale early embeddings that overlook multi-scale spatial structure, rely on spike-domain encoders confined to local convolutions that cannot capture long-range dependencies, and adopt relatively direct decoding heads that handle the transition from discrete spike activations to continuous intensities only crudely.

To address these limitations, we propose MADSNet, a multi-scale axial-attentive dense spiking network for spike camera image reconstruction. The core design principle of MADSNet is to improve representation capacity while preserving the accumulate-only, multiplication-free computation that underlies the energy efficiency of SNNs. Specifically, MADSNet introduces a Multi-Scale Spike Stem (MS3Stem) to jointly capture inter-channel temporal correlations and local-to-large spatial patterns from stacked spike frames. It further employs depthwise-separable SEW blocks and light spike blocks with axial spike attention to build an efficient spike-domain encoder. A dense spike bottleneck is introduced to enhance latent feature reuse and stabilize information propagation in deeper spiking architectures. Finally, spike pixel-shuffle upsampling and a spike-to-continuous fusion block are used to recover fine spatial details and convert discrete spike-domain features into continuous intensity predictions.

MADSNet achieves a favorable balance between reconstruction accuracy and computational efficiency. On the SREDS benchmark, MADSNet reaches 37.232 dB PSNR, improving over the SNN baseline SSIR by +4.30 dB and surpassing the ANN-based Spk2ImgNet by +1.94 dB. At the same time, MADSNet remains much more efficient than Spk2ImgNet, achieving about 15×15\times lower inference latency, 3.4×3.4\times lower GPU memory usage, and only about 1/401/40 of the estimated energy consumption. These results suggest that SNN-based reconstruction does not have to trade substantial accuracy for efficiency; instead, with appropriate architecture design, SNNs can provide a strong accuracy–efficiency balance for spike camera reconstruction.

The main contributions of this work are threefold:

  • We propose MADSNet, a hierarchical spiking encoder-decoder built on the design principle of improving representation capacity while retaining the accumulate-only efficiency of spike-domain computation.

  • We develop a coherent set of spike-oriented modules that progressively enrich the spike-domain representation along the entire pipeline, from multi-scale early embedding, through axial-attentive encoding and dense latent aggregation, to spike-to-continuous decoding.

  • We conduct experiments on gray-scale and color spike reconstruction, demonstrating superior accuracy and efficiency trade-off over existing ANN and SNN baselines.

Related Work

Spike and Event Camera Image Reconstruction

1) Gray-Scale Spike Camera Reconstruction: Early filtering-based methods estimate intensity directly from spike statistics without learning. TFP and TFI [3] infer pixel intensity from spike firing frequency and inter-spike intervals, respectively, while TFSTP [22] and MAHTF [4, 26] exploit short-term plasticity and motion-aligned temporal filtering for higher fidelity, albeit at high latency. Learning-based methods improve quality further: Spk2ImgNet [5] aligns multi-temporal spike features with deformable convolutions but is computationally heavy, whereas SSIR [6] proposes the first SNN-based reconstruction with SEW residual blocks and a reparameterizable PFM, reducing energy to about 1/1019 of Spk2ImgNet but still trailing high-capacity ANNs in fidelity.

2) Color Spike Camera Reconstruction: The color spike camera (CSC) employs a Bayer-pattern Color Filter Array, introducing the additional challenges of spatially missing pixels per channel and inter-channel chromatic correlations. 3DRI [18] fills missing pixels via 3D residual interpolation using spatially and temporally neighboring samples, while subsequent work has developed deep demosaicing networks explicitly tailored for spike camera CFAs [20]. SJRE [9], a self-supervised gray-scale method, can be adapted to color streams by processing each Bayer channel independently, but lacks explicit inter-channel modeling. SSL-CSC [19] proposes a motion-guided pipeline that jointly estimates optical flow across color channels and trains a self-supervised network with a masked spike encoder to suppress quantization noise, establishing the current state-of-the-art among methods that do not require ground-truth supervision. Additionally, recent advances have demonstrated that aggregating long short-term temporal spike signals can further enhance continuous color reconstruction fidelity [21]. These methods, however, are purpose-built for color reconstruction rather than transferred from a general spike architecture.

3) Event Camera Reconstruction: A closely related neuromorphic modality is the event camera, which records per-pixel asynchronous brightness changes as a stream of events, enabling high dynamic range and microsecond temporal resolution [28]. Because event and spike cameras both represent visual information as asynchronous data streams, their deep-learning reconstruction pipelines share methodological similarities, and event-to-video reconstruction has demonstrated significant progress. E2VID [29] is a seminal work for this purpose, employing a recurrent fully convolutional network to convert event streams into video frames. Following E2VID, numerous studies have augmented this paradigm from various angles, including FireNet [30], E2VID++ [31], and FireNet++ [31]. ETNet [32] further incorporates Transformer-based attention into event-to-video reconstruction, improving the modeling of long-range temporal dependencies. HyperE2VID [33] adopts a dynamic neural network architecture and uses hypernetworks to generate per-pixel adaptive filters, achieving state-of-the-art reconstruction quality. However, event cameras record only relative intensity changes and thus cannot recover absolute texture detail, making event-based reconstruction complementary to but distinct from the spike-based reconstruction discussed above.

Spiking Neurons in SNNs

The Integrate-and-Fire (IF) neuron [24] fires a binary spike when its accumulated membrane potential exceeds a threshold, offering event-driven efficiency but limited temporal expressiveness. The Leaky Integrate-and-Fire (LIF) neuron [34] introduces membrane potential decay, enabling implicit short-term memory. To further improve adaptability, Parametric LIF (PLIF) [35] makes the time constant learnable, which is adopted in SSIR to improve training convergence. However, direct training via spatio-temporal backpropagation requires storing neuronal states across all timesteps, imposing significant GPU memory overhead, while the non-differentiable spike generation function necessitates surrogate gradient approximations that introduce inherent estimation errors and hinder stable convergence [36]. To address this, I-LIF [37] allows integer-valued spike emission during training while decomposing to binary spikes at inference, preserving accumulate-only (AC) computation on neuromorphic hardware. IBRA-LIF [38] further introduces a Range Alignment strategy that scales the effective dynamic range of integer spikes, alleviating quantization errors that accumulate across deep layers.

These neuron-level developments improve the representational capacity and training stability of SNNs, but they are orthogonal to architecture-level designs for spike reconstruction. In this work, we adopt IBRA-LIF as the basic activation function for spiking layers and focus on designing an efficient encoder-decoder architecture for gray-scale spike reconstruction.

Method

Spike Generation Mechanism

A spike camera sensor consists of an array of pixels that each independently accumulate incident photons in a continuous, asynchronous manner, mimicking the integrate-and-fire (I&F) dynamics of biological retinal ganglion cells. Specifically, each pixel maintains an internal accumulator whose state at spatial location x\mathbf{x} and time tt evolves as

A(x,t)=∫0tα⋅I(x,τ) dτ mod θ,(1)A(\mathbf{x}, t) = \int_{0}^{t} \alpha\cdot I(\mathbf{x}, \tau)\,\mathrm{d}\tau\bmod\theta, \tag*{(1)}

where I(x,t)I(\mathbf{x}, t) is the scene radiance (light intensity), α\alpha is the quantum efficiency of the photodetector, and θ\theta is the fixed firing threshold. Whenever the accumulated value reaches θ\theta, the pixel fires a binary spike and immediately resets its accumulator to zero, starting the next integration cycle.

The sensor output is read out at a fixed interval Δt\Delta t. Denoting the kk-th readout window as Tk:=((k−1)Δt,kΔt]\mathcal{T}_k := ((k-1)\Delta t, k\Delta t], the kk-th binary spike frame records whether at least one firing event occurred within that window:

Sk(x)=1[∃ t∈Tk:A(x,t)=θ],(2)S_k(\mathbf{x}) = \mathbf{1}[\exists\ t \in\mathcal{T}_k : A(\mathbf{x}, t) = \theta], \tag*{(2)}

where k=1,…,Tk = 1, \ldots, T, and the accumulator is immediately reset, A(x,t)←0A(\mathbf{x}, t) \leftarrow0, upon each firing event. Because the inter-spike interval at each pixel is inversely proportional to the local light intensity, brighter regions produce denser spike trains whereas darker regions produce sparser ones. This design enables ultra-high temporal resolution (up to tens of thousands of spike frames per second) and avoids the motion blur and temporal aliasing of conventional frame-based exposure. In practice, the continuous spike stream is discretized into TT binary frames S∈{0,1}T×H×W\mathbf{S} \in\{0,1\}^{T \times H \times W} over a short temporal window of duration T′ΔtT'\Delta t, which serves as the input to the reconstruction network. Recovering the continuous intensity image I^∈[0,1]H×W\hat{\mathbf{I}} \in[0,1]^{H \times W} from this sparse binary observation is inherently ill-posed, as a single spike sequence is consistent with a range of underlying radiance fields; this motivates the network design described in the following subsections.

Overall Architecture

Given a sequence of TT binary spike frames S∈{0,1}T×H×W\mathbf{S} \in\{0,1\}^{T \times H \times W}, the goal of gray-scale spike reconstruction is to recover the corresponding intensity image I^∈[0,1]H×W\hat{\mathbf{I}} \in[0,1]^{H \times W}. Unlike frame-based image restoration, spike reconstruction requires the network to infer continuous intensity values from sparse, asynchronous, and temporally accumulated binary observations. This poses three main challenges: effective early spike representation, efficient spike-domain feature learning, and smooth conversion from discrete spiking activations to continuous image intensities.

To address these challenges, we propose MADSNet, a hierarchical spiking encoder-decoder architecture. As illustrated in Figure 1, MADSNet first employs a Multi-Scale Spike Stem (MS3Stem) to embed the input spike sequence into a rich spike-domain representation. The encoder then uses light spike blocks to jointly model local structures and long-range spatial dependencies with low computational cost. A dense spike bottleneck is inserted at the latent stage to enhance feature reuse and stabilize information propagation. In the decoder, Spike Pixel-Shuffle Upsampling (SPSU) restores spatial resolution, while a Spike-to-Continuous Fusion Block bridges the discrete spiking backbone and the continuous prediction head. An auxiliary prediction branch is used only during training to provide deep supervision.

Figure 1. Overall architecture of MADSNet and detailed diagrams of each sub-module. Top: the full encoder-decoder pipeline from input spike stream to reconstructed frames. Bottom: detailed structures of MS3Stem,DS-SEW Block,Axial Spike Attention (ASA), Light Spike Block, Dense Spike Bottleneck, Spike Pixel-Shuffle Upsampling (SPSU), and the Spike-to-Continuous Fusion Block.

All spiking convolutional layers in MADSNet are built upon the IBRA-LIF neuron, which improves spike representation during training while preserving accumulate-only inference through binary decomposition.

Unless otherwise specified, all spiking convolutional layers use the IBRA-LIF neuron [38] as the activation function. IBRA-LIF provides range-aligned integer spike representations during training and supports binary spike decomposition during inference, preserving the accumulate-only computation property of SNNs. Since the neuron model is not the main architectural contribution of this work, we follow the original setting of [38] and focus the following description on the proposed architecture.

Multi-Scale Spike Stem

The original SSIR starts from a single spiking convolution that captures mainly local spatial patterns. However, the stacked spike tensor encodes temporal cues in its channel dimension while spatial patterns jointly determine the target intensity image. To exploit this property, we design a three-branch multi-scale spike stem.

Specifically, the input spike tensor X\mathbf{X} is processed in parallel by:

fpt=ϕ(BN⁡(Conv⁡1×1(X))),(3)\mathbf{f}_{\mathrm{pt}} = \phi(\operatorname{BN}(\operatorname{Conv}_{1 \times1}(\mathbf{X}))), \tag*{(3)}
floc=ϕ(BN⁡(Conv⁡3×3(X))),(4)\mathbf{f}_{\mathrm{loc}} = \phi(\operatorname{BN}(\operatorname{Conv}_{3 \times3}(\mathbf{X}))), \tag*{(4)}
flk=ϕ(BN⁡(Conv⁡1×1(ϕ(BN⁡(DWConv⁡5×5(X)))))),(5)\mathbf{f}_{\mathrm{lk}} = \phi(\operatorname{BN}(\operatorname{Conv}_{1 \times1}(\phi(\operatorname{BN}(\operatorname{DWConv}_{5 \times5}(\mathbf{X})))))), \tag*{(5)}

where ϕ(⋅)\phi(\cdot) denotes the IBRA-LIF neuron. The three branches are concatenated and fused by a 1×11 \times1 spiking convolution:

s0=ϕ(BN(Conv1×1([fpt,floc,flk]))).(6)s_0 = \phi(\mathrm{BN}(\mathrm{Conv}_{1 \times1}([f_{\mathrm{pt}}, f_{\mathrm{loc}}, f_{\mathrm{lk}}]))). \tag*{(6)}

The 1×11 \times1 branch models inter-channel (temporal) correlations, the 3×33 \times3 branch captures local spatial textures, and the depthwise-separable 5×55 \times5 branch enlarges the receptive field with limited overhead. This design makes the first-stage spike embedding significantly richer than the single-branch baseline of SSIR.

Efficient Spike-Domain Encoding

Depthwise-Separable SEW Block

The original SSIR adopts a standard bottleneck SEW block. To reduce computation while maintaining residual learning capability, we redesign it as a depthwise-separable SEW block. Given an input feature x\mathbf{x} with CC channels, we first compress the channel dimension:

u1=ϕ(BN(PW1(x))).(7)\mathbf{u}_1 = \phi(\mathrm{BN}(\mathrm{PW}_1(\mathbf{x}))). \tag*{(7)}

where PW1\mathrm{PW}_1 is a 1×11 \times1 pointwise convolution that projects to C/4C/4 channels. Spatial interaction is then modelled by a depthwise convolution:

u2=ϕ(BN(DW3×3(u1))).(8)\mathbf{u}_2 = \phi(\mathrm{BN}(\mathrm{DW}_{3 \times3}(\mathbf{u}_1))). \tag*{(8)}

An inner micro-residual path is introduced to enhance feature propagation:

u3=ϕ(BN(PWin(u2)))+u2.(9)\mathbf{u}_3 = \phi(\mathrm{BN}(\mathrm{PW}_{\mathrm{in}}(\mathbf{u}_2))) + \mathbf{u}_2. \tag*{(9)}

Finally, the feature is projected back to CC channels and added to the input:

y=ϕ(BN(PWout(u3)))+x.(10)\mathbf{y} = \phi(\mathrm{BN}(\mathrm{PW}_{\mathrm{out}}(\mathbf{u}_3))) + \mathbf{x}. \tag*{(10)}

Compared with the original bottleneck block, the proposed DS-SEWBlock decouples spatial and channel mixing, introduces an additional internal gradient path, and substantially reduces parameter count.

Axial Spike Attention

To model long-range spatial dependencies efficiently, we propose an axial spike attention module comprising one channel attention branch and two orthogonal axial spatial branches. Instead of densely computing attention over the full 2-D spatial grid, it factorizes spatial modeling into two 1-D axes.

For channel attention, the input feature is squeezed by global average pooling and processed by two fully-connected layers:

mc=σ(FC2(ϕ(BN(FC1(GAP(x))))) ,(11)\mathbf{m}_{c} = \sigma(\mathrm{FC}_2(\phi(\mathrm{BN}(\mathrm{FC}_1(\mathrm{GAP}(\mathbf{x})))))\,, \tag*{(11)}

where σ(⋅)\sigma(\cdot) denotes the sigmoid function.

For spatial attention, the feature is aggregated along the horizontal and vertical axes independently:

mh=σ(ϕ(BN(DWConv3×1(AvgW(x))))),(12)\mathbf{m}_{h} = \sigma(\phi(\mathrm{BN}(\mathrm{DWConv}_{3 \times1}(\mathrm{AvgW}(\mathbf{x}))))), \tag*{(12)}
mw=σ(ϕ(BN(DWConv1×3(AvgH(x))))).(13)\mathbf{m}_{w} = \sigma(\phi(\mathrm{BN}(\mathrm{DWConv}_{1 \times3}(\mathrm{AvgH}(\mathbf{x}))))). \tag*{(13)}

The final attention-modulated feature is

y=x⊙mc⊙mh⊙mw,(14)\mathbf{y} = \mathbf{x} \odot\mathbf{m}_{c} \odot\mathbf{m}_{h} \odot\mathbf{m}_{w}, \tag*{(14)}

where ⊙\odot denotes element-wise multiplication.

This factorised design captures spatial context along two orthogonal axes, generating the spatial attention at O((H+W)⋅C)\mathcal{O}((H + W) \cdot C) cost rather than the O(H⋅W⋅C⋅k2)\mathcal{O}(H \cdot W \cdot C \cdot k^{2}) of producing a dense 2-D attention map, thus substantially reducing overhead, while the two axial branches provide global horizontal and vertical context that complements the local receptive field of the depthwise-separable convolutions.

  1. Light Spike Block: To build an efficient encoder stage, we introduce a light spike block that combines local residual modelling and attention modulation. Two parallel branches are computed:

f1=D2(D1(x)),f2=S3×3(x).(15)\begin{aligned} f_{1} &= \mathcal{D}_{2}(\mathcal{D}_{1}(x)),\\ f_{2} &= \mathcal{S}_{3 \times3}(x). \tag*{(15)} \end{aligned}
f2=S3×3(x)(16)f_{2} = \mathcal{S}_{3 \times3}(x) \tag*{(16)}

where D1,D2\mathcal{D}_{1}, \mathcal{D}_{2} are DS-SEWBlocks and S3×3\mathcal{S}_{3 \times3} is a standard spiking convolution. The branches are aggregated and refined by ASA:

f=A(f1+f2),(17)f = \mathcal{A}(f_{1} + f_{2}), \tag*{(17)}

and the block output is formed as a spike-domain residual:

y=ϕ(x+BN⁡(f)).(18)y = \phi(x + \operatorname{BN}(f)). \tag*{(18)}

Dense Spike Bottleneck

To improve feature reuse in the latent space, we introduce a dense spike bottleneck composed of three DS-SEWBlocks {B1,B2,B3}\{B_{1}, B_{2}, B_{3}\}. Rather than stacking blocks sequentially, we design a directed dense aggregation built on additive cross-block skips:

x1=B1(x0),x2=B2(x0+x1),x3=B3(x1+x2).(19)\begin{aligned} x_{1} &= B_{1}(x_{0}),\\ x_{2} &= B_{2}(x_{0} + x_{1}),\\ x_{3} &= B_{3}(x_{1} + x_{2}). \tag*{(19)} \end{aligned}
x2=B2(x0+x1)(20)x_{2} = B_{2}(x_{0} + x_{1}) \tag*{(20)}
x3=B3(x1+x2)(21)x_{3} = B_{3}(x_{1} + x_{2}) \tag*{(21)}

The bottleneck output is obtained by a residual projection:

y=x0+ϕ(BN⁡(Conv⁡1×1(x3))).(22)y = x_{0} + \phi(\operatorname{BN}(\operatorname{Conv}_{1 \times1}(x_{3}))). \tag*{(22)}

In contrast to the concatenation-based connectivity of DenseNet, these skips are additive and thus introduce no channel growth, keeping the bottleneck lightweight. The design lets B2B_{2} re-access the initial feature x0x_{0} to prevent information forgetting, while B3B_{3} aggregates the newly produced representations x1x_{1} and x2x_{2}; since x0x_{0} is already reintroduced through the final residual projection, re-feeding it into B3B_{3} would be redundant. The resulting cross-block connections enrich gradient flow and improve training stability for deep spiking architectures.

Spike-Domain Decoder and Continuous Reconstruction

  1. Spike Pixel-Shuffle Upsampling: The baseline SSIR uses transposed convolution for spatial recovery. We instead adopt a learnable sub-pixel upsampling strategy to reconstruct fine spatial details more effectively and avoid checkerboard artefacts. Given the bottleneck feature bb, we first expand its channel dimension by a factor of r2r^{2} (where r=2r = 2 is the upsampling scale):

u′=ϕ(BN⁡(Conv⁡3×3(b))),u′∈RCout⋅r2×Hr×Wr.(23)u' = \phi(\operatorname{BN}(\operatorname{Conv}_{3 \times3}(b))), \qquad u' \in\mathbb{R}^{C_{\mathrm{out}} \cdot r^{2} \times\frac{H}{r} \times\frac{W}{r}}. \tag*{(23)}

The feature is then rearranged by pixel shuffle:

u=PixelShuffle⁡(u′),u∈RCout×H×W.(24)u = \operatorname{PixelShuffle}(u'), \qquad u \in\mathbb{R}^{C_{\mathrm{out}} \times H \times W}. \tag*{(24)}

A lightweight 1×11 \times1 refinement layer is further applied:

u←ϕ(BN⁡(Conv⁡1×1(u))).(25)u \leftarrow\phi(\operatorname{BN}(\operatorname{Conv}_{1 \times1}(u))). \tag*{(25)}

All upsampling coefficients are learned end-to-end, equivalent to a learnable fractional-stride transposed convolution but without checkerboard artefacts.

  1. Spike-to-Continuous Fusion Block: Although the backbone operates in the spiking domain, the final intensity prediction benefits from a smooth transition from discrete spike activations to continuous values. We introduce a fusion block with channel and spatial gating for this purpose.

For an input feature x∈RB×C×H×Wx \in\mathbb{R}^{B \times C \times H \times W}, the channel gate is

gt=σ(FC⁡(GAP⁡(x)))∈RB×C,(26)g_{t} = \sigma(\operatorname{FC}(\operatorname{GAP}(x))) \in\mathbb{R}^{B \times C}, \tag*{(26)}

and the channel-weighted feature is

yt=x⊙gt,(27)y_{t} = x \odot g_{t}, \tag*{(27)}

where gtg_{t} is broadcast to the spatial dimensions. The spatial gate uses a large-kernel convolution:

gs=σ(Conv⁡7×7(x)).(28)g_{s} = \sigma(\operatorname{Conv}_{7 \times7}(x)). \tag*{(28)}

The spatial branch is then modulated by the complementary factor (1−yt)(1 - y_{t}), which down-weights the content already emphasized by the channel-gated path so that the two branches attend to complementary information:

ys=x⊙gs⊙(1−yt).(29)y_{s} = x \odot g_{s} \odot(1 - y_{t}). \tag*{(29)}

The final fused feature sums the two complementary paths:

z=yt+ys.(30)z = y_{t} + y_{s}. \tag*{(30)}

Through this complementary gating, the channel-weighted path yty_{t} supplies channel-wise global context, while the spatial path ysy_{s}, suppressed where yty_{t} is already strong, contributes spatially-varying structure that the channel gate leaves under-emphasized. The two paths are thus complementary rather than redundant, providing a smooth bridge from the discrete spiking backbone to the continuous prediction head.

  1. Prediction Head and Deep Supervision: After upsampling, the decoder feature uu is concatenated with the refined skip connection from the stem via a 1×11 \times1 spiking convolution:

f=ϕ(BN⁡(Conv⁡1×1([u,R(s0)]))),(31)f = \phi(\operatorname{BN}(\operatorname{Conv}_{1 \times1}([u, \mathcal{R}(s_{0})]))), \tag*{(31)}

where R(⋅)\mathcal{R}(\cdot) denotes the skip-connection refinement layer. The fused feature is fed into the fusion block to obtain zz, which is then decoded by the lightweight prediction head:

I^=σ(Conv⁡1×1(ϕ(BN⁡(Conv⁡3×3(z))))).(32)\hat{I} = \sigma(\operatorname{Conv}_{1 \times1}(\phi(\operatorname{BN}(\operatorname{Conv}_{3 \times3}(z))))). \tag*{(32)}

During training, an auxiliary prediction head provides deep supervision:

I^aux=σ(Conv⁡1×1(z)).(33)\hat{I}_{\mathrm{aux}} = \sigma(\operatorname{Conv}_{1 \times1}(z)). \tag*{(33)}

The auxiliary output is included directly in the loss without adding inference-time cost.

Training Objective

For fair comparison with the original SSIR, we adopt the same training objective. The loss combines an ℓ1\ell_1 reconstruction term and a perceptual term for both the main and auxiliary predictions.

For the main prediction:

Lrecmain=∥I^−Igt∥1,(34)\mathcal{L}_{\mathrm{rec}}^{\mathrm{main}}=\left\lVert\hat{I}-I_{\mathrm{gt}}\right\rVert_1, \tag*{(34)}
Lpermain=∥ϕvgg(I^)−ϕvgg(Igt)∥22,(35)\mathcal{L}_{\mathrm{per}}^{\mathrm{main}}=\left\lVert\phi_{\mathrm{vgg}}(\hat{I})-\phi_{\mathrm{vgg}}(I_{\mathrm{gt}})\right\rVert_2^2, \tag*{(35)}

and for the auxiliary prediction:

Lrecaux=∥I^aux−Igt∥1,(36)\mathcal{L}_{\mathrm{rec}}^{\mathrm{aux}}=\left\lVert\hat{I}_{\mathrm{aux}}-I_{\mathrm{gt}}\right\rVert_1, \tag*{(36)}
Lperaux=∥ϕvgg(I^aux)−ϕvgg(Igt)∥22,(37)\mathcal{L}_{\mathrm{per}}^{\mathrm{aux}}=\left\lVert\phi_{\mathrm{vgg}}(\hat{I}_{\mathrm{aux}})-\phi_{\mathrm{vgg}}(I_{\mathrm{gt}})\right\rVert_2^2, \tag*{(37)}

where ϕvgg(⋅)\phi_{\mathrm{vgg}}(\cdot) denotes feature extraction by a pre-trained VGG network and IgtI_{\mathrm{gt}} is the ground-truth intensity frame.

The total training loss is

L=Lrecmain+Lrecaux+λ(Lpermain+Lperaux),(38)\mathcal{L}=\mathcal{L}_{\mathrm{rec}}^{\mathrm{main}}+\mathcal{L}_{\mathrm{rec}}^{\mathrm{aux}}+\lambda\left(\mathcal{L}_{\mathrm{per}}^{\mathrm{main}}+\mathcal{L}_{\mathrm{per}}^{\mathrm{aux}}\right), \tag*{(38)}

where λ=0.2\lambda=0.2 is the perceptual loss balancing coefficient.

Experiments

Experimental Settings

a) Datasets. We evaluate MADSNet on both gray-scale and color spike reconstruction tasks. For the main gray-scale reconstruction task, we follow SSIR [6] and conduct experiments on the simulated SREDS dataset and real-captured spike streams. SREDS [6] is generated from REDS [39] and contains 524,160 image ground truths and around 1×1071 \times10^7 spike frames. Following SSIR, we use the first 22 ground-truth frames in the validation split for evaluation.

To verify the transferability of the proposed design, we further conduct a color reconstruction experiment on RBSS [19] with α=1.0\alpha=1.0 and real-world Bayer-pattern spike streams from BSS [40]. This setting allows us to examine whether a spike-based architecture originally designed for gray-scale reconstruction can be extended to the more challenging color spike reconstruction scenario.

b) Implementation Details. Unless otherwise specified, we follow the training protocol of SSIR. Each training sample contains 160 spike frames and 8 corresponding ground-truth images. Following SSIR, each of the 8 ground-truth frames is reconstructed from its corresponding local window of spike frames within the 160-frame sample. Data augmentation includes random horizontal/vertical flipping and 90∘90^\circ/180∘180^\circ rotations. We optimize the network using Adam with an initial learning rate of 4×10−44 \times10^{-4}. The batch size is set to 8.

For the gray-scale task, we crop each scene to 96×9696 \times96 spatially without overlap, following SSIR. The learning rate decays every 5 epochs from the 20th to the 70th epoch, and training runs for 80 epochs in total.

For the color transfer experiment, we crop each scene to 128×128128 \times128 following SSL-CSC [19], and extend the training schedule to 200 epochs while keeping all other hyperparameters unchanged.

c) Compared Methods. For gray-scale reconstruction, we compare MADSNet with representative conventional and learning-based methods, including TFP [3], TFI [3], TF-STP [22], MAHTF [4], ETNet [32], HyperE2VID [33], Spk2ImgNet [5], and SSIR [6]. ETNet and HyperE2VID are event-camera reconstruction methods included as cross-sensor baselines to situate MADSNet within the broader neuromorphic image reconstruction landscape. For color reconstruction, we compare against TFP [3], TFI [3], TFSTP [22], 3DRI [18], SJRE [9], SSL-CSC [19], ETNet [32], HyperE2VID [33], Spk2ImgNet [5], and SSIR [6] as a direct SNN transfer baseline.

d) Evaluation Metrics. For gray-scale reconstruction on SREDS, we report PSNR, SSIM, and LPIPS [41], and additionally report NIQE [42] and BRISQUE [43]. For color reconstruction on RBSS, we use PSNR, SSIM, and LPIPS. Lower values of LPIPS, NIQE, and BRISQUE indicate better perceptual quality, while higher values of PSNR and SSIM are preferred.

Comparison on Gray-Scale Spike Reconstruction

a) Quantitative Results on SREDS: Table 1 compares MADSNet with representative gray-scale spike reconstruction methods on SREDS. Conventional temporal filtering methods, including TFP, TFI, and TFSTP, achieve limited reconstruction fidelity, showing that simple temporal aggregation is insufficient for recovering high-quality intensity images from sparse spike streams. MAHTF improves the reconstruction quality by exploiting long-range temporal motion information, but still lags behind learning-based methods in reference metrics.

ParadigmMethodPSNR↑SSIM↑LPIPS↓NIQE↓BRISQUE↓
Non-SNNTFP [3]28.0270.6870.2697.44540.656
Non-SNNTFI [3]24.2580.6910.2635.07440.069
Non-SNNTFSTP [22]20.8450.6400.2715.16336.948
Non-SNNMAHTF [4]31.5660.9040.0892.63115.997
ANNETNet [32]31.0880.8990.0953.47714.357
ANNHyperE2VID [33]35.4560.9320.0693.33812.636
ANNSpk2ImgNet [5]35.2930.9370.0583.61020.491
SNNSSIR [6]32.9300.9220.0503.11119.694
SNNMADSNet(Ours)37.2320.9560.0282.92717.550

Table 1. Comparison on the gray-scale spike reconstruction benchmark. Results of existing methods are taken from SSIR.

Among ANN-based methods, the event-based models ET-Net and HyperE2VID adapted for spike streams demonstrate strong capabilities, particularly HyperE2VID achieving 35.456 dB PSNR. Spk2ImgNet similarly achieves a competitive 35.293 dB PSNR.

Compared with the SNN baseline SSIR, MADSNet improves PSNR from 32.930 dB to 37.232 dB and SSIM from 0.922 to 0.956, while reducing LPIPS from 0.050 to 0.028. This corresponds to a massive +4.30 dB PSNR gain within the SNN paradigm. Moreover, MADSNet surpasses both the dedicated spike reconstruction ANN (Spk2ImgNet by +1.94 dB) and the modern event-adapted ANN (HyperE2VID by +1.78 dB). These gains demonstrate that a carefully designed spiking architecture can achieve substantially stronger reconstruction fidelity than even advanced conventional ANN baselines.

For no-reference perceptual metrics, MADSNet obtains competitive NIQE and BRISQUE scores. Although it does not achieve the best value on every no-reference metric, the overall results show that the proposed architecture substantially improves reference fidelity while maintaining excellent perceptual quality.

b) Qualitative Results on Real-Captured Gray-Scale Data: Figure 2 presents visual comparisons on real-captured gray-scale spike streams across three scenes, covering a bicycle, a car, and a decorative doll. The event-camera methods ET-Net and HyperE2VID produce visually plausible reconstructions but tend to introduce blurring or ghosting artifacts in regions with complex texture, reflecting the inherent limitation of event cameras in recovering absolute luminance detail. Spk2ImgNet recovers sharper edges than the event-based methods but still exhibits residual blur on high-frequency elements. SSIR reconstructs recognizable structure but loses fine detail, particularly on thin elements such as bicycle spokes. MADSNet produces the sharpest and most faithful reconstructions across all scenes: bicycle spokes are more precisely delineated, high-frequency fan textures are better preserved with higher local contrast, and fine ornamental details in the doll scene exhibit fewer boundary artifacts. These observations are consistent with the quantitative improvements reported in Table I.

Qualitative comparison on real-captured gray-scale spike streams

Figure 2. Qualitative comparison on real-captured gray-scale spike streams (Binary Spikes, ET-Net, HyperE2VID, Spk2ImgNet, SSIR, Ours). Zoom in for more details. Gamma transformation with parameter 1.5 is used for visualization.

Transfer to Color Spike Reconstruction

Although MADSNet is developed for gray-scale spike reconstruction, its design is not tied to a specific output modality. The multi-scale spiking encoder, axial-attentive feature extraction, and spike-to-continuous decoding are expected to remain useful when the input changes from gray-scale spike streams to Bayer-pattern color spike streams. Therefore, we further investigate whether MADSNet can be effectively transferred to color spike reconstruction, and whether it can serve as a strong SNN baseline for this more challenging task.

Quantitative Results on RBSS.

Table II reports transfer results on the RBSS color reconstruction benchmark. Filtering-based methods TFP, TFI, and TFSTP remain below 24 dB PSNR, confirming their limited capacity to handle the spatial-chromatic coupling of Bayer-pattern spike streams. Among event-camera methods, ETNet achieves only 13.10 dB due to the fundamental modality mismatch between event streams and Bayer spike streams, while HyperE2VID reaches 27.34 dB, benefiting from its adaptive per-pixel filtering design. Spk2ImgNet achieves 29.13 dB, demonstrating the advantage of a spike-specific ANN architecture. SSL-CSC attains the highest PSNR (29.95 dB) and SSIM (0.8417) among all compared methods, benefiting from its motion-guided self-supervised design tailored for color spike reconstruction.

Directly transferring SSIR to color spike reconstruction yields 24.31 dB PSNR and 0.6514 SSIM, showing that a spike-based SNN can generalize to the color domain to a reasonable degree, though with a substantial gap in fine chromatic detail compared with dedicated methods. In contrast, MADSNet achieves 29.36 dB PSNR, 0.8434 SSIM, and 0.1464 LPIPS,

ParadigmMethodPSNR↑\uparrowSSIM↑\uparrowLPIPS↓\downarrow
Non-SNNTFP [3]23.780.62070.3986
Non-SNNTFI [3]23.970.62940.3731
Non-SNNTFSTP [22]23.320.59780.3759
Non-SNN3DRI [18]26.950.80290.1946
ANNSJRE [9]24.840.54990.4050
ANNSSL-CSC [19]29.950.84170.1829
ANNETNet [32]13.100.57010.3695
ANNHyperE2VID [33]27.340.77710.2753
ANNSpk2ImgNet [5]29.130.82190.2324
SNNSSIR [6]24.310.65140.2811
SNNMADSNet (Ours)29.360.84340.1464

caption: Transferability test on RBSS (α=1.0\alpha= 1.0).

:::

improving over SSIR by +5.05 dB in PSNR and outperforming all compared methods in both SSIM and LPIPS. Notably, MADSNet also surpasses the ANN-based Spk2ImgNet in all three metrics and closely approaches SSL-CSC in PSNR while outperforming it in SSIM and LPIPS, despite not being specifically designed for color reconstruction. These results demonstrate the strong transferability of the proposed architecture and confirm that MADSNet serves as a competitive SNN baseline for color spike reconstruction.

b) Qualitative Results on Real-Captured BSS.: Figure 3 presents visual comparisons on real-world BSS color spike streams across three scenes. ETNet produces severely overexposed outputs, reflecting the severe modality gap between event cameras and Bayer spike cameras. HyperE2VID recovers plausible color but introduces noticeable chromatic noise and loses fine structural boundaries. Spk2ImgNet reconstructs coherent color images but still exhibits some color bleeding and texture blurring. SSIR recovers a coarse color approximation with recognizable scene structure, though with noticeably reduced chromatic fidelity and visible mosaic-pattern artifacts attributable to its limited Bayer-layout modeling. MADSNet reconstructs well-structured color images with the most faithful chromatic fidelity and sharpest object boundaries across all three scenes, consistent with its leading quantitative results in Table II.

Qualitative comparison on real-captured color spike streams across Binary Spikes, ET-Net, HyperE2VID, Spk2ImgNet, SSIR, and Ours

Figure 3. Qualitative comparison on real-captured color spike streams (BSS) (Binary Spikes, ET-Net, HyperE2VID, Spk2ImgNet, SSIR, Ours). Zoom in for more details. Gamma transformation with parameter 2.2 is used for visualization.

c) Analysis of Color Transfer.: The quantitative and qualitative results together reveal a clear transferability hierarchy. SSIR transfers to color reconstruction with moderate success (24.31 dB), demonstrating that SNN architectures are not inherently incompatible with the color domain. However, its single-branch 3×33 \times3 spiking stem treats all stacked spike channels uniformly and provides limited receptive-field diversity, making it difficult to fully capture chromatic correlations across the Bayer mosaic. MADSNet transfers substantially better (+5.05 dB over SSIR) because its MS3Stem jointly captures inter-channel and multi-scale spatial patterns, while ASA contributes long-range horizontal and vertical context through its axial modeling. The spike-to-continuous fusion block further adapts discrete spike-domain features to the continuous RGB output space. These components jointly enable MADSNet to recover structured color images without task-specific architectural redesign, establishing it as a strong SNN

baseline for color spike reconstruction.

Ablation Studies

a) Component-Wise Ablation on SREDS: To validate the individual contribution of each proposed component, we conduct a systematic ablation study by removing one module at a time from the full MADSNet and evaluating the resulting model on SREDS. Table III summarizes the results, and we detail the findings for each component below.

VariantPSNR↑SSIM↑LPIPS↓
MADSNet (Full)37.2320.9560.028
w/o MS3Stem (single 3×33 \times3 SConv)35.4060.9480.034
w/o ASA36.6010.9550.030
w/o DS-SEWBlock (standard SEWBlock)36.9900.9540.032
w/o Dense Bottleneck (sequential)36.5660.9540.032
w/o SPSU (transposed conv)36.5810.9540.031
w/o Fusion Block36.0850.9530.031
w/o Deep Supervision37.0890.9550.029
PLIF instead of IBRA-LIF30.2300.8170.124

Table III. Component-wise ablation on SREDS.

MS3Stem. Replacing the proposed three-branch multi-scale stem with a single 3×33 \times3 spiking convolution (as in SSIR) leads to the most significant performance drop among all architectural components, reducing PSNR from 37.232 dB to 35.406 dB (−1.826-1.826 dB) and LPIPS from 0.028 to 0.034. This large degradation confirms that the rich multi-scale feature embedding provided by the parallel 1×11 \times1, 3×33 \times3, and depthwise-separable 5×55 \times5 branches is essential for encoding both temporal inter-channel correlations and multi-scale spatial patterns from the input spike tensor. The single-branch baseline is insufficient for capturing the full spatial and temporal complexity of the stacked spike representation, and the MS3Stem provides an indispensable foundation for the rest of the network.

Axial Spike Attention (ASA). Removing ASA from all LightSpikeBlocks results in a PSNR drop from 37.232 dB to 36.601 dB (−0.631-0.631 dB), with LPIPS increasing from 0.028 to 0.030. Although the improvement from ASA is more modest than that of MS3Stem, it is consistent and meaningful. The factorized axial attention provides the encoder with the ability to model long-range spatial dependencies along the horizontal and vertical axes at O((H+W)⋅C)\mathcal{O}((H + W) \cdot C) complexity, which is important for recovering coherent global structures (e.g., object contours and smooth background regions) in the reconstructed intensity image.

DS-SEWBlock. Replacing the proposed depthwise-separable SEW block with the standard bottleneck SEW block from SSIR reduces PSNR to 36.990 dB (−0.242-0.242 dB) and LPIPS to 0.032. This result confirms that the decoupled spatial and channel mixing strategy of DS-SEWBlock, together with its inner micro-residual path, provides benefits beyond simple parameter reduction. The additional gradient propagation path through the inner micro-residual enhances feature learning and mitigates potential gradient vanishing in the depthwise-separable structure.

Dense Spike Bottleneck. Replacing the dense-connected bottleneck structure with three sequentially stacked DS-SEWBlocks (removing all inter-block shortcut paths) yields 36.566 dB PSNR (−0.666-0.666 dB) and 0.032 LPIPS. The dense connectivity allows B2\mathcal{B}_2 to re-access the initial representation x0x_0 and prevents information forgetting, while B3\mathcal{B}_3 aggregates the adjacent representations x1x_1 and x2x_2. The additional gradient paths introduced by these connections also improve training stability in the deep spiking architecture. The performance gap confirms that feature reuse in the latent space is a critical factor for high-fidelity spike camera reconstruction.

Spike Pixel-Shuffle Upsampling (SPSU). Replacing SPSU with a standard transposed convolution (as in SSIR) reduces PSNR to 36.581 dB (−0.651-0.651 dB) and LPIPS to 0.031. The learnable sub-pixel upsampling avoids the checkerboard artifacts inherent to transposed convolution and provides a more accurate reconstruction of fine spatial details, which is particularly important for recovering high-frequency textures from spike streams.

Spike-to-Continuous Fusion Block. Removing the fusion block results in a PSNR of 36.085 dB (−1.147-1.147 dB) and LPIPS of 0.031, making it the second most impactful component after MS3Stem. This finding underscores the importance of the smooth transition from discrete spiking domain features to continuous intensity values. The joint channel gating and spatially-varying spatial gating of the fusion block enable the network to adaptively weight and recombine features before the prediction head, providing a critical bridge that prevents information loss at the spiking-to-continuous interface.

Deep Supervision. Removing the auxiliary supervision branch yields a modest but consistent drop (37.232→37.08937.232 \to37.089 dB PSNR, −0.143-0.143 dB). Although deep supervision primarily accelerates convergence during training, it also provides additional gradient paths that serve as implicit regularization, leading to marginally better generalization at convergence. Since the auxiliary head is discarded at inference time, this improvement is obtained at zero additional inference cost.

IBRA-LIF vs. PLIF Neuron. Replacing IBRA-LIF with a standard PLIF neuron causes a catastrophic performance collapse (37.232→30.23037.232 \to30.230 dB PSNR, −7.002-7.002 dB; SSIM: 0.956 →\to 0.817; LPIPS: 0.028 →\to 0.124). The nature and significance of this result are discussed in detail in the following paragraph.

Taken together, these ablation results confirm that every proposed component makes a meaningful and statistically consistent contribution to the final reconstruction quality, and that the full MADSNet design represents an effective and well-motivated integration of all proposed modules.

b) Neuron-Architecture Interaction: Although IBRA-LIF is not our architectural contribution, Table III shows that replacing it with PLIF severely degrades MADSNet. We therefore further analyze whether the gain comes from the neuron alone or from its interaction with the proposed architecture.

Table III shows that replacing IBRA-LIF with a standard Parametric LIF (PLIF) neuron causes a dramatic performance drop (37.232 → 30.230 PSNR), which is far larger than the drops caused by removing any individual architectural component. A naive interpretation might attribute this to IBRA-LIF being the dominant factor; however, this interpretation is inconsistent with a critical observation: replacing PLIF with IBRA-LIF in the original SSIR yields only a marginal improvement (32.930 → 33.168 PSNR, +0.24 dB), as shown in Table IV. Moreover, MADSNet with PLIF actually underperforms SSIR with PLIF (30.230 vs. 32.930), meaning the proposed architectural improvements are counterproductive when paired with a neuron that cannot sustain healthy information flow through a deeper network.

ArchitectureNeuronPSNR↑SSIM↑LPIPS↓
SSIR (baseline)PLIF32.9300.9220.050
IBRA-LIF33.1680.9370.030
MADSNet (Ours)PLIF30.2300.8170.124
IBRA-LIF37.2320.9560.028

Table IV. Cross-architecture neuron comparison on SREDS. IBRA-LIF brings only a marginal gain to SSIR, but is essential for preventing performance collapse in the deeper MADSNet architecture.

c) Activation Analysis: Table V further reports the layer-wise non-zero activation rate. MADSNet-PLIF suffers from severe activation collapse in deep stages, with nearly zero activations in the Dense Bottleneck and only 0.74% in SPSU. In contrast, MADSNet-IBRA maintains non-trivial activations across all stages. This explains why the proposed deeper architecture cannot be effectively trained with PLIF, whereas IBRA-LIF provides sufficient representational range to sustain information propagation.

Layer Group NeuronSSIR PLIFSSIR IBRAMADSNet PLIFMADSNet IBRA
SSIR Stem (static_conv)39.6623.53––
Encoder (enc1)80.9436.30––
Bottleneck (eres1)65.0529.42––
Decoder (dec3)83.7913.19––
MADSNet MS3Stem––6.4235.21
LightSpikeBlock-Stem––0.1922.31
DownSampling––0.0215.43
Dense Bottleneck––0.008.68
SPSU (up)––0.749.40
Fusion Block––12.8932.17
Pred Head––8.8623.29
Network Avg.67.3625.614.1620.93

Table V. Layer-wise non-zero activation rate (%) on SREDS validation set. MADSNet-PLIF exhibits severe activation collapse in deep layers, while IBRA-LIF maintains healthy activations throughout.

These results indicate that IBRA-LIF is not the sole source of MADSNet’s performance gain. In SSIR, replacing PLIF with IBRA-LIF improves PSNR by only 0.24 dB. With the same IBRA-LIF neuron, MADSNet still outperforms SSIR by 4.06 dB. Therefore, the main gain comes from the proposed architecture, while IBRA-LIF serves as a necessary activation substrate that enables the deeper spiking architecture to function properly.

Efficiency Analysis

To evaluate the practical efficiency of MADSNet, we compare it with Spk2ImgNet (ANN) and SSIR (SNN) in terms of inference latency, GPU memory footprint, and estimated energy consumption, measured on an NVIDIA RTX 4090 GPU at 256×400256 \times400 input resolution. Energy consumption is estimated following the AC/MAC analysis framework of SSIR [6] based on per-layer spike firing rates. The results are summarized in Table VI.

MethodLatency ↓\downarrow (ms)Memory ↓\downarrow (GB)Energy ↓\downarrow (mJ)Norm. Energy↓\downarrow
Spk2ImgNet305.41.6754690.01
SSIR3.30.3104.61/1019
MADSNet (Ours)20.40.489118.41/40

Table VI. EFFICIENCY ANALYSIS. SPK2IMGNET AND SSIR RESULTS ARE TAKEN FROM SSIR.

The efficiency results highlight two important observations. First, MADSNet incurs higher latency (20.4 ms vs. 3.3 ms), memory (0.489 GB vs. 0.310 GB), and energy (118.4 mJ vs. 4.6 mJ) than the lightweight SSIR baseline. This overhead is the expected consequence of MADSNet’s deeper architecture, which incorporates multi-scale branching, dense bottleneck connections, and axial attention modules—all of which contribute to the substantial performance gains reported in Table I.

The efficiency results show a clear accuracy-efficiency trade-off. Compared with SSIR, MADSNet has higher latency, memory usage, and estimated energy consumption, which is expected due to its multi-scale stem, dense bottleneck, and axial-attentive encoder. In return, MADSNet improves PSNR by +4.30 dB on SREDS and surpasses the ANN-based Spk2ImgNet in reconstruction fidelity.

More importantly, MADSNet still preserves the efficiency advantage of SNNs over ANN-based reconstruction. Compared with Spk2ImgNet, MADSNet is approximately 15×15\times faster, uses 3.4×3.4\times less GPU memory, and consumes only about 1/401/40 of the estimated energy. These results indicate that MADSNet achieves a better balance between reconstruction quality and computational efficiency than existing ANN and SNN baselines.

These advantages stem from the SNN computation paradigm, in which the binary-range integer spike representation of IBRA-LIF enables the replacement of multiply-accumulate (MAC) operations with efficient accumulate-only (AC) operations during inference, substantially reducing the effective arithmetic cost per layer. Furthermore, the depthwise-separable structure of DS-SEWBlock contributes to a reduced parameter count and lower activation volume, partially mitigating the additional overhead introduced by the deeper architecture compared to SSIR.

Conclusion

In this paper, we proposed MADSNet, a multi-scale axial-attentive dense spiking network for spike camera image reconstruction. MADSNet is designed to better exploit the complementary advantages of spike cameras and SNNs: spike cameras provide high-temporal-resolution binary visual streams, while SNNs enable efficient spike-domain computation. To improve reconstruction fidelity beyond existing shallow SNN designs, MADSNet enriches the spike-domain representation along the full encoder–decoder pipeline, from multi-scale early embedding, through axial-attentive encoding and dense latent aggregation, to spike-to-continuous decoding.

Extensive experiments on gray-scale and color spike reconstruction demonstrate the effectiveness and generality of the proposed architecture. On SREDS, MADSNet significantly outperforms the SNN baseline SSIR and surpasses the ANN-based Spk2ImgNet in reconstruction accuracy, while maintaining much lower latency and estimated energy consumption than Spk2ImgNet. Color reconstruction experiments show that MADSNet transfers better than the direct SNN baseline. These results suggest that carefully designed SNN architectures can provide a favorable accuracy–efficiency trade-off for spike camera reconstruction. Nevertheless, MADSNet’s deeper architecture incurs higher latency and energy than lightweight shallow SNNs such as SSIR; reducing this overhead, e.g., through neuromorphic-hardware deployment or further architectural slimming, is a promising direction for future work.

Portrait of Sirui Li

Figure 4.

Sirui Li received the B.S. degree from the School of Mathematical Sciences, Dalian University of Technology, Dalian, China, in 2023, and the M.S. degree from the School of Data Science, The Chinese University of Hong Kong, Shenzhen, China, in 2025. He is currently pursuing the Ph.D. degree with the School of Data Science, The Chinese University of Hong Kong, Shenzhen, China. His research interests include spiking neural networks and their applications in image, EEG, and speech signal processing.

Portrait of Rui Zhao

Figure 5.

Rui Zhao (Member, IEEE) received the B.E. degree in Communication Engineering from Tianjin University, Tianjin, China, in 2020, and the Ph.D. degree in Computer Science from Peking University, Beijing, China, in 2025. He is currently a postdoctoral research fellow with Nanyang Technological University, Singapore. His research interests include computational photography, computer vision, and image processing, particularly for topics related to motion estimation and neuromorphic cameras.

Portrait of Siqi Cai

Figure 6.

Siqi Cai (Senior Member, IEEE) received the Ph.D. degree in intelligent engineering from South China University of Technology, Guangzhou, China, in 2020. She is currently a Professor with Harbin Institute of Technology, Shenzhen, China. From 2020 to 2024, she was a Research Fellow with the Department of Electrical and Computer Engineering, National University of Singapore, Singapore. Her research interests include brain–computer interfaces and multimodal biosignal processing. She served as Workshop Chair of ICASSP 2022 and Entrepreneurship Forum Chair of ICASSP 2024 and ICASSP 2026, and is an Associate Editor of the International Journal of Social Robotics. She has been a Council Member of the Chinese and Oriental Languages Information Processing Society since 2022.

Portrait of Haizhou Li

Figure 7.

Haizhou Li (Fellow, IEEE) received the B.Sc., M.Sc., and Ph.D. degrees in electrical and electronic engineering from South China University of Technology, Guangzhou, China in 1984, 1987, and 1990 respectively. He is currently a Presidential Chair Professor with the School of Data Science, The Chinese University of Hong Kong, Shenzhen, China. He is also an Adjunct Professor with the Department of Electrical and Computer Engineering, National University of Singapore (NUS). His research interests include automatic speech recognition, speaker and language recognition, natural language processing. Dr. Li was the Editor-in-Chief of IEEE/ACM Transactions on Audio, Speech and Language Processing from 2015 to 2018, a Member of the Editorial Board of Computer Speech and Language Processing Technical Committee during 2013–2015, the President of the International Speech Communication Association during 2015–2017, the President of Asia Pacific Signal and Information Processing Association during 2015–2016, the President of Asian Federation of Natural Language Processing during 2017–2018, and Vice President of IEEE Signal Processing Society during 2024–2026. He was the General Chair of ACL 2012, INTERSPEECH 2014, ASRU 2019 and ICASSP 2022. Dr. Li is a Fellow of the ISCA, and a Fellow of the Academy of Engineering Singapore.

Portrait of Tiejun Huang

Figure 8.

Tiejun Huang (Senior Member, IEEE) is currently a Professor with the School of Computer Science, Peking University, Beijing, China, and the Dean of Beijing Academy of Artificial Intelligence (BAAI). He is a recipient of the National Science Fund for Distinguished Young Scholars, Changjiang Distinguished Professorship, and National Talent Program for Technological Innovation Leadership. He received the Ph.D. degree in pattern recognition and intelligent systems from Huazhong (Central China) University of Science and Technology, Wuhan, China, in 1998, and the master’s and bachelor’s degree in computer science from the Wuhan University of Technology, Wuhan, in 1995 and 1992, respectively. His research area includes video coding, image understanding, digital right management, and digital library. He has authored or co-authored over 100 peer-reviewed papers and three books.

Yajing Zheng (Member, IEEE) received the B.S. degree from Sichuan University, Chengdu, China, in 2017, and the Ph.D. degree from the School of Computer Science, Peking University, Beijing, China, in 2022. She is currently a Boya Postdoctoral Fellow with the School of Computer Science, Peking University, and an Honorary Research Fellow with UCL Imaging Neuroscience, University College London, London, U.K. Her research focuses on neuromorphic vision and brain-inspired computing, with particular interests in spike cameras, event-based vision, spiking neural networks, computational neuroscience, and low-latency intelligent perception systems. She serves as the Deputy Secretary-General of the Brain-Inspired Vision Committee of the China Society of Image and Graphics.

References

  1. [1]T. Huang, Y. Zheng, Z. Yu, R. Chen, Y. Li, R. Xiong, L. Ma, J. Zhao, S. Dong, L. Zhu et al., “1000× faster camera and machine vision with ordinary devices,” Engineering, vol. 25, pp. 110–119, 2023.
  2. [2]Y. Zheng, J. Zhang, R. Zhao, J. Ding, S. Chen, W. Wu, R. Xiong, Z. Yu, and T. Huang, “SpikeCV: open a continuous computer vision era,” Science China: Information Sciences, vol. 69, no. 3, p. 132106, 2026.DOI
  3. [3]L. Zhu, S. Dong, T. Huang, and Y. Tian, “A retina-inspired sampling method for visual texture reconstruction,” in IEEE International Conference on Multimedia and Expo, 2019.arxiv.org/abs/1907.08769
  4. [4]J. Zhao, R. Xiong, J. Xie, B. Shi, Z. Yu, W. Gao, and T. Huang, “Reconstructing clear image for high-speed motion scene with a retina-inspired spike camera,” IEEE Transactions on Computational Imaging, vol. 8, pp. 12–27, 2021.DOI
  5. [5]J. Zhao, R. Xiong, H. Liu, J. Zhang, and T. Huang, “Spk2imgnet: Learning to reconstruct dynamic scene from continuous spike stream,” in IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2021, pp. 11 996–12 005.DOI
  6. [6]R. Zhao, R. Xiong, J. Zhang, Z. Yu, S. Zhu, L. Ma, and T. Huang, “Spike camera image reconstruction using deep spiking neural networks,” IEEE Transactions on Circuits and Systems for Video Technology, vol. 34, no. 6, pp. 5207–5212, 2023.DOI
  7. [7]S. Chen, C. Duan, Z. Yu, R. Xiong, and T. Huang, “Self-supervised mutual learning for dynamic scene reconstruction of spiking camera,” in International Joint Conference on Artificial Intelligence, 2022, pp. 2859–2866.DOI
  8. [8]R. Zhao, R. Xiong, J. Zhao, J. Zhang, X. Fan, Z. Yu, and T. Huang, “Boosting spike camera image reconstruction from a perspective of dealing with spike fluctuations,” in IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2024, pp. 24 955–24 965.DOI
  9. [9]S. Chen, Z. Yu, and T. Huang, “Self-supervised joint dynamic scene reconstruction and optical flow estimation for spiking camera,” in Proceedings of the AAAI Conference on Artificial Intelligence, vol. 37, no. 1, 2023, pp. 350–358.DOI
  10. [10]J. Zhang, S. Chen, Y. Zheng, Z. Yu, and T. Huang, “Spike-guided motion deblurring with unknown modal spatiotemporal alignment,” 2024, pp. 25 047–25 057.DOI
  11. [11]L. Hu, R. Zhao, Z. Ding, L. Ma, B. Shi, R. Xiong, and T. Huang, “Optical flow estimation for spiking camera,” in Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, 2022, pp. 17 844–17 853.arxiv.org/abs/2110.03916
  12. [12]R. Zhao, R. Xiong, J. Zhao, Z. Yu, X. Fan, and T. Huang, “Learning optical flow from continuous spike streams,” 2022, pp. 7905–7920.DOI
  13. [13]R. Zhao, R. Xiong, J. Zhang, X. Zhang, Z. Yu, and T. Huang, “Optical flow for spike camera with hierarchical spatial-temporal spike fusion,” 2024, pp. 7496–7504.
  14. [14]R. Zhao, R. Xiong, D. Wang, S. Xuan, J. Zhang, X. Fan, and T. Huang, “Spike camera optical flow estimation based on continuous spike streams,” vol. 48, no. 4, pp. 4756–4770, 2026.
  15. [15]Y. Zhu, Y. Zhang, X. Xie, and T. Huang, “An fpga accelerator for high-speed moving objects detection and tracking with a spike camera,” Neural Computation, vol. 34, no. 8, pp. 1812–1839, 2022.DOI
  16. [16]Y. Zheng, C. Li, J. Zhang, Z. Yu, and T. Huang, “Snntracker: Online high-speed multi-object tracking with spike camera,” IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 48, no. 1, pp. 624–638, 2026.DOI
  17. [17]K. Feng, C. Jia, S. Ma, and W. Gao, “A joint visual compression and perception framework for neuromorphic spiking camera,” IEEE Transactions on Image Processing, 2025.DOI
  18. [18]Y. Dong, J. Zhao, R. Xiong, and T. Huang, “3d residual interpolation for spike camera demosaicing,” in 2022 IEEE International Conference on Image Processing (ICIP), IEEE, 2022, pp. 1461–1465.
  19. [19]Y. Dong, R. Xiong, X. Fan, Z. Yu, Y. Tian, and T. Huang, “Self-supervised learning for color spike camera reconstruction,” in Proceedings of the Computer Vision and Pattern Recognition Conference, 2025, pp. 6231–6240.DOI
  20. [20]Y. Dong, R. Xiong, J. Zhao, J. Zhang, X. Fan, S. Zhu, and T. Huang, “Learning a deep demosaicing network for spike camera with color filter array,” IEEE Transactions on Image Processing, vol. 33, pp. 3634–3647, 2024.DOI
  21. [21]Y. Dong, R. Xiong, J. Zhao, X. Fan, X. Zhang, and T. Huang, “Color spike camera reconstruction via long short-term temporal aggregation of spike signals,” IEEE Transactions on Image Processing, 2025.DOI
  22. [22]Y. Zheng, L. Zheng, Z. Yu, B. Shi, Y. Tian, and T. Huang, “High-speed image reconstruction through short-term plasticity for spiking cameras,” in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2021, pp. 6358–6367.DOI
  23. [23]K. Chen, Y. Zheng, T. Huang, and Z. Yu, “Rethinking high-speed image reconstruction framework with spike camera,” in Proceedings of the AAAI Conference on Artificial Intelligence, vol. 39, no. 2, 2025, pp. 2097–2104.arxiv.org/abs/2501.04477
  24. [24]W. Maass, “Networks of spiking neurons: the third generation of neural network models,” Neural networks, vol. 10, no. 9, pp. 1659–1671, 1997.DOI
  25. [25]Y. Zheng, Z. Yu, S. Wang, and T. Huang, “Spike-based motion estimation for object tracking through bio-inspired unsupervised learning,” IEEE Transactions on Image Processing, vol. 32, pp. 335–349, 2022.DOI
  26. [26]Y. Zheng, L. Zheng, Z. Yu, T. Huang, and S. Wang, “Capture the moment: High-speed imaging with spiking cameras through short-term plasticity,” IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 45, no. 7, pp. 8127–8142, 2023.
  27. [27]J. Zhao, S. Zhang, Z. Yu, and T. Huang, “Recognizing ultra-high-speed moving objects with bio-inspired spike camera,” in Proceedings of the AAAI Conference on Artificial Intelligence, vol. 38, no. 7, 2024, pp. 7478–7486.
  28. [28]C. Gallego, T. Delbrück, G. Orchard, C. Bartolozzi, B. Taba, A. Censi, S. Leutenegger, A. J. Davison, J. Conradt, K. Daniilidis et al., “Event-based vision: A survey,” IEEE transactions on pattern analysis and machine intelligence, vol. 44, no. 1, pp. 154–180, 2020.
  29. [29]H. Rebecq, R. Ranftl, V. Koltun, and D. Scaramuzza, “Events-to-video: Bringing modern computer vision to event cameras,” in Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, 2019, pp. 3857–3866.
  30. [30]H. Scheerlinck, H. Rebecq, D. Gehrig, N. Barnes, R. Mahony, and D. Scaramuzza, “Fast image reconstruction with an event camera,” in Proceedings of the IEEE/CVF winter conference on applications of computer vision, 2020, pp. 156–163.
  31. [31]T. Stoffregen, C. Scheerlinck, D. Scaramuzza, T. Drummond, N. Barnes, L. Kleeman, and R. Mahony, “Reducing the sim-to-real gap for event cameras,” in European Conference on Computer Vision. Springer, 2020, pp. 534–549.arxiv.org/abs/2003.09078
  32. [32]W. Weng, Y. Zhang, and Z. Xiong, “Event-based video reconstruction using transformer,” in Proceedings of the IEEE/CVF International Conference on Computer Vision, 2021, pp. 2563–2572.DOI
  33. [33]B. Ercan, O. Eker, C. Saglam, A. Erdem, and E. Erdem, “HyperEV2ID: Improving event-based video reconstruction via hypernetworks,” IEEE Transactions on Image Processing, vol. 33, pp. 1826–1837, 2024.DOI
  34. [34]Y. Wu, L. Deng, G. Li, J. Zhu, and L. Shi, “Spatio-temporal backpropagation for training high-performance spiking neural networks,” Frontiers in Neuroscience, vol. 12, p. 331, 2018.DOI
  35. [35]W. Fang, Z. Yu, Y. Chen, T. Masquelier, T. Huang, and Y. Tian, “Incorporating learnable membrane time constant to enhance learning of spiking neural networks,” in Proceedings of the IEEE/CVF international conference on computer vision, 2021, pp. 2661–2671.arxiv.org/abs/2007.05785
  36. [36]Y. Li, Y. Guo, S. Zhang, S. Deng, Y. Hu, and S. Gu, “Differentiable spike: Rethinking gradient-descent for training spiking neural networks,” Advances in neural information processing systems, vol. 34, pp. 23 426–23 439, 2021.
  37. [37]X. Luo, M. Yao, Y. Chou, B. Xu, and G. Li, “Integer-valued training and spike-driven inference spiking neural network for high-performance and energy-efficient object detection,” in European Conference on Computer Vision. Springer, 2024, pp. 253–272.arxiv.org/abs/2407.20708
  38. [38]B. Ye, W. Li, D. Huang, M. Yao, B. Li, W. Hu, D. Liang, and K. Shang, “Integer binary-range alignment neuron for spiking neural networks,” arXiv preprint arXiv:2506.05679, 2025.arxiv.org/abs/2506.05679
  39. [39]S. Nah, S. Baik, S. Hong, G. Moon, S. Son, R. Timofte, and K. Mu Lee, “NTIRE 2019 challenge on video deblurring and super-resolution: Dataset and study,” in Proceedings of the IEEE/CVF conference on computer vision and pattern recognition workshops, 2019, pp. 0–0.DOI
  40. [40]Y. Dong, R. Xiong, J. Zhao, J. Zhang, X. Fan, S. Zhu, and T. Huang, “Joint demosaicing and denoising for spike camera,” in Proceedings of the AAAI Conference on Artificial Intelligence, vol. 38, no. 2, 2024, pp. 1582–1590.
  41. [41]R. Zhang, P. Isola, A. A. Efros, E. Shechtman, and O. Wang, “The unreasonable effectiveness of deep features as a perceptual metric,” in Proceedings of the IEEE conference on computer vision and pattern recognition, 2018, pp. 586–595.arxiv.org/abs/1801.03924
  42. [42]A. Mittal, R. Soundararajan, and A. C. Bovik, “Making a “completely blind” image quality analyzer,” IEEE Signal Processing Letters, vol. 20, no. 3, pp. 209–212, 2012.DOI
  43. [43]A. Mittal, A. K. Moorthy, and A. C. Bovik, “No-reference image quality assessment in the spatial domain,” IEEE Transactions on image processing, vol. 21, no. 12, pp. 4695–4708, 2012.DOI

Paper details

vlnc:W_8hk4626j
Version
v1 · latest
Published
Fresh — added to the Hub within the last 30 days
Added
License
CC BY 4.0
Usage

views · 13

downloads · 1

A view counts once per browser per day, a download each time the PDF is requested.

  • Computer Science

Not recorded for this paper

View statement
Looking for related papers…
Contents