WorksEngineering
TOP: Transmission-Oriented Remote Sensing Pansharpening
Abstract
Pansharpening has been extensively studied as a fundamental image fusion task in remote sensing. However, most existing methods focus on improving fusion quality under the implicit assumption that high-resolution observations have already been fully downlinked to the ground. As a result, existing models are highly vulnerable to perturbations introduced by lossy compression used for downlink transmission. To address this limitation, we present, to the best of our knowledge, the first transmission oriented pansharpening (TOP) framework, which unifies compact latent coding, onboard pansharpening, and efficient downlink transmission within a single satellite-toground pipeline. TOP encodes remote-sensing observations into a compact binary latent space, where a lightweight onboard fusion module can efficiently predict the fused binary latent code. Since only the compact fused code is downlinked, the transmission payload is substantially reduced. Overall, TOP enables low-payload downlink and lightweight onboard processing while maintaining high fusion quality under severe bandwidth constraints. Experiments across multispectral and hyperspectral settings show that TOP consistently achieves superior fusion quality at higher compression ratios than representative pansharpening methods paired with standard lossy codecs. For example, on GaoFen-2, TOP attains a fusion PSNR of 31.4 dB at a downlink compression ratio of 50x. This design also points toward transmissionoriented processing for other remote-sensing tasks. The code will be made available at https://github.com/JingZe-Liang/TOP.git.
Introduction
Pansharpening reconstructs a high-spatial-resolution multispectral or hyperspectral image (HR-MSI/HR-HSI) by combining the spatial structures of a panchromatic (PAN) observation with the spectral information of a low-spatial-resolution multispectral or hyperspectral image (LR-MSI/LR-HSI), and underpins remote-sensing image interpretation [1], [2]. From the explicit detail-injection rules of component-substitution and multi-resolution-analysis methods [3, 4, 5] to the nonlinear mappings learned by deep networks [6, 7, 8], these methods share an implicit assumption: the input observations have already been fully delivered to the ground station and are directly available to the fusion model.
In practice, the PAN and LR-MSI/LR-HSI observations are acquired on board and must pass through bandwidth-limited downlink channels [9]. A conventional deployment therefore compresses and transmits them independently and fuses the decompressed images on the ground (first row of Figure 1(a)). Under restrictive bitrate budgets, lossy compression introduces spatial blurring, spectral distortion, and quantization artifacts, whereas pansharpening models are commonly trained on clean observations [6, 7, 8]. This training–deployment mismatch can severely degrade the fusion result (Figure 1(b)), and the codec-based baselines in Sec. IV-C indeed degrade sharply under constrained downlink budgets. Recent work on degradation-robust fusion [10] further highlights the importance of realistic input perturbations.
Figure 1. TOP compared with existing deployment paths for pansharpening. (a) Three ways of delivering a pansharpened image to the ground, with compression ratios and PSNR ranges from WorldView-3 (WV3). (b) Ground truth, the best compress-then-fuse baseline under JPEG2000, and TOP on WV3, GaoFen-2 (GF2), and Pavia Center. (c) PSNR versus downlink compression ratio for methods under HEVC or JPEG2000; onboard fusion followed by compression is evaluated on WV3 only (Sec. S9 of the Supplementary Material).
Fusing on board (second row of Figure 1(a)) does not resolve this issue. The satellite must then generate a full HR-MSI/HR-HSI, which is computationally expensive on resource-constrained platforms, and the fused image must still be compressed with a standard codec such as JPEG2000 [11] or High Efficiency Video Coding (HEVC) [12], whose rate–distortion performance then bounds the final quality [9, 13].
These limitations motivate a different question: rather than deciding whether to compress images before or after pansharpening, can transmission and pansharpening, which have so far been handled separately, benefit from each other? To this end, we propose, to the best of our knowledge, the first transmission-oriented pansharpening (TOP) framework, which jointly designs compact latent coding, onboard pansharpening, and downlink transmission. TOP maps the PAN and LR-MSI/LR-HSI observations into binary latent codes and employs a lightweight latent fusion module to predict the fused HR-MSI/HR-HSI code on board (third row of Figure 1(a)). Only this code is downlinked; the ground station decodes it into the final HR-MSI/HR-HSI or archives it for later reconstruction. TOP thus shifts the communication target from source images to a compact fused code.
TOP is built on a transmission-oriented binary tokenizer (TOBT), which maps PAN, LR-MSI/LR-HSI, and HR-MSI/HR-HSI observations into a shared latent space through an adaptive embedding for different numbers of bands and a Swin-based encoder [14]. Binary spherical quantization (BSQ) [15] then maps each latent entry to one bit. Besides reducing the payload, this bottleneck turns continuous-latent regression into an easier per-entry binary classification for the latent fusion module (§ 5-B). The binary code is transmitted directly, without an entropy model [13] or range asymmetric numeral systems (rANS) coding [16] (§ 5-C), and bit-flip augmentation makes the ground-side decoder robust to latent prediction and transmission errors.
We compare TOP with representative pansharpening methods whose inputs are compressed by HEVC or JPEG2000 before ground-side fusion, on the WorldView-3 (WV3) and GaoFen-2 (GF2) multispectral datasets and the Pavia Center hyperspectral dataset (hereafter Pavia). On all three datasets, TOP delivers a higher PSNR than every compressed baseline while operating at a higher compression ratio (Figure 1(c)); on WV3, for example, it reaches 30.32 dB at 33×, whereas the strongest baseline reaches 27.70 dB at a lower ratio. At these ratios, 1 TB of source observations corresponds to roughly 15.2 GB of downlinked data on WV3 (at 66×), 20.0 GB on GF2, and 41.7 GB on Pavia. Our contributions are summarized as follows:
We firstly introduce and formalize transmission-oriented pansharpening, a system-level design that jointly considers compact binary coding, onboard fusion, downlink communication, and ground-side reconstruction instead of assuming fully available lossless observations.
We build TOP, which performs pansharpening in a compact binary latent space and transmits only the fused HR-MSI/HR-HSI code; its BSQ bottleneck yields a code that is reconstructable, directly transmittable, and predictable for the latent fusion module.
We conduct experiments on multispectral and hyperspectral datasets, showing that TOP achieves superior fusion quality at substantially higher compression ratios than representative pansharpening methods paired with codecs.
Related Work
Pansharpening
Pansharpening fuses the spatial details of a PAN image with the spectral information of an LR-MSI/LR-HSI image [1, 2] and belongs to the broader family of image fusion tasks, which also includes infrared–visible, multi-exposure, and multi-focus fusion [17, 18, 19]. Traditional methods are based on component substitution, such as GS [3], BDSD [4], GSA [20], and PRACS [21]; multi-resolution analysis, such as GLP [5], AWLP [22], and morphological filters [23]; or variational, Bayesian, and low-rank formulations [1, 2, 24, 25]. They are efficient and interpretable, but predefined models may cause spatial or spectral distortions. Deep multispectral models have introduced detail injection [6], dual-branch spatial–spectral modeling [7], frequency-aware or adaptive feature extraction [8, 26, 27], degradation-aware modeling [28], and zero-shot adaptation [29]. Hyperspectral pansharpening must further handle high spectral dimensionality and sensor variability, and relies on Transformer-based fusion and equivariant imaging [30, 31], image-specific priors and band-wise propagation [32, 33, 34], and diffusion-based zero-shot guidance [35, 36]. Degradation-robust fusion methods [10] handle generic low-quality inputs rather than downlink compression artifacts. All these methods assume that the source observations are fully available at the processing platform, whereas TOP performs pansharpening before downlink in a compact binary latent space.
Remote-Sensing Image Compression and Transmission
The growing volume of remote-sensing imagery strains onboard storage and satellite-to-ground communication [9]. Standard codecs such as JPEG2000 [11] and HEVC [12] reduce payloads through transform coding, quantization, and entropy coding. Lossless and near-lossless remote-sensing codecs such as CCSDS 123.0-B-2 [37] target low-complexity, distortion-bounded compression rather than the high-compression regime studied here, and are therefore not included in our comparison. Neural image compression instead learns nonlinear transforms with quantized latents under rate–distortion optimization [13]. Existing pipelines combine compression and pansharpening in one of two orders, either compressing the source observations and fusing on the ground or fusing on board and compressing the fused image, and in both cases treat fusion and compression as separate stages. TOP instead fuses within a compact binary latent space and downlinks the fused code as the payload.
Discrete Visual Representation and Quantization
Transmitting a learned latent requires discretizing it. VQ-VAE [38] assigns latent vectors to a learned codebook whose capacity and utilization are hard to control. Finite scalar quantization [39] maps each dimension to fixed scalar levels, lookup-free quantization [40] derives token indices directly from binary states, and BSQ [15] projects latents onto a unit hypersphere before sign quantization. Such binary codes are highly compact and thus well suited to transmission, so TOP adopts BSQ as a shared latent code for all three modalities. Unlike generative models, which predict discrete tokens with autoregressive or masked-token objectives [40], TOP predicts the target binary code conditionally from the PAN and LR-MSI/LR-HSI codes, and this prediction serves as both the downlink payload and the input to the ground-side decoder.
Methodology
Sec. III-A introduces the notation and the deployment-time pipeline. We then detail the components of TOP, namely the transmission-oriented binary encoder (TOBE; Sec. III-B), BSQ (Sec. III-C), and the transmission-oriented binary decoder (TOBD; Sec. III-E), together with the latent fusion module (Sec. III-D), the training strategy (Sec. III-F), and the overall formulation of TOP (Sec. III-G). Implementation details are provided in Sec. S3 of the Supplementary Material.
Algorithm 1: Deployment-time inference of TOP.
Input: PAN observation ; upsampled LR-MSI/LR-HSI observation ; trained encoder ; quantizer ; latent fusion module ; ground-side decoder
Output: Reconstructed HR-MSI/HR-HSI image
▷ Onboard Encoding, Fusion, and Downlink
▷ Encode PAN
▷ Encode spectral image
▷ Predict binary-state logits
▷ Normalize the fused code
▷ Pack the binary payload
Downlink to the ground station
▷ Ground-Side Reconstruction
▷ Recover the received fused code
▷ Decode the fused HR image
return
System Overview
Let , , and denote the PAN observation, the upsampled LR-MSI/LR-HSI observation, and the target HR-MSI/HR-HSI image, respectively. During deployment, TOP divides pansharpening into onboard latent fusion and ground-side reconstruction (Figure 2). On board, the shared TOBE and the BSQ quantizer encode and into normalized binary latent codes and , which are concatenated along the channel dimension and processed by the lightweight latent fusion module to predict the fused code . During training, this prediction is supervised by , the code obtained by encoding ; during inference, is directly bit-packed and transmitted. At the ground station, the received code is unpacked as , and the decoder reconstructs the fused image , which approximates the ground-truth (GT) image . Algorithm 1 summarizes the deployment-time inference. The variables and define only the training target; neither is available on board during deployment. The fused code is inferred solely from and , so the payload is a predicted fused representation rather than a compressed copy of a pre-existing HR-MSI/HR-HSI.
Figure 2. Overall pipeline of TOP. On board, the observations are encoded by TOBE, binarized, and fused; the fused code is bit-packed, downlinked, and decoded by TOBD or stored on the ground. Downsampling is omitted for clarity; configurations are listed in Tab. X of the Supplementary Material. GELU: Gaussian error linear unit.
Transmission-Oriented Binary Encoder
TOBE maps observations with different numbers of spectral bands into a shared, compact latent feature space before binary quantization (middle-left panel of Figure 2).
Band-Adaptive Embedding
TOBE embeds all modalities with an adaptive convolution that maintains a learnable base kernel with input channels. For an incoming modality with spectral bands, the kernel is resampled along its input-channel dimension:
where the superscripts and select input-channel slices of the adapted and base kernels, , and is the bicubic resampling weight from base channel to adapted channel . PAN and spectral inputs are thus mapped to the same feature dimension with shared weights. TOBD applies the same adaptation along the output-channel dimension.
| Block | Modeling behavior | Main complexity |
| CNN | Fixed, local | |
| Global attention | Adaptive, all tokens | |
| Swin | Adaptive, windowed |
Table 1. Complexity of candidate TOBE backbone blocks ( tokens, width , kernel size , window size ).
Swin-Based Multi-Stage Feature Extraction
The embedded features are processed by a Swin Transformer encoder [14], whose windowed self-attention provides stronger content-adaptive spatial–spectral modeling than CNNs at a much lower cost than global self-attention [41] (Tab. I); the ablation in § 5-A verifies this choice. With denoting the embedded feature, the encoder stages compute
where is a downsampling operator, applied only until the target factor is reached, and is a stack of Swin blocks whose depth varies with (Table X of the Supplementary Material). The output has channels and is spatially reduced by a factor of along each dimension.
| Configuration | WV3 33× | WV3 66× | GF2 50× | Pavia 23.97× |
| Bands//s | 8/8/4 | 8/16/8 | 4/16/8 | 102/64/4 |
| Stages/ | 4/64 | 3/128 | 3/128 | 4/64 |
| Width multipliers | [1, 2, 4, 4] | [1, 2, 4] | [1, 2, 4] | [1, 2, 4, 4] |
| Encoder/decoder blocks | [2, 4, 4, 2]/[3, 6, 6, 3] | [2, 4, 2]/[4, 2, 2] | [2, 4, 2]/[4, 2, 2] | [2, 4, 4, 2]/[4, 4, 2, 2] |
| Embedding kernel | ||||
| BSQ groups | 2 | 2 | 2 | 4 |
| Fusion blocks/input channels | 8/16 | 12/32 | 8/32 | 10/128 |
| Stage-1 batch | 8 | 16 | 16 | 8 |
| Stage-1 MSE/BSQ weights | 1/0.05 | 4/0.1 | 4/0.1 | 1/0.1 |
| Fusion batch | 2 | 8 | 8 | 8 |
Table X. Top architecture and configuration-specific training settings.
Binary Spherical Quantization
Although spatially compressed, is still continuous-valued. TOBT therefore adopts BSQ [15], a binary tokenization technique from generative visual representation learning that maps the latent vector at each location to a binary code on the unit hypersphere. BSQ reduces the payload to one bit per latent entry and provides the latent fusion module with a compact and explicit discrete space that is well suited to fusion (§ 5-B). Let denote the channel vector of at latent location , with . BSQ performs deterministic sign quantization
where acts elementwise, maps zero to , and is trained with a straight-through estimator. Collecting the codes over all locations gives the normalized binary latent code
where arranges the matrix of column vectors into a tensor in a fixed raster order, and denotes the code of any of the three modalities. During training, soft assignment and grouped entropy regularization encourage effective code utilization [15]. In practice, the latent channels are split into several groups, and the entropy terms are computed per group; for simplicity, the main text treats all channels as a single group. Its details are given in § 9 of the Supplementary Material.
Latent Fusion Module
From the concatenated codes and , the latent fusion module predicts two logits for each latent entry:
and the fused code is obtained by selecting a binary state independently for each entry:
where is the slice of for state , the is taken elementwise, and is an all-ones tensor of the latent shape.
Binary Downlink and TOBD
The satellite downlinks the bitstream , where ‘BitPack’ maps negative entries to 0 and nonnegative entries to 1 and serializes them in a fixed order, together with the latent raster shape, whose negligible size is omitted from the compression ratio in Eq. (18). Unlike the original BSQ, which further compresses the codes with an autoregressive prior and arithmetic coding [15], TOP uses no entropy model or rANS codec [16], as supported by § 5-C.
On the ground side, the received bitstream is unpacked and decoded as
where ‘BitUnpack’ inverts ‘BitPack’ by mapping each bit to ; the received code may differ from if transmission errors occur. follows the stage layout of in reverse order, possibly with different numbers of Swin blocks per stage (lower-left panel of Figure 2), and outputs the final HR-MSI/HR-HSI reconstruction . The binary code can also be archived for deferred decoding, which yields considerable storage savings.
Optimization Objectives
We first train TOBT, including its TOBE and TOBD, and then train the latent fusion module with the frozen tokenizer (Figure 3).
Figure 3. Training pipeline of the proposed method. (a) Three-stage TOBT training: stage 1 trains TOBE and TOBD jointly (40k iterations); stages 2 and 3 refine TOBD with TOBE frozen (20k iterations each), stage 3 with random bit flips. (b) Latent fusion training with TOBT frozen. Bold outlines: trainable; hatched: frozen; gray: parameter-free; dashed arrows: loss targets.
Three-Stage TOBT Training
Optimization Objectives
a) Loss Components: For an image-domain target and its reconstruction , the mean squared error (MSE) and Sobel-gradient losses are
where the norms are computed over vectorized entries and is the channel-wise Sobel operator; penalizes edge discrepancies when refining the ground-side decoder. For spectral fidelity, the spectral angle mapper (SAM) loss [42] averages the angle, in radians, between the reconstructed and target spectral vectors over all spatial locations:
Unlike MSE, SAM is insensitive to uniform spectral scaling. The quantization loss follows BSQ [15]: a commitment term pulls the normalized continuous latent toward its binary code, and an entropy term prevents the encoder from collapsing onto a few binary patterns:
where weights the commitment term (Sec. S1 of the Supplementary Material).
b) Training Stages: In stage 1 (Figure 3(a)), TOBE and TOBD are jointly trained on all three modalities to stabilize the shared binary latent space before the decoder is refined, using
where and are weighting coefficients. Stage 2 targets the full TOP setting, in which only the HR code is transmitted: TOBE is frozen, so the quantization loss is dropped, and TOBD is fine-tuned on the HR modality with
where the weights and are listed in § 11 of the Supplementary Material.
Stage 3 copes with binary latent errors caused by imperfect latent fusion or by transmission. It keeps the stage-2 setting but injects random bit flips into the HR code before decoding:
where the entries of the mask are independently drawn from Bernoulli, and an entry equal to 1 reverses the sign of the corresponding latent entry. The flip probability is linearly ramped from to over the first 2000 iterations:
where is the stage-3 iteration index. TOBD thereby learns to correct occasional bit errors by exploiting the redundancy of the code. Under 5% bit flips (Figure 4), the stage-3 decoder suppresses the speckle-like artifacts of the stage-2 decoder and better preserves spatial structures and colors; § 5-D quantifies this effect.

Figure 4. WV3 reduced-resolution MSI reconstruction under 5% bit flips. Left: TOBD after stage 3 (with bit-flip augmentation). Right: TOBD after stage 2 (without). White boxes mark the close-ups below.
Latent Fusion Module Training
With all tokenizer parameters frozen, the PAN, upsampled LR-MSI/LR-HSI, and HR-MSI/HR-HSI images of each training sample are encoded into , , and . The latent fusion module predicts the logits from and as in Eq. (5), and the fixed code supervises it, so the training target uses the same binary representation as the transmitted payload. The module is trained with a binary cross-entropy (BCE) loss on the logits:
where the target has the same shape as , with state slices and (elementwise), and the loss averages the scalar BCE terms over both states and all latent entries (§ 10 of the Supplementary Material). No pixel-domain loss is back-propagated through TOBD during this training.
Transmission-Oriented Pansharpening Formulation
At the system level, TOP can be summarized as
where the downlinked bitstream and the received code are obtained from through the encoding, latent-fusion, and bit-packing steps of Alg. 1 (Eqs. (5)–(7)), collectively represents the fidelity objectives used throughout TOP, and denotes the maximum allowable downlink budget.
In contrast, conventional pansharpening mainly considers
where is a conventional pansharpening model with learnable parameters and is its fusion objective. Unlike this formulation, TOP explicitly incorporates the downlink constraint into the pansharpening problem.
Experiments
Experimental Settings
Datasets
1) Datasets: We use the WV3 (8 bands) and GF2 (4 bands) multispectral datasets from PanCollection [43] and the 102-band Pavia Center hyperspectral dataset distributed through HyperPanCollection and used in Hyper-DSNet [44], all with a spatial scale factor of four. WV3, GF2, and Pavia contain 9,714, 19,809, and 1,512 training samples and 1,080, 2,201, and 168 validation samples, respectively; the validation samples are held-out training patches distinct from the test scenes. Training samples consist of PAN, upsampled LR-MSI/LR-HSI, and HR-MSI/HR-HSI patches together with native LR-MSI/LR-HSI patches. For WV3 and GF2, we test on 20 reduced-resolution scenes of size , simulated following Wald’s protocol [45], and on 20 full-resolution scenes of size without GT. For Pavia, the full test split contains two scenes with HR-HSI references. Details are given in § 12 of the Supplementary Material.
Compared Methods
2) Compared Methods: The multispectral baselines are GS [3], BDSD [4], GLP [5], and MF [23] (traditional); FusionNet [6], U2Net [7], W-FANet [8], ADWM [26], and ARConv [27] (supervised); and ZS-PAN [29] (zero-shot). The hyperspectral baselines are AWLP [22], BDSD-PC [46], BT-H [47], GSA [20], MF [23], and PRACS [21] (traditional); HyperTransformer [30] (supervised); and DIP-HyperKite [32], R-PNN [33], uTDSR [35], DM-ZS [36], and -PNN [34] (zero-shot).
Transmission-Oriented Protocol and Communication Metrics
All baselines follow the ‘compress first, fuse later’ pipeline with JPEG2000 [11] (OpenJPEG, with a search over quality layers to match the target ratio) or HEVC [12] (libx265 with constant rate factor control to match the target ratio). The compression ratio divides the raw sensor-side data volume by the size of the transmitted bitstream. To avoid inflating , the raw volume uses the native, non-upsampled LR-MSI/LR-HSI:
where the PAN image has size , the LR-MSI/LR-HSI has bands and size , is the spatial scale factor, and is the native sensor bit depth (11 for WV3, 10 for GF2, and 13 for Pavia, based on the official specifications). For a codec-based baseline, is the total size of the separately encoded PAN and native LR-MSI/LR-HSI streams; for TOP, it is the size of the single bit-packed fused latent. Using the native spectral image in the numerator is essential because interpolation creates samples without adding sensor information and would otherwise overstate the apparent compression ratio. The bits per pixel per band (bppb) normalizes the payload by the size of the target HR-MSI/HR-HSI:
Evaluation Metrics
On scenes with GT, we report PSNR, SSIM [48], SAM (in degrees) [42], and ERGAS [49]. We also report the no-reference HQNR index and its spectral and spatial distortion components and [50]. In the reduced-resolution tables, they compare each fused image with the LR-MSI/LR-HSI and PAN of the same scene and thus measure consistency with the source observations rather than full-resolution quality. For the lossy baselines, and are computed against the original lossless observations rather than their decompressed counterparts. Otherwise, a codec could distort both the fusion input and the metric anchor in a similar direction and receive an artificially favorable consistency score. Retaining the common lossless anchors makes HQNR terms comparable across codecs and TOP, while PSNR, SSIM, SAM, and ERGAS independently measure agreement with GT. In the result tables, the best and second-best results are shown in boldface and underlined, respectively.
5) Implementation Details: TOP is implemented in PyTorch with bfloat16 mixed precision on an NVIDIA RTX 4090 GPU and optimized with AdamW at an initial learning rate of , with weight decay 0.05 for TOBT and 0.01 for the latent fusion module. Architectures and training schedules are listed in Sec. S3 of the Supplementary Material.
Multimodal Reconstruction of TOBT
Table III reports validation-set reconstruction of PAN, upsampled LR-MSI/LR-HSI, and HR-MSI/HR-HSI images, and Figure 5 shows examples. These modality-wise results isolate the representation capacity available before latent fusion. On WV3, halving the HR-MSI payload from 0.063 to 0.031 bppb lowers reconstruction PSNR from 30.17 to 28.22 dB, exposing the expected rate–fidelity trade-off. LR-MSI reconstruction nevertheless remains above 37 dB and 0.95 SSIM. TOBT also reaches 33.46 dB on GF2 HR-MSI and 34.85 dB on Pavia HR-HSI, supporting the shared design across sensors and spectral dimensionalities. Fine textures become smoother as the ratio increases, while the main structures and spectral appearance remain stable. These results define the tokenizer’s reconstruction ceiling; the end-to-end experiments test how closely the predicted fused codes approach it.
Figure 5. TOBT multi-modality reconstruction results on WV3, GF2, and Pavia. Upper rows: original test crops; lower rows: independent TOBT reconstructions. Ratios are overall TOP compression ratios (Eq. (18)).
Transmission-Oriented Pansharpening
Having established that TOBT preserves the three modalities under a constrained binary capacity, we evaluate whether the complete TOP pipeline can turn that representation into a useful downlink product. The comparison therefore considers fusion fidelity and communication cost jointly: a method is favorable only if it preserves spatial–spectral information while reducing the transmitted payload relative to the codec-based pipelines.
Multispectral Pansharpening on WV3
Table II reports the WV3 reduced-resolution results under JPEG2000. The best baseline reaches 27.70 dB at 21.26×, whereas TOP achieves 30.32 dB at 33× and 28.88 dB at 66×; relative to the best baseline value of each metric, TOP also reduces ERGAS from 8.36 to 6.00 and SAM from 10.39 to 6.00 degrees. Both TOP configurations outperform all baselines on every metric. Figure 6a shows that TOP preserves sharper local structures and more stable spectral appearance, whereas the codec-based baselines suffer from edge degradation and texture loss. Comparisons with all baselines are given in Sec. S5 of the Supplementary Material.
Figure 6. Qualitative comparison under the JPEG2000 transmission-oriented protocol on (a) WV3, (b) GF2, and (c) Pavia. TOP is compared with the six baselines of highest PSNR (averaged over WV3 and GF2 for the multispectral data). Rows show the predictions with GT and the error maps (dark blue: lower error). All baselines are compared in Sec. S5 of the Supplementary Material.
2) Multispectral Pansharpening on GF2: On GF2 (Tab. IV), TOP achieves 31.37 dB PSNR, 0.83 SSIM, 2.63 ERGAS, and 2.46 degrees SAM at 50×. The strongest baseline, U2Net, reaches 28.53 dB and 4.29 degrees at 20.33×, so TOP improves PSNR by 2.84 dB at about 2.5
Implementation Details
times its compression ratio; the zero-shot ZS-PAN reaches 24.52 dB. Figure 6(b) shows that TOP avoids the codec artifacts that affect the baselines.
Tables V and VI report the corresponding reduced-resolution comparisons under HEVC. The TOP rows are identical to those in the JPEG2000 tables because neither codec is part of TOP: HEVC and JPEG2000 are applied only to the source observations transmitted by the pixel-domain baselines, whereas TOP always bit-packs and transmits the same fused binary latent. Repeating the TOP operating point in each table provides a direct comparison against the baseline pipeline induced by that codec. Although the HEVC baselines operate at only 6.69× on WV3 and 8.94× on GF2, TOP exceeds their strongest PSNR values (27.22 and 29.13 dB) by 3.10 and 2.24 dB at 33× and 50×, respectively, and leads on every metric. The persistence of these gains across two sensors and two codecs indicates that the advantage comes from changing the transmitted representation rather than adapting to the artifacts of one particular codec. The corresponding visual comparisons are provided in Sec. S6, and full-resolution results, where TOP attains the highest HQNR and lowest under both codecs, are given in Sec. S7 of the Supplementary Material.
3) Hyperspectral Pansharpening on Pavia: Table VII reports the Pavia results under JPEG2000. The baselines operate at 19.62×, with the best PSNR and HQNR reaching 28.09 dB and 0.83. At the higher ratio of 23.97×, TOP achieves 29.72 dB PSNR, 0.83 SSIM, 5.91 ERGAS, 0.85 HQNR, 6.39 degrees SAM, , and , the best result on every metric. Figure 6(c) shows that TOP also preserves hyperspectral spatial structures under a constrained downlink. The HEVC results, where TOP again outperforms all baselines, are given in Sec. S6 of the Supplementary Material. Across WV3, GF2, and Pavia, the same pattern holds for 8-, 4-, and 102-band data: TOP improves both GT-based fidelity and consistency with the source observations while using a higher compression ratio. The simultaneous gains in PSNR, SAM, , and are important because a higher PSNR alone could mask band-dependent spectral distortion. The following profile analysis examines this balance directly at the band level.
| Method | Ratio ↑ | PSNR ↑ | SSIM ↑ | ERGAS ↓ | HQNR ↑ | ↓ | ↓ | SAM (deg) ↓ |
| AWLP+JPEG2000 [22] | 26.15 | 0.65 | 8.94 | 0.76 | 0.16 | 0.09 | 10.91 | |
| BDSD-PC+JPEG2000 [46] | 27.02 | 0.65 | 8.15 | 0.83 | 0.15 | 0.03 | 9.69 | |
| BT-H+JPEG2000 [47] | 27.25 | 0.72 | 8.60 | 0.66 | 0.30 | 0.06 | 10.16 | |
| GSA+JPEG2000 [20] | 28.09 | 0.69 | 7.54 | 0.78 | 0.16 | 0.07 | 10.20 | |
| MF+JPEG2000 [23] | 26.72 | 0.67 | 8.39 | 0.78 | 0.16 | 0.07 | 10.00 | |
| PRACS+JPEG2000 [21] | 26.58 | 0.62 | 8.59 | 0.81 | 0.15 | 0.05 | 9.73 | |
| uTDSP+JPEG2000 [35] | 19.62× | 25.26 | 0.48 | 10.28 | 0.69 | 0.15 | 0.19 | 12.40 |
| DIP-HyperKite+JPEG2000 [32] | 27.40 | 0.69 | 7.80 | 0.82 | 0.15 | 0.03 | 9.68 | |
| HyperTransformer+JPEG2000 [30] | 19.19 | 0.21 | 20.50 | 0.44 | 0.25 | 0.42 | 34.43 | |
| R-PNN+JPEG2000 [33] | 26.62 | 0.64 | 8.46 | 0.79 | 0.15 | 0.06 | 10.34 | |
| DM-ZS+JPEG2000 [36] | 26.18 | 0.67 | 9.67 | 0.75 | 0.20 | 0.06 | 11.70 | |
| -PNN+JPEG2000 [34] | 25.14 | 0.58 | 10.00 | 0.75 | 0.14 | 0.13 | 11.31 | |
| TOP (ours) | 23.97× | 29.72 | 0.83 | 5.91 | 0.85 | 0.14 | 0.02 | 6.39 |
Table VII. Pavia Results Under JPEG2000 (Mean over Two Full Test Scenes; Std. Omitted)
4) Spectral Profile Analysis: Image-level metrics summarize the final reconstruction but do not show how compression and fusion redistribute errors across spectral bands. Figure 7 therefore compares the normalized band-wise mean over a fixed ROI with the GT profile, with all baselines paired with JPEG2000. In all four settings, the TOP curve follows the GT profile most closely and its residual strip remains nearer to zero, whereas the codec-based baselines show larger band-dependent amplitude errors. This trend persists for WV3 at both 33× and 66× and extends to GF2 and the 102-band Pavia data. The spectral-profile evidence thus complements SAM and : the quantitative gains are not produced by spatial sharpening at the expense of spectral consistency.
Figure 7. Spectral profile comparison on WV3, GF2, and Pavia. (a, b) WV3 with TOP at 33× and 66×, (c) GF2 at 50×, and (d) Pavia at 23.97×. Upper plots: normalized band-wise mean over a fixed ROI; lower strips: deviation from GT; best-ranked baseline of each method family.
Ablation Studies
We ablate four design choices of TOP on the WV3 reduced-resolution data: the tokenizer backbone, BSQ, an additional entropy model, and stage-3 bit-flip training. These components address different system constraints, so they cannot be judged by reconstruction accuracy alone: the backbone controls representation quality and computation, BSQ affects payload and latent predictability, entropy coding trades bitrate for sender-side overhead, and stage 3 controls sensitivity to erroneous bits. Figure 8 summarizes these accuracy–payload–complexity trade-offs, and the numerical values are listed in panels (a)–(e) of Table XVI in the Supplementary Material.
Figure 8. Ablation studies of the key design choices of TOP. WV3 reduced-resolution data; blue: design adopted in TOP, gray: alternative. (a) Backbone (HR-MSI reconstruction). (b) BSQ: reconstruction–fusion PSNR gap (left), payload and fusion-module complexity (right). (c) Additional entropy model with rANS. (d) Degradation under random bit flips after stages 2 and 3. Bars in (b) and (c) are scaled per pair. Values are listed in § 16 of the Supplementary Material.
| (a) Architecture reconstruction | ||||
| Architecture | Params (M) ↓ | GFLOPs ↓ | PSNR ± Std. ↑ | SSIM ± Std. ↑ |
| CNN | 66.721 | 169.96 | 27.42 ± 2.02 | 0.75 ± 0.06 |
| ViT | 62.709 | 219.46 | 27.05 ± 1.82 | 0.75 ± 0.06 |
| Swin | 65.609 | 171.72 | 28.92 ± 2.07 | 0.83 ± 0.04 |
| (b) BSQ payload | ||||
| Method | bppb ↓ | Payload/latent entry ↓ | ||
| TOBT | 0.0625 | 1 bit | ||
| TOBT w/o BSQ | 2.000 | 32 bits | ||
| (c) BSQ reconstruction, fusion, and latent-module complexity | ||||
| Method | Recon. PSNR ↑ | Fusion PSNR ↑ | Params (M) ↓ | GFLOPs ↓ |
| TOBT | 31.59 | 30.32 | 0.61 | 0.31 |
| TOBT w/o BSQ | 34.70 | 32.11 | 19.33 | 9.88 |
| (d) Direct packing versus entropy coding | ||||
| Method | bppb ↓ | Add. params ↓ | Add. GFLOPs/sample ↓ | |
| TOP | 0.062500 | 0.00M | 0.00 | |
| TOBEM+rans | 0.060260 | 10.329096M | 74.769797 | |
| (e) Bit-flip robustness on 20 scenes | ||||
| Stage | Flip | ΔPSNR (dB) ↓ | ΔSAM (deg) ↓ | |
| Stage 2 | 3% | 3.24 | 0.36 | |
| Stage 3 | 3% | 0.76 | −0.11 | |
| Stage 2 | 5% | 5.17 | 0.64 | |
| Stage 3 | 5% | 1.61 | −0.07 |
Table XVI. Numerical ablations on WV3 reduced-resolution data.
Architecture Choice
We compare CNN, Vision Transformer (ViT) [41], and Swin [14] backbones with 66.721M, 62.709M, and 65.609M parameters, respectively, keeping the latent configuration, quantization, and training fixed; this Swin variant is matched to the parameter budget and differs from the dataset-specific tokenizer configurations listed in Table X of the Supplementary Material. As shown in Figure 8(a), Swin achieves the best HR-MSI reconstruction at a computational cost close to that of the CNN and lower than that of ViT, whereas ViT adds computation without a corresponding gain. Swin is therefore adopted for TOBE and TOBD.
Effectiveness of BSQ
BSQ reduces the payload to one bit per latent entry (Figure 8(b)). To test whether it also makes the target latent easier to predict, we compare the fusion PSNR with the PSNR of directly encoding and decoding the GT image with the same tokenizer; their gap indicates how closely latent fusion approaches the reconstruction capability of the latent space. Both PSNRs are computed on the 20 WV3 reduced-resolution test scenes, so the fusion PSNR with BSQ is the 30.32 dB of TOP at 33× in Table II, whereas Table III reports reconstruction on the validation set. Removing BSQ improves reconstruction, but even with a deliberately enlarged and fully trained fusion module, the continuous-latent variant leaves a 2.59 dB gap between reconstruction and fusion. BSQ reduces this gap to 1.27 dB while shrinking the fusion module from 19.33M to 0.61M parameters and from 9.88 to 0.31 giga floating-point operations (GFLOPs). BSQ thus trades part of the reconstruction capacity for a compact payload and a more predictable fusion target.
| Method | Ratio ↑ | PSNR ± Std. ↑ | SSIM ± Std. ↑ | ERGAS ± Std. ↓ | HQNR ± Std. ↑ | ± Std. ↓ | ± Std. ↓ | SAM (deg) ± Std. ↓ |
| BDSD+JPEG2000 [4] | 25.44 ± 3.85 | 0.55 ± 0.16 | 10.85 ± 2.31 | 0.17 ± 0.06 | 0.68 ± 0.06 | 0.48 ± 0.12 | 11.20 ± 2.44 | |
| FusionNet+JPEG2000 [6] | 26.50 ± 3.69 | 0.73 ± 0.09 | 9.39 ± 2.31 | 0.36 ± 0.08 | 0.54 ± 0.08 | 0.23 ± 0.09 | 10.51 ± 2.34 | |
| GLP+JPEG2000 [5] | 26.53 ± 3.84 | 0.68 ± 0.11 | 9.56 ± 2.04 | 0.27 ± 0.06 | 0.63 ± 0.05 | 0.29 ± 0.11 | 10.64 ± 2.42 | |
| GS+JPEG2000 [3] | 27.70 ± 3.74 | 0.71 ± 0.11 | 8.36 ± 1.78 | 0.47 ± 0.09 | 0.43 ± 0.08 | 0.18 ± 0.09 | 10.52 ± 2.49 | |
| MF+JPEG2000 [23] | 21.26× | 26.59 ± 3.88 | 0.67 ± 0.12 | 9.49 ± 2.08 | 0.27 ± 0.08 | 0.62 ± 0.07 | 0.29 ± 0.12 | 10.44 ± 2.34 |
| U2Net+JPEG2000 [7] | 26.62 ± 3.63 | 0.70 ± 0.10 | 9.34 ± 2.05 | 0.33 ± 0.08 | 0.56 ± 0.08 | 0.26 ± 0.09 | 10.39 ± 2.23 | |
| WFANet+JPEG2000 [8] | 25.64 ± 3.92 | 0.64 ± 0.13 | 10.54 ± 2.45 | 0.20 ± 0.08 | 0.68 ± 0.07 | 0.38 ± 0.11 | 10.62 ± 2.34 | |
| ZS-PAN+JPEG2000 [29] | 26.02 ± 3.54 | 0.62 ± 0.14 | 9.92 ± 1.70 | 0.23 ± 0.09 | 0.65 ± 0.08 | 0.37 ± 0.14 | 11.00 ± 2.44 | |
| ADWM+JPEG2000 [26] | 26.02 ± 5.52 | 0.71 ± 0.14 | 11.23 ± 5.66 | 0.38 ± 0.12 | 0.53 ± 0.10 | 0.21 ± 0.12 | 15.19 ± 7.37 | |
| ARConv+JPEG2000 [27] | 26.70 ± 3.84 | 0.70 ± 0.11 | 9.33 ± 1.95 | 0.29 ± 0.08 | 0.60 ± 0.07 | 0.29 ± 0.10 | 10.43 ± 2.30 | |
| TOP (ours) | 33× | 30.32 ± 2.83 | 0.82 ± 0.04 | 6.00 ± 1.23 | 0.76 ± 0.14 | 0.19 ± 0.11 | 0.08 ± 0.06 | 6.00 ± 1.38 |
| TOP (ours) | 66× | 28.88 ± 3.06 | 0.75 ± 0.07 | 7.13 ± 1.48 | 0.69 ± 0.13 | 0.23 ± 0.11 | 0.11 ± 0.05 | 6.69 ± 1.56 |
Table II. WV3 reduced-resolution results under JPEG2000 (mean ± std.).
| Dataset | Modality | bppb ↓ | PSNR ± Std. ↑ | SSIM ± Std. ↑ |
| WV3 (33×) | HR-MSI | 0.063 | 30.17 ± 4.42 | 0.83 ± 0.08 |
| LR-MSI | 0.063 | 39.61 ± 3.18 | 0.97 ± 0.01 | |
| PAN | 0.500 | 34.88 ± 5.01 | 0.91 ± 0.04 | |
| WV3 (66×) | HR-MSI | 0.031 | 28.22 ± 4.59 | 0.74 ± 0.12 |
| LR-MSI | 0.031 | 37.14 ± 3.73 | 0.95 ± 0.02 | |
| PAN | 0.250 | 31.94 ± 5.43 | 0.83 ± 0.08 | |
| GF2 | HR-MSI | 0.063 | 33.46 ± 3.75 | 0.87 ± 0.06 |
| LR-MSI | 0.063 | 40.31 ± 2.85 | 0.97 ± 0.02 | |
| PAN | 0.250 | 33.46 ± 3.94 | 0.87 ± 0.05 | |
| Pavia | HR-HSI | 0.039 | 34.85 ± 3.50 | 0.92 ± 0.02 |
| LR-HSI | 0.039 | 43.90 ± 2.57 | 0.99 ± 0.00 | |
| PAN | 4.000 | 40.86 ± 2.93 | 0.97 ± 0.01 |
Table III. TOBT Reconstruction on the Validation Sets (Mean ± Std.; BPPB: Modality-Specific Payload, Eq. (19))
Trade-Off of an Additional Entropy Model
A neural entropy model, which drives rANS [16] or arithmetic coding with estimated symbol probabilities [13], is most effective when the latent retains considerable statistical redundancy, whereas TOBT already compresses the HR information
Multispectral Pansharpening on GF2
| Method | Ratio ↑ | PSNR ± Std. ↑ | SSIM ± Std. ↑ | ERGAS ± Std. ↓ | HQNR ± Std. ↑ | ± Std. ↓ | ± Std. ↓ | SAM (deg) ± Std. ↓ |
| BDSD+JPEG2000 [4] | 20.33× | 24.33 ± 2.19 | 0.63 ± 0.08 | 5.78 ± 0.99 | 0.09 ± 0.02 | 0.80 ± 0.02 | 0.57 ± 0.06 | 5.32 ± 0.76 |
| FusionNet+JPEG2000 [6] | 26.73 ± 2.16 | 0.81 ± 0.03 | 4.36 ± 0.74 | 0.46 ± 0.06 | 0.50 ± 0.06 | 0.08 ± 0.03 | 4.97 ± 0.80 | |
| GLP+JPEG2000 [5] | 24.56 ± 2.13 | 0.69 ± 0.08 | 5.62 ± 0.96 | 0.12 ± 0.03 | 0.79 ± 0.03 | 0.46 ± 0.08 | 5.28 ± 0.77 | |
| GS+JPEG2000 [3] | 26.99 ± 1.77 | 0.78 ± 0.05 | 4.16 ± 0.64 | 0.53 ± 0.07 | 0.36 ± 0.06 | 0.18 ± 0.05 | 5.36 ± 1.14 | |
| MF+JPEG2000 [23] | 25.04 ± 2.18 | 0.74 ± 0.06 | 5.32 ± 0.93 | 0.20 ± 0.04 | 0.73 ± 0.04 | 0.29 ± 0.08 | 5.20 ± 0.77 | |
| U2Net+JPEG2000 [7] | 28.53 ± 1.65 | 0.81 ± 0.03 | 3.41 ± 0.45 | 0.63 ± 0.07 | 0.31 ± 0.06 | 0.09 ± 0.04 | 4.29 ± 0.60 | |
| WFANet+JPEG2000 [8] | 25.65 ± 2.23 | 0.76 ± 0.04 | 4.88 ± 0.85 | 0.27 ± 0.03 | 0.65 ± 0.04 | 0.23 ± 0.05 | 4.99 ± 0.76 | |
| ADWM+JPEG2000 [26] | 25.95 ± 2.29 | 0.79 ± 0.04 | 4.74 ± 0.89 | 0.38 ± 0.05 | 0.56 ± 0.05 | 0.13 ± 0.05 | 5.10 ± 0.84 | |
| ARConv+JPEG2000 [27] | 24.82 ± 2.24 | 0.72 ± 0.06 | 5.41 ± 0.96 | 0.14 ± 0.02 | 0.77 ± 0.02 | 0.37 ± 0.06 | 5.10 ± 0.75 | |
| ZS-PAN+JPEG2000 [29] | 24.52 ± 2.05 | 0.67 ± 0.08 | 5.63 ± 0.90 | 0.12 ± 0.03 | 0.78 ± 0.03 | 0.47 ± 0.08 | 5.34 ± 0.77 | |
| TOP (ours) | 50× | 31.37 ± 1.90 | 0.83 ± 0.05 | 2.63 ± 0.50 | 0.83 ± 0.03 | 0.11 ± 0.03 | 0.07 ± 0.02 | 2.46 ± 0.42 |
Table IV. GF2 Reduced-Resolution Results Under JPEG2000 (Mean ± Std.).
Hyperspectral Pansharpening on Pavia
into a tight binary bottleneck. We therefore code the fused binary latent that TOP downlinks for WV3, which uses 0.0625 bppb with direct bit packing, with a recent 10.329096M-parameter dictionary-based entropy model [51] and rANS. As shown in Figure 8(c), the payload decreases only from 0.062500 to 0.060260 bppb (about 3.58%), at the cost of 74.769797 additional GFLOPs/sample for probability prediction alone, plus GPU synchronization, data transfer, CPU rANS coding,
Spectral Profile Analysis
and packet assembly on the sender side. This overhead is disproportionate to the gain, which supports the entropy-coding-free design of TOP.
Stage-3 Training Strategy on Robustness
The fused code is both the output of onboard fusion and the downlink payload, so a wrong bit may come from imperfect latent prediction or from transmission errors. We randomly flip 3% or 5% of the signs of the fused code on the 20 WV3 reduced-resolution test scenes and decode it with TOBD after stage 2 or stage 3, keeping the encoder, latent fusion module, and unperturbed code unchanged. As shown in Figure 8(d), stage-3 training reduces the PSNR drop from 3.24 to 0.76 dB at 3% flips and from 5.17 to 1.61 dB at 5% flips. Under the same flips, stage 2 increases SAM by 0.36 and 0.64 degrees, whereas after stage 3 SAM changes by only −0.11 and −0.07 degrees. The small negative values can be interpreted as negligible sample-level variation rather than a benefit from corruption; the relevant result is that spectral fidelity no longer
deteriorates systematically. Bit-flip training therefore makes TOBD markedly less sensitive to bit errors.
Discussion
Mechanism of Latent-Space Fusion
TOP works because one binary latent space serves reconstruction, fusion, and transmission. In stage 1, the shared tokenizer must reconstruct PAN, LR-MSI/LR-HSI, and HR-MSI/HR-HSI images under the same binary capacity, so it keeps the structures needed for both spatial detail and spectral recovery and discards pixel-level redundancy that is costly to transmit. In this space, the latent fusion module only has to predict a binary state at each aligned latent location, instead of synthesizing every output pixel and band.
The BSQ ablation shows the cost and the benefit of the design (§ 5-B). Without BSQ, the tokenizer reconstructs better on its own, but the fusion target is continuous. It then needs a much larger fusion module and still leaves a larger reconstruction–fusion gap. BSQ gives up part of the standalone reconstruction capacity and obtains a target that is compact and easy to predict. The training stages build on this. Stage 2 fine-tunes TOBD on the HR code, and stage 3 adds bit-flip augmentation so that TOBD tolerates wrong bits from imperfect latent prediction or from the channel. With TOBT frozen, latent-fusion training then makes the onboard prediction match the HR code. The fused code is therefore already the payload for the channel, not an intermediate image that must be compressed again.
Improvements Over Existing Pipelines
Ground-side fusion after compression and onboard pixel-domain fusion before compression are both limited by an objective mismatch. In the first, the codec discards information before fusion, and the fusion model is usually trained on clean inputs, so a better fusion model cannot recover what the downlink has already removed. In the second, the fusion model produces a full-resolution image without knowing how a generic codec will quantize it, and the codec minimizes image distortion without knowing which structures matter for pansharpening. Either way, two stages with different objectives are concatenated.
TOP removes this mismatch by making the fused latent code the payload. Tokenization sets what can be represented within the bit budget, onboard fusion predicts that representation, and the ground decoder reconstructs the image from the same code that was optimized during training. This agrees with the results against compress-then-fuse baselines in the main text and against onboard fusion followed by JPEG2000 or HEVC in Sec. S9 of the Supplementary Material. The gain therefore comes from coordinating representation, fusion, and transmission, not from one fusion network or one codec alone.
Deployment Flexibility
The binary payload keeps onboard processing simple. It ends with sign selection and bit packing, and it needs neither full-resolution reconstruction nor a learned entropy model (§ 5-C). The fixed payload size is known before transmission, which helps buffer allocation and link scheduling, and decoding stays on the ground.
The shared encoder supports two transmission policies. By default, TOP sends the fused latent. If downstream tasks need the source observations, the satellite can instead send the PAN and spectral latents, and the ground station reconstructs them and applies an existing fusion method. Table VIII shows that this policy remains viable: on TOBT-decoded inputs, FusionNet reaches 29.95 dB on GF2 at and 29.32 dB on WV3 at . These ratios are about half those of TOP because two equal-sized source latents are sent. TOP sends only the fused latent, at and , while improving PSNR by 1.42 dB on GF2 and 1.00 dB on WV3. The choice is therefore explicit: transmit the source latents for reuse, or transmit the fused latent for the best rate–quality trade-off.
Table VIII. Ground-Side Fusion After TOBT-Only Transmission (Mean Std.)
TOP also has two practical limitations: (1) the tokenizer is trained per sensor, and (2) training takes several stages. Sec. S10 of the Supplementary Material discusses them.
Conclusion
This work presents TOP, which treats pansharpening for satellite downlink as a joint design of compact latent coding, onboard fusion, and transmission. TOBE with a BSQ bottleneck produces a binary latent code that can be reconstructed, predicted, and bit-packed directly. TOBD decodes the fused code on the ground and can tolerate occasional bit errors. On WV3, GF2, and Pavia datasets, TOP produces higher fusion quality than pansharpening methods paired with common codecs, at compression ratios 1.2 to 9.9 times those of these baselines. For example, on WV3 it reaches 30.32 dB PSNR at , whereas the best JPEG2000 baseline reaches 27.70 dB at . These findings suggest that future remote-sensing systems could benefit from co-designing task processing and transmission, rather than treating downlink transmission as a separate stage. Future work will simplify the training procedure and improve generalization across sensors.
Supplementary Material for TOP: Transmission-Oriented Remote Sensing Pansharpening
Zi-Han Cao; Jing-Ze Liang; Shao-Jie Wu; Jin-Cheng Di; Liang-Jian Deng; Gemine Vivone
This supplement details BSQ and the latent-fusion loss (Secs. S1–S2), implementation and evaluation protocols (Secs. S3–S4), and additional JPEG2000, HEVC, full-resolution, ablation, and onboard-fusion results (Secs. S5–S9). Sec. S10 discusses limitations. In all tables, bold and underlined entries are the best and second best; repeated-sample results are mean standard deviation. Baseline references appear in the main manuscript. In reduced-resolution figures, prediction and error-map rows alternate; darker blue indicates lower error, and green boxes mark enlarged regions.
Details of Binary Spherical Quantization
Following BSQ [15], inference and downlink use only the hard normalized tensor ; the soft assignments below are training auxiliaries.
a) Hard Spherical Binary Code: For at latent location ,
where and reshape maps the raster-ordered vectors to the latent grid. For each of the three modalities, define
with numerical constant . Positive normalization preserves the sign, and the straight-through estimator (STE) is
Thus, the forward value is , while the identity surrogate Jacobian is used only for back-propagation; sg denotes stop-gradient. Bit packing stores signs, and unpacking restores the two normalized values. The corresponding implicit, unlearned codebook is
b) Geometry and Packed Payload: The spherical normalization makes binary disagreement directly measurable. For two codewords, with Hamming distance ,
A sign error therefore causes a fixed, bounded latent displacement, and latent prediction errors have the same form as transmission bit errors. The packed tensor contains bits, giving for target bands. Its payload is fixed by and requires no codebook indices, floating-point magnitudes, or probability tables. The dot-product identity also keeps similarity in the fixed interval across latent dimensions, so changing the rate configuration does not change the scale presented to the decoder.
c) Soft Assignment for Entropy Regularization: During training, inverse temperature defines the nearest-code distribution
The second form follows because codeword-independent squared-norm terms cancel. The Cartesian binary codebook factorizes by channel:
In practice, split the channels into disjoint groups of channels. Let , for , and . Then
This marginal preserves the global scaling and reduces explicit assignment from states to grouped states.
d) Entropy and Commitment Terms: For the batch and latent locations ,
while the entropy of average code usage is
where . The first term favors decisive local assignments; the second prevents collapse through broad group-code usage. TOP minimizes
and the commitment term
where is the full batch and latent grid. This moves the continuous feature toward the fixed hard code. Their combination is Eq. (10) of the main manuscript. Tab. IX gives all training coefficients; they do not change the deterministic hard code. The effective coefficients of , , and are for WV3 33×, for WV3 66×, and for GF2 and Pavia.
| Coefficient | WV3 33× | WV3 66× | GF2 50× | Pavia 23.97× |
| , , , | 0.1 | 0.1 | 1 | 1 |
| 1 | 1 | 1 | 1 | |
| 0.05 | 0.1 | 0.1 | 0.1 |
Table IX. BSQ training coefficients; training uses hard BSQ with STE, whereas inference uses hard BSQ only.
e) Boundary With Entropy Coding: The entropy regularizer is used only for optimization; it produces no probability model, side information, or arithmetic-coded stream. Deployment discards , , and and directly bit-packs the signs of . A neural entropy model would additionally require probability inference, packet coding, and decoding.
Binary Cross-Entropy Loss of the Latent Fusion Module
Sec. III-D defines the single logit tensor . Its first axis represents the negative or positive state, and is one logit. For each target entry , define
and lift it to two complementary channels:
Thus, and . The resulting matches , so requires no broadcasting. For scalar logit and target ,
which evaluates the exponential only at a nonpositive argument and avoids an explicit sigmoid. The latent-fusion objective is
This is complementary one-vs-rest BCE, rather than softmax cross-entropy. The frozen values and map to and . At inference, and , preserving the training state convention; no sigmoid precedes .
Implementation Details
All configurations use Swin-based encoders/decoders, grouped BSQ with eight channels per group, and a band-adaptive stride-2 embedding. The kernel layout is output×input×height×width and is interpolated along the input-channel dimension. The factor applies to both spatial axes, and stage counts include stages without scale changes. Tab. X retains all architecture and configuration-specific training values.
The fusion module concatenates spectral and PAN codes, maps channels to two logits per target bit, and uses 64-channel residual blocks with two convolutions, residual addition, and GELU [52]. With the tokenizer frozen, it is trained to convergence using only Sec. S2 (weight 1; image-domain losses disabled), seed 2025.
All tokenizers use EMA weights and three stages. Stage 1 jointly trains encoder and decoder for 40,000 iterations on mixed PAN, upsampled LR spectral, and HR reference images. Stages 2 and 3 initialize from the preceding EMA, freeze the encoder, and refine the decoder for 20,000 iterations each with batch 16 and MSE/Sobel-gradient/SAM weights 1/0.1/0.1; quantizer loss is disabled. Stage 3 ramps bit-flip probability from to over 2,000 iterations and holds it fixed. The stage-3 EMA is final. AdamW uses learning rate , 500 tokenizer warm-up iterations, and weight decay 0.05/0.01 for tokenizer/fusion. No adversarial, perceptual, or LeCAM loss is used. Training uses PyTorch, bfloat16, and one NVIDIA RTX 4090.
Payload is . WV3 33× uses 0.5 bit/pixel (0.0625 bppb); WV3 66× and GF2 50× use 0.25 bit/pixel (0.03125 and 0.0625 bppb); Pavia uses 4 bits/pixel, i.e., bppb and 23.9688×. WV3 33× sets z_channel. Pavia pads to at most 128 channels while using 102 spectral and 64 latent channels.
Datasets and Evaluation Protocols
WV3 (8 bands) and GF2 (4 bands) come from PanCollection [43],1 and the 102-band ROSIS Pavia Center data come from HyperPanCollection/HSI and Hyper-DSNet [44].2 All use scale factor four. Tab. XI gives every split used in training and evaluation; HR denotes the common spatial size of the reference or upsampled spectral image and PAN, and LR denotes the native spectral size.
Arrays are channel-first . The keys gt/GT, lms/LMS, ms/MS, and pan/PAN denote the HR reference, upsampled LR spectral image, native LR spectral image, and single-band PAN; WV3/GF2 use lowercase and Pavia uppercase. Communication accounting uses the native ms/MS. WV3/GF2 reduced-resolution tests follow Wald’s protocol [45]; their GT supports PSNR, SSIM, ERGAS, and SAM, whereas full-resolution tests lack GT. Whenever HQNR, , and are reported, their anchors are the original spectral and PAN observations. Pavia Full retains GT, so both metric groups are available and are averaged over its two scenes. Its MAT variable has shape and is permuted to before adding the sample dimension.
| Dataset | Split | HR | LR | GT | ||
| WV3 | Train | 9714 | 8 | 64 | 16 | Yes |
| WV3 | Val. | 1080 | 8 | 64 | 16 | Yes |
| WV3 | Reduced | 20 | 8 | 256 | 64 | Yes |
| WV3 | Full | 20 | 8 | 512 | 128 | No |
| GF2 | Train | 19809 | 4 | 64 | 16 | Yes |
| GF2 | Val. | 2201 | 4 | 64 | 16 | Yes |
| GF2 | Reduced | 20 | 4 | 256 | 64 | Yes |
| GF2 | Full | 20 | 4 | 512 | 128 | No |
| Pavia | Train | 1512 | 102 | 64 | 16 | Yes |
| Pavia | Val. | 168 | 102 | 64 | 16 | Yes |
| Pavia | Full | 2 | 102 | 400 | 100 | Yes |
Table XI. Dataset splits and spatial dimensions.
Qualitative Comparisons With All Baselines Under JPEG2000
For clarity, Figure 6 of the main manuscript shows only the best-performing baselines of each dataset. Figure 9, Figure 10, and Figure 11 provide the comparisons with all baselines under the JPEG2000 protocol for the same WV3, GF2, and Pavia scenes.

Figure 9. WV3 reduced-resolution JPEG2000 comparison. Baselines: 21.26×; TOP: 33×.

Figure 10. GF2 reduced-resolution JPEG2000 comparison. Baselines: 20.33×; TOP: 50×.

Figure 11. Pavia Full JPEG2000 comparison. Baselines: 19.62×; TOP: 23.97×.
Comparisons Under the HEVC Protocol
The main manuscript reports the reduced-resolution WV3 and GF2 HEVC results in Table V and Table VI. Table XII gives the Pavia results, while Figure 12–Figure 14 provide the corresponding visual comparisons. The pixel-domain baselines operate at 6.69×, 8.94×, and 7.79× on WV3, GF2, and Pavia, whereas TOP uses 33×, 50×, and 23.97×, respectively. Metric anchors follow § 12; across all three datasets, TOP provides the strongest quantitative results and preserves clearer structures and more stable spectral appearance.
| Method | Ratio ↑ | PSNR ± Std. ↑ | SSIM ± Std. ↑ | ERGAS ± Std. ↓ | HQNR ± Std. ↑ | ± Std. ↓ | ± Std. ↓ | SAM (deg) ± Std. ↓ |
| BDSD+HEVC | 6.69× | 26.53 ± 3.65 | 0.60 ± 0.14 | 9.56 ± 2.05 | 0.31 ± 0.08 | 0.50 ± 0.10 | 0.39 ± 0.08 | 10.72 ± 2.52 |
| FusionNet+HEVC | 26.59 ± 3.55 | 0.62 ± 0.12 | 9.31 ± 2.07 | 0.34 ± 0.09 | 0.49 ± 0.11 | 0.34 ± 0.07 | 9.71 ± 2.36 | |
| GLP+HEVC | 26.79 ± 3.58 | 0.62 ± 0.13 | 9.24 ± 1.97 | 0.31 ± 0.07 | 0.51 ± 0.09 | 0.37 ± 0.07 | 9.52 ± 2.30 | |
| GS+HEVC | 27.22 ± 3.53 | 0.64 ± 0.12 | 8.82 ± 1.91 | 0.38 ± 0.10 | 0.46 ± 0.11 | 0.31 ± 0.06 | 9.79 ± 2.36 | |
| MF+HEVC | 26.92 ± 3.58 | 0.62 ± 0.13 | 9.13 ± 2.00 | 0.32 ± 0.09 | 0.51 ± 0.10 | 0.35 ± 0.07 | 9.35 ± 2.26 | |
| U2Net+HEVC | 26.65 ± 3.54 | 0.61 ± 0.13 | 9.36 ± 2.05 | 0.33 ± 0.09 | 0.49 ± 0.11 | 0.36 ± 0.07 | 9.48 ± 2.29 | |
| WFANet+HEVC | 26.24 ± 3.70 | 0.59 ± 0.15 | 9.82 ± 2.17 | 0.27 ± 0.07 | 0.55 ± 0.09 | 0.41 ± 0.08 | 9.65 ± 2.35 | |
| ZS-PAN+HEVC | 26.57 ± 3.44 | 0.60 ± 0.14 | 9.40 ± 1.85 | 0.28 ± 0.09 | 0.54 ± 0.11 | 0.40 ± 0.08 | 9.60 ± 2.32 | |
| ADWM+HEVC | 26.83 ± 3.70 | 0.63 ± 0.13 | 9.18 ± 2.19 | 0.37 ± 0.09 | 0.46 ± 0.11 | 0.32 ± 0.07 | 10.13 ± 3.12 | |
| ARConv+HEVC | 26.64 ± 3.59 | 0.61 ± 0.14 | 9.39 ± 2.00 | 0.30 ± 0.08 | 0.52 ± 0.10 | 0.38 ± 0.08 | 9.49 ± 2.31 | |
| TOP (ours) | 33× | 30.32 ± 2.83 | 0.82 ± 0.04 | 6.00 ± 1.23 | 0.76 ± 0.14 | 0.19 ± 0.11 | 0.08 ± 0.06 | 6.00 ± 1.38 |
| TOP (ours) | 66× | 28.88 ± 3.06 | 0.75 ± 0.07 | 7.13 ± 1.48 | 0.69 ± 0.13 | 0.23 ± 0.11 | 0.11 ± 0.05 | 6.69 ± 1.56 |
Table V. WV3 Reduced-Resolution Results Under HEVC (Mean ± Std.).
| Method | Ratio ↑ | PSNR ± Std. ↑ | SSIM ± Std. ↑ | ERGAS ± Std. ↓ | HQNR ± Std. ↑ | ± Std. ↓ | ± Std. ↓ | SAM (deg) ± Std. ↓ |
| BDSD+HEVC | 8.94× | 28.03 ± 1.91 | 0.70 ± 0.07 | 3.77 ± 0.65 | 0.43 ± 0.06 | 0.40 ± 0.05 | 0.29 ± 0.06 | 3.95 ± 0.59 |
| FusionNet+HEVC | 28.68 ± 1.84 | 0.73 ± 0.06 | 3.47 ± 0.56 | 0.50 ± 0.06 | 0.37 ± 0.05 | 0.21 ± 0.04 | 3.85 ± 0.57 | |
| GLP+HEVC | 27.78 ± 1.86 | 0.69 ± 0.07 | 3.88 ± 0.65 | 0.37 ± 0.06 | 0.46 ± 0.05 | 0.32 ± 0.06 | 3.91 ± 0.57 | |
| GS+HEVC | 28.89 ± 1.71 | 0.73 ± 0.06 | 3.36 ± 0.54 | 0.61 ± 0.05 | 0.31 ± 0.04 | 0.11 ± 0.05 | 3.81 ± 0.60 | |
| MF+HEVC | 28.16 ± 1.89 | 0.71 ± 0.07 | 3.72 ± 0.62 | 0.43 ± 0.06 | 0.42 ± 0.05 | 0.26 ± 0.04 | 3.87 ± 0.57 | |
| U2Net+HEVC | 29.13 ± 1.72 | 0.72 ± 0.06 | 3.25 ± 0.52 | 0.61 ± 0.05 | 0.29 ± 0.04 | 0.14 ± 0.04 | 3.55 ± 0.49 | |
| WFANet+HEVC | 28.05 ± 1.91 | 0.69 ± 0.07 | 3.72 ± 0.61 | 0.41 ± 0.05 | 0.42 ± 0.05 | 0.29 ± 0.05 | 3.82 ± 0.56 | |
| ADWM+HEVC | 28.52 ± 1.87 | 0.72 ± 0.06 | 3.55 ± 0.58 | 0.47 ± 0.05 | 0.38 ± 0.05 | 0.24 ± 0.04 | 3.86 ± 0.57 | |
| ARConv+HEVC | 27.50 ± 1.98 | 0.67 ± 0.08 | 3.98 ± 0.68 | 0.34 ± 0.06 | 0.47 ± 0.05 | 0.36 ± 0.05 | 3.91 ± 0.57 | |
| ZS-PAN+HEVC | 27.72 ± 1.85 | 0.69 ± 0.07 | 3.88 ± 0.64 | 0.37 ± 0.05 | 0.45 ± 0.05 | 0.33 ± 0.05 | 3.93 ± 0.57 | |
| TOP (ours) | 50× | 31.37 ± 1.90 | 0.83 ± 0.05 | 2.63 ± 0.50 | 0.83 ± 0.03 | 0.11 ± 0.03 | 0.07 ± 0.02 | 2.46 ± 0.42 |
Table VI. GF2 Reduced-Resolution Results Under HEVC (Mean ± Std.).

Figure 12. WV3 reduced-resolution HEVC comparison. Baselines: 6.69×; TOP: 33×.
| Method | Ratio | PSNR | SSIM | ERGAS | HQNR | SAM (deg) | ||
| AWLP+HEVC | 7.79 | 24.47 | 0.57 | 10.79 | 0.67 | 0.16 | 0.20 | 13.36 |
| BDSD-PC+HEVC | 24.28 | 0.49 | 11.05 | 0.64 | 0.16 | 0.24 | 12.32 | |
| BT-H+HEVC | 25.11 | 0.67 | 10.38 | 0.60 | 0.33 | 0.11 | 12.65 | |
| GSA+HEVC | 26.64 | 0.64 | 8.74 | 0.74 | 0.17 | 0.11 | 12.22 | |
| MF+HEVC | 24.77 | 0.59 | 10.43 | 0.70 | 0.16 | 0.16 | 12.67 | |
| PRACS+HEVC | 25.10 | 0.56 | 10.07 | 0.73 | 0.17 | 0.12 | 12.33 | |
| uTDSP+HEVC | 24.50 | 0.44 | 11.45 | 0.58 | 0.16 | 0.31 | 13.88 | |
| DIP-HyperKite+HEVC | 25.23 | 0.61 | 9.94 | 0.77 | 0.17 | 0.08 | 12.61 | |
| HyperTransformer+HEVC | 18.71 | 0.17 | 21.71 | 0.40 | 0.19 | 0.50 | 35.16 | |
| R-PNN+HEVC | 24.67 | 0.57 | 10.59 | 0.71 | 0.16 | 0.16 | 12.87 | |
| DM-ZS+HEVC | 24.56 | 0.64 | 11.95 | 0.70 | 0.23 | 0.09 | 13.60 | |
| -PNN+HEVC | 23.86 | 0.50 | 11.63 | 0.62 | 0.16 | 0.27 | 13.36 | |
| TOP (ours) | 23.97 | 29.72 | 0.83 | 5.91 | 0.85 | 0.14 | 0.02 | 6.39 |
Table XII. PAVIA FULL HEVC RESULTS AVERAGED OVER TWO SCENES.

Figure 14. Pavia Full HEVC comparison. Baselines: 7.79 (displayed as 8); TOP: 23.97 (displayed as 24).
Full-Resolution Validation on WV3 and GF2
Full-resolution HR-MSI references are unavailable, so WV3/GF2 evaluation uses HQNR, , and against the
original lossless PAN and LR-MSI. Pixel-domain baselines receive codec-decoded observations, TOP transmits its fu slightly lower , but lower HQNR and weaker spectral consistency. Figure 15–18 show the corresponding qualitative trend.

Figure 15. WV3 full-resolution JPEG2000 comparison. Baselines: 28.43; TOP: 33 and 66.

Figure 13. GF2 reduced-resolution HEVC comparison. Baselines: 8.94; TOP: 50.
Numerical Results of the Ablation Studies
All ablations use WV3 reduced-resolution data. Table XVI reports (a) backbone, (b)–(c) BSQ payload/quality/complexity, (d) direct packing versus entropy model+rANS on the same latent, and (e) stage-3 robustness. The continuous-latent model is fully retrained with a larger fusion module; its entries use 32-bit latents. Both PSNR columns in (c) use the same 20 test scenes, unlike the validation value in Table III. In (e), PSNR is
| Method | Ratio | HQNR | ||
| JPEG2000 | ||||
| BDSD+JPEG2000 | 28.43 | 0.7925 | 0.1636 | 0.0524 |
| FusionNet+JPEG2000 | 0.7336 | 0.1839 | 0.1011 | |
| GLP+JPEG2000 | 0.7342 | 0.2043 | 0.0769 | |
| GS+JPEG2000 | 0.7983 | 0.1756 | 0.0322 | |
| MF+JPEG2000 | 0.7508 | 0.1938 | 0.0686 | |
| U2Net+JPEG2000 | 0.7143 | 0.1880 | 0.1204 | |
| WFANet+JPEG2000 | 0.6447 | 0.2216 | 0.1723 | |
| ADWM+JPEG2000 | 0.7386 | 0.1842 | 0.0947 | |
| ARConv+JPEG2000 | 0.6718 | 0.2158 | 0.1436 | |
| ZS-PAN+JPEG2000 | 0.7082 | 0.2049 | 0.1101 | |
| TOP (ours, 33) | 33 | 0.8724 | 0.0984 | 0.0329 |
| TOP (ours, 66) | 66 | 0.8481 | 0.1184 | 0.0389 |
| HEVC | ||||
| BDSD+HEVC | 23.64 | 0.3821 | 0.4182 | 0.3502 |
| FusionNet+HEVC | 0.3937 | 0.4198 | 0.3236 | |
| GLP+HEVC | 0.3791 | 0.4466 | 0.3175 | |
| GS+HEVC | 0.5420 | 0.3300 | 0.1939 | |
| MF+HEVC | 0.4007 | 0.4286 | 0.3033 | |
| U2Net+HEVC | 0.4108 | 0.4066 | 0.3111 | |
| WFANet+HEVC | 0.3340 | 0.4637 | 0.3826 | |
| ADWM+HEVC | 0.4383 | 0.3835 | 0.2900 | |
| ARConv+HEVC | 0.3550 | 0.4510 | 0.3583 | |
| ZS-PAN+HEVC | 0.3795 | 0.4382 | 0.3309 | |
| TOP (ours, 33) | 33 | 0.8724 | 0.0984 | 0.0329 |
| TOP (ours, 66) | 66 | 0.8481 | 0.1184 | 0.0389 |
Table XIII. WV3 FULL-RESOLUTION RESULTS UNDER BOTH CODECS.
| Method | Ratio | HQNR | ||
| JPEG2000 | ||||
| BDSD+JPEG2000 | 27.46 | 0.8278 | 0.1366 | 0.0416 |
| FusionNet+JPEG2000 | 0.8304 | 0.1366 | 0.0382 | |
| GLP+JPEG2000 | 0.7959 | 0.1726 | 0.0383 | |
| GS+JPEG2000 | 0.7112 | 0.1787 | 0.1344 | |
| MF+JPEG2000 | 0.7977 | 0.1689 | 0.0406 | |
| U2Net+JPEG2000 | 0.7999 | 0.1301 | 0.0800 | |
| WFANet+JPEG2000 | 0.8150 | 0.1535 | 0.0374 | |
| ADWM+JPEG2000 | 0.8311 | 0.1382 | 0.0355 | |
| ARConv+JPEG2000 | 0.7298 | 0.1846 | 0.1048 | |
| ZS-PAN+JPEG2000 | 0.8094 | 0.1597 | 0.0367 | |
| TOP (ours) | 50 | 0.8511 | 0.1031 | 0.0521 |
| HEVC | ||||
| BDSD+HEVC | 31.69 | 0.5165 | 0.3248 | 0.2365 |
| FusionNet+HEVC | 0.5704 | 0.3048 | 0.1801 | |
| GLP+HEVC | 0.4497 | 0.3841 | 0.2709 | |
| GS+HEVC | 0.6606 | 0.2905 | 0.0694 | |
| MF+HEVC | 0.4947 | 0.3616 | 0.2258 | |
| U2Net+HEVC | 0.6819 | 0.2453 | 0.0971 | |
| WFANet+HEVC | 0.4861 | 0.3494 | 0.2536 | |
| ADWM+HEVC | 0.5499 | 0.3164 | 0.1964 | |
| ARConv+HEVC | 0.4152 | 0.3978 | 0.3116 | |
| ZS-PAN+HEVC | 0.4748 | 0.3592 | 0.2600 | |
| TOP (ours) | 50 | 0.8511 | 0.1031 | 0.0521 |
Table XIV. GF2 FULL-RESOLUTION RESULTS UNDER BOTH CODECS.

Figure 16. WV3 full-resolution HEVC comparison. Baselines: 23.64; TOP: 33 and 66.

Figure 17. GF2 full-resolution JPEG2000 comparison. Baselines: 27.46; TOP: 50.
no-flip minus perturbed PSNR and SAM is perturbed minus no-flip SAM; lower is better.
Panel (a) isolates the backbone choice. Swin improves reconstruction PSNR by 1.50 dB over CNN and 1.87 dB over ViT, while its 171.72 GFLOPs remain close to CNN and below ViT; the SSIM increase from 0.75 to 0.83 confirms better structural preservation. It therefore provides the strongest representation without using the most expensive candidate.
Panels (b)–(c) clarify the BSQ trade-off. Removing BSQ raises reconstruction and fusion PSNR, but changes each latent entry from one to 32 bits and requires about 31.7 the fusion parameters and 31.9 the GFLOPs. Its reconstruction–fusion gap also grows from 1.27 to 2.59 dB. BSQ therefore sacrifices some reconstruction ceiling for a target that is much smaller and easier for the onboard module to predict.
Panels (d)–(e) address the remaining communication risks. Entropy modeling saves only 3.58% bppb while adding 10.33M parameters and 74.77 GFLOPs per sample, supporting direct packing. Stage-3 training reduces the PSNR loss under 3% and 5% flips by 76.5% and 68.9%, and leaves SAM nearly unchanged. The small negative SAM values reflect sample-level variation rather than a benefit from corruption; the meaningful result is the absence of systematic spectral degradation. The three components consequently have distinct roles: BSQ controls payload and prediction complexity, direct packing avoids disproportionate sender overhead, and decoder refinement limits residual bit errors.
Comparison With Onboard Pansharpening Followed by Downlink Compression
This comparison tests whether moving conventional pansharpening on board is sufficient without redesigning the transmitted representation. Each baseline fuses the uncompressed PAN and LR-MSI into a full-resolution WV3 MSI and then applies JPEG2000 or HEVC; TOP predicts and bit-packs one fused latent. All methods use the same 20 scenes and ratio definition, so their payloads are measured against the same native observations.
Table XV reports every quality metric and measured fusion time. Against JPEG2000, TOP improves the strongest baseline PSNR by 1.08 dB at 1.65 its compression ratio, while raising the best HQNR from 0.6356 to 0.7552. Against HEVC, it improves the strongest PSNR by 1.66 dB at about 4.98 the ratio. Its lower and under both codecs show that the rate gain does not trade spectral consistency for spatial sharpness.
| Method | Ratio ↑ | Fusion time (ms/img) ↓ | HQNR ↑ | ↓ | ↓ | PSNR ↑ | SSIM ↑ | ERGAS ↓ | SAM (deg) ↓ |
| JPEG2000 Codec | |||||||||
| WFANet | 20.00× | 185.624 | 0.5847 ± 0.0989 | 0.2622 ± 0.1046 | 0.2103 ± 0.0365 | 29.2451 ± 3.9892 | 0.6964 ± 0.1230 | 7.1451 ± 1.4832 | 8.0026 ± 1.8765 |
| U2Net | 20.03× | 114.427 | 0.5830 ± 0.0999 | 0.2658 ± 0.1057 | 0.2088 ± 0.0367 | 29.2254 ± 3.9889 | 0.6960 ± 0.1230 | 7.1554 ± 1.4812 | 8.0102 ± 1.8692 |
| FusionNet | 19.99× | 1.444 | 0.5824 ± 0.0994 | 0.2691 ± 0.1064 | 0.2057 ± 0.0367 | 29.2050 ± 3.9294 | 0.6961 ± 0.1223 | 7.1534 ± 1.4757 | 7.9752 ± 1.9140 |
| ADWM | 19.97× | 47.665 | 0.5794 ± 0.0982 | 0.2676 ± 0.1035 | 0.2116 ± 0.0398 | 29.1645 ± 3.9383 | 0.6947 ± 0.1238 | 7.1916 ± 1.4841 | 8.1475 ± 1.8269 |
| ARConv | 20.01× | 80.710 | 0.5879 ± 0.1001 | 0.2655 ± 0.1065 | 0.2024 ± 0.0360 | 29.1799 ± 3.9359 | 0.6952 ± 0.1231 | 7.1642 ± 1.4738 | 7.8361 ± 1.8401 |
| GS | 20.05× | - | 0.5464 ± 0.0867 | 0.2990 ± 0.1064 | 0.2197 ± 0.0475 | 28.4811 ± 3.7461 | 0.6658 ± 0.1306 | 7.7017 ± 1.5986 | 7.6836 ± 1.9582 |
| BDSD | 19.98× | - | 0.5857 ± 0.0975 | 0.2670 ± 0.1130 | 0.2020 ± 0.0308 | 28.6829 ± 3.7466 | 0.6830 ± 0.1192 | 7.5294 ± 1.6008 | 7.9723 ± 1.8917 |
| GLP | 19.97× | - | 0.6212 ± 0.0939 | 0.2483 ± 0.1012 | 0.1747 ± 0.0399 | 28.9107 ± 3.7803 | 0.6869 ± 0.1196 | 7.3476 ± 1.5406 | 7.6171 ± 1.8623 |
| MF | 20.02× | - | 0.6356 ± 0.0922 | 0.2277 ± 0.0945 | 0.1786 ± 0.0435 | 28.7851 ± 3.7437 | 0.6848 ± 0.1226 | 7.4207 ± 1.5290 | 7.6305 ± 1.8427 |
| ZS-PAN | 20.07× | 36863.683 | 0.5629 ± 0.1090 | 0.3016 ± 0.1267 | 0.1942 ± 0.0393 | 28.5151 ± 3.5605 | 0.6838 ± 0.1216 | 7.4629 ± 1.4911 | 8.0665 ± 1.8173 |
| TOP | 33× | 32.511 | 0.7552 ± 0.1422 | 0.1863 ± 0.1145 | 0.0789 ± 0.0564 | 30.3237 ± 2.8339 | 0.8193 ± 0.0387 | 6.0036 ± 1.2298 | 5.9989 ± 1.3813 |
| HEVC Codec | |||||||||
| WFANet | 6.65× | 185.624 | 0.5355 ± 0.1369 | 0.3004 ± 0.1321 | 0.2437 ± 0.0586 | 28.6397 ± 3.4463 | 0.6983 ± 0.0965 | 7.3943 ± 1.4707 | 8.4511 ± 1.8924 |
| U2Net | 6.68× | 114.427 | 0.5267 ± 0.1348 | 0.3073 ± 0.1322 | 0.2484 ± 0.0575 | 28.5693 ± 3.4832 | 0.6952 ± 0.0992 | 7.4309 ± 1.4827 | 8.5067 ± 1.9079 |
| FusionNet | 6.66× | 1.444 | 0.5281 ± 0.1415 | 0.3115 ± 0.1406 | 0.2423 ± 0.0577 | 28.5932 ± 3.4080 | 0.6971 ± 0.0965 | 7.4091 ± 1.4595 | 8.4481 ± 1.8467 |
| ADWM | 6.68× | 47.665 | 0.5232 ± 0.1290 | 0.3083 ± 0.1269 | 0.2515 ± 0.0584 | 28.5146 ± 3.4834 | 0.6923 ± 0.1028 | 7.4620 ± 1.4281 | 8.6357 ± 1.8846 |
| ARConv | 6.63× | 80.710 | 0.5424 ± 0.1423 | 0.3012 ± 0.1366 | 0.2332 ± 0.0596 | 28.6597 ± 3.3866 | 0.6995 ± 0.0964 | 7.3682 ± 1.4539 | 8.2661 ± 1.7866 |
| GS | 6.65× | - | 0.5099 ± 0.1286 | 0.3311 ± 0.1399 | 0.2440 ± 0.0530 | 28.1831 ± 3.4005 | 0.6690 ± 0.1047 | 7.8659 ± 1.4707 | 7.8659 ± 1.9401 |
| BDSD | 6.60× | - | 0.5598 ± 0.1453 | 0.2814 ± 0.1393 | 0.2311 ± 0.0611 | 28.4264 ± 3.3264 | 0.6910 ± 0.0900 | 7.6359 ± 1.6836 | 8.3336 ± 1.8504 |
| GLP | 6.60× | - | 0.5822 ± 0.1529 | 0.2744 ± 0.1399 | 0.2085 ± 0.0643 | 28.6502 ± 3.3278 | 0.6950 ± 0.0904 | 7.4536 ± 1.6026 | 8.0500 ± 1.8544 |
| MF | 6.62× | - | 0.5881 ± 0.1475 | 0.2635 ± 0.1339 | 0.2146 ± 0.0667 | 28.4843 ± 3.3915 | 0.6858 ± 0.0948 | 7.5802 ± 1.6303 | 8.1932 ± 1.8319 |
| ZS-PAN | 6.66× | 36863.683 | 0.5327 ± 0.1424 | 0.3137 ± 0.1384 | 0.2329 ± 0.0634 | 28.2744 ± 3.2000 | 0.6888 ± 0.0956 | 7.6057 ± 1.5630 | 8.5449 ± 1.8477 |
| TOP | 33× | 32.511 | 0.7552 ± 0.1422 | 0.1863 ± 0.1145 | 0.0789 ± 0.0564 | 30.3237 ± 2.8339 | 0.8193 ± 0.0387 | 6.0036 ± 1.2298 | 5.9989 ± 1.3813 |
Table XV. Onboard pansharpening followed by JPEG2000/HEVC on WV3. Full-reference metrics use GT; HQNR terms use the original LR-MSI/PAN. Dashes denote omitted CPU-only fusion times.
The payload difference follows from where compression occurs. After pixel-domain fusion, a baseline must represent all samples of the full-resolution MSI, and its final size depends on how strongly a generic codec can quantize that image. TOP instead transmits binary decisions, whose size is fixed before observing image content. For WV3 at 33×, this is 0.5 bit per spatial pixel, or 0.0625 bppb across eight bands. The comparison therefore evaluates two deployment representations, rather than only two fusion networks.
TOP has the same row in both codec blocks because it uses neither codec. The changing baseline rows show that onboard pixel-domain fusion still exposes the final product to a separately optimized lossy stage. The timing column measures fusion inference rather than complete encode–transmit–decode latency; among the reported neural baselines, only FusionNet is faster than TOP, while traditional CPU timings are omitted. Thus, the table supports the rate–quality advantage and moderate onboard fusion cost, but does not claim hardware-independent end-to-end latency or energy.
Limitations
a) Sensor-Specific Training: The band-adaptive embedding accepts different band counts, but dimensions and learned parameters remain configuration-specific. Each WV3, GF2,
and Pavia setting in Tab. X therefore uses its own tokenizer and fusion module. A new sensor, or a ratio requiring a different latent configuration, requires one-time ground-side retraining without additional onboard computation. A sensor-agnostic tokenizer remains open.
b) Multi-Stage Training: TOBT uses 40,000 joint encoder–decoder iterations, two 20,000-iteration decoder-refinement stages (the last with bit flips), and separate fusion training (Tab. X). These stages learn the latent space, adapt decoding to the transmitted HR code, and improve bit-error robustness, as supported by Sec. V-D. The added training and tuning effort motivates a simpler future procedure.
References
- [1]L.-J. Deng, G. Vivone, W. Guo, M. Dalla Mura, and J. Chanussot, “A variational pansharpening approach based on reproducible kernel Hilbert space and Heaviside function,” IEEE Transactions on Image Processing, vol. 27, no. 9, pp. 4330–4344, 2018.
- [2]Z.-C. Wu, T.-Z. Huang, L.-J. Deng, J. Huang, J. Chanussot, and G. Vivone, “LRTCFPan: Low-rank tensor completion based framework for pansharpening,” IEEE Transactions on Image Processing, vol. 32, pp. 1640–1655, 2023.DOI
- [3]C. A. Laben and B. V. Brower, “Process for enhancing the spatial resolution of multispectral imagery using pan-sharpening,” U.S. Patent 6,011,875, 2000.
- [4]A. Garzelli, F. Nencini, and L. Capobianco, “Optimal MMSE pan sharpening of very high resolution multispectral images,” IEEE Transactions on Geoscience and Remote Sensing, vol. 46, no. 1, pp. 228–236, 2008.DOI
- [5]B. Aiazzi, L. Alparone, S. Baronti, and A. Garzelli, “Context-driven fusion of high spatial and spectral resolution images based on oversampled multiresolution analysis,” IEEE Transactions on Geoscience and Remote Sensing, vol. 40, no. 10, pp. 2300–2312, 2002.DOI
- [6]L.-J. Deng, G. Vivone, C. Jin, and J. Chanussot, “Detail injection-based deep convolutional neural networks for pansharpening,” IEEE Transactions on Geoscience and Remote Sensing, vol. 59, no. 8, pp. 6995–7010, 2021.DOI
- [7]S. Peng, C. Guo, X. Wu, and L.-J. Deng, “U2Net: A general framework with spatio-spectral-integrated double U-net for image fusion,” in Proceedings of the ACM International Conference on Multimedia (ACM MM), 2023, pp. 3219–3227.DOI
- [8]J. Huang, R. Huang, J. Xu, S. Peng, Y. Duan, and L.-J. Deng, “Wavelet-assisted multi-frequency attention network for pansharpening,” in Proceedings of the AAAI Conference on Artificial Intelligence (AAAI), vol. 39, 2025, pp. 3662–3670.
- [9]S. Xiang, Q. Liang, and P. Tang, “Task-oriented compression framework for remote sensing satellite data transmission,” IEEE Transactions on Industrial Informatics, vol. 20, no. 3, pp. 3487–3496, 2024.DOI
- [10]T. Lv, H. Jiang, and Y. Liu, “STAFuse: Scene-text aggregation-guided composite degradation-robust infrared and visible image fusion,” IEEE Transactions on Image Processing, vol. 35, pp. 8603–8618, 2026.DOI
- [11]D. S. Taubman and M. W. Marcellin, “JPEG2000: Standard for interactive imaging,” Proceedings of the IEEE, vol. 90, no. 8, pp. 1336–1357, 2002.DOI
- [12]G. J. Sullivan, J.-R. Ohm, W.-J. Han, and T. Wiegand, “Overview of the high efficiency video coding (HEVC) standard,” IEEE Transactions on Circuits and Systems for Video Technology, vol. 22, no. 12, pp. 1649–1668, 2012.DOI
- [13]J. Ballé, D. Minnen, S. Singh, S. J. Hwang, and N. Johnston, “Variational image compression with a scale hyperprior,” in International Conference on Learning Representations (ICLR), 2018.arxiv.org/abs/1802.01436
- [14]Z. Liu, Y. Lin, Y. Cao, H. Hu, W. Wei, Z. Zhang, S. Lin, and B. Guo, “Swin transformer: Hierarchical vision transformer using shifted windows,” in Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV), 2021, pp. 10012–10022.arxiv.org/abs/2103.14030
- [15]Y. Zhao, Y. Xiong, and P. Krähenbühl, “Image and video tokenization with binary spherical quantization,” in International Conference on Learning Representations (ICLR), 2025.arxiv.org/abs/2406.07548
- [16]J. Duda, “Asymmetric numeral systems: Entropy coding combining speed of Huffman coding with compression rate of arithmetic coding,” arXiv preprint arXiv:1311.2540, 2013.arxiv.org/abs/1311.2540
- [17]X. Li, X. Li, L. Tan, H. Li, and T. Ye, “UMCFuse: A unified multiple complex scenes infrared and visible image fusion framework,” IEEE Transactions on Image Processing, vol. 34, pp. 6231–6245, 2025.DOI
- [18]J. Luo, W. Ren, X. Gao, and X. Cao, “Multi-exposure image fusion via deformable self-attention,” IEEE Transactions on Image Processing, vol. 32, pp. 1529–1540, 2023.DOI
- [19]J. Li, X. Guo, G. Lu, B. Zhang, Y. Xu, F. Wu, and D. Zhang, “DRPL: Deep regression pair learning for multi-focus image fusion,” IEEE Transactions on Image Processing, vol. 29, pp. 4816–4831, 2020.DOI
- [20]B. Aiazzi, S. Baronti, and M. Selva, “Improving component substitution pansharpening through multivariate regression of MS+Pan data,” IEEE Transactions on Geoscience and Remote Sensing, vol. 45, no. 10, pp. 3230–3239, 2007.DOI
- [21]J. Choi, K. Yu, and Y. Kim, “A new adaptive component-substitution-based satellite image fusion by using partial replacement,” IEEE Transactions on Geoscience and Remote Sensing, vol. 49, no. 1, pp. 295–309, 2011.DOI
- [22]X. Otazu, M. Gonzalez-Audicana, O. Fors, and J. Nunez, “Introduction of sensor spectral response into image fusion methods: Application to wavelet-based methods,” IEEE Transactions on Geoscience and Remote Sensing, vol. 43, no. 10, pp. 2376–2385, 2005.DOI
- [23]R. Restaino, G. Vivone, M. Dalla Mura, and J. Chanussot, “Fusion of multispectral and panchromatic images based on morphological operators,” IEEE Transactions on Image Processing, vol. 25, no. 6, pp. 2882–2895, 2016.DOI
- [24]T. Wang, F. Fang, F. Li, and G. Zhang, “High-quality bayesian pansharpening,” IEEE Transactions on Image Processing, vol. 28, no. 1, pp. 227–239, 2019.DOI
- [25]H. Lu, Y. Yang, S. Huang, W. Tu, and W. Wan, “A unified pansharpening model based on band-adaptive gradient and detail correction,” IEEE Transactions on Image Processing, vol. 31, pp. 918–933, 2022.
- [26]J. Huang, H. Chen, J. Ren, S. Peng, and L.-J. Deng, “A general adaptive dual-level weighting mechanism for remote sensing pansharpening,” in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2025, pp. 7474–7456.arxiv.org/abs/2503.13214
- [27]X. Wang, Z. Zheng, J. Shao, Y. Duan, and L.-J. Deng, “Adaptive rectangular convolution for remote sensing pansharpening,” in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2025, pp. 17872–17881.arxiv.org/abs/2503.00467
- [28]B. Xu, Y. Zheng, L. Wang, S. Mazhar, Z. Huang, and J. Liu, “DTEA: Degradation-aware taylor expansion approximation network for pansharpening,” IEEE Transactions on Image Processing, vol. 35, pp. 7581–7596, 2026.DOI
- [29]Q. Cao, L.-J. Deng, W. Wang, J. Hou, and G. Vivone, “Zero-shot semi-supervised learning for pansharpening,” Information Fusion, vol. 101, p. 102001, 2024.DOI
- [30]W. G. C. Bandara and V. M. Patel, “HyperTransformer: A textural and spectral feature fusion transformer for pansharpening,” in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2022, pp. 1767–1777.arxiv.org/abs/2203.02503
- [31]N. Wang, A. Guo, R. Dian, and S. Li, “Equivariant high-resolution hyperspectral imaging via mosaiced and PAN image fusion,” IEEE Transactions on Image Processing, vol. 35, pp. 1246–1260, 2026.DOI
- [32]W. G. C. Bandara, J. M. J. Valanarasu, and V. M. Patel, “Hyperspectral pansharpening based on improved deep image prior and residual reconstruction,” IEEE Transactions on Geoscience and Remote Sensing, vol. 60, pp. 1–16, 2022.arxiv.org/abs/2107.02630
- [33]G. Guarino, M. Ciotola, G. Vivone, and G. Scarpa, “Band-wise hyperspectral image pansharpening using CNN model propagation,” IEEE Transactions on Geoscience and Remote Sensing, vol. 62, pp. 1–18, 2024.arxiv.org/abs/2311.06510
- [34]G. Guarino, M. Ciotola, G. Vivone, G. Poggi, and G. Scarpa, “Zero-shot hyperspectral pansharpening using hysteresis-based tuning for spectral quality control,” IEEE Transactions on Geoscience and Remote Sensing, vol. 63, pp. 1–19, 2025.DOI
- [35]H. Jiang and Z. Chen, “Hyperspectral pansharpening with transformer-based spectral diffusion priors,” in Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision (WACV) Workshops, 2025, pp. 581–590.DOI
- [36]J.-L. Xiao, T.-Z. Huang, L.-J. Deng, G. Lin, Z. Cao, C. Li, and Q. Zhao, “Hyperspectral pansharpening via diffusion models with iteratively zero-shot guidance,” in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2025, pp. 12669–12678.DOI
- [37]M. Hernández-Cabronero, A. B. Kiely, M. Klimesh, I. Blanes, J. Ligo, E. Magli, and J. Serra-Sagristà, “The CCSDS 123.0-B-2 “Low-complexity lossless and near-lossless multispectral and hyperspectral image compression” standard: A comprehensive review,” IEEE Geoscience and Remote Sensing Magazine, vol. 9, no. 4, pp. 102–119, 2021.
- [38]A. van den Oord, O. Vinyals, and K. Kavukcuoglu, “Neural discrete representation learning,” in Advances in Neural Information Processing Systems, vol. 30, 2017.arxiv.org/abs/1711.00937
- [39]F. Mentzer, D. Minnen, E. Augustsson, and M. Tschannen, “Finite scalar quantization: VQ-VAE made simple,” in International Conference on Learning Representations (ICLR), 2024.arxiv.org/abs/2309.15505
- [40]L. Yu, J. Lezama, N. B. Gundavarapu, L. Versari, K. Sohn, D. Minnen, Y. Cheng, A. Gupta, X. Gu, A. G. Hauptmann, B. Gong, M.-H. Yang, I. Essa, D. A. Ross, and L. Jiang, “Learner: A neural network-based entropy model beats diffusion—tokenizer is key to visual generation,” in International Conference on Learning Representations (ICLR), 2024.
- [41]A. Dosovitskiy, L. Beyer, A. Kolesnikov, D. Weissenborn, X. Zhai, T. Unterthiner, M. Dehghani, M. Minderer, G. Heigold, S. Gelly, J. Uszkoreit, and N. Houlsby, “An image is worth 16x16 words: Transformers for image recognition at scale,” in International Conference on Learning Representations (ICLR), 2021.
- [42]F. A. Kruse, A. B. Lefkoff, J. W. Boardman, K. B. Heidebrecht, A. T. Shapiro, P. J. Barloon, and A. F. H. Goetz, “The spectral image processing system (SIPS)—interactive visualization and analysis of imaging spectrometer data,” Remote Sensing of Environment, vol. 44, no. 2–3, pp. 145–163, 1993.
- [43]L.-J. Deng, G. Vivone, M. E. Paoletti, G. Scarpa, J. He, Y. Zhang, J. Chanussot, and A. Plaza, “Machine learning in pansharpening: A benchmark, from shallow to deep networks,” IEEE Geoscience and Remote Sensing Magazine, vol. 10, no. 3, pp. 279–315, 2022.DOI
- [44]Y.-W. Zhuo, T.-J. Zhang, J.-F. Hu, H.-X. Dou, T.-Z. Huang, and L.-J. Deng, “A deep-shallow fusion network with multidetail extractor and spectral attention for hyperspectral pansharpening,” IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing, vol. 15, pp. 7539–7555, 2022.DOI
- [45]L. Wald, T. Ranchin, and M. Mangolini, “Fusion of satellite images of different spatial resolutions: Assessing the quality of resulting images,” Photogrammetric Engineering & Remote Sensing, vol. 63, no. 6, pp. 691–699, 1997.
- [46]G. Vivone, “Robust band-dependent spatial-detail approaches for panchromatic sharpening,” IEEE Transactions on Geoscience and Remote Sensing, vol. 57, no. 9, pp. 6421–6433, 2019.DOI
- [47]S. Lolli, L. Alparone, A. Garzelli, and G. Vivone, “Haze correction for contrast-based multispectral pansharpening,” IEEE Geoscience and Remote Sensing Letters, vol. 14, no. 12, pp. 2255–2259, 2017.DOI
- [48]Z. Wang, A. C. Bovik, H. R. Sheikh, and E. P. Simoncelli, “Image quality assessment: From error visibility to structural similarity,” IEEE Transactions on Image Processing, vol. 13, no. 4, pp. 600–612, 2004.pubmed.ncbi.nlm.nih.gov/15376593
- [49]G. Vivone, L. Alparone, J. Chanussot, M. Dalla Mura, A. Garzelli, G. A. Licciardi, R. Restaino, and L. Wald, “A critical comparison among pansharpening algorithms,” IEEE Transactions on Geoscience and Remote Sensing, vol. 53, no. 5, pp. 2565–2586, 2015.DOI
- [50]B. Aiazzi, L. Alparone, S. Baronti, R. Carlà, A. Garzelli, and L. Santurri, “Full scale assessment of pansharpening methods and data products,” in Image and Signal Processing for Remote Sensing XX, vol. 9244. SPIE, 2014, p. 924402.DOI
- [51]J. Lu, L. Zhang, X. Zhou, M. Li, W. Li, and S. Gu, “Learned image compression with dictionary-based entropy model,” in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2025, pp. 12850–12859.arxiv.org/abs/2504.00496
- [52]D. Hendrycks and K. Gimpel, "Gaussian error linear units (GELUs)," arXiv preprint arXiv:1606.08415, 2016.






