betaFeedback

WorksEngineering

Zero-shot diffusion image compression with sectioned codebooks and one channel code

Abstract

Zero-shot diffusion codecs such as Turbo-DDCM compress images without any training. In each coded step, the encoder selects noise vectors from one codebook and transmits a single index for the selected set. We propose sectioned codebooks. A coded step is divided into sections, and each section has its own large codebook and transmits its best vector as a short field of bits. A model based on order statistics quantifies how well the bits of a step are used and predicts the gain of sectioned codebooks from measurements of Turbo-DDCM alone. Fast Walsh-Hadamard transforms make sections of 2202^{20} vectors practical. At equal bits, sectioned codebooks improve the PSNR of Turbo-DDCM by 0.18 to 0.51 dB on three datasets, improve LPIPS and DISTS, and save 16 to 21% of the bits at equal PSNR. Codecs that are robust to channel errors by design are usually compared only with unprotected codecs. We compare them, at equal total bits, with a codec protected by one channel code. With one BCH code, the sectioned codec outperforms all reported results of Robust Turbo-DDCM and BitFlat on two datasets, by up to 7.74 dB at the highest bit error rate. At the lowest bit error rate its margin over BitFlat is only 0.08 to 0.16 dB. When the code fails, the receiver can still decode the fields as received. Additional experiments cover burst errors and packet loss, and all observed gains of robust codecs stay within a limit derived from the channel capacity.

Introduction

Zero-shot diffusion codecs compress images with a pretrained diffusion model and need no training of their own [1, 2, 3]. The decoder runs the reverse diffusion process, and the encoder controls this process through the noise that is added in each step. In denoising diffusion codebook models (DDCM) [3], both sides generate the same random codebook from a shared seed, and the bitstream identifies the noise vectors that the encoder has selected. Turbo-DDCM [4] is a fast codec of this family. In each coded step it combines about a hundred vectors of a Gaussian codebook with 16 384 entries and transmits the selected set as a single index, which we call the joint index.

It is not known how efficiently this bitstream uses its bits. Turbo-DDCM keeps the codebook at the size of the latent, and its authors list a theoretical account of the method as an open problem [4]. The bitstream is also fragile, because a single flipped bit of the joint index changes most of the selected vectors of a step. Recent work redesigns the bitstream so that it tolerates bit errors. Robust Turbo-DDCM [5] writes every selected vector as a separate field, so that a bit error corrupts only one vector. BitFlat [6] assigns every field a small codebook of its own, protects each field with a short code and discards the vectors whose field is detected as corrupted. For packet loss, learned codecs are trained with random loss of parts of their latent representation [7, 8] [9]. These robust codecs are compared with unprotected codecs or with each other. The classical alternative, an unmodified codec protected by one channel code at the same total number of bits, is missing from these comparisons, although robustness by design also costs bits. The separate fields of Robust Turbo-DDCM, for example, need 41 to 47% more bits than the joint index for the same vectors.

In this paper, we replace the joint index of Turbo-DDCM with sectioned codebooks (Fig. 1). A coded step is divided into sections. Each section has its own codebook and contributes the single vector that best matches the image, which is written as a field of fixed length. To explain why this helps, we introduce the steering of a step, a measure of how strongly the transmitted noise points towards the part of the image that is still missing. A model based on order statistics shows that a section steers more per bit as its codebook grows and that large sections are more efficient than the joint index. Since a codebook with a million vectors cannot be stored, we construct the sections from Hadamard matrices, for which a fast transform computes all correlations of a block. At exactly the bits of Turbo-DDCM, sections of 2202^{20} vectors improve the PSNR by 0.18 to 0.51 dB on three datasets, improve LPIPS and DISTS, and reduce the bitrate by 16 to 21% at equal PSNR. The model predicts these gains from measurements of Turbo-DDCM alone.

When we protect the sectioned bitstream with one BCH code and compare it with the robust codecs at equal total bits. The code is designed once for the highest bit error rate (BER) at which the robust codecs are tested. Even the original Turbo-DDCM protected in this way outperforms Robust Turbo-DDCM at every BER from 10−410^{-4} to 10−210^{-2}. The sectioned codec with the same code outperforms all reported results of Robust Turbo-DDCM and BitFlat in PSNR, LPIPS and DISTS. Its margin over BitFlat is 4.14 to 5.05 dB at a BER of 10−210^{-2} and 0.08 to 0.16 dB at 10−410^{-4}, where the comparison with reported values is not conclusive. Even when the code is designed for twice the highest reported BER, the sectioned codec matches Turbo-DDCM with a code for the reported range. When the code fails, the receiver can still decode the fields as received, which a joint index does not allow.

The main contributions of this paper are as follows.

  • We propose sectioned codebooks for zero-shot diffusion compression, together with a model of the steering of a coded step that predicts their gain over the joint index.

  • We construct sections of up to 2202^{20} vectors from Hadamard matrices and obtain consistent rate-distortion gains over Turbo-DDCM on three datasets.

  • We show that a codec protected by one channel code is a stronger baseline than the unprotected codecs used so far, and that the sectioned codec with one BCH code outperforms the reported results of two robust diffusion codecs.

Additional results on burst errors and packet loss, and a limit on the gain that robustness by design can achieve, are given in the appendices.

Zero-shot diffusion codecs differ in how they transmit the noise of a step. DiffC [1] uses reverse channel coding, and its practical implementation codes the noise in chunks of coordinates that are searched separately [2]. DDCM [3] selects one vector of a Gaussian codebook per step, and Turbo-DDCM [4] combines many vectors per step and codes their set jointly. Later codecs change the trajectory that the selections follow [10] or distribute the selections over time, space and channels [11]. In the latter, the channels of the latent are split into groups and one codebook vector is selected for each group. Chunks and channel groups partition the coordinates of the noise, whereas every vector of a section spans all coordinates and the selected vectors are added. BitFlat [6] gives every field a codebook of 512 or 1 024 vectors in order to confine bit errors. It does not compare this layout with the joint index on an error-free channel and does not consider larger codebooks.

Sectioned codebooks have the structure of sparse regression codes. These codes were introduced for the Gaussian channel [12] and achieve the rate-distortion function of Gaussian sources with codewords that add one column from each section of a random matrix [13, 14, 15]. Transmitting the positions of the largest entries of a randomly rotated vector also achieves this function [16], which is the principle behind the selection in Turbo-DDCM. Algebraic codebooks in speech coding place one signed pulse in each of several tracks [17]. Design matrices built from Hadamard matrices make the decoders of sparse superposition codes fast [18, 19], and randomized Hadamard transforms serve as fast random projections [20, 21]. Our codec uses a sparse regression code with signs and equal weights as the noise of a diffusion step. The theory of these codes addresses Gaussian sources and long blocks and does not cover this use. Two concurrent preprints are related to parts of our work. H-DDCM [22] replaces the Gaussian codebook of Turbo-DDCM with randomized Hadamard dictionaries, from which the best vectors are selected and coded jointly. Reeves and Venkataramanan [23] bound the distortion of sparse regression codes with the order statistics that also appear in our model. Neither work studies sections in a diffusion codec or their protection by a channel code.

Protecting an image bitstream with error-correcting and erasure codes, including unequal protection of embedded bitstreams, is well established for wavelet coders [24, 25, 26, 27]. Robust codecs take a different route. Learned codecs are trained to tolerate packet loss [7, 8, 28] [9], and the robust diffusion codecs redesign the bitstream [5, 6]. The separation theorem states that separate source and channel coding is optimal for a known channel [29], and the broadcast approach quantifies what is lost when the channel state is unknown [30, 31, 32]. We use the classical protection as a baseline for robust diffusion codecs, which previous comparisons do not include.

Proposed method

This section reviews the coded step of Turbo-DDCM, introduces a measure of how well a step uses its bits, and presents sectioned codebooks, their fast construction and their protection by one channel code. Figure 1 illustrates the difference between Turbo-DDCM and the proposed codec.

Figure 1. One coded step. (a) Turbo-DDCM selects the best MM vectors of one codebook and codes the selected set as a joint index. (b) The proposed codec selects the best vector of each section and writes one field per section. One BCH code protects all fields of an image.

Turbo-DDCM

Turbo-DDCM [4] uses a pretrained latent diffusion model [33]. The image is mapped to a latent x0x_0 of D=16 384D = 16\,384 values, and the reverse process runs for 30 steps from a fixed starting point. A regular reverse step adds fresh Gaussian noise. A coded step instead adds a noise vector zz that the encoder selects and the decoder reconstructs from the bitstream. The encoder knows the current prediction x^0\hat{x}_0 of the latent and computes the residual r=x0−x^0r = x_0 - \hat{x}_0. Both sides generate a codebook of J=DJ = D Gaussian vectors c1,…,cJc_1,\ldots,c_J from a shared seed. The encoder selects the MM vectors with the largest correlation magnitude ∣⟨cj,r⟩∣\lvert\langle c_j,r\rangle\rvert, assigns each the sign of its correlation, and sets zz to their sum, scaled to unit variance. The bits of the step are the joint index, which identifies the selected set among the (JM)\binom{J}{M} possible sets, and the MM signs. The last steps of the reverse process carry no bits.

Steering of a coded step

The transmitted noise is useful through its component along the residual. For a random codebook, the normalized correlation gj=⟨cj,r⟩/∥r∥g_j = \langle c_j,r\rangle/\lVert r\rVert of a vector is a standard normal variable regardless of the residual. The signed sum of the selected vectors has length ∑selected∣gj∣\sum_{\mathrm{selected}}\lvert g_j\rvert along the residual and a squared length of about MDMD in the orthogonal directions. This gives the squared correlation between the noise and the residual,

⟨z,r⟩2∥z∥2∥r∥2=ED+E,E=1M(∑selected∣gj∣)2.(1)\frac{\langle z,r\rangle^2}{\lVert z\rVert^2\lVert r\rVert^2} = \frac{E}{D+E}, \qquad E = \frac{1}{M}\left(\sum_{\mathrm{selected}}\lvert g_j\rvert\right)^2 . \tag*{(1)}

We call EE the steering of the step. Random noise has E≈1E \approx1, and the coded steps of Turbo-DDCM reach several hundred.

The steering that bb bits can achieve is bounded. A description with bb bits cannot approximate the direction of the residual better than the rate-distortion function of a Gaussian vector allows [29], which gives E≤D(22b/D−1)E \le D (2^{2b/D} - 1). For the 900 bits per step that serve as an example in this section, the bound is 1 296. The joint index is far from this bound. The MM largest of JJ standard normal magnitudes exceed the threshold aa that a fraction M/JM/J of all magnitudes exceeds, and their mean is gˉ=2φ(a)J/M\bar{g} = 2\varphi(a)J/M, where φ\varphi is the normal density. The steering is then E=Mgˉ2E = M\bar{g}^{2}. The 91 vectors that Turbo-DDCM transmits with 900 bits give E=859E = 859, which is 66% of the bound.

Sectioned codebooks

In the proposed codec, a coded step has LL sections. Each section is a separate codebook of 2w2^{w} vectors and contributes the vector with the largest correlation magnitude, together with the sign of its correlation. The selection of a section is written as a field of w+1w+1 bits, and the bits of the step are the concatenated fields. The noise is the normalized sum of the LL selected vectors, as in Turbo-DDCM, and the rest of the codec is unchanged.

From (1), sections of equal size give

E=Lμw2,(2)E = L\mu_w^2, \tag*{(2)}

where μw\mu_w is the expected maximum of 2w2^w standard normal magnitudes. The steering per bit, μw2/(w+1)\mu_w^2/(w+1), increases with the section size. It is 1.08 for 2102^{10} vectors, 1.14 for 2142^{14}, 1.19 for 2202^{20} and 1.22 for 2242^{24}, compared with 0.95 for the joint index (Figure 2a). Large sections are more efficient, and their size is limited only by computation. With sections of 2202^{20} vectors, 900 bits hold 43 fields and give E=1 074E = 1\,074, which is 83% of the bound. When the bits of a step are not a multiple of the field length, the sections have two sizes and we add their contributions μw2\mu_w^2, which overestimates EE by at most 0.5% for the layouts used here.

Figure 2. Sectioned codebooks on Kodak. (a) Steering per bit as a function of the section size. Lines show the model and markers show measurements. (b) Measured and predicted PSNR gain over Turbo-DDCM with standard errors. Open markers lie outside the range of the fit. (c) Rate-distortion curves with 24 coded steps. Gray is Turbo-DDCM.

Sectioned codebooks differ from Turbo-DDCM in the number of candidate vectors and in the way the selection is written, and the model separates the two effects. If the best vectors of the whole pool of all sections were selected and written as one joint index, each vector would cost about log⁡2e≈1.4\log_2 e \approx1.4 bits more than a field. At equal bits, such an index holds 40 vectors instead of 43 and gives E=1 041E = 1\,041. This suggests that most of the gain over Turbo-DDCM comes from the larger pool and a smaller part from the fields, which the ablation in Section IV-C confirms.

Fast sections from Hadamard blocks

A section of 2202^{20} Gaussian vectors has 2342^{34} values and can be neither stored nor generated for every step. We therefore construct sections from blocks of DD vectors that need no storage. Let HH be the Hadamard matrix of order DD, let π\pi be a permutation of its columns drawn once per step, and let vv be a vector of DD standard normal values drawn once per block. The vectors of the block are

cj[i]=v[i]H[j,π(i)],j=1,…,D.(3)c_j[i] = v[i]H[j,\pi(i)], \qquad j=1,\ldots,D. \tag*{(3)}

Each vector has independent standard normal coordinates, as the model assumes. The correlations of all DD vectors with the residual are obtained with one fast Walsh–Hadamard transform of the values v[i] r[i]v[i]\,r[i], taken in the order π\pi, which costs Dlog⁡2DD\log_2 D additions. A section of 2w≥D2^w \ge D vectors consists of 2w/D2^w/D blocks with independent vv, and a smaller section uses the first 2w2^w vectors of one block. The decoder needs no transform, because it regenerates only the LL selected vectors from (3).

The vectors of a block share the magnitudes of vv and are not mutually independent. This does not affect a section, which contributes a single vector, and a control experiment with independent Gaussian vectors showed no significant difference (Section IV-C). Dictionaries based on Hadamard matrices are known from sparse superposition codes [18, 19] and from fast random projections [20, 21]. The Gaussian values vv are specific to our use, in which the sum of the selected vectors replaces Gaussian noise.

Protection with one channel code

For transmission over a noisy channel, we protect the whole bitstream of an image with a single channel code. Of a total of nn bits, kk bits are used for the compressed bitstream and the remaining bits for the parity of a shortened binary BCH code [34, 35] that corrects up to tt bit errors. A bounded-distance decoder returns the bitstream exactly when at most tt of the transmitted bits are flipped. For independent bit errors with rate pp, the probability Pf(p)P_{\mathrm{f}}(p) that decoding fails is a binomial tail. If decoding fails, the receiver displays a mid-gray image. With Q(k)Q(k) the mean quality of the codec at kk bits and Q0Q_0 the quality of the mid-gray image, the expected quality is

S(p)=(1−Pf(p))Q(k)+Pf(p)Q0.(4)S(p) = (1-P_{\mathrm{f}}(p))Q(k) + P_{\mathrm{f}}(p)Q_0. \tag*{(4)}

This expression is exact and requires no channel simulation. The code is designed once for the highest tested BER. Among the operating points and codes that fit into nn bits, we select the pair that maximizes (4) at that BER. Appendix A describes the codes in detail.

A sectioned bitstream allows a better reaction to a decoding failure than a mid-gray image. The code is systematic, and every bit pattern of a field identifies a valid vector of its section. The receiver can still decode the fields as received. In this case, a flipped bit replaces the vector of one section and leaves the other sections unchanged. We refer to this option as the fallback.

Experiments

Setup

We implemented sectioned codebooks in the public code of Turbo-DDCM [4], with its pretrained model, weights and seed. Only the selection of the noise differs from the original codec. Following the robust codecs [5, 6], we used the 24 Kodak images [36], the 100 images of the DIV2K validation set [37] and the 250 images of the CLIC 2020 professional test set [38], each cropped to its central 512×512512 \times512 pixels. Quality is measured with PSNR, LPIPS [39] and DISTS [40]. Unless stated otherwise, the codecs use 24 coded steps, as the robust codecs do, and sections of 2202^{20} vectors. The comparisons with Turbo-DDCM are made per image at exactly the same number of bits, and differences are reported with the standard error of their mean over the images.

For bit errors, we compare with the results reported for Robust Turbo-DDCM [5] and BitFlat [6] at BERs from 10−410^{-4} to 10−210^{-2}. BitFlat was reported with one parity bit per field, which is its main configuration, and with a Hamming or SECDED code per field. Since the code of BitFlat is not public, we take its results from Tables I and IV of [6]. To check that our evaluation matches the reported one, we reproduced Robust Turbo-DDCM, whose code is public. Our PSNR values are within 0.15 dB of the reported ones on Kodak and within 0.09 dB on DIV2K (Appendix C). The reported values for CLIC could not be reproduced, and on that dataset we compare only with our reproduction of Robust Turbo-DDCM. The BCH codes are designed for a BER of 10−210^{-2}, the highest tested rate, and the total number of bits is that of the compared robust codec.

Compression performance

Table I compares sectioned codebooks with Turbo-DDCM at equal bits. Sectioned codebooks improve the PSNR in every row, by 0.18 to 0.51 dB, and they also improve LPIPS and DISTS. At 21 600 bits per image, the gain is 0.30 dB on Kodak, 0.45 dB on DIV2K and 0.39 dB on CLIC, and the PSNR is higher on 24 of 24, 100 of 100 and 245 of 250 images. The gain is largest at low rates and decreases as the rate grows, on Kodak from 0.51 dB at 0.040 bpp to 0.18 dB at 0.194 bpp, where the rate-distortion curve of the codec becomes flat. The rows with 19, 27 and 29 coded steps use the number of steps that Turbo-DDCM selects by default at these rates, and the gain is preserved (0.18 to 0.31 dB). In terms of rate, the sectioned codec reaches the PSNR of Turbo-DDCM with 16 to 21% fewer bits (Fig. 2c).

Model validation and ablation

The steering model predicts these gains from measurements of Turbo-DDCM alone. We fitted the PSNR of Turbo-DDCM on Kodak as a quadratic function of the logarithm of its steering, using its 32 operating points with 24 coded steps, and read the expected gain of a sectioned layout from this curve at the steering given by (2). The first eight predictions were made before the sectioned codec was run. For the 14 layouts measured on Kodak, with sections of 2102^{10} to 2202^{20} vectors, the measured gain differs from the prediction by at most 0.05 dB (Fig. 2b). The gain also grows with the section size as predicted, from 0.17 dB for 2102^{10} vectors to 0.30 dB for 2202^{20} vectors at 21 600 bits.

Table II separates the two sources of the gain. We also recorded the correlation between the transmitted noise and the residual in every coded step, which gives the steering directly from (1). For Turbo-DDCM and all sectioned layouts, the measured steering deviates from the model by only −0.2 to +0.1%. The size of the pool is the main factor. With the selection rule and the joint index of Turbo-DDCM, pools of 60 and 2 696 Hadamard blocks gain 0.20 and 0.26 dB. Writing the selection as fields adds a smaller part. Sections on these two pools gain 0.22 and 0.30 dB, and the difference to the joint index on the same pool is significant only for the larger pool, where it is +0.04 dB with a standard error of 0.01 dB. The table also shows that several vectors should not be taken from the same block. One block with its best 91 vectors is 0.17 dB below Turbo-DDCM, because the vectors of a block share the values vv and their sum is no longer Gaussian. Sections avoid this, because each section has its own blocks and contributes a single vector. Finally, sections of 2102^{10} independent Gaussian vectors and Hadamard-based sections of the same size differ by −0.04 dB in PSNR with a standard error of 0.05 dB, which is not significant.

The encoding cost depends on the section size. At 900 bits per step, the encoder transforms 2 696 blocks per step with 6.2×1086.2 \times10^{8} additions, while Turbo-DDCM computes a product with a dense codebook of D×JD \times J values, which takes 2.7×1082.7 \times10^{8} multiplications. On a laptop GPU, encoding an image took 5.6 s with sections of 2202^{20} vectors, 3.7 s with sections of 2172^{17} vectors and 3.7 s with Turbo-DDCM. Sections of 2172^{17} vectors thus keep 0.29 of the 0.30 dB at the encoding time of Turbo-DDCM. The cost of the transforms grows with the number of sections. At 0.194 bpp, the highest rate tested, encoding with sections of 2202^{20} vectors took 8.9 s compared with 3.8 s. The

bppCoded stepsPSNR (dB) BasePSNR (dB) OursGainLPIPS BaseLPIPS OursDISTS BaseDISTS OursImages
\multicolumn{10}{l}{Kodak, 24 images}
0.04042423.2123.72+0.51 (0.05)0.2210.1840.1420.11724
0.05582424.0024.36+0.36 (0.04)0.1630.1350.1020.08424
0.05912424.1024.46+0.36 (0.03)0.1550.1300.0960.07924
0.08242424.6424.95+0.30 (0.03)0.1200.1040.0740.06524
0.09992424.9125.17+0.26 (0.02)0.1070.0970.0650.06024
0.06461924.0024.31+0.31 (0.03)0.1370.1200.0800.07124
0.12572725.3025.55+0.26 (0.02)0.0970.0880.0610.05624
0.15232925.5925.81+0.22 (0.02)0.0900.0830.0570.05424
0.19392925.7925.96+0.18 (0.01)0.0840.0800.0540.05224
\multicolumn{10}{l}{DIV2K, 100 images}
0.05912423.1823.69+0.51 (0.03)0.1710.1420.1160.096100
0.08242423.9424.38+0.45 (0.02)0.1300.1110.0870.075100
0.09922424.3024.67+0.37 (0.02)0.1150.1010.0760.06899
0.15232925.2025.49+0.30 (0.01)0.0940.0860.0660.06199
0.19392925.4725.71+0.24 (0.01)0.0860.0820.0610.05799
\multicolumn{10}{l}{CLIC, 250 images}
0.05912426.4226.89+0.47 (0.02)0.1190.1010.0890.075243
0.08242427.1027.49+0.39 (0.01)0.0930.0820.0700.062245

Table 1. Sectioned codebooks with 2202^{20} vectors per section (Ours) and Turbo-DDCM (Base) at equal bits. The PSNR gain is given with its standard error. The last column is the number of images with a higher PSNR.

CodebookCandidate vectorsBits per stepSteering modelSteering measuredPSNR diff. (dB)Time (s)
Joint index over the best vectors of one pool
Turbo-DDCM1.6×1041.6 \times10^{4}919008598593.7
One Hadamard block1.6×1041.6 \times10^{4}91900859−0.17-0.17 (0.02)3.5
8 blocks1.3×1051.3 \times10^{5}67892931+0.09 (0.01)3.5
60 blocks9.8×1059.8 \times10^{5}54892979+0.20 (0.02)3.5
2 696 blocks4.4×1074.4 \times10^{7}408971 041+0.26 (0.02)5.8
Sectioned codebooks, one vector per section
2102^{10} vectors per section8.3×1048.3 \times10^{4}82900969+0.17 (0.02)3.5
2142^{14} vectors per section9.8×1059.8 \times10^{5}609001 025+0.22 (0.02)3.5
2172^{17} vectors per section6.6×1066.6 \times10^{6}509001 054+0.29 (0.02)3.7
2202^{20} vectors per section4.4×1074.4 \times10^{7}439001 074+0.30 (0.03)5.6

Table 2. Ablation on Kodak with 24 coded steps at about 900 bits per step. PSNR differences are relative to Turbo-DDCM and are given with standard errors. Time is the encoding time per image.

decoder regenerates only the selected vectors and needs no transform.

Robustness to bit errors

The BCH-protected codecs are compared with the robust codecs in Table III and Figure 3. At 26 280 bits per image, the sectioned codec spends 21 600 bits on the image and 4 680 bits on parity, which corrects up to 319 bit errors. Its quality is constant up to a BER of 10−210^{-2}, whereas the quality of the robust codecs decreases with the BER.

Figure 3. PSNR as a function of the BER on Kodak at about 0.10 bpp. The BCH codes are designed for a BER of 10−210^{-2}. Beyond this BER, the sectioned codec may start to fail, and its performance can degrade rapidly.

Turbo-DDCM protected by one BCH code already outperforms Robust Turbo-DDCM at every BER, by 0.48 to 7.29 dB on the two datasets. The separate fields of Robust Turbo-DDCM need so many bits that the joint index can carry more vectors and their protection within the same total. BitFlat with parity fields is 0.21 to 0.40 dB better than this baseline at a BER of 10−410^{-4} and worse from a BER of 10−310^{-3} on (Appendix B).

The sectioned codec with the same code has the highest PSNR at every BER in Table III. At exactly equal total bits, it outperforms Robust Turbo-DDCM by 0.84 to 7.74 dB and BitFlat with Hamming or SECDED fields by 0.37 to 1.73 dB. Its margin over BitFlat with parity fields is 4.14 to 5.05 dB at a BER of 10−210^{-2} and 0.08 to 0.16 dB at 10−410^{-4}. The margin at 10−410^{-4} is of the size of the deviations of our reproduction, so the two codecs should be regarded as equivalent at this BER. LPIPS and DISTS give the same ranking, and the sectioned codec is better in all 180 comparisons with reported values (Appendix B). Compared with our reproduction of Robust Turbo-DDCM, it is better by 0.83 to 7.24 dB on Kodak, 1.11 to 7.83 dB on DIV2K and 1.13 to 9.30 dB on CLIC.

The steering model explains the ranking at low BER. The parity fields of BitFlat are sections of 512 vectors, and their 1 089 bits per step give E=1 045E = 1\,045. This is more than the 859 of the joint index with the bits that the code leaves for it, and less than the 1 074 of sections of 2202^{20} vectors with the same bits.

Beyond the design range

A code designed for a BER of 10−210^{-2} fails when the channel is worse. The BCH-protected Turbo-DDCM then loses the image, while the robust codecs still deliver a degraded image. The sectioned codec can use the fallback of Section III-E, which we measured on Kodak (Figure 3). At a BER of 2×10−22 \times10^{-2}, the fallback gives 14.9 dB with sections of 2202^{20} vectors, compared with 12.8 dB for a mid-gray image and 15.8 dB for our reproduction of Robust Turbo-DDCM. The difference to Robust Turbo-DDCM shrinks from −3.1-3.1 to −0.9-0.9 dB at this BER and from −0.7-0.7 to −0.5-0.5 dB at 5×10−25 \times10^{-2}. Smaller sections have shorter fields, so a smaller fraction of the selected vectors is corrupted at a given BER. With sections of 2142^{14} vectors, the fallback is on par with Robust Turbo-DDCM, with differences of 0.0 and +0.2 dB, and with sections of 2102^{10} vectors it is above Robust Turbo-DDCM, with differences of +0.9 and +0.9 dB. Without errors, these smaller sections gain 0.22 and 0.17 dB over Turbo-DDCM instead of 0.30 dB. The section size is thus a trade-off between quality within the design range and quality beyond it.

The design range can also be extended. A code designed for a BER of 2×10−22 \times10^{-2}, twice the highest reported BER, needs more parity and leaves 736 bits per step for the image at 26 280 bits. With sectioned codebooks, this design reaches 24.67 dB on Kodak up to a BER of 2×10−22 \times10^{-2}. The difference to Turbo-DDCM with the code designed for 10−210^{-2} is +0.02 dB on Kodak and +0.06 dB on DIV2K. In other words, the gain of sectioned codebooks pays for a design range that is twice as wide. At a BER of 2×10−22 \times10^{-2}, this design is 8.8 dB better than our reproduction of Robust Turbo-DDCM on Kodak. It still outperforms the reported results of Robust Turbo-DDCM and of BitFlat with Hamming or SECDED fields at every BER, and those of BitFlat with parity fields from a BER of 10−310^{-3} on, while BitFlat with parity fields is 0.18 to 0.39 dB better at 10−410^{-4}.

Burst errors and packet loss

We tested two further channels and summarize the results here. When bit errors occur in bursts, a BCH code designed for independent errors is not suitable, and we use a Reed-Solomon code instead. With this code, the sectioned codec outperforms our reproduction of Robust Turbo-DDCM on all 12 burst channels tested, by 0.39 to 4.30 dB, although the margin becomes small for very long bursts (Appendix D). For packet loss, we protected the learned code MLCC++ [41] with one erasure code and compared it with the robust codec Every Packet Counts [9]. The protected codec is 0.14 to 0.92 dB better on Kodak and 0.24 to 0.86 dB worse on images of two megapixels, where the comparison is only indicative because the image sets may differ (Appendix E).

Discussion

Sectioned codebooks improve Turbo-DDCM because they draw the noise from a much larger pool of vectors. The joint index is an efficient description of a set, but the set is selected from a codebook that is small for the number of bits spent on it, and the dense codebook of Turbo-DDCM cannot grow. The steering model also indicates the remaining potential. Sections of 2202^{20} vectors reach 83% of the bound, and larger sections close the gap only slowly. Two extensions concern the encoder alone. The sections of a step currently choose simultaneously for the same residual, whereas the encoders of sparse regression codes let each section react to the choices of the previous ones [14]. The model also treats all steps alike, although the steps of a diffusion process differ in their effect on the image.

The results on error resilience raise the question of when robustness by design can be better than a channel code. Appendix § 4.6 derives a limit on this advantage, which we call the budget of a robust codec. A robust codec cannot deliver more than the reference codec would deliver at the capacity of the channel. Its possible gain over a code designed for the worst channel state depends on the range of capacity that the channel states span and on the slope of the rate-distortion curve. For BERs up to 10−210^{-2}, the budget is a few tenths of a decibel, and all gains of robust codecs that we observed lie within it. Robustness by design can be expected to pay off when the capacity range is wide, when the blocks are short, or when no useful bound on the channel state is known. We therefore suggest that evaluations of robust codecs include a codec protected by one channel code at equal total bits.

Codecbpp10−410^{-4}3×10−43 \times10^{-4}10−310^{-3}3×10−33 \times10^{-3}10−210^{-2}
Kodak, 24 images
Robust Turbo-DDCM [5]0.100324.1023.7022.5120.7617.73
BitFlat, parity [6]0.099724.8524.6123.9622.8320.40
BitFlat, Hamming [6]0.100024.5824.5824.5624.4723.29
BitFlat, SECDED [6]0.100324.4624.4624.4524.3823.66
Turbo-DDCM + BCH0.099724.6424.6424.6424.6424.62
Sectioned + BCH (ours)0.099724.9524.9524.9524.9524.93
Robust Turbo-DDCM [5]0.072823.6223.2322.2320.6017.74
BitFlat, parity [6]0.073524.3624.1523.6422.6120.32
Turbo-DDCM + BCH0.072824.1024.1024.1024.1024.10
Sectioned + BCH (ours)0.072824.4624.4624.4624.4624.46
DIV2K, 100 images
Robust Turbo-DDCM [5]0.100323.2822.8621.6219.7016.64
BitFlat, parity [6]0.099724.2224.0023.3322.0419.31
BitFlat, Hamming [6]0.100023.8423.8423.8223.7122.64
BitFlat, SECDED [6]0.100323.7023.7023.6923.6322.91
Turbo-DDCM + BCH0.099723.9223.9223.9223.9223.92
Sectioned + BCH (ours)0.099724.3824.3824.3824.3824.36
Robust Turbo-DDCM [5]0.072822.6222.2521.1519.4316.51
BitFlat, parity [6]0.073523.6123.4322.8221.6019.21
Turbo-DDCM + BCH0.072823.1823.1823.1823.1823.17
Sectioned + BCH (ours)0.072823.6923.6923.6923.6923.69

Table 3. PSNR (dB) under independent bit errors. The values of the robust codecs are those reported in [6]. The BCH-protected codecs use the smallest number of bits of each group. The best value of each group is in bold.

Our study has some limitations as well. BitFlat and Every Packet Counts are not publicly available, so the comparisons with them rely on reported values. The sectioned codec was evaluated with one diffusion model, on crops of 512×512512 \times512 pixels and at rates up to 0.194 bpp. The fallback and the burst channels were tested on Kodak only. With sections of 2202^{20} vectors, encoding is slower than with Turbo-DDCM, and the encoding times come from a research implementation on a single device. The channel code is a BCH code with bounded-distance decoding, and a modern code would leave more bits for the image.

Conclusion

We proposed sectioned codebooks for zero-shot diffusion image compression. Each section of a coded step selects its best vector from a large codebook of its own, and a model based on order statistics predicts the resulting gain over the joint index of Turbo-DDCM. With sections of 2202^{20} vectors built from Hadamard matrices, the PSNR improves by 0.18 to 0.51 dB at equal bits on three datasets, which corresponds to 16 to 21% fewer bits at equal PSNR. Protected by one BCH code, the sectioned codec outperforms the reported results of Robust Turbo-DDCM and BitFlat at all tested BERs, with a clear margin at high BERs and a marginal one at the lowest. The gain is also sufficient to double the BER range of the code without falling below Turbo-DDCM. A codec protected by one channel code is thus a simple and strong baseline for the evaluation of robust codecs. Future work will study encoders that choose the sections of a step sequentially and the extension to other diffusion models and larger images.

Declarations

a) Funding: This research did not receive any specific grant from funding agencies in the public, commercial, or not-for-profit sectors.

b) Competing interests: The author declares no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper.

c) Data and code availability: The implementation and the numerical results of all experiments will be released with the article. The images and the pretrained codecs are available from their original distributors under their respective terms.

Declaration of generative AI and AI-assisted technologies in the manuscript preparation process

During the preparation of this work the author used Claude to assist with the experimental analysis, interpretation of results, and drafting of the manuscript. The author reviewed and edited the content generated by AI and takes full responsibility for the final content of the manuscript.

Appendix A: Comparison rules and channel codes

An image is transmitted with a total of nn channel bits. The channel has a state θ\theta that is unknown to the sender, which is the BER of a binary channel or the loss rate of a packet channel, and a robust codec is tested over a range 0≤θ≤θˉ0 \le\theta\le\bar{\theta}. A robust codec spends all nn bits on a representation that tolerates errors. The protected codec spends kk bits on the bitstream of an unmodified codec and the remaining bits on the parity of one channel code.

The comparison follows four rules. The totals are equal, and every transmitted bit of the protected codec counts, including parity, padding and packet headers. The images and quality measures are those of the reported results. The code is designed once for the tested range, and the same bitstream is evaluated at every channel state. For bit errors, the code is designed for the highest state θˉ\bar{\theta}. For packet loss, where the robust codec is trained for a uniformly distributed loss rate, the design maximizes the expected quality under that distribution. Finally, the evaluation is exact, because the failure probability in (4) is the tail of a known distribution.

For independent bit errors we use shortened binary BCH codes [34, 35]. The primitive narrow-sense BCH code of length 2m−12^m-1 that corrects tt errors has r(t)≤mtr(t) \le mt parity bits, given by the sizes of the cyclotomic cosets of 1,…,2t1,\ldots,2t modulo 2m−12^m-1, and shortening preserves both numbers. For a total of nn bits and an operating point of kk bits, tt is the largest value with k+r(t)≤nk+r(t) \le n, and m=15m=15 suffices for the totals used here. The decoding fails when more than tt of the k+r(t)k+r(t) transmitted bits are flipped, so that

Pf(p)=Pr⁡[Bin⁡(k+r(t),p)>t].(5)P_{\mathrm{f}}(p)=\Pr\left[\operatorname{Bin}(k+r(t),p)>t\right]. \tag*{(5)}

A word with more than tt errors could also lie within distance tt of another codeword and be accepted as valid. The fraction of such words is below 2−252^{-25} for all codes used here, and we neglect this case. We verified the decoding rule by transmitting the bitstreams of all images through an independent implementation of the codes [42]. Every word was recovered exactly when it had at most tt errors, and no wrong bitstream was accepted.

A stronger code than BCH with bounded-distance decoding would leave more bits for the image. To bound this benefit, we use the converse of Polyanskiy, Poor and Verdú [43], Theorem 35. No code with 2k2^k messages and nn bits fails less often than one that assigns every message the 2−n2^{-n} lightest error patterns. We compute this lower bound on the failure probability exactly and refer to the resulting design as the limit of any code.

For burst errors we use a channel with two states of the Gilbert-Elliott type [44, 45]. The number of flipped bits is then no longer binomial, and its exact distribution follows from a forward recursion over the bits. The code for bursts is a Reed-Solomon code [46], which corrects any tt corrupted symbols with 2t2t parity symbols. For packet loss, the bitstream is spread over NN packets with a maximum-distance-separable code, so that any KK received packets recover it [47]. The failure probability for independent loss with rate qq is Pr⁡[Bin⁡(N,1−q)<K]\Pr[\operatorname{Bin}(N,1-q)<K].

Appendix B: Detailed results for bit errors

This appendix describes the reported configurations of the robust codecs and gives the complete comparison in PSNR, LPIPS and DISTS.

Figure 4 shows how the compared codecs use the bits of an image. Robust Turbo-DDCM [5] sends each selected vector as a field of 15 bits. BitFlat [6] gives every field a codebook of 512 vectors, which shortens the field, and adds a parity bit, a shortened Hamming code or a SECDED code to every field. The reported results cover six configurations with 24 coded steps. These are Robust Turbo-DDCM at 26 280 and 19 080 bits per image, BitFlat with parity, Hamming and SECDED fields at 26 136, 26 208 and 26 280 bits, and BitFlat with parity fields at 19 272 bits. The BERs are 10−410^{-4}, 3×10−43 \times10^{-4}, 10−310^{-3}, 3×10−33 \times10^{-3} and 10−210^{-2}.

Figure 4. Bits of one image with 24 coded steps in five codecs. Lengths are proportional to bits. The BCH codes are designed for a BER of 10−210^{-2}.

Table IV gives the results for Turbo-DDCM with one BCH code. At 26 136 bits, the code protects 91 vectors per step in 21 600 bits with 4 530 parity bits, which correct up to

Robust codecbppPSNR (dB) 131030100LPIPS (10−310^{-3}) 131030100DISTS (10−310^{-3}) 131030100
Kodak, 24 images
Robust Turbo-DDCM0.0728+0.48+0.87+1.87+3.50+6.36−28−42−85−166−343−21−28−49−86−164
BitFlat, parity0.0735−0.26−0.05+0.46+1.49+3.78+21+14−2−41−156+13+11+2−18−72
Robust Turbo-DDCM0.1003+0.54+0.94+2.13+3.88+6.91−29−42−87−170−360−17−24−48−86−175
BitFlat, parity0.0997−0.21+0.03+0.68+1.81+4.22+11+4−16−55−166+5+2−9−29−73
BitFlat, Hamming0.1000+0.06+0.06+0.08+0.17+1.35−4−4−4−7−43−3−3−3−5−22
BitFlat, SECDED0.1003+0.18+0.18+0.19+0.26+0.98−11−11−11−13−34−7−7−7−8−18
DIV2K, 100 images
Robust Turbo-DDCM0.0728+0.56+0.93+2.03+3.75+6.66−29−39−80−156−330−20−26−45−83−167
BitFlat, parity0.0735−0.40−0.22+0.39+1.61+3.99+24+19+1−39−141+18+15+5−14−62
Robust Turbo-DDCM0.1003+0.66+1.08+2.32+4.24+7.29−28−40−79−157−339−19−26−46−83−172
BitFlat, parity0.0997−0.30−0.08+0.59+1.88+4.61+15+10−8−48−158+11+8−2−22−75
BitFlat, Hamming0.1000+0.10+0.10+0.12+0.23+1.29−5−5−6−9−38−5−5−5−7−23
BitFlat, SECDED0.1003+0.24+0.24+0.25+0.31+1.02−12−12−12−14−34−9−9−9−10−21

Table 4. Turbo-DDCM with one BCH code minus the reported values of the robust codecs, at equal total bits. The columns are the BERs in units of 10−410^{-4}. Positive PSNR differences and negative LPIPS and DISTS differences favour the BCH-protected codec.

309 errors. The code rate is 0.83, and the image is lost with probability 2×10−32 \times10^{-3} at a BER of 10−210^{-2}. This baseline is better than Robust Turbo-DDCM in all three measures at every BER. Against BitFlat with parity fields, the PSNR difference on Kodak is −0.21-0.21 dB at 10−410^{-4} and +0.03+0.03 dB at 3×10−43 \times10^{-4}, and it grows to +4.22+4.22 dB at 10−210^{-2}. Against the Hamming and SECDED fields, the baseline is 0.06 to 0.26 dB better up to 3×10−33 \times10^{-3}, which is within the accuracy of the comparison, and 0.98 and 1.35 dB better at 10−210^{-2}. A code in every field spends four or five check bits on ten information bits, whereas one code over the whole bitstream reaches a failure probability of 3×10−43 \times10^{-4} at a code rate of 0.82. The results on DIV2K follow the same pattern.

Table 5 gives the results for the sectioned codec. On Kodak, the sectioned bitstream has exactly the bits of the joint index that it replaces, so the codes and their failure probabilities are unchanged and only the quality of a received image improves. On DIV2K, two of the six designs use the nearest available sectioned layout, which differs by 9 bits per step. All differences in the table favour the sectioned codec.

Robust codecbppPSNR (dB) 131030100LPIPS (10−310^{-3}) 131030100DISTS (10−310^{-3}) 131030100
Kodak, 24 images
Robust Turbo-DDCM0.0728+0.84+1.23+2.23+3.86+6.72−53−67−110−191−368−38−45−66−103−181
BitFlat, parity0.0735+0.10+0.31+0.82+1.85+4.14−4−11−27−66−181−4−6−15−35−89
Robust Turbo-DDCM0.1003+0.85+1.25+2.44+4.19+7.22−45−58−103−186−376−26−33−57−95−185
BitFlat, parity0.0997+0.10+0.34+0.99+2.12+4.53−5−12−32−71−182−4−7−18−38−83
BitFlat, Hamming0.1000+0.37+0.37+0.39+0.48+1.65−20−20−20−23−59−12−12−12−14−31
BitFlat, SECDED0.1003+0.49+0.49+0.50+0.57+1.29−27−27−27−29−50−16−16−16−17−28
DIV2K, 100 images
Robust Turbo-DDCM0.0728+1.07+1.44+2.54+4.26+7.18−58−68−109−185−360−40−46−65−103−187
BitFlat, parity0.0735+0.08+0.26+0.87+2.09+4.48−2−7−25−65−168−1−4−14−33−82
Robust Turbo-DDCM0.1003+1.10+1.52+2.76+4.68+7.74−47−59−98−176−357−31−38−58−95−184
BitFlat, parity0.0997+0.16+0.38+1.05+2.34+5.05−5−10−28−68−176−2−5−15−35−87
BitFlat, Hamming0.1000+0.54+0.54+0.56+0.67+1.73−24−24−25−28−57−17−17−17−19−34
BitFlat, SECDED0.1003+0.68+0.68+0.69+0.75+1.47−31−31−31−33−52−21−21−21−22−33

Table 5. Sectioned codec with one BCH code minus the reported values of the robust codecs, in the format of Table IV.

The code must be designed for a maximum BER, and a wider range costs quality. For Turbo-DDCM, a design for a BER of 2×10−22 \times10^{-2} has 72 vectors per step and a code rate of 0.68, and it is about 0.30 dB below the design for 10−210^{-2}. With sectioned codebooks, the design for 2×10−22 \times10^{-2} uses 736 bits per step and a code that corrects 613 errors, and it is 0.28 dB below the sectioned design for 10−210^{-2}. Beyond the design range the code fails as a whole. The design for 10−210^{-2} at 26 136 bits loses the image with probability 0.59 at a BER of 1.2% and 1.00 at 1.5%.

Reproduction of Robust Turbo-DDCM

The comparison with BitFlat relies on reported values, so our evaluation must match the reported one in images and quality measures. Table I of [6] includes Robust Turbo-DDCM, whose code is public. We reproduced this codec under the reported settings, with independent bit flips applied to its bitstream. Table VI lists the differences between our PSNR values and the reported ones. They do not exceed 0.15 dB on Kodak and 0.09 dB on DIV2K, and the differences in LPIPS and DISTS are at most 0.005 and 0.009. Differences of this size are within the variation between channel realizations. The agreement holds for a codec that we can run and is not a confidence interval for the reported values of BitFlat. Differences of one or two tenths of a decibel should be regarded as inconclusive.

For the third dataset of [6], the CLIC 2020 professional test set [38], we could not reproduce the reported values. On the central crops of its 250 images, our PSNR for Robust Turbo-DDCM is 0.60 dB higher than the reported value at a BER of 10−410^{-4}, which suggests that the reported values were obtained on different images. We therefore do not use the reported values for this dataset and compare the codecs in our own evaluation. In this comparison, Turbo-DDCM with one BCH code is 0.74

ImagesVectors10−410^{-4}3×10−43 \times10^{-4}10−310^{-3}3×10−33 \times10^{-3}10−210^{-2}
Kodak73−0.05-0.05−0.07-0.07−0.01-0.01+0.02+0.02−0.03-0.03
53+0.01+0.01+0.07+0.07+0.12+0.12−0.15-0.15+0.10+0.10
DIV2K730.00−0.08-0.08+0.07+0.07−0.07-0.07−0.09-0.09

Table VI. Our reproduction of Robust Turbo-DDCM minus the reported PSNR (dB), with 73 vectors per step (0.1003 bpp) and, on Kodak, also with 53 vectors (0.0728 bpp).

to 8.91 dB better than Robust Turbo-DDCM, and the sectioned codec is 1.13 to 9.30 dB better.

Burst errors

Both papers on robust diffusion codecs leave burst errors untested. We model bursts with a two-state channel [44, 45]. The good state flips no bits, and the bad state flips every bit with probability 0.5. The mean duration of the bad state is 8, 64, 512 or 4096 bits, and the fraction of time spent in it is set so that the average BER is 10−310^{-3}, 3×10−33 \times10^{-3} or 10−210^{-2}. We measured Robust Turbo-DDCM on these channels at 19080 bits per image and computed the failure probabilities of the codes exactly.

A BCH code designed for independent errors is not suitable for this channel (Figure 5). At an average BER of 10−210^{-2}, it loses 10.2% of the images with bursts of 8 bits and 28.1% with bursts of 512 bits. A Reed-Solomon code with symbols of 11 bits, designed once for all tested channels at 10−210^{-2}, keeps Turbo-DDCM on par with or better than Robust Turbo-DDCM on every channel, at a cost of 0.76 dB when the errors are independent. Its advantage decreases with the burst length, from 3.94 dB for bursts of 8 bits to 1.09 dB for bursts of 512 bits at an average BER of 10−210^{-2}. For bursts of 4096 bits, a fifth of the bitstream, the advantage is 0.05 to 0.44 dB, while the measured mean of the robust codec varies by 0.27 to 0.46 dB between realizations, so no clear advantage remains. A robust codec benefits from burstiness, because a burst that flips many bits of one field still corrupts only one vector, whereas a long burst can exceed the correction capability of any code that the bit budget allows.

Figure 5. PSNR on Kodak at 19 080 bits per image under burst errors, as a function of the mean burst length. The leftmost point of each panel is the channel with independent errors. The Reed-Solomon code is designed once for all channels of the right panel.

Sectioned codebooks add 0.36 dB to both coded designs wherever the code holds. With the Reed-Solomon code, the sectioned codec is better than Robust Turbo-DDCM on all 12 burst channels, by 0.39 to 4.30 dB in general and by 0.39 to 0.79 dB for the longest bursts.

Packet loss

We repeated the comparison for packet loss with a learned codec optimized for mean squared error and an erasure code. The robust codec is Every Packet Counts (EPC) [9], which is trained to reconstruct the image from the latent packets that arrive, for loss rates between 0 and 30%. Its code and weights are not public, so we use the reported mean PSNR at independent loss rates of 5, 10 and 20%, on the Kodak images at 0.125, 0.217 and 0.346 bpp and on images of the CLIC 2020 dataset at 0.090, 0.148 and 0.228 bpp. EPC is built on a variant of the HPCM codec [48]. Our reference codec is MLIC++ [41], a codec of the same generation, so the two systems do not share a base codec.

The protected codec compresses every image with MLIC++ at one operating point for the whole dataset and uses the actual entropy coder. The bitstream is spread over packets of at most 1500 bytes with one maximum-distance-separable code, and the packet headers count in the rate. Following the training range of EPC, the design maximizes the expected PSNR for a loss rate that is uniform between 0 and 30%. An image is sent in at least eight packets on Kodak and at least 32 packets on the larger images. Two differences in accounting favour EPC. Its rates include no packet headers, and it assumes that the packet with the hyperprior always arrives. The larger images are the 41 images of the CLIC 2020 professional validation set [49] at full size, with 2.2 megapixels on average. EPC only states that it uses the CLIC 2020 dataset at 2K resolution, so the comparison on these images is indicative. We also include unequal protection [24, 25, 26], in which the bitstream of MLIC++ is divided into at most four levels with decreasing protection along the decoding order.

Table VII shows the results. On Kodak, one erasure code is better than EPC in all nine cases, by 0.14 to 0.92 dB, and unequal protection adds 0.15 to 0.34 dB. On the larger images the order is reversed, and one code is 0.24 to 0.86 dB worse than EPC. The reported values themselves differ between the two datasets. At 5% loss, EPC is 1.84 to 2.83 dB below error-free MLIC++ at the same rate on Kodak and only 0.54 to 0.68 dB below it on the larger images. A code that must survive a loss of 30% gives up about a third of the rate, which costs 1.33 to 2.06 dB on Kodak and 1.40 to 1.45 dB on the larger images. The two datasets differ in the images, in the number of packets per image and possibly in the selection of images behind the reported values. We report the reversal as an observation and do not attribute it to the image size.

ImagesbppEPC, reported 5%EPC, reported 10%EPC, reported 20%One code minus EPC 5%One code minus EPC 10%One code minus EPC 20%Graded minus one code 5%Graded minus one code 10%Graded minus one code 20%Budget 5%Budget 10%Budget 20%
Kodak0.12527.5527.4727.21+0.51+0.52+0.14+0.18+0.19+0.341.121.191.57
0.21728.9828.8428.56+0.73+0.82+0.51+0.15+0.16+0.291.281.221.38
0.34630.4330.2729.93+0.77+0.92+0.86+0.23+0.22+0.201.811.581.57
CLIC0.09030.5530.3830.01−0.86−0.69−0.441.401.400.81
0.14831.9331.7931.45−0.84−0.70−0.361.451.450.68
0.22833.2833.1432.78−0.74−0.59−0.241.421.420.73

Table VII. MLIC++ protected by erasure codes designed for loss rates from 0 to 30%, compared with the reported PSNR (dB) of Every Packet Counts at independent loss rates of 5, 10 and 20%. Graded denotes unequal protection. The budget is defined in Appendix F.

The budget of a robust codec

This appendix derives a limit on the gain that a robust codec can achieve over a codec protected by a channel code, and compares the observed gains with this limit.

Let C(θ)C(\theta) be the capacity of the channel per transmitted bit, which is 1−h(p)1-h(p) for independent bit errors with the binary entropy function hh and 1−q1-q for packet loss. Whatever the scheme, the receiver obtains at most nC(θ)nC(\theta) bits of information about the image from nn channel uses in state θ\theta [29]. No scheme can therefore deliver a better expected quality than the distortion-rate function of the images allows at nC(θ)nC(\theta) bits. This function is unknown, and we use the quality of the reference codec at that number of bits instead,

G(θ)=Q(nC(θ)),(6)G(\theta) = Q(nC(\theta)), \tag*{(6)}
B(θ)=G(θ)−S(θ).(7)B(\theta) = G(\theta) - S(\theta). \tag*{(7)}

We call B(θ)B(\theta) the budget at state θ\theta. A robust codec whose gain over the protected codec exceeds the budget delivers more than the reference codec could deliver over an error-free channel of the same capacity. Such a gain would be an advance in compression and not in resilience. The budget is a yardstick and not a bound for every robust codec, because a codec with a better rate-distortion performance than the reference codec can exceed it.

The size of the budget follows from two numbers. If the quality of the codec grows by ss decibels per doubling of the rate, and the code carries a fraction ρ<1\rho< 1 of the capacity at θˉ\bar{\theta}, then

B(θ)≈slog⁡2C(θ)ρC(θˉ).(8)B(\theta) \approx s \log_{2} \frac{C(\theta)}{\rho C(\bar{\theta})}. \tag*{(8)}

For BERs up to 10−210^{-2} the capacity decreases from 1 to 0.919, and for loss rates up to 30% it decreases to 0.7. With the slopes of the codecs used here, 1.1 dB per doubling for Turbo-DDCM and 1.7 to 2.8 dB for MLIC++, an ideal code leaves a robust codec at most 0.14 dB at a vanishing BER and about one decibel at a vanishing loss rate.

Figure 6 compares the gains of the robust codecs with the budget. For bit errors at 26 136 bits on Kodak and a BER of 10−410^{-4}, the budget of Turbo-DDCM with the BCH code is 0.26 dB, and BitFlat with parity fields gains 0.21 dB, which is 78% of the budget. At 10−210^{-2} the budget shrinks to 0.14 dB, while BitFlat with parity fields is 4.22 dB below the protected codec. With the sectioned codec as the reference, the budget at 10−410^{-4} is 0.22 to 0.46 dB, and no reported value of a robust codec exceeds the protected codec. For packet loss the budget is larger, with 1.12 to 1.81 dB on Kodak and 0.68 to 1.45 dB on the larger images (Table VII). EPC gains nothing on Kodak and 34 to 61% of the budget on the larger images. All observed gains of the robust codecs are within the budget.

Figure 6. The budget (7) and the PSNR gains of the robust codecs over the protected codec. Left: bit errors on Kodak at about 0.10 bpp. A negative gain means that the protected codec is better. Right: packet loss on images of two megapixels.

References

  1. [1]L. Theis, T. Salimans, M. D. Hoffman, and F. Mentzer, “Lossy compression with Gaussian diffusion,” 2022, preprint, arXiv:2206.08889v2.arxiv.org/abs/2206.08889
  2. [2]J. Vonderfecht and F. Liu, “Lossy compression with pretrained diffusion models,” in 13th International Conference on Learning Representations (ICLR), 2025, arXiv:2501.09815v1.arxiv.org/abs/2501.09815
  3. [3]G. Ohayon, H. Manor, T. Michaeli, and M. Elad, “Compressed image generation with denoising diffusion codebook models,” in Proceedings of the 42nd International Conference on Machine Learning (ICML), 2025, arXiv:2502.01189v4.arxiv.org/abs/2502.01189
  4. [4]A. Vaisman, G. Ohayon, H. Manor, M. Elad, and T. Michaeli, “Turbo-DDCM: Fast and flexible zero-shot diffusion-based image compression,” in 14th International Conference on Learning Representations (ICLR), 2026, arXiv:2511.06424v2.arxiv.org/abs/2511.06424
  5. [5]A. Vaisman, G. Pomerants, and R. Lapid, “On the robustness of diffusion-based image compression to bit-flip errors,” 2026, preprint, arXiv:2604.05743v2; accepted at the AIGENS workshop at CVPR 2026.arxiv.org/abs/2604.05743
  6. [6]X. Deng, L. Zhou, H. Chen, S. Wu, L. Kong, N. Li, H. Zhao, and D. Ma, “BitFlat: Error-localized bitstream design for robust diffusion image compression,” 2026, preprint, arXiv:2610.25682v1.arxiv.org/abs/2610.25682
  7. [7]H. Sha, M. Dong, Q. Luo, M. Lu, H. Chen, and Z. Ma, “Towards loss-resilient image coding for unstable satellite networks,” Proceedings of the AAAI Conference on Artificial Intelligence, vol. 39, no. 12, pp. 12506 to 12514, 2025.
  8. [8]S. Wang, J. Dai, X. Qin, K. Yang, K. Niu, and P. Zhang, “ResiComp: Loss-resilient image compression via dual-functional masked visual token modeling,” IEEE Transactions on Circuits and Systems for Video Technology, vol. 35, no. 7, pp. 7181 to 7195, 2025. Y. Wei, C. Zhou, Y. Shi, J. Wang, and G. Lu, “Every packet counts: Dispersing information for loss-resilient learned image compression,” in Proceedings of the 34th ACM International Conference on Multimedia, 2026, pp. 11 356 to 11 364.
  9. [10]F. Zhang and L. Xu, “Trajectory-consistent denoising diffusion codebook models for zero-shot high-fidelity image compression at ultra-low bitrates,” in Proceedings of the Thirty-Fifth International Joint Conference on Artificial Intelligence (IJCAI), 2026, pp. 2064 to 2072.DOI
  10. [11]Y. Cheng, F. Meng, H. Qi, H. Shen, Z. Zhang, Y. Wang, and Y. Liang, “A spatio-temporal hierarchical diffusion framework for training-free perceptual remote sensing image compression,” IEEE Transactions on Geoscience and Remote Sensing, vol. 64, pp. 1 to 17, 2026.DOI
  11. [12]A. Joseph and A. R. Barron, “Least squares superposition codes of moderate dictionary size are reliable at rates up to capacity,” IEEE Transactions on Information Theory, vol. 58, no. 5, pp. 2541 to 2557, 2012.
  12. [13]R. Venkataramanan, A. Joseph, and S. Tatikonda, “Lossy compression via sparse linear regression: Performance under minimum-distance encoding,” IEEE Transactions on Information Theory, vol. 60, no. 6, pp. 3254 to 3264, 2014.
  13. [14]R. Venkataramanan, T. Sarkar, and S. Tatikonda, “Lossy compression via sparse linear regression: Computationally efficient encoding and decoding,” IEEE Transactions on Information Theory, vol. 60, no. 6, pp. 3265 to 3278, 2014.
  14. [15]R. Venkataramanan, S. Tatikonda, and A. Barron, “Sparse regression codes,” Foundations and Trends in Communications and Information Theory, vol. 15, no. 1-2, pp. 1 to 195, 2019.
  15. [16]A. No and T. Weissman, “Rateless lossy compression via the extremes,” IEEE Transactions on Information Theory, vol. 62, no. 10, pp. 5484 to 5495, 2016.
  16. [17]R. Salami, C. Laflamme, J.-P. Adoul, A. Kataoka, S. Hayashi, T. Moriya, C. Lamblin, D. Massaloux, S. Proust, P. Kroon, and Y. Shoham, “Design and description of CS-ACELP: A toll quality 8 kb/s speech coder,” IEEE Transactions on Speech and Audio Processing, vol. 6, no. 2, pp. 116 to 130, 1998.DOI
  17. [18]C. Rush, A. Greig, and R. Venkataramanan, “Capacity-achieving sparse superposition codes via approximate message passing decoding,” IEEE Transactions on Information Theory, vol. 63, no. 3, pp. 1476 to 1500, 2017.
  18. [19]J. Barbier and F. Krzakala, “Approximate message-passing decoder and capacity achieving sparse superposition codes,” IEEE Transactions on Information Theory, vol. 63, no. 8, pp. 4894 to 4927, 2017.DOI
  19. [20]N. Ailon and B. Chazelle, “The fast Johnson-Lindenstrauss transform and approximate nearest neighbors,” SIAM Journal on Computing, vol. 39, no. 1, pp. 302 to 322, 2009.
  20. [21]J. A. Tropp, “Improved analysis of the subsampled randomized Hadamard transform,” Advances in Adaptive Data Analysis, vol. 3, no. 1-2, pp. 115 to 126, 2011.arxiv.org/abs/1011.1595
  21. [22]O. Ronai, V. Kulikov, and T. Michaeli, “H-DDCM: Hadamard codebooks for fast generative image and video compression,” 2026, preprint, arXiv:2610.18125v1.arxiv.org/abs/2610.18125
  22. [23]G. Reeves and R. Venkataramanan, “Lossy compression via sparse regression codes: Generalized construction and finite-length bounds,” 2026, preprint, arXiv:2608.14494v1.arxiv.org/abs/2608.14494
  23. [24]A. Albanese, J. Blömer, J. Edmonds, M. Luby, and M. Sudan, “Priority encoding transmission,” IEEE Transactions on Information Theory, vol. 42, no. 6, pp. 1737 to 1744, 1996.
  24. [25]A. E. Mohr, E. A. Riskin, and R. E. Ladner, “Unequal loss protection: Graceful degradation of image quality over packet erasure channels through forward error correction,” IEEE Journal on Selected Areas in Communications, vol. 18, no. 6, pp. 819 to 828, 2000.DOI
  25. [26]S. Dumitrescu, X. Wu, and Z. Wang, “Globally optimal uneven error-protected packetization of scalable code streams,” IEEE Transactions on Multimedia, vol. 6, no. 2, pp. 230 to 239, 2004.
  26. [27]R. Hamzaoui, V. Stanković, and Z. Xiong, “Optimized error protection of scalable image bit streams,” IEEE Signal Processing Magazine, vol. 22, no. 6, pp. 91 to 107, 2005.
  27. [28]Y. Cheng, Z. Zhang, H. Li, A. Arapin, Y. Zhang, Q. Zhang, Y. Liu, K. Du, X. Zhang, F. Y. Yan, A. Mazumdar, N. Teamster, and J. Jiang, “GRACE: Loss-resilient real-time video through neural codecs,” in 21st USENIX Symposium on Networked Systems Design and Implementation (NSDI 24), 2024, accessed 11 November 2026. [Online]. Available: https://www.usenix.org/conference/nsdi24/presentation/chengarxiv.org/abs/2305.12333
  28. [29]T. M. Cover and J. A. Thomas, Elements of Information Theory, 2nd ed. Hoboken, NJ: Wiley, 2006.
  29. [30]S. Shamai and A. Steiner, “A broadcast approach for a single-user slowly fading MIMO channel,” IEEE Transactions on Information Theory, vol. 49, no. 10, pp. 2617 to 2635, 2003.DOI
  30. [31]C. Tian, A. Steiner, S. Shamai, and S. N. Diggavi, “Successive refinement via broadcast: Optimizing expected distortion of a Gaussian source over a Gaussian fading channel,” IEEE Transactions on Information Theory, vol. 54, no. 7, pp. 2903 to 2918, 2008.DOI
  31. [32]A. Tajer, A. Steiner, and S. Shamai (Shitz), “The broadcast approach in communication networks,” Entropy, vol. 23, no. 1, p. 120, 2021.DOI
  32. [33]R. Rombach, A. Blattmann, D. Lorenz, P. Esser, and B. Ommer, “High-resolution image synthesis with latent diffusion models,” in 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2022, pp. 10 674 to 10 685.arxiv.org/abs/2112.10752
  33. [34]R. C. Bose and D. K. Ray-Chaudhuri, “On a class of error correcting binary group codes,” Information and Control, vol. 3, no. 1, pp. 68 to 79, 1960.
  34. [35]S. Lin and D. J. Costello, Error Control Coding, 2nd ed. Upper Saddle River, NJ: Pearson Prentice Hall, 2004.
  35. [36][dataset] Eastman Kodak Company, “Kodak lossless true color image suite,” 1993, accessed 10 November 2026. [Online]. Available: https://rok.us/graphics/kodak/
  36. [37]E. Agustsson and R. Timofte, “NTIRE 2017 challenge on single image super-resolution: Dataset and study,” in 2017 IEEE Conference on Computer Vision and Pattern Recognition Workshops (CVPRW), 2017, pp. 1122 to 1131.DOI
  37. [38][dataset] Workshop and Challenge on Learned Image Compression, “CLIC 2020 professional test dataset,” 2020, accessed 13 November 2026. [Online]. Available: https://data.vision.ee.ethz.ch/cvl/clic/test/CLIC2020Professional_test.zip
  38. [39]R. Zhang, P. Isola, A. A. Efros, E. Shechtman, and O. Wang, “The unreasonable effectiveness of deep features as a perceptual metric,” in 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2018, pp. 586 to 595.arxiv.org/abs/1801.03924
  39. [40]K. Ding, K. Ma, S. Wang, and E. P. Simoncelli, “Image quality assessment: Unifying structure and texture similarity,” IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 44, no. 5, pp. 2567 to 2581, 2022.
  40. [41]W. Jiang, J. Yang, Y. Zhai, F. Gao, and R. Wang, “MLIC++: Linear complexity multi-reference entropy modeling for learned image compression,” ACM Transactions on Multimedia Computing, Communications, and Applications, vol. 21, no. 5, pp. 1 to 25, 2025.DOI
  41. [42]M. Hostetter, “Galois: A performant NumPy extension for Galois fields,” 2020, software, version 0.4.11. Accessed 12 November 2026. [Online]. Available: https://github.com/mhostetter/galois
  42. [43]Y. Polyanskiy, H. V. Poor, and S. Verdú, “Channel coding rate in the finite blocklength regime,” IEEE Transactions on Information Theory, vol. 56, no. 5, pp. 2307 to 2359, 2010.DOI
  43. [44]E. N. Gilbert, “Capacity of a burst-noise channel,” Bell System Technical Journal, vol. 39, no. 5, pp. 1253 to 1265, 1960.DOI
  44. [45]E. O. Elliott, “Estimates of error rates for codes on burst-noise channels,” Bell System Technical Journal, vol. 42, no. 5, pp. 1977 to 1997, 1963.DOI
  45. [46]I. S. Reed and G. Solomon, “Polynomial codes over certain finite fields,” Journal of the Society for Industrial and Applied Mathematics, vol. 8, no. 2, pp. 300 to 304, 1960.DOI
  46. [47]L. Rizzo, “Effective erasure codes for reliable computer communication protocols,” ACM SIGCOMM Computer Communication Review, vol. 27, no. 2, pp. 24 to 36, 1997.DOI
  47. [48]Y. Li, H. Zhang, L. Li, and D. Liu, “Learned image compression with hierarchical progressive context modeling,” 2025, preprint, arXiv:2507.19125v1; presented at ICCV 2025.arxiv.org/abs/2507.19125
  48. [49][dataset] Workshop and Challenge on Learned Image Compression, “CLIC 2020 professional validation dataset,” 2020, accessed 10 November 2026. [Online]. Available: https://data.vision.ee.ethz.ch/cvl/clic/professional_valid_2020.zip

Paper details

vlnc:W_frxsdpb4
Version
v1 · latest
Published
Fresh — added to the Hub within the last 30 days
Added
License
CC BY 4.0
Usage

views · 0

downloads · 0

A view counts once per browser per day, a download each time the PDF is requested.

  • Engineering

Claude

View statement
Looking for related papers…
Contents