WorksEngineering
Zero-shot diffusion image compression with sectioned codebooks and one channel code
Abstract
Zero-shot diffusion codecs such as Turbo-DDCM compress images without any training. In each coded step, the encoder selects noise vectors from one codebook and transmits a single index for the selected set. We propose sectioned codebooks. A coded step is divided into sections, and each section has its own large codebook and transmits its best vector as a short field of bits. A model based on order statistics quantifies how well the bits of a step are used and predicts the gain of sectioned codebooks from measurements of Turbo-DDCM alone. Fast Walsh-Hadamard transforms make sections of vectors practical. At equal bits, sectioned codebooks improve the PSNR of Turbo-DDCM by 0.18 to 0.51 dB on three datasets, improve LPIPS and DISTS, and save 16 to 21% of the bits at equal PSNR. Codecs that are robust to channel errors by design are usually compared only with unprotected codecs. We compare them, at equal total bits, with a codec protected by one channel code. With one BCH code, the sectioned codec outperforms all reported results of Robust Turbo-DDCM and BitFlat on two datasets, by up to 7.74 dB at the highest bit error rate. At the lowest bit error rate its margin over BitFlat is only 0.08 to 0.16 dB. When the code fails, the receiver can still decode the fields as received. Additional experiments cover burst errors and packet loss, and all observed gains of robust codecs stay within a limit derived from the channel capacity.
Introduction
Zero-shot diffusion codecs compress images with a pretrained diffusion model and need no training of their own [1, 2, 3]. The decoder runs the reverse diffusion process, and the encoder controls this process through the noise that is added in each step. In denoising diffusion codebook models (DDCM) [3], both sides generate the same random codebook from a shared seed, and the bitstream identifies the noise vectors that the encoder has selected. Turbo-DDCM [4] is a fast codec of this family. In each coded step it combines about a hundred vectors of a Gaussian codebook with 16 384 entries and transmits the selected set as a single index, which we call the joint index.
It is not known how efficiently this bitstream uses its bits. Turbo-DDCM keeps the codebook at the size of the latent, and its authors list a theoretical account of the method as an open problem [4]. The bitstream is also fragile, because a single flipped bit of the joint index changes most of the selected vectors of a step. Recent work redesigns the bitstream so that it tolerates bit errors. Robust Turbo-DDCM [5] writes every selected vector as a separate field, so that a bit error corrupts only one vector. BitFlat [6] assigns every field a small codebook of its own, protects each field with a short code and discards the vectors whose field is detected as corrupted. For packet loss, learned codecs are trained with random loss of parts of their latent representation [7, 8] [9]. These robust codecs are compared with unprotected codecs or with each other. The classical alternative, an unmodified codec protected by one channel code at the same total number of bits, is missing from these comparisons, although robustness by design also costs bits. The separate fields of Robust Turbo-DDCM, for example, need 41 to 47% more bits than the joint index for the same vectors.
In this paper, we replace the joint index of Turbo-DDCM with sectioned codebooks (Fig. 1). A coded step is divided into sections. Each section has its own codebook and contributes the single vector that best matches the image, which is written as a field of fixed length. To explain why this helps, we introduce the steering of a step, a measure of how strongly the transmitted noise points towards the part of the image that is still missing. A model based on order statistics shows that a section steers more per bit as its codebook grows and that large sections are more efficient than the joint index. Since a codebook with a million vectors cannot be stored, we construct the sections from Hadamard matrices, for which a fast transform computes all correlations of a block. At exactly the bits of Turbo-DDCM, sections of vectors improve the PSNR by 0.18 to 0.51 dB on three datasets, improve LPIPS and DISTS, and reduce the bitrate by 16 to 21% at equal PSNR. The model predicts these gains from measurements of Turbo-DDCM alone.
When we protect the sectioned bitstream with one BCH code and compare it with the robust codecs at equal total bits. The code is designed once for the highest bit error rate (BER) at which the robust codecs are tested. Even the original Turbo-DDCM protected in this way outperforms Robust Turbo-DDCM at every BER from to . The sectioned codec with the same code outperforms all reported results of Robust Turbo-DDCM and BitFlat in PSNR, LPIPS and DISTS. Its margin over BitFlat is 4.14 to 5.05 dB at a BER of and 0.08 to 0.16 dB at , where the comparison with reported values is not conclusive. Even when the code is designed for twice the highest reported BER, the sectioned codec matches Turbo-DDCM with a code for the reported range. When the code fails, the receiver can still decode the fields as received, which a joint index does not allow.
The main contributions of this paper are as follows.
We propose sectioned codebooks for zero-shot diffusion compression, together with a model of the steering of a coded step that predicts their gain over the joint index.
We construct sections of up to vectors from Hadamard matrices and obtain consistent rate-distortion gains over Turbo-DDCM on three datasets.
We show that a codec protected by one channel code is a stronger baseline than the unprotected codecs used so far, and that the sectioned codec with one BCH code outperforms the reported results of two robust diffusion codecs.
Additional results on burst errors and packet loss, and a limit on the gain that robustness by design can achieve, are given in the appendices.
Related work
Zero-shot diffusion codecs differ in how they transmit the noise of a step. DiffC [1] uses reverse channel coding, and its practical implementation codes the noise in chunks of coordinates that are searched separately [2]. DDCM [3] selects one vector of a Gaussian codebook per step, and Turbo-DDCM [4] combines many vectors per step and codes their set jointly. Later codecs change the trajectory that the selections follow [10] or distribute the selections over time, space and channels [11]. In the latter, the channels of the latent are split into groups and one codebook vector is selected for each group. Chunks and channel groups partition the coordinates of the noise, whereas every vector of a section spans all coordinates and the selected vectors are added. BitFlat [6] gives every field a codebook of 512 or 1 024 vectors in order to confine bit errors. It does not compare this layout with the joint index on an error-free channel and does not consider larger codebooks.
Sectioned codebooks have the structure of sparse regression codes. These codes were introduced for the Gaussian channel [12] and achieve the rate-distortion function of Gaussian sources with codewords that add one column from each section of a random matrix [13, 14, 15]. Transmitting the positions of the largest entries of a randomly rotated vector also achieves this function [16], which is the principle behind the selection in Turbo-DDCM. Algebraic codebooks in speech coding place one signed pulse in each of several tracks [17]. Design matrices built from Hadamard matrices make the decoders of sparse superposition codes fast [18, 19], and randomized Hadamard transforms serve as fast random projections [20, 21]. Our codec uses a sparse regression code with signs and equal weights as the noise of a diffusion step. The theory of these codes addresses Gaussian sources and long blocks and does not cover this use. Two concurrent preprints are related to parts of our work. H-DDCM [22] replaces the Gaussian codebook of Turbo-DDCM with randomized Hadamard dictionaries, from which the best vectors are selected and coded jointly. Reeves and Venkataramanan [23] bound the distortion of sparse regression codes with the order statistics that also appear in our model. Neither work studies sections in a diffusion codec or their protection by a channel code.
Protecting an image bitstream with error-correcting and erasure codes, including unequal protection of embedded bitstreams, is well established for wavelet coders [24, 25, 26, 27]. Robust codecs take a different route. Learned codecs are trained to tolerate packet loss [7, 8, 28] [9], and the robust diffusion codecs redesign the bitstream [5, 6]. The separation theorem states that separate source and channel coding is optimal for a known channel [29], and the broadcast approach quantifies what is lost when the channel state is unknown [30, 31, 32]. We use the classical protection as a baseline for robust diffusion codecs, which previous comparisons do not include.
Proposed method
This section reviews the coded step of Turbo-DDCM, introduces a measure of how well a step uses its bits, and presents sectioned codebooks, their fast construction and their protection by one channel code. Figure 1 illustrates the difference between Turbo-DDCM and the proposed codec.
Figure 1. One coded step. (a) Turbo-DDCM selects the best vectors of one codebook and codes the selected set as a joint index. (b) The proposed codec selects the best vector of each section and writes one field per section. One BCH code protects all fields of an image.
Turbo-DDCM
Turbo-DDCM [4] uses a pretrained latent diffusion model [33]. The image is mapped to a latent of values, and the reverse process runs for 30 steps from a fixed starting point. A regular reverse step adds fresh Gaussian noise. A coded step instead adds a noise vector that the encoder selects and the decoder reconstructs from the bitstream. The encoder knows the current prediction of the latent and computes the residual . Both sides generate a codebook of Gaussian vectors from a shared seed. The encoder selects the vectors with the largest correlation magnitude , assigns each the sign of its correlation, and sets to their sum, scaled to unit variance. The bits of the step are the joint index, which identifies the selected set among the possible sets, and the signs. The last steps of the reverse process carry no bits.
Steering of a coded step
The transmitted noise is useful through its component along the residual. For a random codebook, the normalized correlation of a vector is a standard normal variable regardless of the residual. The signed sum of the selected vectors has length along the residual and a squared length of about in the orthogonal directions. This gives the squared correlation between the noise and the residual,
We call the steering of the step. Random noise has , and the coded steps of Turbo-DDCM reach several hundred.
The steering that bits can achieve is bounded. A description with bits cannot approximate the direction of the residual better than the rate-distortion function of a Gaussian vector allows [29], which gives . For the 900 bits per step that serve as an example in this section, the bound is 1 296. The joint index is far from this bound. The largest of standard normal magnitudes exceed the threshold that a fraction of all magnitudes exceeds, and their mean is , where is the normal density. The steering is then . The 91 vectors that Turbo-DDCM transmits with 900 bits give , which is 66% of the bound.
Sectioned codebooks
In the proposed codec, a coded step has sections. Each section is a separate codebook of vectors and contributes the vector with the largest correlation magnitude, together with the sign of its correlation. The selection of a section is written as a field of bits, and the bits of the step are the concatenated fields. The noise is the normalized sum of the selected vectors, as in Turbo-DDCM, and the rest of the codec is unchanged.
From (1), sections of equal size give
where is the expected maximum of standard normal magnitudes. The steering per bit, , increases with the section size. It is 1.08 for vectors, 1.14 for , 1.19 for and 1.22 for , compared with 0.95 for the joint index (Figure 2a). Large sections are more efficient, and their size is limited only by computation. With sections of vectors, 900 bits hold 43 fields and give , which is 83% of the bound. When the bits of a step are not a multiple of the field length, the sections have two sizes and we add their contributions , which overestimates by at most 0.5% for the layouts used here.
Figure 2. Sectioned codebooks on Kodak. (a) Steering per bit as a function of the section size. Lines show the model and markers show measurements. (b) Measured and predicted PSNR gain over Turbo-DDCM with standard errors. Open markers lie outside the range of the fit. (c) Rate-distortion curves with 24 coded steps. Gray is Turbo-DDCM.
Sectioned codebooks differ from Turbo-DDCM in the number of candidate vectors and in the way the selection is written, and the model separates the two effects. If the best vectors of the whole pool of all sections were selected and written as one joint index, each vector would cost about bits more than a field. At equal bits, such an index holds 40 vectors instead of 43 and gives . This suggests that most of the gain over Turbo-DDCM comes from the larger pool and a smaller part from the fields, which the ablation in Section IV-C confirms.
Fast sections from Hadamard blocks
A section of Gaussian vectors has values and can be neither stored nor generated for every step. We therefore construct sections from blocks of vectors that need no storage. Let be the Hadamard matrix of order , let be a permutation of its columns drawn once per step, and let be a vector of standard normal values drawn once per block. The vectors of the block are
Each vector has independent standard normal coordinates, as the model assumes. The correlations of all vectors with the residual are obtained with one fast Walsh–Hadamard transform of the values , taken in the order , which costs additions. A section of vectors consists of blocks with independent , and a smaller section uses the first vectors of one block. The decoder needs no transform, because it regenerates only the selected vectors from (3).
The vectors of a block share the magnitudes of and are not mutually independent. This does not affect a section, which contributes a single vector, and a control experiment with independent Gaussian vectors showed no significant difference (Section IV-C). Dictionaries based on Hadamard matrices are known from sparse superposition codes [18, 19] and from fast random projections [20, 21]. The Gaussian values are specific to our use, in which the sum of the selected vectors replaces Gaussian noise.
Protection with one channel code
For transmission over a noisy channel, we protect the whole bitstream of an image with a single channel code. Of a total of bits, bits are used for the compressed bitstream and the remaining bits for the parity of a shortened binary BCH code [34, 35] that corrects up to bit errors. A bounded-distance decoder returns the bitstream exactly when at most of the transmitted bits are flipped. For independent bit errors with rate , the probability that decoding fails is a binomial tail. If decoding fails, the receiver displays a mid-gray image. With the mean quality of the codec at bits and the quality of the mid-gray image, the expected quality is
This expression is exact and requires no channel simulation. The code is designed once for the highest tested BER. Among the operating points and codes that fit into bits, we select the pair that maximizes (4) at that BER. Appendix A describes the codes in detail.
A sectioned bitstream allows a better reaction to a decoding failure than a mid-gray image. The code is systematic, and every bit pattern of a field identifies a valid vector of its section. The receiver can still decode the fields as received. In this case, a flipped bit replaces the vector of one section and leaves the other sections unchanged. We refer to this option as the fallback.
Experiments
Setup
We implemented sectioned codebooks in the public code of Turbo-DDCM [4], with its pretrained model, weights and seed. Only the selection of the noise differs from the original codec. Following the robust codecs [5, 6], we used the 24 Kodak images [36], the 100 images of the DIV2K validation set [37] and the 250 images of the CLIC 2020 professional test set [38], each cropped to its central pixels. Quality is measured with PSNR, LPIPS [39] and DISTS [40]. Unless stated otherwise, the codecs use 24 coded steps, as the robust codecs do, and sections of vectors. The comparisons with Turbo-DDCM are made per image at exactly the same number of bits, and differences are reported with the standard error of their mean over the images.
For bit errors, we compare with the results reported for Robust Turbo-DDCM [5] and BitFlat [6] at BERs from to . BitFlat was reported with one parity bit per field, which is its main configuration, and with a Hamming or SECDED code per field. Since the code of BitFlat is not public, we take its results from Tables I and IV of [6]. To check that our evaluation matches the reported one, we reproduced Robust Turbo-DDCM, whose code is public. Our PSNR values are within 0.15 dB of the reported ones on Kodak and within 0.09 dB on DIV2K (Appendix C). The reported values for CLIC could not be reproduced, and on that dataset we compare only with our reproduction of Robust Turbo-DDCM. The BCH codes are designed for a BER of , the highest tested rate, and the total number of bits is that of the compared robust codec.
Compression performance
Table I compares sectioned codebooks with Turbo-DDCM at equal bits. Sectioned codebooks improve the PSNR in every row, by 0.18 to 0.51 dB, and they also improve LPIPS and DISTS. At 21 600 bits per image, the gain is 0.30 dB on Kodak, 0.45 dB on DIV2K and 0.39 dB on CLIC, and the PSNR is higher on 24 of 24, 100 of 100 and 245 of 250 images. The gain is largest at low rates and decreases as the rate grows, on Kodak from 0.51 dB at 0.040 bpp to 0.18 dB at 0.194 bpp, where the rate-distortion curve of the codec becomes flat. The rows with 19, 27 and 29 coded steps use the number of steps that Turbo-DDCM selects by default at these rates, and the gain is preserved (0.18 to 0.31 dB). In terms of rate, the sectioned codec reaches the PSNR of Turbo-DDCM with 16 to 21% fewer bits (Fig. 2c).
Model validation and ablation
The steering model predicts these gains from measurements of Turbo-DDCM alone. We fitted the PSNR of Turbo-DDCM on Kodak as a quadratic function of the logarithm of its steering, using its 32 operating points with 24 coded steps, and read the expected gain of a sectioned layout from this curve at the steering given by (2). The first eight predictions were made before the sectioned codec was run. For the 14 layouts measured on Kodak, with sections of to vectors, the measured gain differs from the prediction by at most 0.05 dB (Fig. 2b). The gain also grows with the section size as predicted, from 0.17 dB for vectors to 0.30 dB for vectors at 21 600 bits.
Table II separates the two sources of the gain. We also recorded the correlation between the transmitted noise and the residual in every coded step, which gives the steering directly from (1). For Turbo-DDCM and all sectioned layouts, the measured steering deviates from the model by only −0.2 to +0.1%. The size of the pool is the main factor. With the selection rule and the joint index of Turbo-DDCM, pools of 60 and 2 696 Hadamard blocks gain 0.20 and 0.26 dB. Writing the selection as fields adds a smaller part. Sections on these two pools gain 0.22 and 0.30 dB, and the difference to the joint index on the same pool is significant only for the larger pool, where it is +0.04 dB with a standard error of 0.01 dB. The table also shows that several vectors should not be taken from the same block. One block with its best 91 vectors is 0.17 dB below Turbo-DDCM, because the vectors of a block share the values and their sum is no longer Gaussian. Sections avoid this, because each section has its own blocks and contributes a single vector. Finally, sections of independent Gaussian vectors and Hadamard-based sections of the same size differ by −0.04 dB in PSNR with a standard error of 0.05 dB, which is not significant.
The encoding cost depends on the section size. At 900 bits per step, the encoder transforms 2 696 blocks per step with additions, while Turbo-DDCM computes a product with a dense codebook of values, which takes multiplications. On a laptop GPU, encoding an image took 5.6 s with sections of vectors, 3.7 s with sections of vectors and 3.7 s with Turbo-DDCM. Sections of vectors thus keep 0.29 of the 0.30 dB at the encoding time of Turbo-DDCM. The cost of the transforms grows with the number of sections. At 0.194 bpp, the highest rate tested, encoding with sections of vectors took 8.9 s compared with 3.8 s. The
| bpp | Coded steps | PSNR (dB) Base | PSNR (dB) Ours | Gain | LPIPS Base | LPIPS Ours | DISTS Base | DISTS Ours | Images |
| \multicolumn{10}{l}{Kodak, 24 images} | |||||||||
| 0.0404 | 24 | 23.21 | 23.72 | +0.51 (0.05) | 0.221 | 0.184 | 0.142 | 0.117 | 24 |
| 0.0558 | 24 | 24.00 | 24.36 | +0.36 (0.04) | 0.163 | 0.135 | 0.102 | 0.084 | 24 |
| 0.0591 | 24 | 24.10 | 24.46 | +0.36 (0.03) | 0.155 | 0.130 | 0.096 | 0.079 | 24 |
| 0.0824 | 24 | 24.64 | 24.95 | +0.30 (0.03) | 0.120 | 0.104 | 0.074 | 0.065 | 24 |
| 0.0999 | 24 | 24.91 | 25.17 | +0.26 (0.02) | 0.107 | 0.097 | 0.065 | 0.060 | 24 |
| 0.0646 | 19 | 24.00 | 24.31 | +0.31 (0.03) | 0.137 | 0.120 | 0.080 | 0.071 | 24 |
| 0.1257 | 27 | 25.30 | 25.55 | +0.26 (0.02) | 0.097 | 0.088 | 0.061 | 0.056 | 24 |
| 0.1523 | 29 | 25.59 | 25.81 | +0.22 (0.02) | 0.090 | 0.083 | 0.057 | 0.054 | 24 |
| 0.1939 | 29 | 25.79 | 25.96 | +0.18 (0.01) | 0.084 | 0.080 | 0.054 | 0.052 | 24 |
| \multicolumn{10}{l}{DIV2K, 100 images} | |||||||||
| 0.0591 | 24 | 23.18 | 23.69 | +0.51 (0.03) | 0.171 | 0.142 | 0.116 | 0.096 | 100 |
| 0.0824 | 24 | 23.94 | 24.38 | +0.45 (0.02) | 0.130 | 0.111 | 0.087 | 0.075 | 100 |
| 0.0992 | 24 | 24.30 | 24.67 | +0.37 (0.02) | 0.115 | 0.101 | 0.076 | 0.068 | 99 |
| 0.1523 | 29 | 25.20 | 25.49 | +0.30 (0.01) | 0.094 | 0.086 | 0.066 | 0.061 | 99 |
| 0.1939 | 29 | 25.47 | 25.71 | +0.24 (0.01) | 0.086 | 0.082 | 0.061 | 0.057 | 99 |
| \multicolumn{10}{l}{CLIC, 250 images} | |||||||||
| 0.0591 | 24 | 26.42 | 26.89 | +0.47 (0.02) | 0.119 | 0.101 | 0.089 | 0.075 | 243 |
| 0.0824 | 24 | 27.10 | 27.49 | +0.39 (0.01) | 0.093 | 0.082 | 0.070 | 0.062 | 245 |
Table 1. Sectioned codebooks with vectors per section (Ours) and Turbo-DDCM (Base) at equal bits. The PSNR gain is given with its standard error. The last column is the number of images with a higher PSNR.
| Codebook | Candidate vectors | Bits per step | Steering model | Steering measured | PSNR diff. (dB) | Time (s) |
| Joint index over the best vectors of one pool | ||||||
| Turbo-DDCM | 91 | 900 | 859 | 859 | 3.7 | |
| One Hadamard block | 91 | 900 | 859 | (0.02) | 3.5 | |
| 8 blocks | 67 | 892 | 931 | +0.09 (0.01) | 3.5 | |
| 60 blocks | 54 | 892 | 979 | +0.20 (0.02) | 3.5 | |
| 2 696 blocks | 40 | 897 | 1 041 | +0.26 (0.02) | 5.8 | |
| Sectioned codebooks, one vector per section | ||||||
| vectors per section | 82 | 900 | 969 | +0.17 (0.02) | 3.5 | |
| vectors per section | 60 | 900 | 1 025 | +0.22 (0.02) | 3.5 | |
| vectors per section | 50 | 900 | 1 054 | +0.29 (0.02) | 3.7 | |
| vectors per section | 43 | 900 | 1 074 | +0.30 (0.03) | 5.6 |
Table 2. Ablation on Kodak with 24 coded steps at about 900 bits per step. PSNR differences are relative to Turbo-DDCM and are given with standard errors. Time is the encoding time per image.
decoder regenerates only the selected vectors and needs no transform.
Robustness to bit errors
The BCH-protected codecs are compared with the robust codecs in Table III and Figure 3. At 26 280 bits per image, the sectioned codec spends 21 600 bits on the image and 4 680 bits on parity, which corrects up to 319 bit errors. Its quality is constant up to a BER of , whereas the quality of the robust codecs decreases with the BER.
Figure 3. PSNR as a function of the BER on Kodak at about 0.10 bpp. The BCH codes are designed for a BER of . Beyond this BER, the sectioned codec may start to fail, and its performance can degrade rapidly.
Turbo-DDCM protected by one BCH code already outperforms Robust Turbo-DDCM at every BER, by 0.48 to 7.29 dB on the two datasets. The separate fields of Robust Turbo-DDCM need so many bits that the joint index can carry more vectors and their protection within the same total. BitFlat with parity fields is 0.21 to 0.40 dB better than this baseline at a BER of and worse from a BER of on (Appendix B).
The sectioned codec with the same code has the highest PSNR at every BER in Table III. At exactly equal total bits, it outperforms Robust Turbo-DDCM by 0.84 to 7.74 dB and BitFlat with Hamming or SECDED fields by 0.37 to 1.73 dB. Its margin over BitFlat with parity fields is 4.14 to 5.05 dB at a BER of and 0.08 to 0.16 dB at . The margin at is of the size of the deviations of our reproduction, so the two codecs should be regarded as equivalent at this BER. LPIPS and DISTS give the same ranking, and the sectioned codec is better in all 180 comparisons with reported values (Appendix B). Compared with our reproduction of Robust Turbo-DDCM, it is better by 0.83 to 7.24 dB on Kodak, 1.11 to 7.83 dB on DIV2K and 1.13 to 9.30 dB on CLIC.
The steering model explains the ranking at low BER. The parity fields of BitFlat are sections of 512 vectors, and their 1 089 bits per step give . This is more than the 859 of the joint index with the bits that the code leaves for it, and less than the 1 074 of sections of vectors with the same bits.
Beyond the design range
A code designed for a BER of fails when the channel is worse. The BCH-protected Turbo-DDCM then loses the image, while the robust codecs still deliver a degraded image. The sectioned codec can use the fallback of Section III-E, which we measured on Kodak (Figure 3). At a BER of , the fallback gives 14.9 dB with sections of vectors, compared with 12.8 dB for a mid-gray image and 15.8 dB for our reproduction of Robust Turbo-DDCM. The difference to Robust Turbo-DDCM shrinks from to dB at this BER and from to dB at . Smaller sections have shorter fields, so a smaller fraction of the selected vectors is corrupted at a given BER. With sections of vectors, the fallback is on par with Robust Turbo-DDCM, with differences of 0.0 and +0.2 dB, and with sections of vectors it is above Robust Turbo-DDCM, with differences of +0.9 and +0.9 dB. Without errors, these smaller sections gain 0.22 and 0.17 dB over Turbo-DDCM instead of 0.30 dB. The section size is thus a trade-off between quality within the design range and quality beyond it.
The design range can also be extended. A code designed for a BER of , twice the highest reported BER, needs more parity and leaves 736 bits per step for the image at 26 280 bits. With sectioned codebooks, this design reaches 24.67 dB on Kodak up to a BER of . The difference to Turbo-DDCM with the code designed for is +0.02 dB on Kodak and +0.06 dB on DIV2K. In other words, the gain of sectioned codebooks pays for a design range that is twice as wide. At a BER of , this design is 8.8 dB better than our reproduction of Robust Turbo-DDCM on Kodak. It still outperforms the reported results of Robust Turbo-DDCM and of BitFlat with Hamming or SECDED fields at every BER, and those of BitFlat with parity fields from a BER of on, while BitFlat with parity fields is 0.18 to 0.39 dB better at .
Burst errors and packet loss
We tested two further channels and summarize the results here. When bit errors occur in bursts, a BCH code designed for independent errors is not suitable, and we use a Reed-Solomon code instead. With this code, the sectioned codec outperforms our reproduction of Robust Turbo-DDCM on all 12 burst channels tested, by 0.39 to 4.30 dB, although the margin becomes small for very long bursts (Appendix D). For packet loss, we protected the learned code MLCC++ [41] with one erasure code and compared it with the robust codec Every Packet Counts [9]. The protected codec is 0.14 to 0.92 dB better on Kodak and 0.24 to 0.86 dB worse on images of two megapixels, where the comparison is only indicative because the image sets may differ (Appendix E).
Discussion
Sectioned codebooks improve Turbo-DDCM because they draw the noise from a much larger pool of vectors. The joint index is an efficient description of a set, but the set is selected from a codebook that is small for the number of bits spent on it, and the dense codebook of Turbo-DDCM cannot grow. The steering model also indicates the remaining potential. Sections of vectors reach 83% of the bound, and larger sections close the gap only slowly. Two extensions concern the encoder alone. The sections of a step currently choose simultaneously for the same residual, whereas the encoders of sparse regression codes let each section react to the choices of the previous ones [14]. The model also treats all steps alike, although the steps of a diffusion process differ in their effect on the image.
The results on error resilience raise the question of when robustness by design can be better than a channel code. Appendix § 4.6 derives a limit on this advantage, which we call the budget of a robust codec. A robust codec cannot deliver more than the reference codec would deliver at the capacity of the channel. Its possible gain over a code designed for the worst channel state depends on the range of capacity that the channel states span and on the slope of the rate-distortion curve. For BERs up to , the budget is a few tenths of a decibel, and all gains of robust codecs that we observed lie within it. Robustness by design can be expected to pay off when the capacity range is wide, when the blocks are short, or when no useful bound on the channel state is known. We therefore suggest that evaluations of robust codecs include a codec protected by one channel code at equal total bits.
| Codec | bpp | |||||
| Kodak, 24 images | ||||||
| Robust Turbo-DDCM [5] | 0.1003 | 24.10 | 23.70 | 22.51 | 20.76 | 17.73 |
| BitFlat, parity [6] | 0.0997 | 24.85 | 24.61 | 23.96 | 22.83 | 20.40 |
| BitFlat, Hamming [6] | 0.1000 | 24.58 | 24.58 | 24.56 | 24.47 | 23.29 |
| BitFlat, SECDED [6] | 0.1003 | 24.46 | 24.46 | 24.45 | 24.38 | 23.66 |
| Turbo-DDCM + BCH | 0.0997 | 24.64 | 24.64 | 24.64 | 24.64 | 24.62 |
| Sectioned + BCH (ours) | 0.0997 | 24.95 | 24.95 | 24.95 | 24.95 | 24.93 |
| Robust Turbo-DDCM [5] | 0.0728 | 23.62 | 23.23 | 22.23 | 20.60 | 17.74 |
| BitFlat, parity [6] | 0.0735 | 24.36 | 24.15 | 23.64 | 22.61 | 20.32 |
| Turbo-DDCM + BCH | 0.0728 | 24.10 | 24.10 | 24.10 | 24.10 | 24.10 |
| Sectioned + BCH (ours) | 0.0728 | 24.46 | 24.46 | 24.46 | 24.46 | 24.46 |
| DIV2K, 100 images | ||||||
| Robust Turbo-DDCM [5] | 0.1003 | 23.28 | 22.86 | 21.62 | 19.70 | 16.64 |
| BitFlat, parity [6] | 0.0997 | 24.22 | 24.00 | 23.33 | 22.04 | 19.31 |
| BitFlat, Hamming [6] | 0.1000 | 23.84 | 23.84 | 23.82 | 23.71 | 22.64 |
| BitFlat, SECDED [6] | 0.1003 | 23.70 | 23.70 | 23.69 | 23.63 | 22.91 |
| Turbo-DDCM + BCH | 0.0997 | 23.92 | 23.92 | 23.92 | 23.92 | 23.92 |
| Sectioned + BCH (ours) | 0.0997 | 24.38 | 24.38 | 24.38 | 24.38 | 24.36 |
| Robust Turbo-DDCM [5] | 0.0728 | 22.62 | 22.25 | 21.15 | 19.43 | 16.51 |
| BitFlat, parity [6] | 0.0735 | 23.61 | 23.43 | 22.82 | 21.60 | 19.21 |
| Turbo-DDCM + BCH | 0.0728 | 23.18 | 23.18 | 23.18 | 23.18 | 23.17 |
| Sectioned + BCH (ours) | 0.0728 | 23.69 | 23.69 | 23.69 | 23.69 | 23.69 |
Table 3. PSNR (dB) under independent bit errors. The values of the robust codecs are those reported in [6]. The BCH-protected codecs use the smallest number of bits of each group. The best value of each group is in bold.
Our study has some limitations as well. BitFlat and Every Packet Counts are not publicly available, so the comparisons with them rely on reported values. The sectioned codec was evaluated with one diffusion model, on crops of pixels and at rates up to 0.194 bpp. The fallback and the burst channels were tested on Kodak only. With sections of vectors, encoding is slower than with Turbo-DDCM, and the encoding times come from a research implementation on a single device. The channel code is a BCH code with bounded-distance decoding, and a modern code would leave more bits for the image.
Conclusion
We proposed sectioned codebooks for zero-shot diffusion image compression. Each section of a coded step selects its best vector from a large codebook of its own, and a model based on order statistics predicts the resulting gain over the joint index of Turbo-DDCM. With sections of vectors built from Hadamard matrices, the PSNR improves by 0.18 to 0.51 dB at equal bits on three datasets, which corresponds to 16 to 21% fewer bits at equal PSNR. Protected by one BCH code, the sectioned codec outperforms the reported results of Robust Turbo-DDCM and BitFlat at all tested BERs, with a clear margin at high BERs and a marginal one at the lowest. The gain is also sufficient to double the BER range of the code without falling below Turbo-DDCM. A codec protected by one channel code is thus a simple and strong baseline for the evaluation of robust codecs. Future work will study encoders that choose the sections of a step sequentially and the extension to other diffusion models and larger images.
Declarations
a) Funding: This research did not receive any specific grant from funding agencies in the public, commercial, or not-for-profit sectors.
b) Competing interests: The author declares no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper.
c) Data and code availability: The implementation and the numerical results of all experiments will be released with the article. The images and the pretrained codecs are available from their original distributors under their respective terms.
Declaration of generative AI and AI-assisted technologies in the manuscript preparation process
During the preparation of this work the author used Claude to assist with the experimental analysis, interpretation of results, and drafting of the manuscript. The author reviewed and edited the content generated by AI and takes full responsibility for the final content of the manuscript.
Appendix A: Comparison rules and channel codes
An image is transmitted with a total of channel bits. The channel has a state that is unknown to the sender, which is the BER of a binary channel or the loss rate of a packet channel, and a robust codec is tested over a range . A robust codec spends all bits on a representation that tolerates errors. The protected codec spends bits on the bitstream of an unmodified codec and the remaining bits on the parity of one channel code.
The comparison follows four rules. The totals are equal, and every transmitted bit of the protected codec counts, including parity, padding and packet headers. The images and quality measures are those of the reported results. The code is designed once for the tested range, and the same bitstream is evaluated at every channel state. For bit errors, the code is designed for the highest state . For packet loss, where the robust codec is trained for a uniformly distributed loss rate, the design maximizes the expected quality under that distribution. Finally, the evaluation is exact, because the failure probability in (4) is the tail of a known distribution.
For independent bit errors we use shortened binary BCH codes [34, 35]. The primitive narrow-sense BCH code of length that corrects errors has parity bits, given by the sizes of the cyclotomic cosets of modulo , and shortening preserves both numbers. For a total of bits and an operating point of bits, is the largest value with , and suffices for the totals used here. The decoding fails when more than of the transmitted bits are flipped, so that
A word with more than errors could also lie within distance of another codeword and be accepted as valid. The fraction of such words is below for all codes used here, and we neglect this case. We verified the decoding rule by transmitting the bitstreams of all images through an independent implementation of the codes [42]. Every word was recovered exactly when it had at most errors, and no wrong bitstream was accepted.
A stronger code than BCH with bounded-distance decoding would leave more bits for the image. To bound this benefit, we use the converse of Polyanskiy, Poor and Verdú [43], Theorem 35. No code with messages and bits fails less often than one that assigns every message the lightest error patterns. We compute this lower bound on the failure probability exactly and refer to the resulting design as the limit of any code.
For burst errors we use a channel with two states of the Gilbert-Elliott type [44, 45]. The number of flipped bits is then no longer binomial, and its exact distribution follows from a forward recursion over the bits. The code for bursts is a Reed-Solomon code [46], which corrects any corrupted symbols with parity symbols. For packet loss, the bitstream is spread over packets with a maximum-distance-separable code, so that any received packets recover it [47]. The failure probability for independent loss with rate is .
Appendix B: Detailed results for bit errors
This appendix describes the reported configurations of the robust codecs and gives the complete comparison in PSNR, LPIPS and DISTS.
Figure 4 shows how the compared codecs use the bits of an image. Robust Turbo-DDCM [5] sends each selected vector as a field of 15 bits. BitFlat [6] gives every field a codebook of 512 vectors, which shortens the field, and adds a parity bit, a shortened Hamming code or a SECDED code to every field. The reported results cover six configurations with 24 coded steps. These are Robust Turbo-DDCM at 26 280 and 19 080 bits per image, BitFlat with parity, Hamming and SECDED fields at 26 136, 26 208 and 26 280 bits, and BitFlat with parity fields at 19 272 bits. The BERs are , , , and .
Figure 4. Bits of one image with 24 coded steps in five codecs. Lengths are proportional to bits. The BCH codes are designed for a BER of .
Table IV gives the results for Turbo-DDCM with one BCH code. At 26 136 bits, the code protects 91 vectors per step in 21 600 bits with 4 530 parity bits, which correct up to
| Robust codec | bpp | PSNR (dB) 1 | 3 | 10 | 30 | 100 | LPIPS () 1 | 3 | 10 | 30 | 100 | DISTS () 1 | 3 | 10 | 30 | 100 |
| Kodak, 24 images | ||||||||||||||||
| Robust Turbo-DDCM | 0.0728 | +0.48 | +0.87 | +1.87 | +3.50 | +6.36 | −28 | −42 | −85 | −166 | −343 | −21 | −28 | −49 | −86 | −164 |
| BitFlat, parity | 0.0735 | −0.26 | −0.05 | +0.46 | +1.49 | +3.78 | +21 | +14 | −2 | −41 | −156 | +13 | +11 | +2 | −18 | −72 |
| Robust Turbo-DDCM | 0.1003 | +0.54 | +0.94 | +2.13 | +3.88 | +6.91 | −29 | −42 | −87 | −170 | −360 | −17 | −24 | −48 | −86 | −175 |
| BitFlat, parity | 0.0997 | −0.21 | +0.03 | +0.68 | +1.81 | +4.22 | +11 | +4 | −16 | −55 | −166 | +5 | +2 | −9 | −29 | −73 |
| BitFlat, Hamming | 0.1000 | +0.06 | +0.06 | +0.08 | +0.17 | +1.35 | −4 | −4 | −4 | −7 | −43 | −3 | −3 | −3 | −5 | −22 |
| BitFlat, SECDED | 0.1003 | +0.18 | +0.18 | +0.19 | +0.26 | +0.98 | −11 | −11 | −11 | −13 | −34 | −7 | −7 | −7 | −8 | −18 |
| DIV2K, 100 images | ||||||||||||||||
| Robust Turbo-DDCM | 0.0728 | +0.56 | +0.93 | +2.03 | +3.75 | +6.66 | −29 | −39 | −80 | −156 | −330 | −20 | −26 | −45 | −83 | −167 |
| BitFlat, parity | 0.0735 | −0.40 | −0.22 | +0.39 | +1.61 | +3.99 | +24 | +19 | +1 | −39 | −141 | +18 | +15 | +5 | −14 | −62 |
| Robust Turbo-DDCM | 0.1003 | +0.66 | +1.08 | +2.32 | +4.24 | +7.29 | −28 | −40 | −79 | −157 | −339 | −19 | −26 | −46 | −83 | −172 |
| BitFlat, parity | 0.0997 | −0.30 | −0.08 | +0.59 | +1.88 | +4.61 | +15 | +10 | −8 | −48 | −158 | +11 | +8 | −2 | −22 | −75 |
| BitFlat, Hamming | 0.1000 | +0.10 | +0.10 | +0.12 | +0.23 | +1.29 | −5 | −5 | −6 | −9 | −38 | −5 | −5 | −5 | −7 | −23 |
| BitFlat, SECDED | 0.1003 | +0.24 | +0.24 | +0.25 | +0.31 | +1.02 | −12 | −12 | −12 | −14 | −34 | −9 | −9 | −9 | −10 | −21 |
Table 4. Turbo-DDCM with one BCH code minus the reported values of the robust codecs, at equal total bits. The columns are the BERs in units of . Positive PSNR differences and negative LPIPS and DISTS differences favour the BCH-protected codec.
309 errors. The code rate is 0.83, and the image is lost with probability at a BER of . This baseline is better than Robust Turbo-DDCM in all three measures at every BER. Against BitFlat with parity fields, the PSNR difference on Kodak is dB at and dB at , and it grows to dB at . Against the Hamming and SECDED fields, the baseline is 0.06 to 0.26 dB better up to , which is within the accuracy of the comparison, and 0.98 and 1.35 dB better at . A code in every field spends four or five check bits on ten information bits, whereas one code over the whole bitstream reaches a failure probability of at a code rate of 0.82. The results on DIV2K follow the same pattern.
Table 5 gives the results for the sectioned codec. On Kodak, the sectioned bitstream has exactly the bits of the joint index that it replaces, so the codes and their failure probabilities are unchanged and only the quality of a received image improves. On DIV2K, two of the six designs use the nearest available sectioned layout, which differs by 9 bits per step. All differences in the table favour the sectioned codec.
| Robust codec | bpp | PSNR (dB) 1 | 3 | 10 | 30 | 100 | LPIPS () 1 | 3 | 10 | 30 | 100 | DISTS () 1 | 3 | 10 | 30 | 100 |
| Kodak, 24 images | ||||||||||||||||
| Robust Turbo-DDCM | 0.0728 | +0.84 | +1.23 | +2.23 | +3.86 | +6.72 | −53 | −67 | −110 | −191 | −368 | −38 | −45 | −66 | −103 | −181 |
| BitFlat, parity | 0.0735 | +0.10 | +0.31 | +0.82 | +1.85 | +4.14 | −4 | −11 | −27 | −66 | −181 | −4 | −6 | −15 | −35 | −89 |
| Robust Turbo-DDCM | 0.1003 | +0.85 | +1.25 | +2.44 | +4.19 | +7.22 | −45 | −58 | −103 | −186 | −376 | −26 | −33 | −57 | −95 | −185 |
| BitFlat, parity | 0.0997 | +0.10 | +0.34 | +0.99 | +2.12 | +4.53 | −5 | −12 | −32 | −71 | −182 | −4 | −7 | −18 | −38 | −83 |
| BitFlat, Hamming | 0.1000 | +0.37 | +0.37 | +0.39 | +0.48 | +1.65 | −20 | −20 | −20 | −23 | −59 | −12 | −12 | −12 | −14 | −31 |
| BitFlat, SECDED | 0.1003 | +0.49 | +0.49 | +0.50 | +0.57 | +1.29 | −27 | −27 | −27 | −29 | −50 | −16 | −16 | −16 | −17 | −28 |
| DIV2K, 100 images | ||||||||||||||||
| Robust Turbo-DDCM | 0.0728 | +1.07 | +1.44 | +2.54 | +4.26 | +7.18 | −58 | −68 | −109 | −185 | −360 | −40 | −46 | −65 | −103 | −187 |
| BitFlat, parity | 0.0735 | +0.08 | +0.26 | +0.87 | +2.09 | +4.48 | −2 | −7 | −25 | −65 | −168 | −1 | −4 | −14 | −33 | −82 |
| Robust Turbo-DDCM | 0.1003 | +1.10 | +1.52 | +2.76 | +4.68 | +7.74 | −47 | −59 | −98 | −176 | −357 | −31 | −38 | −58 | −95 | −184 |
| BitFlat, parity | 0.0997 | +0.16 | +0.38 | +1.05 | +2.34 | +5.05 | −5 | −10 | −28 | −68 | −176 | −2 | −5 | −15 | −35 | −87 |
| BitFlat, Hamming | 0.1000 | +0.54 | +0.54 | +0.56 | +0.67 | +1.73 | −24 | −24 | −25 | −28 | −57 | −17 | −17 | −17 | −19 | −34 |
| BitFlat, SECDED | 0.1003 | +0.68 | +0.68 | +0.69 | +0.75 | +1.47 | −31 | −31 | −31 | −33 | −52 | −21 | −21 | −21 | −22 | −33 |
Table 5. Sectioned codec with one BCH code minus the reported values of the robust codecs, in the format of Table IV.
The code must be designed for a maximum BER, and a wider range costs quality. For Turbo-DDCM, a design for a BER of has 72 vectors per step and a code rate of 0.68, and it is about 0.30 dB below the design for . With sectioned codebooks, the design for uses 736 bits per step and a code that corrects 613 errors, and it is 0.28 dB below the sectioned design for . Beyond the design range the code fails as a whole. The design for at 26 136 bits loses the image with probability 0.59 at a BER of 1.2% and 1.00 at 1.5%.
Reproduction of Robust Turbo-DDCM
The comparison with BitFlat relies on reported values, so our evaluation must match the reported one in images and quality measures. Table I of [6] includes Robust Turbo-DDCM, whose code is public. We reproduced this codec under the reported settings, with independent bit flips applied to its bitstream. Table VI lists the differences between our PSNR values and the reported ones. They do not exceed 0.15 dB on Kodak and 0.09 dB on DIV2K, and the differences in LPIPS and DISTS are at most 0.005 and 0.009. Differences of this size are within the variation between channel realizations. The agreement holds for a codec that we can run and is not a confidence interval for the reported values of BitFlat. Differences of one or two tenths of a decibel should be regarded as inconclusive.
For the third dataset of [6], the CLIC 2020 professional test set [38], we could not reproduce the reported values. On the central crops of its 250 images, our PSNR for Robust Turbo-DDCM is 0.60 dB higher than the reported value at a BER of , which suggests that the reported values were obtained on different images. We therefore do not use the reported values for this dataset and compare the codecs in our own evaluation. In this comparison, Turbo-DDCM with one BCH code is 0.74
| Images | Vectors | |||||
| Kodak | 73 | |||||
| 53 | ||||||
| DIV2K | 73 | 0.00 |
Table VI. Our reproduction of Robust Turbo-DDCM minus the reported PSNR (dB), with 73 vectors per step (0.1003 bpp) and, on Kodak, also with 53 vectors (0.0728 bpp).
to 8.91 dB better than Robust Turbo-DDCM, and the sectioned codec is 1.13 to 9.30 dB better.
Burst errors
Both papers on robust diffusion codecs leave burst errors untested. We model bursts with a two-state channel [44, 45]. The good state flips no bits, and the bad state flips every bit with probability 0.5. The mean duration of the bad state is 8, 64, 512 or 4096 bits, and the fraction of time spent in it is set so that the average BER is , or . We measured Robust Turbo-DDCM on these channels at 19080 bits per image and computed the failure probabilities of the codes exactly.
A BCH code designed for independent errors is not suitable for this channel (Figure 5). At an average BER of , it loses 10.2% of the images with bursts of 8 bits and 28.1% with bursts of 512 bits. A Reed-Solomon code with symbols of 11 bits, designed once for all tested channels at , keeps Turbo-DDCM on par with or better than Robust Turbo-DDCM on every channel, at a cost of 0.76 dB when the errors are independent. Its advantage decreases with the burst length, from 3.94 dB for bursts of 8 bits to 1.09 dB for bursts of 512 bits at an average BER of . For bursts of 4096 bits, a fifth of the bitstream, the advantage is 0.05 to 0.44 dB, while the measured mean of the robust codec varies by 0.27 to 0.46 dB between realizations, so no clear advantage remains. A robust codec benefits from burstiness, because a burst that flips many bits of one field still corrupts only one vector, whereas a long burst can exceed the correction capability of any code that the bit budget allows.
Figure 5. PSNR on Kodak at 19 080 bits per image under burst errors, as a function of the mean burst length. The leftmost point of each panel is the channel with independent errors. The Reed-Solomon code is designed once for all channels of the right panel.
Sectioned codebooks add 0.36 dB to both coded designs wherever the code holds. With the Reed-Solomon code, the sectioned codec is better than Robust Turbo-DDCM on all 12 burst channels, by 0.39 to 4.30 dB in general and by 0.39 to 0.79 dB for the longest bursts.
Packet loss
We repeated the comparison for packet loss with a learned codec optimized for mean squared error and an erasure code. The robust codec is Every Packet Counts (EPC) [9], which is trained to reconstruct the image from the latent packets that arrive, for loss rates between 0 and 30%. Its code and weights are not public, so we use the reported mean PSNR at independent loss rates of 5, 10 and 20%, on the Kodak images at 0.125, 0.217 and 0.346 bpp and on images of the CLIC 2020 dataset at 0.090, 0.148 and 0.228 bpp. EPC is built on a variant of the HPCM codec [48]. Our reference codec is MLIC++ [41], a codec of the same generation, so the two systems do not share a base codec.
The protected codec compresses every image with MLIC++ at one operating point for the whole dataset and uses the actual entropy coder. The bitstream is spread over packets of at most 1500 bytes with one maximum-distance-separable code, and the packet headers count in the rate. Following the training range of EPC, the design maximizes the expected PSNR for a loss rate that is uniform between 0 and 30%. An image is sent in at least eight packets on Kodak and at least 32 packets on the larger images. Two differences in accounting favour EPC. Its rates include no packet headers, and it assumes that the packet with the hyperprior always arrives. The larger images are the 41 images of the CLIC 2020 professional validation set [49] at full size, with 2.2 megapixels on average. EPC only states that it uses the CLIC 2020 dataset at 2K resolution, so the comparison on these images is indicative. We also include unequal protection [24, 25, 26], in which the bitstream of MLIC++ is divided into at most four levels with decreasing protection along the decoding order.
Table VII shows the results. On Kodak, one erasure code is better than EPC in all nine cases, by 0.14 to 0.92 dB, and unequal protection adds 0.15 to 0.34 dB. On the larger images the order is reversed, and one code is 0.24 to 0.86 dB worse than EPC. The reported values themselves differ between the two datasets. At 5% loss, EPC is 1.84 to 2.83 dB below error-free MLIC++ at the same rate on Kodak and only 0.54 to 0.68 dB below it on the larger images. A code that must survive a loss of 30% gives up about a third of the rate, which costs 1.33 to 2.06 dB on Kodak and 1.40 to 1.45 dB on the larger images. The two datasets differ in the images, in the number of packets per image and possibly in the selection of images behind the reported values. We report the reversal as an observation and do not attribute it to the image size.
| Images | bpp | EPC, reported 5% | EPC, reported 10% | EPC, reported 20% | One code minus EPC 5% | One code minus EPC 10% | One code minus EPC 20% | Graded minus one code 5% | Graded minus one code 10% | Graded minus one code 20% | Budget 5% | Budget 10% | Budget 20% |
| Kodak | 0.125 | 27.55 | 27.47 | 27.21 | +0.51 | +0.52 | +0.14 | +0.18 | +0.19 | +0.34 | 1.12 | 1.19 | 1.57 |
| 0.217 | 28.98 | 28.84 | 28.56 | +0.73 | +0.82 | +0.51 | +0.15 | +0.16 | +0.29 | 1.28 | 1.22 | 1.38 | |
| 0.346 | 30.43 | 30.27 | 29.93 | +0.77 | +0.92 | +0.86 | +0.23 | +0.22 | +0.20 | 1.81 | 1.58 | 1.57 | |
| CLIC | 0.090 | 30.55 | 30.38 | 30.01 | −0.86 | −0.69 | −0.44 | 1.40 | 1.40 | 0.81 | |||
| 0.148 | 31.93 | 31.79 | 31.45 | −0.84 | −0.70 | −0.36 | 1.45 | 1.45 | 0.68 | ||||
| 0.228 | 33.28 | 33.14 | 32.78 | −0.74 | −0.59 | −0.24 | 1.42 | 1.42 | 0.73 |
Table VII. MLIC++ protected by erasure codes designed for loss rates from 0 to 30%, compared with the reported PSNR (dB) of Every Packet Counts at independent loss rates of 5, 10 and 20%. Graded denotes unequal protection. The budget is defined in Appendix F.
The budget of a robust codec
This appendix derives a limit on the gain that a robust codec can achieve over a codec protected by a channel code, and compares the observed gains with this limit.
Let be the capacity of the channel per transmitted bit, which is for independent bit errors with the binary entropy function and for packet loss. Whatever the scheme, the receiver obtains at most bits of information about the image from channel uses in state [29]. No scheme can therefore deliver a better expected quality than the distortion-rate function of the images allows at bits. This function is unknown, and we use the quality of the reference codec at that number of bits instead,
We call the budget at state . A robust codec whose gain over the protected codec exceeds the budget delivers more than the reference codec could deliver over an error-free channel of the same capacity. Such a gain would be an advance in compression and not in resilience. The budget is a yardstick and not a bound for every robust codec, because a codec with a better rate-distortion performance than the reference codec can exceed it.
The size of the budget follows from two numbers. If the quality of the codec grows by decibels per doubling of the rate, and the code carries a fraction of the capacity at , then
For BERs up to the capacity decreases from 1 to 0.919, and for loss rates up to 30% it decreases to 0.7. With the slopes of the codecs used here, 1.1 dB per doubling for Turbo-DDCM and 1.7 to 2.8 dB for MLIC++, an ideal code leaves a robust codec at most 0.14 dB at a vanishing BER and about one decibel at a vanishing loss rate.
Figure 6 compares the gains of the robust codecs with the budget. For bit errors at 26 136 bits on Kodak and a BER of , the budget of Turbo-DDCM with the BCH code is 0.26 dB, and BitFlat with parity fields gains 0.21 dB, which is 78% of the budget. At the budget shrinks to 0.14 dB, while BitFlat with parity fields is 4.22 dB below the protected codec. With the sectioned codec as the reference, the budget at is 0.22 to 0.46 dB, and no reported value of a robust codec exceeds the protected codec. For packet loss the budget is larger, with 1.12 to 1.81 dB on Kodak and 0.68 to 1.45 dB on the larger images (Table VII). EPC gains nothing on Kodak and 34 to 61% of the budget on the larger images. All observed gains of the robust codecs are within the budget.
Figure 6. The budget (7) and the PSNR gains of the robust codecs over the protected codec. Left: bit errors on Kodak at about 0.10 bpp. A negative gain means that the protected codec is better. Right: packet loss on images of two megapixels.
References
- [1]L. Theis, T. Salimans, M. D. Hoffman, and F. Mentzer, “Lossy compression with Gaussian diffusion,” 2022, preprint, arXiv:2206.08889v2.arxiv.org/abs/2206.08889
- [2]J. Vonderfecht and F. Liu, “Lossy compression with pretrained diffusion models,” in 13th International Conference on Learning Representations (ICLR), 2025, arXiv:2501.09815v1.arxiv.org/abs/2501.09815
- [3]G. Ohayon, H. Manor, T. Michaeli, and M. Elad, “Compressed image generation with denoising diffusion codebook models,” in Proceedings of the 42nd International Conference on Machine Learning (ICML), 2025, arXiv:2502.01189v4.arxiv.org/abs/2502.01189
- [4]A. Vaisman, G. Ohayon, H. Manor, M. Elad, and T. Michaeli, “Turbo-DDCM: Fast and flexible zero-shot diffusion-based image compression,” in 14th International Conference on Learning Representations (ICLR), 2026, arXiv:2511.06424v2.arxiv.org/abs/2511.06424
- [5]A. Vaisman, G. Pomerants, and R. Lapid, “On the robustness of diffusion-based image compression to bit-flip errors,” 2026, preprint, arXiv:2604.05743v2; accepted at the AIGENS workshop at CVPR 2026.arxiv.org/abs/2604.05743
- [6]X. Deng, L. Zhou, H. Chen, S. Wu, L. Kong, N. Li, H. Zhao, and D. Ma, “BitFlat: Error-localized bitstream design for robust diffusion image compression,” 2026, preprint, arXiv:2610.25682v1.arxiv.org/abs/2610.25682
- [7]H. Sha, M. Dong, Q. Luo, M. Lu, H. Chen, and Z. Ma, “Towards loss-resilient image coding for unstable satellite networks,” Proceedings of the AAAI Conference on Artificial Intelligence, vol. 39, no. 12, pp. 12506 to 12514, 2025.
- [8]S. Wang, J. Dai, X. Qin, K. Yang, K. Niu, and P. Zhang, “ResiComp: Loss-resilient image compression via dual-functional masked visual token modeling,” IEEE Transactions on Circuits and Systems for Video Technology, vol. 35, no. 7, pp. 7181 to 7195, 2025. Y. Wei, C. Zhou, Y. Shi, J. Wang, and G. Lu, “Every packet counts: Dispersing information for loss-resilient learned image compression,” in Proceedings of the 34th ACM International Conference on Multimedia, 2026, pp. 11 356 to 11 364.
- [10]F. Zhang and L. Xu, “Trajectory-consistent denoising diffusion codebook models for zero-shot high-fidelity image compression at ultra-low bitrates,” in Proceedings of the Thirty-Fifth International Joint Conference on Artificial Intelligence (IJCAI), 2026, pp. 2064 to 2072.DOI
- [11]Y. Cheng, F. Meng, H. Qi, H. Shen, Z. Zhang, Y. Wang, and Y. Liang, “A spatio-temporal hierarchical diffusion framework for training-free perceptual remote sensing image compression,” IEEE Transactions on Geoscience and Remote Sensing, vol. 64, pp. 1 to 17, 2026.DOI
- [12]A. Joseph and A. R. Barron, “Least squares superposition codes of moderate dictionary size are reliable at rates up to capacity,” IEEE Transactions on Information Theory, vol. 58, no. 5, pp. 2541 to 2557, 2012.
- [13]R. Venkataramanan, A. Joseph, and S. Tatikonda, “Lossy compression via sparse linear regression: Performance under minimum-distance encoding,” IEEE Transactions on Information Theory, vol. 60, no. 6, pp. 3254 to 3264, 2014.
- [14]R. Venkataramanan, T. Sarkar, and S. Tatikonda, “Lossy compression via sparse linear regression: Computationally efficient encoding and decoding,” IEEE Transactions on Information Theory, vol. 60, no. 6, pp. 3265 to 3278, 2014.
- [15]R. Venkataramanan, S. Tatikonda, and A. Barron, “Sparse regression codes,” Foundations and Trends in Communications and Information Theory, vol. 15, no. 1-2, pp. 1 to 195, 2019.
- [16]A. No and T. Weissman, “Rateless lossy compression via the extremes,” IEEE Transactions on Information Theory, vol. 62, no. 10, pp. 5484 to 5495, 2016.
- [17]R. Salami, C. Laflamme, J.-P. Adoul, A. Kataoka, S. Hayashi, T. Moriya, C. Lamblin, D. Massaloux, S. Proust, P. Kroon, and Y. Shoham, “Design and description of CS-ACELP: A toll quality 8 kb/s speech coder,” IEEE Transactions on Speech and Audio Processing, vol. 6, no. 2, pp. 116 to 130, 1998.DOI
- [18]C. Rush, A. Greig, and R. Venkataramanan, “Capacity-achieving sparse superposition codes via approximate message passing decoding,” IEEE Transactions on Information Theory, vol. 63, no. 3, pp. 1476 to 1500, 2017.
- [19]J. Barbier and F. Krzakala, “Approximate message-passing decoder and capacity achieving sparse superposition codes,” IEEE Transactions on Information Theory, vol. 63, no. 8, pp. 4894 to 4927, 2017.DOI
- [20]N. Ailon and B. Chazelle, “The fast Johnson-Lindenstrauss transform and approximate nearest neighbors,” SIAM Journal on Computing, vol. 39, no. 1, pp. 302 to 322, 2009.
- [21]J. A. Tropp, “Improved analysis of the subsampled randomized Hadamard transform,” Advances in Adaptive Data Analysis, vol. 3, no. 1-2, pp. 115 to 126, 2011.arxiv.org/abs/1011.1595
- [22]O. Ronai, V. Kulikov, and T. Michaeli, “H-DDCM: Hadamard codebooks for fast generative image and video compression,” 2026, preprint, arXiv:2610.18125v1.arxiv.org/abs/2610.18125
- [23]G. Reeves and R. Venkataramanan, “Lossy compression via sparse regression codes: Generalized construction and finite-length bounds,” 2026, preprint, arXiv:2608.14494v1.arxiv.org/abs/2608.14494
- [24]A. Albanese, J. Blömer, J. Edmonds, M. Luby, and M. Sudan, “Priority encoding transmission,” IEEE Transactions on Information Theory, vol. 42, no. 6, pp. 1737 to 1744, 1996.
- [25]A. E. Mohr, E. A. Riskin, and R. E. Ladner, “Unequal loss protection: Graceful degradation of image quality over packet erasure channels through forward error correction,” IEEE Journal on Selected Areas in Communications, vol. 18, no. 6, pp. 819 to 828, 2000.DOI
- [26]S. Dumitrescu, X. Wu, and Z. Wang, “Globally optimal uneven error-protected packetization of scalable code streams,” IEEE Transactions on Multimedia, vol. 6, no. 2, pp. 230 to 239, 2004.
- [27]R. Hamzaoui, V. Stanković, and Z. Xiong, “Optimized error protection of scalable image bit streams,” IEEE Signal Processing Magazine, vol. 22, no. 6, pp. 91 to 107, 2005.
- [28]Y. Cheng, Z. Zhang, H. Li, A. Arapin, Y. Zhang, Q. Zhang, Y. Liu, K. Du, X. Zhang, F. Y. Yan, A. Mazumdar, N. Teamster, and J. Jiang, “GRACE: Loss-resilient real-time video through neural codecs,” in 21st USENIX Symposium on Networked Systems Design and Implementation (NSDI 24), 2024, accessed 11 November 2026. [Online]. Available: https://www.usenix.org/conference/nsdi24/presentation/chengarxiv.org/abs/2305.12333
- [29]T. M. Cover and J. A. Thomas, Elements of Information Theory, 2nd ed. Hoboken, NJ: Wiley, 2006.
- [30]S. Shamai and A. Steiner, “A broadcast approach for a single-user slowly fading MIMO channel,” IEEE Transactions on Information Theory, vol. 49, no. 10, pp. 2617 to 2635, 2003.DOI
- [31]C. Tian, A. Steiner, S. Shamai, and S. N. Diggavi, “Successive refinement via broadcast: Optimizing expected distortion of a Gaussian source over a Gaussian fading channel,” IEEE Transactions on Information Theory, vol. 54, no. 7, pp. 2903 to 2918, 2008.DOI
- [32]A. Tajer, A. Steiner, and S. Shamai (Shitz), “The broadcast approach in communication networks,” Entropy, vol. 23, no. 1, p. 120, 2021.DOI
- [33]R. Rombach, A. Blattmann, D. Lorenz, P. Esser, and B. Ommer, “High-resolution image synthesis with latent diffusion models,” in 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2022, pp. 10 674 to 10 685.arxiv.org/abs/2112.10752
- [34]R. C. Bose and D. K. Ray-Chaudhuri, “On a class of error correcting binary group codes,” Information and Control, vol. 3, no. 1, pp. 68 to 79, 1960.
- [35]S. Lin and D. J. Costello, Error Control Coding, 2nd ed. Upper Saddle River, NJ: Pearson Prentice Hall, 2004.
- [36][dataset] Eastman Kodak Company, “Kodak lossless true color image suite,” 1993, accessed 10 November 2026. [Online]. Available: https://rok.us/graphics/kodak/
- [37]E. Agustsson and R. Timofte, “NTIRE 2017 challenge on single image super-resolution: Dataset and study,” in 2017 IEEE Conference on Computer Vision and Pattern Recognition Workshops (CVPRW), 2017, pp. 1122 to 1131.DOI
- [38][dataset] Workshop and Challenge on Learned Image Compression, “CLIC 2020 professional test dataset,” 2020, accessed 13 November 2026. [Online]. Available: https://data.vision.ee.ethz.ch/cvl/clic/test/CLIC2020Professional_test.zip
- [39]R. Zhang, P. Isola, A. A. Efros, E. Shechtman, and O. Wang, “The unreasonable effectiveness of deep features as a perceptual metric,” in 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2018, pp. 586 to 595.arxiv.org/abs/1801.03924
- [40]K. Ding, K. Ma, S. Wang, and E. P. Simoncelli, “Image quality assessment: Unifying structure and texture similarity,” IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 44, no. 5, pp. 2567 to 2581, 2022.
- [41]W. Jiang, J. Yang, Y. Zhai, F. Gao, and R. Wang, “MLIC++: Linear complexity multi-reference entropy modeling for learned image compression,” ACM Transactions on Multimedia Computing, Communications, and Applications, vol. 21, no. 5, pp. 1 to 25, 2025.DOI
- [42]M. Hostetter, “Galois: A performant NumPy extension for Galois fields,” 2020, software, version 0.4.11. Accessed 12 November 2026. [Online]. Available: https://github.com/mhostetter/galois
- [43]Y. Polyanskiy, H. V. Poor, and S. Verdú, “Channel coding rate in the finite blocklength regime,” IEEE Transactions on Information Theory, vol. 56, no. 5, pp. 2307 to 2359, 2010.DOI
- [44]E. N. Gilbert, “Capacity of a burst-noise channel,” Bell System Technical Journal, vol. 39, no. 5, pp. 1253 to 1265, 1960.DOI
- [45]E. O. Elliott, “Estimates of error rates for codes on burst-noise channels,” Bell System Technical Journal, vol. 42, no. 5, pp. 1977 to 1997, 1963.DOI
- [46]I. S. Reed and G. Solomon, “Polynomial codes over certain finite fields,” Journal of the Society for Industrial and Applied Mathematics, vol. 8, no. 2, pp. 300 to 304, 1960.DOI
- [47]L. Rizzo, “Effective erasure codes for reliable computer communication protocols,” ACM SIGCOMM Computer Communication Review, vol. 27, no. 2, pp. 24 to 36, 1997.DOI
- [48]Y. Li, H. Zhang, L. Li, and D. Liu, “Learned image compression with hierarchical progressive context modeling,” 2025, preprint, arXiv:2507.19125v1; presented at ICCV 2025.arxiv.org/abs/2507.19125
- [49][dataset] Workshop and Challenge on Learned Image Compression, “CLIC 2020 professional validation dataset,” 2020, accessed 10 November 2026. [Online]. Available: https://data.vision.ee.ethz.ch/cvl/clic/professional_valid_2020.zip





