WorksComputer Science
Flex-DDCM: Zero-Shot Task-Oriented Image Compression via Diffusion Trajectory Guidance
Abstract
Recent zero-shot image compression methods exploit the powerful priors of pretrained diffusion models to produce perceptually pleasing reconstructions without compression-specific training. However, image compression serves not only human viewing but also machine analysis, while existing zero-shot codecs have limited flexibility in meeting the diverse reconstruction requirements of these applications. In this paper, we propose Flex-DDCM, a framework for zero-shot task-oriented image compression via diffusion trajectory guidance. Building on Turbo-DDCM, we adapt the coded diffusion trajectory through task-guided noise selection, embedding the effects of task guidance into the compressed representation. Guidance terms can be applied individually or combined, with timestep-dependent strengths controlling their influence throughout denoising. This design enables task adaptation without introducing additional decoding complexity. Experiments demonstrate flexible trade-offs among pixel fidelity, perceptual quality, and downstream task performance, with competitive performance against trained task-oriented codecs and faster decoding than Diff-ICMH.
Introduction
Zero-shot image compression has recently shown promising perceptual reconstruction quality by reusing pretrained diffusion models, avoiding the need for compression-specific training [1, 2, 3, 4]. These methods differ in how they represent and encode image information. For instance, DiffC [2], subsequently made practical by [4], uses reverse-channel coding (RCC) [5] to progressively communicate noisy samples of the image, followed by denoising to obtain a reconstruction. DDCM [3] instead steers reverse diffusion toward the input image by selecting noise vectors from shared, reproducible codebooks and encoding their indices. Turbo-DDCM [6] efficiently combines multiple codebook vectors at each step and encodes the selected combinations together with quantized coefficients, reducing both the required number of denoising steps and the coding overhead. However, image compression serves not only human viewing but also machine analysis, giving rise to diverse requirements for pixel fidelity, perceptual quality, and semantic information preservation. Although zero-shot codecs offer perceptually pleasing reconstructions, their flexibility in adapting to these diverse requirements remains limited.
To meet these diverse requirements, researchers have explored how to balance different reconstruction objectives within a single compression model. For example, MRIC [7] and CRDR [8] utilize conditional generators to adjust the trade-off between pixel fidelity and perceptual quality, with CRDR additionally supporting bitrate control. Liu et al. [9] extend such control to machine vision, balancing pixel fidelity and downstream task performance through scalable coding and reconstruction interpolation. MPA [10] supports reconstruction for pixel fidelity, perceptual quality, and machine vision through shared and task-specific feature paths. For machine vision applications, some studies have explored image compression frameworks that support multiple downstream tasks [11, 12, 13, 14, 15, 16, 17, 18, 19]. TransTIC [16] and Adapt-ICMH [17] adapt pretrained codecs using lightweight prompts and adapters, respectively, although each task requires separate adaptation. MT-MoA [18] enables joint multi-task adaptation through shared adapter pools and dynamic routing, supporting task-conditioned reconstructions from a single bitstream. Diff-ICMH [19] combines generative priors with semantic consistency training to support multiple downstream tasks from a single codec and bitstream without task-specific adaptation, while maintaining perceptual quality. However, most of these methods require compression-specific training or fine-tuning.
In this paper, we propose Flex-DDCM, a framework for zero-shot task-oriented image compression via diffusion trajectory guidance. Our key idea is to adapt the coded diffusion trajectory to the desired reconstruction objective. In codebook-based diffusion compression, the encoder determines this trajectory through a sequence of noise selections, providing a means of embedding task preferences into the compressed representation. Building on Turbo-DDCM [6], we develop a task-guided selection criterion that combines the reconstruction residual with descent directions derived from task objectives. Different guidance terms can be applied individually or combined to balance competing reconstruction objectives. Motivated by the progression from large-scale structures to fine details in diffusion generation [20], we further use timestep-dependent guidance strengths to emphasize task-specific refinement at later denoising stages. As illustrated in Figure 1, the resulting guidance is conveyed through the transmitted noise selections, enabling task adaptation without additional decoding complexity.
Figure 1. Overview of Flex-DDCM for task-oriented image compression. Task-guided noise selection during encoding enables adaptation to different task requirements, such as pixel fidelity and object detection performance. This adaptation introduces no additional decoding complexity.
Our contributions are summarized as follows:
We propose Flex-DDCM, a framework for zero-shot task-oriented image compression that uses objective gradients to guide encoder-side noise selection, without additional decoding complexity.
We develop a flexible guidance formulation that supports individual and combined reconstruction objectives, with timestep-dependent strengths controlling their influence throughout denoising.
Experiments demonstrate flexible trade-offs among pixel fidelity, perceptual quality, and downstream task performance, with competitive performance against trained task-oriented codecs and faster decoding than Diff-ICMH [19].
Method
Task-guided Noise Selection
Flex-DDCM builds on Turbo-DDCM [6]. Given an image with VAE latent , Turbo-DDCM selects atoms from a shared, reproducible Gaussian codebook at each coded timestep, where is the latent dimension and is the codebook size. Selection uses the largest absolute correlations with the residual , where is the clean latent predicted from . The selected atoms and their quantized coefficients form a sparse vector that determines the reverse update:
where is the reverse mean and is the noise scale. Here, is the original diffusion timestep, and denotes the next sampled timestep. The selected indices are encoded as an unordered set with their coefficients, allowing the decoder to reproduce the trajectory.
As shown in Figure 2, Flex-DDCM adjusts the residual used for atom selection with local descent directions of differentiable task objectives . At the current clean prediction, we compute

Figure 2. Task-guided noise selection in Flex-DDCM. Normalized task gradients are weighted and combined with the latent residual to select codebook atoms, whose indices and signs convey task guidance.
and combine the normalized directions with the latent residual:
Here, contains the active objectives, weights their relative contributions, sets each guidance strength, specifies its temporal profile, and ensures numerical stability. Residual-norm scaling makes the guidance strength relative to the reconstruction residual. A single objective uses and .
We select the atoms most correlated with the guided residual in absolute value and assign signed unit coefficients:
where returns the indices of the largest entries. The noise and reverse update follow (1). Task guidance is thus conveyed through the transmitted indices and signs; the decoder evaluates neither task objectives nor their gradients, introducing no additional decoding complexity.
Guidance Scheduling
Motivated by the progression from coarse structures to fine details in diffusion generation [20], we vary guidance strength across timesteps. Each objective uses one of three profiles:
where is the cumulative signal coefficient and SNR denotes signal-to-noise ratio. The latter two profiles emphasize later denoising steps. We normalize over the timesteps at which codebook indices are transmitted:
Each profile then has unit mean, so controls the average guidance strength. Section III-C compares the schedules.
Task Objectives
For pixel fidelity (D), we use mean squared reconstruction error:
| DiffC | PerCo (SD)† | DiffEIC† | Diff-ICMH† | Turbo-DDCM | Flex-DDCM-D | Flex-DDCM-M | Flex-DDCM-D+M | |
| Enc. (s) | 3.560 | 0.365 | 0.129 | 0.160 | 1.068 | 1.939 | 1.631 | 2.537 |
| Dec. (s) | 3.210 | 1.170 | 3.291 | 6.466 | 0.986 | 0.994 | 1.032 | 1.033 |
Table 1. Average runtime on 512 × 512 Kodak images using an NVIDIA RTX 4090. Enc. and Dec. denote encoding and decoding time. † Trained or fine-tuned for compression.
where is the VAE decoder and are the image dimensions. For perceptual quality (P), we use only the latent residual for selection, without task-gradient guidance, as in Turbo-DDCM [6].
For machine vision (M), following Diff-ICMH [19], we use semantic consistency to preserve features shared across downstream tasks. Let be the middle-block feature map of the frozen diffusion UNet, evaluated on the clean latent at timestep 0 with null-text conditioning. We define the semantic guidance objective as the average cosine distance:
where denotes the feature vector at spatial location , is the number of locations, and cos is cosine similarity. Reference features are computed once and cached. The D+M configuration jointly applies distortion and semantic guidance using in (3), with separate strengths and schedules.
Experiments
Experimental Settings
a) Evaluation Datasets and Metrics: We evaluate our approach in terms of reconstruction quality and downstream machine vision performance on three public datasets. On Kodak [21], we assess signal fidelity using PSNR and perceptual quality using LPIPS [22]. We evaluate object detection and instance segmentation on COCO 2017 [23] using Faster R-CNN [24] and Mask R-CNN [25], respectively, both with R50-FPN backbones in Detectron2 [26]. We report bounding-box AP and mask AP for the two tasks. On Flickr30K, we use BEiT-3 [27] for multimodal retrieval and report Recall@1 in both directions: image-to-text (text retrieval) and text-to-image (image retrieval). Following prior work [3, 6], we center-crop Kodak images to 512 × 512 without resizing. COCO 2017 and Flickr30K images are directly resized to 512 × 512 using bicubic interpolation.
b) Baseline Methods: We compare our method with zero-shot diffusion-based codecs, including DiffC [4], Turbo-DDCM [6], and PSC [1] with distortion- and perception-oriented reconstruction (PSC-D and PSC-P). These methods reuse pretrained diffusion models without compression-specific training. The P configuration is represented by Turbo-DDCM in the comparisons. We also compare with the traditional codec BPG [28] and codecs trained or fine-tuned for image compression, including PerCo [29], MS-ILLM [30], DiffEIC [31], and StableCodec [32]. For machine vision, we further include the task-oriented codecs Adapt-ICMH [17] and Diff-ICMH [19].
c) Implementation Details: We use pretrained Stable Diffusion 2.1-base [33] with sampling steps, , bit per coefficient (), and ranging from 45 to 100. We adopt Turbo-DDCM’s [6] timestep schedule and bitrate-dependent , followed by uncoded deterministic DDIM steps (). Flex-DDCM-D uses MSE guidance (, inverse-timestep weighting), while Flex-DDCM-M uses semantic guidance (, square-root SNR weighting) and RAM++ [34] tag conditioning. Flex-DDCM-D+M combines both terms with and the same strengths, schedules, and tag conditioning. Following Diff-ICMH [19], we losslessly encode and transmit tags as 13-bit indices into a shared dictionary; the decoder recovers comma-separated text for conditioning. Flex-DDCM bpp uses the actual image and tag file sizes in bits, including headers and padding, divided by the image pixel count.
Main Results
a) Quantitative Results: Figure 3 shows how task-oriented noise selection controls the trade-off between reconstruction fidelity and downstream task performance. Relative to Turbo-DDCM, distortion guidance consistently improves PSNR. Semantic guidance improves downstream performance, particularly at low bitrates, while maintaining comparable LPIPS. Combining the two objectives allows Ours-D+M to retain most of the PSNR gains of Ours-D while improving its downstream performance. Ours-M is also competitive with trained task-oriented codecs. Compared with Diff-ICMH [19], it achieves comparable text and image retrieval performance on Flickr30K and higher detection and segmentation AP on COCO 2017 at moderate and high bitrates, without requiring compression-specific training.

Figure 3. Performance comparison on Kodak, COCO 2017, and Flickr30K. Colored areas represent the adjustable range of Flex-DDCM.
b) Runtime: Table I shows that encoder-only task guidance increases encoding time while retaining Turbo-DDCM’s decoding procedure. Despite this encoding overhead, Flex-DDCM supports different reconstruction objectives while retaining efficient decoding. For machine vision, Flex-DDCM-M decodes substantially faster than Diff-ICMH [19] while maintaining competitive downstream performance.
Ablation Studies
a) Guidance Strength and Schedule: As shown in Figure 4, inverse-timestep guidance achieves the largest bitrate savings for distortion, while square-root SNR guidance performs best for detection and segmentation. Both improve over the best constant-guidance setting. The optimal strengths are for distortion and and 0.1 for detection and segmentation, respectively.
Figure 4. Effect of guidance strength and temporal scheduling on Kodak distortion and COCO 2017 object detection and instance segmentation.
b) Tag Conditioning and Classifier-Free Guidance: As shown in Table II, removing tags increases BD-rate for both object detection and instance segmentation. Following Diff-ICMH [19], we set the CFG scale to 5.0 for the CFG variant. Among the evaluated settings, RAM++ tag conditioning without CFG achieves the lowest BD-rate on both tasks.
| Variant | Detection (%) | Segmentation (%) |
| w/o tags | +8.06 | +7.32 |
| w/ CFG | +22.37 | +23.04 |
| Flex-DDCM-M (Ours) | 0.00 | 0.00 |
Table II. ABLATION OF TAG CONDITIONING AND CLASSIFIER-FREE GUIDANCE (CFG) ON COCO 2017. BD-RATE IS RELATIVE TO FLEX-DDCM-M (LOWER IS BETTER).
Conclusion
We presented Flex-DDCM to adapt zero-shot diffusion compression to diverse reconstruction requirements. Task-objective gradients guide encoder-side noise selection, conveying task preferences through the compressed representation without introducing additional decoding complexity. Guidance terms can be used individually or combined, with timestep-dependent strengths controlling their influence during denoising. Experiments demonstrate flexible trade-offs among pixel fidelity, perceptual quality, and downstream task performance. Flex-DDCM also achieves competitive downstream performance against trained task-oriented codecs, with faster decoding than Diff-ICMH.
References
References
- [1]N. Elata, T. Michaeli, and M. Elad, “PSC: Posterior sampling-based compression,” Transactions on Machine Learning Research, 2025. [Online]. Available: https://openreview.net/forum?id=OsqgU6Jz4tarxiv.org/abs/2407.09896
- [2]L. Theis, T. Salimans, M. D. Hoffman, and F. Mentzer, “Lossy compression with Gaussian diffusion,” 2022.arxiv.org/abs/2206.08889
- [3]G. Ouyang, H. Manor, T. Michaeli, and M. Elad, “Compressed image generation with denoising diffusion codebook models,” in Proceedings of the 42nd International Conference on Machine Learning, ser. Proceedings of Machine Learning Research, vol. 267, 2025, pp. 47 044–47 089. [Online]. Available: https://proceedings.mlr.press/v267/ohayon25a.html
- [4]J. Vonderfecht and F. Liu, “Lossy compression with pretrained diffusion models,” in The Thirteenth International Conference on Learning Representations, 2025. [Online]. Available: https://openreview.net/forum?id=raUnLeoZ04arxiv.org/abs/2501.09815
- [5]L. Theis and N. Y. Ahmed, “Algorithms for the communication of samples,” in International Conference on Machine Learning. PMLR, 2022, pp. 21 308–21 328.arxiv.org/abs/2110.12805
- [6]A. Vaisman, G. Ohayon, H. Manor, M. Elad, and T. Michaeli, “Turbo-DDCM: Fast and flexible zero-shot diffusion-based image compression,” in The Fourteenth International Conference on Learning Representations, 2026. [Online]. Available: https://openreview.net/forum?id=eIF1OvC9Z4arxiv.org/abs/2511.06424
- [7]E. Agustsson, D. Minnen, G. Toderici, and F. Mentzer, “Multi-realism image compression with a conditional generator,” in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2023, pp. 22 324–22 333.arxiv.org/abs/2212.13824
- [8]S. Liu, T. Miyazaki, and S. Omachi, “Controlling rate, distortion, and realism: Towards a single comprehensive neural image compression model,” in Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, 2024, pp. 2900–2909.
- [9]J. Liu, R. Feng, Y. Qi, Q. Chen, Z. Chen, W. Zeng, and X. Jin, “Rate-distortion-condition controllable versatile neural image compression,” in European Conference on Computer Vision. Springer, 2025, pp. 329–348.arxiv.org/abs/2407.11700
- [10]X. Zhang, P. Guo, M. Lu, and Z. Ma, “All-in-One Image Coding for Joint Human-Machine Vision with Multi-Path Aggregation,” in Advances in Neural Information Processing Systems, vol. 37, 2024. [Online]. Available: https://proceedings.neurips.cc/paper_files/paper/2024/hash/8395fd356059eaa92afd39e3952a677-Abstract-Conference.htmlarxiv.org/abs/2409.19660
- [11]R. Feng, X. Jin, Z. Guo, R. Feng, Y. Gao, T. He, Z. Zhang, S. Sun, and Z. Chen, “Image coding for machines with omnipotent feature learning,” in ECCV. Springer, 2022, pp. 510–528.arxiv.org/abs/2207.01932
- [12]R. Feng, J. Liu, X. Jin, X. Pan, H. Sun, and Z. Chen, “Prompt-ICM: A unified framework towards image coding for machines with task-driven prompts,” arXiv preprint arXiv:2305.02578, 2023.arxiv.org/abs/2305.02578
- [13]S. Guo, L. Sui, C. Zhang, Z. Chen, W. Yang, and L. Duan, “A unified image compression method for human perception and multiple vision tasks,” in Computer Vision – ECCV 2024. Springer, 2024, pp. 342–359.DOI
- [14]Z. Tang, X. Yi, and H. Wang, “Toward learned image compression for multiple semantic analysis tasks,” IEEE Transactions on Broadcasting, vol. 71, no. 4, pp. 1022–1033, 2025.
- [15]K. Yuan, Q. Liu, X. Shen, H. He, W. Yang, and S. Wang, “Unified coding for both human perception and generalized machine analytics with CLIP supervision,” in Proceedings of the AAAI Conference on Artificial Intelligence, vol. 39, no. 9, 2025, pp. 9517–9525.
- [16]Y.-H. Chen, Y.-C. Weng, C.-H. Kao, C. Chien, W.-C. Chiu, and W.-H. Peng, “Transferring transformer-based image compression from human perception to machine perception,” in Proceedings of the IEEE/CVF International Conference on Computer Vision, 2023, pp. 23 297–23 307.DOI
- [17]H. Li, S. Li, S. Ding, W. Dai, M. Cao, C. Li, J. Zou, and H. Xiong, “Image compression for machine and human vision with spatial-frequency adaptation,” in European Conference on Computer Vision. Springer, 2024, pp. 382–399.arxiv.org/abs/2407.09853
- [18]Y. Yu, Y. Xing, Z. Huang, M. Hong, B. Chen, Y. Wang, and W. Yang, “Mt-moa: A unified parameter-efficient multi-task adaptation framework toward image coding for machines,” IEEE Transactions on Multimedia, 2026.DOI
- [19]R. Feng, Y. Qi, J. Liu, Y. Gao, X. Li, X. Jin, and Z. Chen, “Diff-ICMH: Harmonizing machine and human vision in image compression with generative prior,” in Advances in Neural Information Processing Systems, vol. 38, 2025. [Online]. Available: https://papers.nips.cc/paper_files/paper/2025/hash/5c33e9adee21dae9e03f34ec4865d-Abstract-Conference.htmlarxiv.org/abs/2511.22549
- [20]J. Ho, A. Jain, and P. Abbeel, “Denoising diffusion probabilistic models,” in Advances in Neural Information Processing Systems, vol. 33, 2020, pp. 6840–6851.arxiv.org/abs/2006.11239
- [21]E. Kodak, “Kodak lossless true color image suite (photocd pcd0992),” http://r0k.us/graphics/kodak, 1993.
- [22]R. Zhang, P. Isola, A. A. Efros, E. Shechtman, and O. Wang, “The unreasonable effectiveness of deep features as a perceptual metric,” in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2018, pp. 586–595.arxiv.org/abs/1801.03924
- [23]T.-Y. Lin, M. Maire, S. Belongie, J. Hays, P. Perona, D. Ramanan, P. Dollár, and C. L. Zitnick, “Microsoft coco: Common objects in context,” in Computer Vision–ECCV 2014: 13th European Conference, Zurich, Switzerland, September 6-12, 2014, Proceedings, Part V 13. Springer, 2014, pp. 740–755.arxiv.org/abs/1405.0312
- [24]S. Ren, K. He, R. Girshick, and J. Sun, “Faster r-cnn: Towards real-time object detection with region proposal networks,” NeurIPS, vol. 28, pp. 91–99, 2015.arxiv.org/abs/1506.01497
- [25]K. He, G. Gkioxari, P. Dollár, and R. Girshick, “Mask r-cnn,” in ICCV, 2017, pp. 2961–2969.
- [26]Y. Wu, A. Kirillov, F. Massa, W.-Y. Lo, and R. Girshick, “Detectron2,” https://github.com/facebookresearch/detectron2, 2019.
- [27]W. Wang, H. Bao, L. Dong, J. Bi, J. Bjorck, Z. Peng, Q. Liu, K. Aggarwal, O. K. Mohammed, S. Singhal, S. Som, and F. Wei, “Image as a foreign language: BEiT pretraining for vision and vision-language tasks,” in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2023, pp. 19 175–19 186.
- [28]F. Bellard, “Better portable graphics (bpg) image format,” https://bellard.org/bpg/, accessed: 2024-05-22.
- [29]M. Careil, M. J. Muckley, J. Verbeek, and S. Lathuilière, “Towards image compression with perfect realism at ultra-low bitrates,” in The Twelfth International Conference on Learning Representations, 2024.arxiv.org/abs/2310.10325
- [30]M. J. Muckley, A. El-Nouby, K. Ullrich, H. Jégou, and J. Verbeek, “Improving statistical fidelity for neural image compression with implicit local likelihood models,” in International Conference on Machine Learning. PMLR, 2023, pp. 25 426–25 443.arxiv.org/abs/2301.11189
- [31]Z. Li, Y. Zhou, H. Wei, C. Ge, and J. Jiang, “Toward extreme image compression with latent feature guidance and diffusion prior,” IEEE Transactions on Circuits and Systems for Video Technology, vol. 35, no. 1, pp. 888–899, 2025.DOI
- [32]T. Zhang, X. Luo, L. Li, and D. Liu, “Stablecodec: Taming one-step diffusion for extreme image compression,” in Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV), October 2025, pp. 17 379–17 389.arxiv.org/abs/2506.21977
- [33]R. Rombach, A. Blattmann, D. Lorenz, P. Esser, and B. Ommer, “High-resolution image synthesis with latent diffusion models,” in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2022, pp. 10 684–10 695.arxiv.org/abs/2112.10752
- [34]X. Huang, Y.-J. Huang, Y. Zhang, W. Tian, R. Feng, Y. Zhang, Y. Xie, Y. Li, and L. Zhang, “Open-set image tagging with multi-grained text supervision,” in Proceedings of the 33rd ACM International Conference on Multimedia. ACM, 2025, pp. 4117–4126. [Online]. Available: https://doi.org/10.1145/3746027.3755316

