WorksComputer Science
Diffusion Sensitivity Geometry and Strong Duality for Optimal Privacy-Utility Tradeoffs
Abstract
Existing approaches to differential privacy in diffusion models rely on loose sensitivity bounds and isotropic Gaussian noise, leading to unnecessary estimation error. We develop a framework for the optimal privacy-utility tradeoff of ideal diffusion denoisers, replacing conservative bounds with exact sensitivity characterizations and provably optimal noise calibration. First, we show that worst-case sensitivity over datasets of arbitrary size can be characterized exactly using only two records. For a data domain of size , this reduces the analysis to at most constraints, regardless of dataset size. Our characterization can be arbitrarily tighter than conventional diameter-based sensitivity bounds, revealing potentially unbounded improvements in noise calibration. Second, we obtain exact privacy guarantees for Rényi, approximate, and Gaussian differential privacy, as well as mutual-information privacy for binary records, avoiding the looseness of previous generic privacy conversions. Finally, we show that the minimum achievable weighted mean-squared error can be computed through a semidefinite program and prove strong duality, providing certificates of global optimality.
Introduction
Diffusion models [1, 2] now lead generative modeling of images and other high-dimensional data. Their training sets often contain sensitive content, revealed by the trained models; deep networks are shown to regenerate near-copies of training images [3], and an adversary can infer whether an individual record was used for training from the model’s denoising behavior [4, 5]. Differential privacy (DP) [6] is the standard remedy. Existing private diffusion models are trained with DP-SGD [7, 8, 9], which clips per-example gradients during training and adds isotropic Gaussian noise scaled to a fixed clipping norm. This calibration ignores the geometry of what is being protected and can add far more noise than the guarantee requires.
We ask what the optimal privacy-utility tradeoff is when privacy is enforced at the denoiser itself. We study explicitly the empirical risk minimizing denoiser (denoted as the ideal denoiser), which materializes as a softmax-weighted average of the training records [5]. We release it through a Gaussian mechanism with a general covariance and measure model utility by a weighted mean-squared error. We answer exactly how far one record can move the denoiser, how much privacy a given provides, and how to find the which provides the best privacy-utility tradeoff. Our contributions follow:
Sensitivity (Section III). The worst-case effect of one record, over datasets of every size, is generated by two-record datasets (Lemma 1, Theorem 1); for a fixed query and a domain of points, at most constraints are needed. When one covariance must serve every query, the worst case is the largest Mahalanobis distance between two points of the domain (Corollary 1), which can be far smaller than a naive diameter-based bound (Fig. 1).
Privacy accounting (Section IV). We demonstrate that a single quantity fixes the Rényi divergence between adjacent datasets at every order, the smallest zCDP and Gaussian DP parameters (Theorem 2), and the exact curve for standard differential privacy, with no loss from generic conversions.
Optimal noise (Section V). The least-error covariance under any of these budgets solves a semidefinite program whose conic dual has an exact gap formula; strong duality holds with both optima attained (Theorem 3), so dual solutions certify global optimality.
Mutual-information privacy (Section VI). For binary records, mutual-information DP reduces to the capacity of a binary-input Gaussian channel whose capacity is strictly increasing in the signal-to-noise ratio (SNR) by the I-MMSE identity, giving another exact instance of the same program.
Related Work
Privacy-utility tradeoffs. Information theory has long studied the tension between disclosure and utility, as exemplified with Yamamoto’s source coding problem with a secret component [10]. It has since been studied through the privacy funnel [11], MMSE estimation under privacy constraints [12, 13], and estimation guarantees under disclosure limits [14]. Under DP, Geng and Viswanath show that the optimal additive noise for a scalar query follows a staircase-shaped distribution rather than the Gaussian law [15]. However, none of these works directly apply to the diffusion model, which we show leads to interesting characterizations of privacy-utility tradeoffs.
Gaussian mechanisms and accounting. We consider approximate DP [6], Rényi DP [16], zCDP [17], Gaussian DP [18], and the mutual-information formulation of Cuff and Yu [19]; our exact curve follows the analytic calibration of Balle and Wang [20]. Correlated Gaussian noise has been optimized for linear marginal queries through the matrix mechanism [21] and for unbiased mean estimation [22]. In those problems the sensitivity follows directly from the query; here the release is a specific nonlinear function of the data, and its sensitivity set must first be characterized.
Information-estimation relations. The I-MMSE identity [23] has been extended to mismatched estimation [24], arbitrary additive perturbations through score functions [25], Poisson and semimartingale channels [26, 27], pointwise relations [28], Gaussian networks and matrix-valued SNR [29, 30], and channels with feedback and memory [31, 32]. We use the identity to show that the binary-input channel induced by our mechanism has capacity strictly increasing in the SNR, which makes mutual-information privacy an exact sensitivity budget.
The Ideal Denoiser
Normalized empirical posterior mean
Let be a public (nonempty) bounded domain. A dataset is a (nonempty) finite multiset in . Fix a public query and noise scale . Define
The denominator is a sum of the weights of . If is drawn uniformly from and with , Bayes’ rule implies that . In particular .
Write , where is the cumulative signal coefficient. For an ideal predictor , we obtain
The entire numerator is divided by . Equations (1)–(2) use the rescaled query and . The empirical posterior mean formulation is related to the ideal diffusion noise predictor setting studied in DIME [5].
Mechanism and a linear objective
Release only
The covariance is the same for all private datasets, independent of the data and of the Gaussian draw. It may depend on public information and the fixed query. Let be a fixed linear map and its error weight. For , the expected additional quadratic loss is exactly
The cross term against any baseline reconstruction error has mean zero. Mean prediction MSE uses . If noise is added to instead, the reconstruction cost is . Conversely, mean noise is equivalent to noise in the ideal noise prediction.
We assume to obtain compactness and for ease of computing matrix square roots. A semidefinite task weight still defines a valid SDP, but may leave unpenalized directions with unbounded variance and an unattained infimum. Adding a specified small positive weight or separately bounding those directions changes the problem. Equation (5) measures exact quadratic response or one-step reconstruction error. i.e., we do not use a nonlinear full sampler.
From a Fixed Dataset to Global Sensitivity
Exact deletion
For , let delete record . Define . Since all Gaussian weights are positive,
Indeed, . This shows that the relevant direction depends on the posterior responsibility and the location of the deleted record relative to the remaining mean. In particular,
The posterior spread also determines the query Jacobian:
obtained by differentiating .
Adjacency Notions
Add/Remove
1) Add/Remove: For global add/remove privacy, our domain consists of all nonempty finite multisets in . Define
Replacement
2) Replacement: Define
For the replacement adjacency at fixed dataset size , write neighboring datasets as and , and form the larger multiset . Then
An add/remove Mahalanobis squared bound therefore implies a replacement bound by the triangle inequality.
A finite exact reduction for one fixed query
Lemma 1 ([33]). For any convex function and set , it holds that
Proof. For a convex and any , write with , , . Convexity gives us . Taking sup over on the left, we get . The reverse inequality is trivial since .
Theorem 1 (Two record reduction). For fixed , and the add/remove adjacency notion, define
The closed symmetric convex hull of equals that generated by all . Consequently, for every ,
Proof. Denote . We first observe that , so . Take valid 2-point dataset in . Then exactly, so every itself is some . It remains to show that . Consider some arbitrary . We first show that . Taking any , then clearly lies in the line between and for some . We can expand
where
Then, adding the constants up we have
where the last inequality comes from Jensen’s inequality . Thus, can be written as a convex combination of for and , which proves that . (13) follows directly from Lemma 1.
For a finite public domain of size , at most nonzero constraints suffice for an exact finite SDP, despite the unbounded dataset size. Explicitly, the constraint of dataset mean shifts can be represented with the unique terms generated from finite domain .
Corollary 1 (Uniformly over all query inputs). If a single covariance must work for every , then
Proof. Clearly, is in both the convex hull the set , and any element is a convex combination of and in , so , so from Theorem 1 we get that . Fix . Denote for . Then it holds that
As , . Since the quadratic product is continuous, it holds that any product can be written as a limit of terms , which by Theorem 1 is bounded above by . Since limits preserve inequality, it holds that .
Rényi DP and Related Gaussian Privacy Definitions
For a chosen adjacency relation, write for the output distribution and let be the full neighboring mean difference set (e.g., Equation (9)), or a set with the same closed symmetric convex hull. For ,
A mechanism is -RDP if for every ordered neighboring pair [16]. It is -DP if for every such pair and measurable [6].
Theorem 2 (Exact Gaussian privacy curve). For a common positive definite covariance, set . The smallest valid RDP parameter at every order is
Thus, one order- budget is equivalent to , and several budgets are equivalent to , where is the collection of Rényi orders being considered. The same mechanism is exactly -zCDP and -GDP in the sense of their smallest parameters [17, 18].
Proof. The Rényi divergence between two Gaussian distributions with common covariance matrix has closed form [16, 34]
for all orders . Thus, the Rényi divergence for specific between noised outputs of adjacent datasets has lowest upper bound of . A mechanism satisfies -zCDP if and only if its Rényi Divergence between neighboring outputs is bounded by for every [17], which means we directly get that the mechanism satisfies -zCDP. GDP measures privacy by how well the best possible hypothesis test can distinguish a mechanism’s output on two neighboring datasets, and says a mechanism is -GDP if that best test is never more powerful than distinguishing and . Testing vs. is equivalent by invertible whitening map to testing two isotropic Gaussians separated by . The worst case neighbor mean shift is captured exactly by so the mechanism satisfies -GDP.
A general RDP guarantee implies -DP for [16]. Define , regarded as a log-likelihood random variable for two Gaussian laws representing mechanisms characterized by adjacent datasets. Optimizing this conversion over (16) gives the budget
Let the standard Gaussian CDF. For and , the log-likelihood ratio is affine in a Gaussian, so for it holds that , . Each tail probability is then a single Gaussian CDF:
( )-DP requires that , so we define
The maximizing event in is . For , increases continuously from zero to one, since
Thus, the exact worst-case is . We conclude that -DP is exactly given by when . For any nonzero shift, has an unbounded upper tail, i.e., finite Gaussian covariance cannot give pure DP at finite .
The Covariance Primal and Its Conic Dual
Let describe the full symmetric convex hull of a specified sensitivity set. Take , , and a public noise floor . All matrices are real and symmetric, and .
The primal is
For the ordinary Schur complement gives . For , the generalized Schur complement gives both and . Equivalently, . With , exactly minimizes the loss among the specified common covariance Gaussian mechanisms satisfying RDP when the sensitivity hull is exact. Theorem 1 makes a finite public exact for a single fixed query; Corollary 1 handles a uniform-query covariance.
Theorem 3 (Conic dual, gap, and strong duality). Introduce
A dual of is
For any feasible pair, its objective gap equals
For finite , , , and finite shifts, both programs attain their optima and their values agree.
Proof. We first show a block-trace identity. Block multiplication gives
so . We dualize the constraints each with and with and, giving us the Lagrangian
We can simplify using our block-trace identity
The dual function is . However, in the dual function is unconstrained so unless the coefficient with is zero the infimum is , giving us the constraint . When the constraint holds, , demonstrating that is the Lagrangian dual of . Let be primal-feasible and be dual-feasible, so in particular . We get that , writing and using the block-trace identity we write as:
Moving the dual-objective terms to the left-hand side gives us (19) for any feasible pair. We are left with the strong duality claim. Slater’s condition tells us that in a convex program if there exists a point in the primal with strict feasibility and the primal has boundedness from below then the primal and dual optimum exist and agree [35]. Since and clearly , so we have boundedness from below. Consider for some constant . Clearly and for all so we have strict feasibility. as along any direction in the closed feasible set, so we conclude that the optimum is attainable by Extreme Value Theorem [36].
MI-DP: Exact Programs
Definition and conditional channel capacity
Fix a database size and product domain (for the denoiser, ). For random , Cuff–Yu MI-DP is
All information is in nats. The supremum includes all joint priors, with arbitrary record dependence [37].
Let where is the coordinate/record whose privacy we are measuring and is the fixed value of all the other records . Let be the channel . Here is the channel input, i.e. the value of record , and the output is , the mechanism run on the dataset with record equal to and all other records fixed by . Let be a probability distribution on , and let and . Then
This is easy to see if one conditions any prior on : the conditional information is an average of channel informations, each at most the corresponding capacity. Conversely, choose deterministically and any distribution for the remaining record. Taking suprema leads to (21).
Exact binary record calibration and its dual
Suppose every record has two possible values. Then when the Gaussian means differ by , we let . Whitening and translating reduces its (informative) coordinate to
By reflection symmetry and concavity of mutual information in the input distribution, we know that the uniform sign is capacity-achieving. Its capacity is
The real Gaussian I–MMSE identity [23] gives
Also and as : a sign detector has vanishing error, so its conditional entropy vanishes. Thus, for ,
This gives an exact instance of and using the full replacement sensitivity set. For singular covariance define when , and otherwise. The same equivalence uses in place of the inverse quadratic: disjoint supports give , and common supports reduce to the nonsingular Gaussian channel. For add/remove adjacency, the choice of suffices for , following directly from the triangle inequality in Equation (11).
Conclusion
We characterized the optimal privacy-utility tradeoff for releasing the ideal diffusion denoiser through a Gaussian mechanism with a general covariance. The worst-case sensitivity over datasets of any size is generated by pairs of records, specifically the largest Mahalanobis distance between two points of the data domain. This sensitivity quantity fixes the mechanism’s Rényi, zCDP, Gaussian, and approximate DP guarantees exactly, as well as its mutual-information privacy for binary records. Finding the least-error covariance under any of these budgets is a semidefinite program whose strong dual certifies global optimality. Together, these results replace conservative sensitivity bounds with exact privacy accounting and provably optimal noise design for privacy-preserving diffusion denoisers.
AI Use Acknowledgment
In this work, we used generative AI tools to generate and optimize the code that was used to run the simulation and experiments (Figure 1).

Figure 1. Histogram of the ratio of naive sensitivity bound to theoretical diffusion-specialized sensitivity bound over Gaussian mechanisms with random covariance matrices for a simulated convex hull . Our theoretical bound achieves sensitivity analysis on average tighter with up to gain in tail cases.
References
- [1]Jonathan Ho, Ajay Jain, and Pieter Abbeel. Denoising diffusion probabilistic models. Advances in neural information processing systems, 33:6840–6851, 2020.arxiv.org/abs/2006.11239
- [2]Yang Song, Jascha Sohl-Dickstein, Diederik P Kingma, Abhishek Kumar, Stefano Ermon, and Ben Poole. Score-based generative modeling through stochastic differential equations. arXiv preprint arXiv:2011.13456, 2020.arxiv.org/abs/2011.13456
- [3]Nicholas Carlini, Florian Tramer, Eric Wallace, Matthew Jagielski, Ariel Herbert-Voss, Katherine Lee, Adam Roberts, Tom Brown, Dawn Song, Ulfar Erlingsson, et al. Extracting training data from large language models. In 30th USENIX security symposium (USENIX Security 21), pages 2633–2650, 2021.arxiv.org/abs/2012.07805
- [4]Mingxing Rao, Bowen Qu, and Daniel Moyer. Score-based membership inference on diffusion models. arXiv preprint arXiv:2509.25003, 2025.arxiv.org/abs/2509.25003
- [5]Tue Do and Daniel Alabi. DIME: Query-efficient framework for membership inference on diffusion models. arXiv preprint arXiv:2608.22824, 2026.
- [6]Cynthia Dwork, Frank McSherry, Kobbi Nissim, and Adam D. Smith. Calibrating noise to sensitivity in private data analysis. In TCC, 2006.
- [7]Martín Abadi, Andy Chu, Ian Goodfellow, H. Brendan McMahan, Ilya Mironov, Kunal Talwar, and Li Zhang. Deep learning with differential privacy. In Proceedings of the 2016 ACM SIGSAC Conference on Computer and Communications Security, pages 308–318, 2016.DOI
- [8]Tim Dockhorn, Tianshi Cao, Arash Vahdat, and Karsten Kreis. Differentially private diffusion models. Transactions on Machine Learning Research, 2023.arxiv.org/abs/2210.09929
- [9]Sahra Ghalebikesabi, Leonard Berrada, Sven Gowal, Ira Ktena, Robert Stanforth, Jamie Hayes, Soham De, Samuel L Smith, Olivia Wiles, and Borja Balle. Differentially private diffusion models generate useful synthetic images. arXiv preprint arXiv:2302.13861, 2023.arxiv.org/abs/2302.13861
- [10]Hirosuke Yamamoto. A source coding problem for sources with additional outputs to keep secret from the receiver or wiretappers. IEEE Transactions on Information Theory, 29(6):918–923, November 1983.DOI
- [11]Ali Makhdoumi, Salman Salamatian, Nadia Fawaz, and Muriel Médard. From the information bottleneck to the privacy funnel. In Proceedings of the IEEE Information Theory Workshop (ITW), pages 501–505, 2014.arxiv.org/abs/1402.1774
- [12]Shahab Asoodeh, Fady Alajaji, and Tamás Linder. Privacy-aware MMSE estimation. In Proceedings of the IEEE International Symposium on Information Theory (ISIT), pages 1989–1993, 2016.arxiv.org/abs/1601.07417
- [13]Shahab Asoodeh, Mario Diaz, Fady Alajaji, and Tamás Linder. Estimation efficiency under privacy constraints. IEEE Transactions on Information Theory, 65(3):1512–1534, March 2019.DOI
- [14]Hao Wang, Lisa Vo, Flávio P. Calmon, Muriel Médard, Ken R. Duffy, and Mayank Varia. Privacy with estimation guarantees. IEEE Transactions on Information Theory, 65(12):8025–8042, December 2019.
- [15]Quan Geng and Pramod Viswanath. The optimal noise-adding mechanism in differential privacy. IEEE Transactions on Information Theory, 62(2):925–951, February 2016.DOI
- [16]Ilya Mironov. Rényi differential privacy. In 2017 IEEE 30th Computer Security Foundations Symposium (CSF), pages 263–275. IEEE, 2017.DOI
- [17]Mark Bun and Thomas Steinke. Concentrated differential privacy: Simplifications, extensions, and lower bounds. In Martin Hirt and Adam Smith, editors, Theory of Cryptography (TCC 2016-B), Part I, volume 9985 of Lecture Notes in Computer Science, pages 635–658. Springer, 2016.arxiv.org/abs/1605.02065
- [18]Jinshuo Dong, Aaron Roth, and Weijie J. Su. Gaussian differential privacy. Journal of the Royal Statistical Society: Series B (Statistical Methodology), 84(1):3–37, 2022.
- [19]Paul Cuff and Lanqing Yu. Differential privacy as a mutual information constraint. In Proceedings of the 2016 ACM SIGSAC Conference on Computer and Communications Security, Vienna, Austria, October 24-28, 2016, pages 43–54. ACM, 2016.DOI
- [20]Borja Balle and Yu-Xiang Wang. Improving the Gaussian mechanism for differential privacy: Analytical calibration and optimal denoising. In Jennifer Dy and Andreas Krause, editors, Proceedings of the 35th International Conference on Machine Learning, volume 80 of Proceedings of Machine Learning Research, pages 394–403. PMLR, 2018.arxiv.org/abs/1805.06530
- [21]Yingtai Xiao, Guanlin He, Danfeng Zhang, and Daniel Kifer. An optimal and scalable matrix mechanism for noisy marginals under convex loss functions. In Advances in Neural Information Processing Systems, volume 36, 2023.DOI
- [22]Aleksandar Nikolov and Haohua Tang. General Gaussian noise mechanisms and their optimality for unbiased mean estimation. In 15th Innovations in Theoretical Computer Science Conference (ITCS 2024), volume 287 of Leibniz International Proceedings in Informatics (LIPIcs), pages 85:1–85:23. Schloss Dagstuhl – Leibniz-Zentrum für Informatik, 2024.arxiv.org/abs/2301.13850
- [23]Dongning Guo, Shlomo Shamai, and Sergio Verdú. Mutual information and minimum mean-square error in gaussian channels. IEEE transactions on information theory, 51(4):1261–1282, 2005.arxiv.org/abs/cs/0412108
- [24]Sergio Verdú. Mismatched estimation and relative entropy. IEEE Transactions on Information Theory, 56(8):3712–3720, 2010.
- [25]Dongning Guo. Relative entropy and score function: New information-estimation relations through arbitrary additive perturbation. In 2009 IEEE International Symposium on Information Theory, pages 814–818. IEEE, 2009.
- [26]Rami Atar and Tsachy Weissman. Mutual information, relative entropy, and estimation in the poisson channel. IEEE Transactions on Information Theory, 58(3):1302–1318, 2012.
- [27]Jiantao Jiao, Kartik Venkat, and Tsachy Weissman. Mutual information, relative entropy and estimation error in semimartingale channels. IEEE Transactions on Information Theory, 64(10):6662–6671, 2018.
- [28]Kartik Venkat and Tsachy Weissman. Pointwise relations between information and estimation in gaussian noise. In 2012 IEEE International Symposium on Information Theory Proceedings, pages 701–705, 2012.
- [29]Alex Dytso, Ronit Bustin, H. Vincent Poor, and Shlomo Shamai (Shitz). A view of information-estimation relations in gaussian networks. Entropy, 19(8), 2017.
- [30]Galen Reeves, Henry D. Pfister, and Alex Dytso. Mutual information as a function of matrix snr for linear gaussian channels. In 2018 IEEE International Symposium on Information Theory (ISIT), pages 1754–1758, 2018.DOI
- [31]Guangyue Han and Jian Song. Extensions of the i-mmse relation. In 2014 IEEE International Symposium on Information Theory, pages 2202–2206, 2014.arxiv.org/abs/1506.07230
- [32]Guangyue Han and Jian Song. Extensions of the i-mmse relationship to gaussian channels with feedback and memory. IEEE Transactions on Information Theory, 62(10):5422–5445, 2016.arxiv.org/abs/1401.3527
- [33]R Tyrrell Rockafellar. Convex analysis, volume 28. Princeton university press, 1997.
- [34]Tim Van Erven and Peter Harremos. Rényi divergence and kullback-leibler divergence. IEEE transactions on information theory, 60(7):3797–3820, 2014.
- [35]Lieven Vandenberghe and Stephen Boyd. Convex optimization, volume 1. Cambridge university press Cambridge, 2004.
- [36]Walter Rudin. Principles of mathematical analysis. 2021.
- [37]Paul Cuff and Lanqing Yu. Differential privacy as a mutual information constraint. In Proceedings of the 2016 ACM SIGSAC Conference on Computer and Communications Security, pages 43–54. ACM, 2016.DOI