Introduction
Institutions often hold complementary datasets that capture different aspects of the same downstream learning task. For example, a breast screening clinic may hold primarily negative cases, while a specialized diagnostic center may hold a much higher proportion of confirmed cancer cases [35 , 43 ] . Similarly, organizations trying to detect cybersecurity incidents frequently share data to provide a comprehensive view of the cybersecurity landscape, as attackers may target different organizations with different exploits at different times [13 ] . In such settings, pooling data may enable institutions to learn a useful classifier that no entity could learn on its own. Indeed, this potential was highlighted and analyzed in early theoretical works on distributed machine learning [3 ] . Realizing the potential benefit of collaboration, however, often requires institutions to commit resources upfront through legal agreements, financial compensation, governance approvals, and dedicated infrastructure. Before incurring these costs, institutions need to assess the utility of combining their datasets: will pooling improve model accuracy or other downstream metrics enough to justify the collaboration? Yet this assessment depends on joint properties of the datasets, generally hidden from each institution’s local view. Institutions face a circular dilemma: they must judge whether collaboration is worthwhile before committing, yet doing so may require the very data sharing whose value they seek to assess.
Privacy requirements further complicate this dilemma. Hospitals, for example, must protect sensitive patient records even when sharing them would help evaluate a collaboration. Institutions may also hesitate to disclose valuable data to a prospective partner who could use them independently and decline to collaborate. When no trusted third party is available, each party must therefore assess whether collaboration is promising while limiting what it reveals about its data. This tension between the need to preserve data privacy while enabling agents to assess the utility of collaboration motivates the main conceptual question of this paper: How can agents estimate whether it is beneficial to combine their datasets while preserving the privacy of individual records? What algorithms can achieve this goal, and what are their fundamental limits?
How to Assess the Benefits of Data Sharing?
In this section, we propose a learning-theoretic notion that agents can use to assess the value of data sharing and provide two real-world scenarios in which it can inform the decision to collaborate. Our focus throughout the paper is on binary classification on a data space (or feature space) X ⊆ R d \mathcal{X} \subseteq\mathbb{R}^{d} X ⊆ R d of size m m m for some d ≥ 1 d \ge1 d ≥ 1 and a fixed publicly known hypothesis class H ⊆ { ± 1 } X \mathcal{H} \subseteq\{\pm1\}^{\mathcal{X}} H ⊆ { ± 1 } X . We consider a setting with two agents, Alice (A A A ) and Bob (B B B ) who respectively hold a labeled dataset S A , S B ∈ ( X × { ± 1 } ) n S_A, S_B \in(\mathcal{X} \times\{\pm1\})^n S A , S B ∈ ( X × { ± 1 } ) n of size n ∈ N n \in\mathbb{N} n ∈ N .
We focus on a basic question that arises before joint training: can a single classifier in a given hypothesis class explain both parties’ data? We formalize this question using their joint empirical loss defined as
opt H ( S A , S B ) : = min h ∈ H ∑ U ∈ { A , B } ∑ ( x , y ) ∈ S U 1 { h ( x ) ≠ y } . (1) \operatorname{opt}_{\mathcal{H}}(S_A,S_B) := \min_{h \in\mathcal{H}} \sum_{U \in\{A,B\}} \sum_{(x,y) \in S_U} \mathbf{1}\{h(x) \ne y\}. \tag*{(1)} opt H ( S A , S B ) := h ∈ H min U ∈ { A , B } ∑ ( x , y ) ∈ S U ∑ 1 { h ( x ) = y } . (1)
We use opt H ( S A , S B ) \operatorname{opt}_{\mathcal{H}}(S_A,S_B) opt H ( S A , S B ) , the number of mistakes made on the pooled data by the best classifier in H \mathcal{H} H , to assess the value of combining the data. It is zero when one hypothesis explains both datasets, and large when every hypothesis in H \mathcal{H} H makes many mistakes on their union. A large joint loss may indicate conflicting labeling practices, selection biases causing missing features and labels, or that the hypothesis class is not complex enough to fit the combined data.
We therefore focus on distinguishing between the following:
opt H ( S A , S B ) = 0 opt H ( S A , S B ) ≥ t (2) \operatorname{opt}_{\mathcal{H}}(S_A,S_B) = 0
\qquad\qquad
\operatorname{opt}_{\mathcal{H}}(S_A,S_B) \ge t \tag*{(2)} opt H ( S A , S B ) = 0 opt H ( S A , S B ) ≥ t (2)
for some t > 0 t > 0 t > 0 . When opt H ( S A , S B ) = 0 \operatorname{opt}_{\mathcal{H}}(S_A,S_B) = 0 opt H ( S A , S B ) = 0 , Alice and Bob agree to collaborate. When opt H ( S A , S B ) ≥ t \operatorname{opt}_{\mathcal{H}}(S_A,S_B) \ge t opt H ( S A , S B ) ≥ t , Alice and Bob must reassess2 . Finally, if 0 < opt H ( S A , S B ) < t 0 < \operatorname{opt}_{\mathcal{H}}(S_A,S_B) < t 0 < opt H ( S A , S B ) < t , we make no claims, as is standard in distribution testing. We now present two examples that further motivate the property-testing problem in (2) .
Example 1 (Learning from disparate label sets). Suppose Alice is a screening clinician and Bob is a diagnostician. They want to develop a classifier to determine whether a patient in a target population should be referred for further diagnostic testing. Their datasets reflect different stages of care: Alice’s screening dataset S A S_A S A mainly contains records of patients who were not referred, whereas Bob’s diagnostic dataset S B S_B S B contains only records of patients who were referred and received more thorough testing [24 ] . Since the records contain sensitive patient information, neither agent is willing to disclose individual records to assess a possible collaboration, and no trusted third party is available. To isolate this selection bias, suppose Alice’s labels are all − 1 -1 − 1 , and Bob’s are all + 1 +1 + 1 . Then, neither can train a useful model alone; they must assess opt H ( S A , S B ) \operatorname{opt}_{\mathcal{H}}(S_A,S_B) opt H ( S A , S B ) to determine whether they can train a low-error model on their joint data.
Example 2 (Identifying conflicting labeling practices). Now suppose Alice has labeled spam and non-spam emails in her dataset. However, her data may be limited or imbalanced, so she wants to augment it with Bob’s curated, labeled email dataset. The challenge is that Bob may have a different view of what constitutes spam, so his labels could conflict with Alice’s data [29 ] . For example, Alice may label unsolicited promotional emails as spam, while Bob uses that label for fraudulent or malicious messages. Before joint training, Alice wants to know whether a single classifier can achieve a sufficiently low test error under both labeling policies. To isolate conflicting labeling practices, suppose the agents encounter the same distribution of emails but may assign different labels to the same email. In this case, opt H ( S A , S B ) = 0 \operatorname{opt}_{\mathcal{H}}(S_A,S_B)=0 opt H ( S A , S B ) = 0 indicates that, under the hypothesis class H H H , Alice’s and Bob’s labeling practices are compatible, implying a collaboration may be worthwhile. On the other end, opt H ( S A , S B ) ≥ t \operatorname{opt}_{\mathcal{H}}(S_A,S_B)\ge t opt H ( S A , S B ) ≥ t points to differing labeling practices.
Problem Formalization
In this section, we formalize our problem and state our main technical question. We first consider the disjoint-label setting as in Example 1, where Alice holds only negatively labeled examples, and Bob holds only positively labeled ones. Since the labels are fixed by the party holding each record, we suppress them and write S A , S B ∈ X n S_A, S_B \in\mathcal{X}^n S A , S B ∈ X n . This setting isolates the joint aspect of the problem: when H \mathcal{H} H contains the constant predictors, each local dataset is perfectly realizable, but their union need not be. Thus, the local optimal errors alone do not determine whether a common classifier can explain both datasets. For our algorithms, we later show that they extend to datasets with arbitrary label mixtures (see Corollaries 4.8 and 3.13 ).
The definition of opt H \operatorname{opt}_{\mathcal{H}} opt H in Equation (1) in the disjoint-label setting can be rewritten as
opt H ( S A , S B ) ≜ min h ∈ H { ∑ x A ∈ S A 1 { h ( x A ) ≠ − 1 } + ∑ x B ∈ S B 1 { h ( x B ) ≠ + 1 } } . (3) \operatorname{opt}_{\mathcal{H}}(S_A,S_B) \triangleq\min_{h\in\mathcal{H}}\left\{\sum_{x_A\in S_A}\mathbf{1}\{h(x_A)\ne-1\}+\sum_{x_B\in S_B}\mathbf{1}\{h(x_B)\ne+1\}\right\}. \tag*{(3)} opt H ( S A , S B ) ≜ h ∈ H min { x A ∈ S A ∑ 1 { h ( x A ) = − 1 } + x B ∈ S B ∑ 1 { h ( x B ) = + 1 } } . (3)
To assess a potential collaboration, the agents first agree on an error threshold t ∈ [ 2 n ] t\in[2n] t ∈ [ 2 n ] . They seek a private assessment algorithm that rejects with high probability when every classifier in H \mathcal{H} H makes at least t t t mistakes on the pooled data, i.e., opt H ( S A , S B ) ≥ t \operatorname{opt}_{\mathcal{H}}(S_A,S_B)\ge t opt H ( S A , S B ) ≥ t . We call this requirement soundness. However, a trivial algorithm that always rejects also satisfies soundness. We therefore also require the algorithm to accept with high probability when the pooled data are jointly realizable, i.e., opt H ( S A , S B ) = 0 \operatorname{opt}_{\mathcal{H}}(S_A,S_B)=0 opt H ( S A , S B ) = 0 , a requirement we call completeness. Finally, to protect the privacy of individual records throughout the interaction, we use differential privacy [17 ] in its two-party form [8 , 37 ] , which requires each agent’s view to be differentially private with respect to the other agent’s dataset (see Definition 2.1). We combine these pieces and formally define the two-party algorithm for dataset valuation as follows:
Definition 1.2 (ε \varepsilon ε -DP tester with resolution t t t ). Fix a data space X \mathcal{X} X , a hypothesis class H ⊂ { ± 1 } X \mathcal{H} \subset\{\pm1\}^{\mathcal{X}} H ⊂ { ± 1 } X , a dataset size n ∈ N n \in\mathbb{N} n ∈ N , a privacy parameter ε > 0 \varepsilon> 0 ε > 0 , and t ∈ [ 2 n ] t \in[2n] t ∈ [ 2 n ] . Let Π \Pi Π be an almost-surely terminating interactive two-party protocol on all pairs of finite multisets over X \mathcal{X} X , with common output in { M a t c h , F a r } \{\mathrm{Match}, \mathrm{Far}\} { Match , Far } . We interpret these outputs as acceptance or rejection of the proposed collaboration at this preliminary stage, respectively. We call Π \Pi Π an ε \varepsilon ε -private tester with resolution t t t for H \mathcal{H} H at dataset size n n n if it satisfies the following requirements:
Privacy. Π \Pi Π satisfies ε \varepsilon ε -two-party differential privacy as in Definition 2.1 for every input S A , S B ∈ X n S_A, S_B \in\mathcal{X}^n S A , S B ∈ X n
Soundness. For every S A , S B ∈ X n S_A, S_B \in\mathcal{X}^n S A , S B ∈ X n s.t. opt H ( S A , S B ) ≥ t \operatorname{opt}_{\mathcal{H}}(S_A, S_B) \ge t opt H ( S A , S B ) ≥ t , we have Pr [ Π ( S A , S B ) = F a r ] ≥ 0.9 \Pr[\Pi(S_A, S_B) = \mathrm{Far}] \ge0.9 Pr [ Π ( S A , S B ) = Far ] ≥ 0.9 .
Completeness. For every S A , S B ∈ X n S_A, S_B \in\mathcal{X}^n S A , S B ∈ X n , s.t. opt H ( S A , S B ) = 0 \operatorname{opt}_{\mathcal{H}}(S_A, S_B) = 0 opt H ( S A , S B ) = 0 , we have Pr [ Π ( S A , S B ) = M a t c h ] ≥ 0.9 \Pr[\Pi(S_A, S_B) = \mathrm{Match}] \ge0.9 Pr [ Π ( S A , S B ) = Match ] ≥ 0.9 .
The probabilities are over the protocol’s private and public randomness.
We seek the smallest achievable resolution t t t , since smaller values let agents enforce stricter error tolerances. Without privacy constraints, the agents can exchange their datasets and compute opt H ( S A , S B ) \operatorname{opt}_{\mathcal{H}}(S_A, S_B) opt H ( S A , S B ) exactly, achieving t = 1 t = 1 t = 1 , the best possible resolution. A trusted curator could also ensure differential privacy by releasing opt H ( S A , S B ) + Lap ( 1 / ε ) \operatorname{opt}_{\mathcal{H}}(S_A, S_B) + \operatorname{Lap}(1/\varepsilon) opt H ( S A , S B ) + Lap ( 1/ ε ) , since the joint loss has sensitivity one. This gives a tester with resolution t c e n t r a l = O ( 1 / ε ) t_{\mathrm{central}} = O(1/\varepsilon) t central = O ( 1/ ε ) , independent of the dimension and hypothesis class. However, we require information-theoretic two-party differential privacy without a trusted curator. This motivates the main technical question that our paper seeks to answer:
For fixed H \mathcal{H} H and ε \varepsilon ε , what is the smallest resolution t t t that an ε \varepsilon ε -DP private tester can guarantee?
Our Results
We answer the above technical question for two fundamental classes of hypotheses: thresholds on the real line and half-spaces in R d \mathbb{R}^d R d . For each class, we establish both upper and lower bounds on the optimal resolution t t t of any ε \varepsilon ε -private tester (summarized in Table 1 ). Notably, our main interactive protocols achieve a resolution t t t that is *completely independent of the dataset size n n n , and all of our upper bounds extend immediately from the disjoint-label setting to arbitrary mixed-label datasets (see Corollaries 4.8 and 3.13 ).
Table 1. Bounds on the error gap t t t for pure ε \varepsilon ε -two-party DP testing with success probability at least 0.9, up to universal constant factors. † The halfspace lower bound holds for some input domains, d ≥ d 0 d \ge d_{0} d ≥ d 0 , m ≳ d m \gtrsim d m ≳ d , and sufficiently small ε \varepsilon ε .
Thresholds on Real Line
We start with the simplest nontrivial class, thresholds on the real line, where each hypothesis labels a point positive exactly when it lies at or above a cutoff θ \theta θ :
H t h r = { h θ : R → { ± 1 } ∣ ∃ θ ∈ R s.t. h θ ( x ) = 1 for x ≥ θ and h θ ( x ) = − 1 for x < θ } . (4) \mathcal{H}_{\mathrm{thr}} = \{h_\theta: \mathbb{R} \to\{\pm1\} \mid\exists\theta\in\mathbb{R}\ \text{s.t.}\ h_\theta(x) = 1\ \text{for}\ x \ge\theta\ \text{and}\ h_\theta(x) = -1\ \text{for}\ x < \theta\}. \tag*{(4)} H thr = { h θ : R → { ± 1 } ∣ ∃ θ ∈ R s.t. h θ ( x ) = 1 for x ≥ θ and h θ ( x ) = − 1 for x < θ } . (4)
Our main result for this class is as follows:
Theorem 1.3 (Informal Version of Theorems 3.1 and 3.12 ). Fix m , n ∈ N m, n \in\mathbb{N} m , n ∈ N and ε ∈ ( 0 , 1 ] \varepsilon\in(0, 1] ε ∈ ( 0 , 1 ] . For every domain X ⊂ R \mathcal{X} \subset\mathbb{R} X ⊂ R with ∣ X ∣ = m |\mathcal{X}| = m ∣ X ∣ = m , provided n , t ≳ log log m ε log ( log log m ε ) n, t \gtrsim\frac{\log\log m}{\varepsilon}\log\left(\frac{\log\log m}{\varepsilon}\right) n , t ≳ ε l o g l o g m log ( ε l o g l o g m ) , there exists an interactive, randomized, and polynomial-time algorithm that is an ε \varepsilon ε -DP tester with resolution t t t for H t h r \mathcal{H}_{\mathrm{thr}} H thr . Moreover, for every domain X ⊂ R \mathcal{X} \subset\mathbb{R} X ⊂ R with ∣ X ∣ = m |\mathcal{X}| = m ∣ X ∣ = m , every ε \varepsilon ε -DP tester with resolution t ∈ [ n ] t \in[n] t ∈ [ n ] for H t h r \mathcal{H}_{\mathrm{thr}} H thr requires t ≳ log log m ε t \gtrsim\frac{\log\log m}{\varepsilon} t ≳ ε l o g l o g m .
Next, we mention a few implications of this result:
Minimal overhead compared to the centralized/non-private setting: Recall that a trusted curator achieves resolution O ( 1 / ε ) O(1/\varepsilon) O ( 1/ ε ) for every hypothesis class. In comparison, two-party differential privacy introduces only an O ~ ( log log m ) \widetilde{O}(\log\log m) O ( log log m ) dependence on the domain size m m m , negligible in most practical settings.
Exponential improvement using an interactive protocol: Interaction is provably required to achieve the near optimal Θ ( log log m ε log ( log log m ε ) ) \Theta\left(\frac{\log\log m}{\varepsilon}\log\left(\frac{\log\log m}{\varepsilon}\right)\right) Θ ( ε l o g l o g m log ( ε l o g l o g m ) ) resolution: without interaction, any ε \varepsilon ε -private tester requires t ≳ log m ε t \gtrsim\frac{\log m}{\varepsilon} t ≳ ε l o g m , yielding an exponential separation between interactive and non-interactive private testing (see Remark 3.4).
Exponential separation between private testing and private learning: A natural approach to the testing problem in Equation (3) is to privately learn a good classifier and then privately evaluate its error on each dataset. However, any pure ε \varepsilon ε -DP threshold learner, whether proper or improper, requires Ω ( log m ε ) \Omega\left(\frac{\log m}{\varepsilon}\right) Ω ( ε l o g m ) samples at constant accuracy and confidence, even when a trusted curator holds the pooled data [9 , 22 ] . In contrast, Theorem 1.3 gives a private tester using only O ( log log m ε log ( log log m ε ) ) O\left(\frac{\log\log m}{\varepsilon}\log\left(\frac{\log\log m}{\varepsilon}\right)\right) O ( ε l o g l o g m log ( ε l o g l o g m ) ) samples, yielding an exponential separation in the dependence on m m m . Qualitatively similar separations arise in non-private settings, where testing or estimating optimal prediction error can require fewer samples or label queries than learning a predictor [33 , 2 , 5 , 34 ] .
Halfspaces in Higher Dimension
The next fundamental hypothesis class we consider is halfspaces in R d \mathbb{R}^{d} R d defined as follows:
H h s = { h w : R d → { ± 1 } ∣ ∃ w ∈ R d s.t. h w ( x ) = sign ( ⟨ w , x ⟩ ) } . (5) \mathcal{H}_{\mathrm{hs}} = \{h_{w}: \mathbb{R}^{d} \to\{\pm1\} \mid\exists\, w \in\mathbb{R}^{d}\ \text{s.t. } h_{w}(x) = \operatorname{sign}(\langle w,x\rangle)\}. \tag*{(5)} H hs = { h w : R d → { ± 1 } ∣ ∃ w ∈ R d s.t. h w ( x ) = sign (⟨ w , x ⟩)} . (5)
Our main result for this class is as follows:
Theorem 1.4 (Informal Version of Theorems 4.1, 4.7 and E.4). Fix d ≥ 2 d \ge2 d ≥ 2 , m , n ∈ N m,n \in\mathbb{N} m , n ∈ N , and ε ∈ ( 0 , 1 ] \varepsilon\in(0,1] ε ∈ ( 0 , 1 ] . For every domain X ⊂ R d \mathcal{X} \subset\mathbb{R}^{d} X ⊂ R d with ∣ X ∣ = m |\mathcal{X}| = m ∣ X ∣ = m , provided n , t ≳ d 2 log m ε ( d + log log ( m ) ) n,t \gtrsim\frac{d^{2}\log m}{\varepsilon}(d+\log\log(m)) n , t ≳ ε d 2 l o g m ( d + log log ( m )) , there exists an interactive ε \varepsilon ε -DP private algorithm with resolution t t t for H h s \mathcal{H}_{\mathrm{hs}} H hs .
Moreover, for every sufficiently large d d d and every m ≳ d m \gtrsim d m ≳ d , there exists a public domain X d , m ⊆ R d \mathcal{X}_{d,m} \subseteq\mathbb{R}^{d} X d , m ⊆ R d with ∣ X d , m ∣ = m |\mathcal{X}_{d,m}| = m ∣ X d , m ∣ = m , chosen only from ( d , m ) (d,m) ( d , m ) , such that for every sufficiently small ε > 0 \varepsilon> 0 ε > 0 and every n ∈ N n \in\mathbb{N} n ∈ N with n ε ≳ 1 n\varepsilon\gtrsim1 n ε ≳ 1 , every ε \varepsilon ε -private tester over X d , m \mathcal{X}_{d,m} X d , m with resolution t ∈ [ n ] t \in[n] t ∈ [ n ] requires t ≳ min { d , n ε } ε log ( 2 + min { m d , n ε d 2 } ) t \gtrsim\frac{\min\{d,\sqrt{n\varepsilon}\}}{\varepsilon}\log\left(2+\min\left\{\frac{m}{d},\frac{n\varepsilon}{d^{2}}\right\}\right) t ≳ ε m i n { d , n ε } log ( 2 + min { d m , d 2 n ε } ) .
Next, we mention a few implications of this result:
Sharp jump in domain-size dependence: Comparing Theorem 1.3 and Theorem 1.4 reveals that, as we move from one-dimensional thresholds (d = 1 d=1 d = 1 ) to halfspaces in any fixed dimension d ≥ d 0 d \ge d_{0} d ≥ d 0 for some constant d 0 d_{0} d 0 , the dependence of the testing resolution t t t on the domain size m m m jumps exponentially from log log m \log\log m log log m to log m \log m log m . Our lower bound confirms that this log m \log m log m dependence is tight up to secondary logarithmic factors in the stated parameter regime.
Dimension dependence despite a one-bit output: Our lower bound implies that, even though opt H ( S A , S B ) \operatorname{opt}_{\mathsf{H}}(\mathcal{S}_{A},\mathcal{S}_{B}) opt H ( S A , S B ) is a scalar, two-party pure DP testing incurs a resolution error that scales with d d d , whereas a centralized algorithm incurs only t c e n t r a l = O ( 1 / ε ) t_{\mathrm{central}} = O(1/\varepsilon) t central = O ( 1/ ε ) . This gap reflects the privacy cost of decentralization, as the agents must replace a trusted curator with a protocol that protects each party’s records from the other throughout their interaction.
The privacy cost even with abundant data: Even with arbitrarily large datasets, two-party private testing can require resolution Ω ( d log ( m / d ) / ε ) \Omega(d\log(m/d)/\varepsilon) Ω ( d log ( m / d ) / ε ) on the same fixed public domain. Thus, increasing the number of records does not eliminate this worst-case cost in absolute resolution, although the corresponding fraction of errors decreases.
Complementary bounds for smaller datasets. An interesting aspect of both of our main results is that the achievable resolution t t t does not depend on n n n . For halfspaces, however, our main tester from Theorem 1.4 provides a nontrivial guarantee only when n ≳ d 2 log m ε ( d + log log ( m ) ) n \gtrsim\frac{d^{2}\log m}{\varepsilon}(d+\log\log(m)) n ≳ ε d 2 l o g m ( d + log log ( m )) . For smaller datasets, we complement it with two simple testers whose resolution grows with n n n . The first one is an application of the net mechanism of [7 ] : each party privately releases a small synthetic dataset, which gives a non-interactive tester with resolution t = O ( ( n 2 d log ( 2 m ) / ε ) 1 / 3 ) t=O((n^{2}d\log(2m)/\varepsilon)^{1/3}) t = O (( n 2 d log ( 2 m ) / ε ) 1/3 ) (see Theorem 4.11 ). The second one targets the coarsest nontrivial resolution, i.e., t = n t=n t = n . It turns out that opt H h s ( S A , S B ) = n \operatorname{opt}_{\mathcal{H}_{\mathrm{hs}}}(S_{A},S_{B})=n opt H hs ( S A , S B ) = n happens only when Alice and Bob have the same number of records on every ray through the origin, and this can be tested with shared randomness as long as n ≳ 1 / ε 2 n\gtrsim1/\varepsilon^{2} n ≳ 1/ ε 2 (see Theorem 4.9 ). Putting the three testers together, for every n ≳ 1 / ε 2 n\gtrsim1/\varepsilon^{2} n ≳ 1/ ε 2 we get (ignoring smaller logarithmic factors),
t = O ( min { d 2 log ( 2 m ) ε ( d + log ( log ( 2 m ) ) ) , ( n 2 d log ( 2 m ) ε ) 1 / 3 , n } ) . t=O\left(\min\left\{\frac{d^{2}\log(2m)}{\varepsilon}(d+\log(\log(2m))),\left(\frac{n^{2}d\log(2m)}{\varepsilon}\right)^{1/3},n\right\}\right). t = O ( min { ε d 2 log ( 2 m ) ( d + log ( log ( 2 m ))) , ( ε n 2 d log ( 2 m ) ) 1/3 , n } ) .
The t = n t=n t = n tester is the best of the three when n ≲ d log ( 2 m ) / ε n\lesssim d\log(2m)/\varepsilon n ≲ d log ( 2 m ) / ε , the synthetic-data tester when d log ( 2 m ) / ε ≲ n ≲ d 4 log ( 2 m ) / ε d\log(2m)/\varepsilon\lesssim n\lesssim d^{4}\log(2m)/\varepsilon d log ( 2 m ) / ε ≲ n ≲ d 4 log ( 2 m ) / ε , and the geometric protocol for larger n n n ; see Figure 1 .
Figure 1. Upper and lower bounds on the resolution t t t for private halfspace testing as a function of dataset size n n n , illustrated for m = 2 d m = 2^d m = 2 d (for d > 1 / ε d > 1/\varepsilon d > 1/ ε , hiding constant and log d \log d log d factors). Since opt H h s ( S A , S B ) ≤ n \operatorname{opt}_{\mathcal{H}_{\mathrm{hs}}}(S_A,S_B) \le n opt H hs ( S A , S B ) ≤ n , the region t > n t > n t > n is trivial; the upper bound (blue) starts at n ≍ 1 ε 2 n \asymp\frac{1}{\varepsilon^2} n ≍ ε 2 1 with t = n t=n t = n (Theorem 4.9 ), transitions to the synthetic-data bound at n ≍ d log m ε = d 2 ε n \asymp\frac{d\log m}{\varepsilon} = \frac{d^2}{\varepsilon} n ≍ ε d l o g m = ε d 2 (Theorem 4.11 ), and reaches the n n n -independent geometric bound at n ≍ d 4 log m ε n \asymp\frac{d^4\log m}{\varepsilon} n ≍ ε d 4 l o g m (Theorem 4.7 ). The lower bound (orange) starts at n ≍ 1 ε n \asymp\frac{1}{\varepsilon} n ≍ ε 1 (where t ≍ 1 ε t \asymp\frac{1}{\varepsilon} t ≍ ε 1 ) and grows across two regimes (Theorem E.4 ) before saturating at d log ( m / d ) ε \frac{d\log(m/d)}{\varepsilon} ε d l o g ( m / d ) for n ≳ m d log ( m / d ) ε n \gtrsim\frac{md\log(m/d)}{\varepsilon} n ≳ ε m d l o g ( m / d ) (Theorem 4.1 ).
Technical Overview
In this section, we outline the main ideas behind our lower bounds and private testing algorithms. We first explain in § 1.5.1 the idea behind our lower bound, which is based on the connection between opt H \operatorname{opt}_{\mathcal{H}} opt H and error-correcting codes. We then describe in § 1.5.2 , our threshold algorithm which uses private binary search over prefix lengths to compare two ordered blocks of data. Finally, in § 1.5.3 , we explain the ideas behind our algorithm for halfspaces, which is based on the cutting-plane method in the dual space, i.e., classifier space.
Lower Bounds: Coding Against Erasures
Our lower bounds use the connection between information complexity and DP [37 ] . We consider a communication problem in which Alice and Bob receive secrets v A , v B ∈ V v_{A},v_{B}\in\mathcal{V} v A , v B ∈ V in a finite domain V \mathcal{V} V , and exchange messages to compute a binary function F : V × V → { 0 , 1 } F:\mathcal{V}\times\mathcal{V}\to\{0,1\} F : V × V → { 0 , 1 } . For a protocol computing F F F on inputs ( V A , V B ) ∼ μ (V_A,V_B)\sim\mu ( V A , V B ) ∼ μ , let Π \Pi Π denote its public transcript, including public randomness, exchanged messages, and final output. The protocol’s internal information cost is defined as IC μ ( Π , F ) : = I ( V A ; Π ∣ V B ) + I ( V B ; Π ∣ V A ) \operatorname{IC}_{\mu}(\Pi,F):=I(V_A;\Pi\mid V_B)+I(V_B;\Pi\mid V_A) IC μ ( Π , F ) := I ( V A ; Π ∣ V B ) + I ( V B ; Π ∣ V A ) . Intuitively, it measures what the transcript reveals about each input, given the other.
To force a private tester to incur a large information cost, we choose F F F and μ \mu μ such that every protocol Π \Pi Π computing F F F with bounded error has large IC μ ( Π , F ) \operatorname{IC}_{\mu}(\Pi,F) IC μ ( Π , F ) . To relate the communication problem to our problem, we construct local encoders E A , E B : V → X n E_A,E_B:\mathcal{V}\to\mathcal{X}^n E A , E B : V → X n , producing datasets S A = E A ( v A ) S_A=E_A(v_A) S A = E A ( v A ) and S B = E B ( v B ) S_B=E_B(v_B) S B = E B ( v B ) that satisfy
F ( v A , v B ) = 0 ⟹ opt H ( S A , S B ) = 0 , and F ( v A , v B ) = 1 ⟹ opt H ( S A , S B ) ≥ t . F(v_A,v_B)=0 \quad\Longrightarrow\quad\operatorname{opt}_{\mathcal{H}}(S_A,S_B)=0,\qquad\text{and}\qquad F(v_A,v_B)=1 \quad\Longrightarrow\quad\operatorname{opt}_{\mathcal{H}}(S_A,S_B)\ge t. F ( v A , v B ) = 0 ⟹ opt H ( S A , S B ) = 0 , and F ( v A , v B ) = 1 ⟹ opt H ( S A , S B ) ≥ t .
Running a private tester with resolution t t t on these datasets therefore yields a bounded-error protocol Π \Pi Π for F F F . To capture the sensitivity of the encoders to the secrets, we define the encoding diameters Δ U ( n , t ) : = max v , v ′ ∈ V d h a m ( E U ( v ) , E U ( v ′ ) ) \Delta_U(n,t):=\max_{v,v'\in\mathcal{V}}d_{\mathrm{ham}}(E_U(v),E_U(v')) Δ U ( n , t ) := max v , v ′ ∈ V d ham ( E U ( v ) , E U ( v ′ )) for U ∈ { A , B } U\in\{A,B\} U ∈ { A , B } , and write Δ = Δ ( n , t ) : = Δ A ( n , t ) + Δ B ( n , t ) \Delta=\Delta(n,t):=\Delta_A(n,t)+\Delta_B(n,t) Δ = Δ ( n , t ) := Δ A ( n , t ) + Δ B ( n , t ) . The delicacy is that we need to design an encoding function that relates the encoding diameters to n , t n,t n , t .
Then, we can invoke existing results [37 ] (also see Lemma 2.2) that show that IC μ ( Π , F ) ≤ O ( ε Δ ) \operatorname{IC}_{\mu}(\Pi,F)\le O(\varepsilon\Delta) IC μ ( Π , F ) ≤ O ( ε Δ ) . Comparing this upper bound with the information required to compute F F F forces Δ \Delta Δ to be large. Our encoders upper-bound Δ \Delta Δ in terms of the loss gap t t t , turning this requirement into a lower bound on t t t .
To design encoders for Alice and Bob, we link joint empirical loss to robustness against record erasures. For any hypothesis class H \mathcal{H} H , opt H ( S A , S B ) \operatorname{opt}_{\mathcal{H}}(S_A,S_B) opt H ( S A , S B ) equals the minimum number of records that must be removed from the pooled data to make it realizable. Thus, ensuring opt H ( S A , S B ) ≥ t \operatorname{opt}_{\mathcal{H}}(S_A,S_B)\ge t opt H ( S A , S B ) ≥ t requires the pooled data to remain non-realizable after removing any set of fewer than t t t records. This suggests using error-correcting codes to make non-realizability robust to erasures while keeping the encoding diameter small. Repetition suffices for thresholds, while our halfspace construction uses an expander code.
For thresholds on a domain X ⊂ R \mathcal{X} \subset\mathbb{R} X ⊂ R of size m m m , Alice and Bob hold inputs v A , v B ∈ V = X v_A,v_B \in\mathcal{V} = \mathcal{X} v A , v B ∈ V = X . We use the Greater-Than predicate F ( v A , v B ) = 1 { v A ≥ v B } F(v_A,v_B) = 1\{v_A \ge v_B\} F ( v A , v B ) = 1 { v A ≥ v B } . Alice creates t t t copies of ( v A , − 1 ) (v_A,-1) ( v A , − 1 ) , and Bob creates t t t copies of ( v B , + 1 ) (v_B,+1) ( v B , + 1 ) . The pooled data are realizable when v A < v B v_A < v_B v A < v B , whereas every threshold misclassifies an entire block when v A ≥ v B v_A \ge v_B v A ≥ v B . This gives a loss gap of t t t with encoding diameter Δ = O ( t ) \Delta= O(t) Δ = O ( t ) . Since Greater-Than has information complexity Ω ( log log m ) \Omega(\log\log m) Ω ( log log m ) under a suitable input distribution [10 ] , we use prior results connecting DP to information complexity [37 ] (Lemma 2.2) to compare this lower bound with an O ( ε t ) O(\varepsilon t) O ( εt ) privacy bound, yielding t = Ω ( log log ( m ) / ε ) t = \Omega(\log\log(m)/\varepsilon) t = Ω ( log log ( m ) / ε ) .
For halfspaces , Alice and Bob hold vectors v A , v B ∈ V = F q k v_A,v_B \in\mathcal{V} = \mathbb{F}_q^k v A , v B ∈ V = F q k , where k = Θ ( d ) k = \Theta(d) k = Θ ( d ) and q q q is a power of two. Our encoders map these inputs to labeled data points in a public feature domain X ⊂ R d \mathcal{X} \subset\mathbb{R}^d X ⊂ R d of size m m m . We use the trace inner-product predicate F ( v A , v B ) = Tr F q / F 2 ( ∑ i = 1 k v A , i v B , i ) F(v_A,v_B) = \operatorname{Tr}_{\mathbb{F}_q/\mathbb{F}_2}(\sum_{i=1}^{k}v_{A,i}v_{B,i}) F ( v A , v B ) = Tr F q / F 2 ( ∑ i = 1 k v A , i v B , i ) , whose information complexity is Ω ( k log q ) \Omega(k\log q) Ω ( k log q ) (Lemma B.6 ). To construct the datasets, we view halfspace consistency as a linear feasibility problem, with classifier parameters as variables and one constraint per labeled record. We use this correspondence in reverse, building an LP and converting its constraints into records. For each i ∈ [ k ] i \in[k] i ∈ [ k ] , Alice supplies one input-dependent, or private , constraint fixing the LP representation of v A , i v_{A,i} v A , i , and Bob does the same for v B , i v_{B,i} v B , i . The input-independent public constraints force an output variable z z z to equal F ( v A , v B ) F(v_A,v_B) F ( v A , v B ) once the private constraints fix the inputs. They also require z = 0 z = 0 z = 0 , making the LP feasible iff F ( v A , v B ) = 0 F(v_A,v_B) = 0 F ( v A , v B ) = 0 .
To obtain a loss gap of t t t , we could repeat every constraint t t t times, as for thresholds. However, the resulting diameter Δ = O ( k t ) \Delta= O(kt) Δ = O ( k t ) cancels the dimension factor in the information comparison k log q ≲ ε k t k\log q \lesssim\varepsilon kt k log q ≲ ε k t (see § 4.1.1 ). We instead encode v A v_A v A and v B v_B v B using an expander code with Θ ( k ) \Theta(k) Θ ( k ) symbols and constant-size checks [21 , 18 ] . Private constraints fix the codeword symbols, while public constraints enforce the code checks. Our key step shows that, with public constraints enforced, fewer than D = Θ ( d ) D = \Theta(d) D = Θ ( d ) private deletions cannot change the codewords, even through fractional LP solutions. Indeed, unique-neighbor expansion gives any nonempty set of fewer than D D D deviating symbols a check touching exactly one of them; the unchanged symbols force it back to its original value. Thus, repeating each private constraint ⌈ t / D ⌉ \lceil t/D\rceil ⌈ t / D ⌉ times and each public constraint t t t times prevents fewer than t t t record deletions from restoring realizability when F ( v A , v B ) = 1 F(v_A,v_B) = 1 F ( v A , v B ) = 1 . Only private copies affect the diameter, giving Δ = O ( d ⌈ t / D ⌉ ) = O ( t + d ) \Delta= O(d\lceil t/D\rceil) = O(t+d) Δ = O ( d ⌈ t / D ⌉) = O ( t + d ) . Finally, we represent field elements as polygon vertices in R 2 \mathbb{R}^2 R 2 ; restricting each symbol to this polygon lets one supporting inequality fix its value. Together with sparse checks, this keeps the dimension O ( d ) O(d) O ( d ) and the universe size O ( d O ( 1 ) ) O(d^{O(1)}) O ( d O ( 1 ) ) . Choosing q = ( m / d ) Θ ( 1 ) q = (m/d)^{\Theta(1)} q = ( m / d ) Θ ( 1 ) and applying the information bound yields t = Ω ( d log ( m / d ) / ε ) t = \Omega(d\log(m/d)/\varepsilon) t = Ω ( d log ( m / d ) / ε ) under the parameter conditions of Theorem 4.1 .
Algorithm for Thresholds: A Robust and Private Greater-Than Protocol
We describe the algorithm in the simpler setting where Alice holds only negative records, Bob holds only positive records, and their feature supports are disjoint (Lemma 3.5 handles overlapping supports, and Corollary 3.13 extends the algorithm to mixed labels.) Also, recall that thresholds label points below their respective cutoffs as negative and those at or above as positive (see Equation (4) ). For thresholds with ∣ X ∣ = m |\mathcal{X}| = m ∣ X ∣ = m and t ≥ 2 t \ge2 t ≥ 2 , we reduce the promise problem to comparing two witness blocks of k = ⌊ t / 2 ⌋ k = \lfloor t/2\rfloor k = ⌊ t /2 ⌋ records from the size-n n n datasets (Figure 2 ). We prove a simple structural result that implies that if opt H t h r ( S A , S B ) = 0 \operatorname{opt}_{\mathcal{H}_{\mathrm{thr}}}(S_A,S_B) = 0 opt H thr ( S A , S B ) = 0 , then all of Alice’s points are smaller than all of Bob’s points, and if opt H t h r ( S A , S B ) ≥ t \operatorname{opt}_{\mathcal{H}_{\mathrm{thr}}}(S_A,S_B) \ge t opt H thr ( S A , S B ) ≥ t , then every retained point of Alice exceeds every retained point of Bob. See Figure 2 for intuition. Therefore, k k k points are sufficient to distinguish between Match and Far. Note that if 0 < opt H t h r < t 0 < \operatorname{opt}_{\mathcal{H}_{\mathrm{thr}}} < t 0 < opt H thr < t , both answers can happen, as is standard in distribution testing.
Figure 2. Reduction to witness blocks for thresholds with disjoint supports. Alice retains her largest k = ⌊ t / 2 ⌋ k = \lfloor t/2 \rfloor k = ⌊ t /2 ⌋ records, and Bob retains his smallest k k k records. Solid points are retained and used by our algorithm; faded points are discarded and never be used by our algorithm. In Match, Alice’s retained block lies entirely below Bob’s; in Far, their order is reversed. The illustration uses n = 8 n = 8 n = 8 , t = 6 t = 6 t = 6 , and k = 3 k = 3 k = 3 .
Given this reduction, a natural idea is for Alice and Bob to compare their blocks using a privately chosen point. Alice applies the exponential mechanism over the full public domain X \mathcal{X} X , favoring points near her block’s median, and sends her choice to Bob, who privately counts how many of his records fall below it. Selecting among O ( m ) O(m) O ( m ) candidates incurs score error O ( log ( m ) / ε ) O(\log(m)/\varepsilon) O ( log ( m ) / ε ) . Taking k k k large enough to dominate this error and Bob’s counting noise ensures, with constant failure probability, that Alice’s choice lies within her block’s range and Bob’s comparison identifies the correct order. This yields resolution O ( log ( m ) / ε ) O(\log(m)/\varepsilon) O ( log ( m ) / ε ) , but leaves a gap with respect to our Ω ( log log ( m ) / ε ) \Omega(\log\log(m)/\varepsilon) Ω ( log log ( m ) / ε ) lower bound.
To compare the blocks with a better dependence on m m m , we build on the standard Greater-Than protocol for comparing two numbers [42 Chapter 3] . We adapt its search over prefix lengths to handle blocks of records while preserving privacy. Specifically, we encode each point’s rank in the public domain as a binary string of length b = O ( log m ) b = O(\log m) b = O ( log m ) and organize these strings into a binary trie; see Figure 3. Each node at depth ℓ \ell ℓ contains the strings sharing a particular prefix of length ℓ \ell ℓ . When each party holds a single distinct string, their first differing bit determines their order. Their prefixes agree before that bit and disagree from that bit onward, so binary search locates it using O ( log b ) = O ( log log m ) O(\log b) = O(\log\log m) O ( log b ) = O ( log log m ) prefix-equality tests. At that position, the string with bit 0 precedes the string with bit 1.
For blocks of records, however, the first separation may involve too few points to overcome privacy noise. We therefore search for a depth at which sufficiently many records have left the shared prefix. To measure this, let A p ℓ A_p^\ell A p ℓ and B p ℓ B_p^\ell B p ℓ contain the records in the respective blocks with prefix p ∈ { 0 , 1 } ℓ p \in\{0,1\}^{\ell} p ∈ { 0 , 1 } ℓ , and define M ( ℓ ) : = max p min { ∣ A p ℓ ∣ , ∣ B p ℓ ∣ } M(\ell) := \max_p \min\{|A_p^\ell|, |B_p^\ell|\} M ( ℓ ) := max p min { ∣ A p ℓ ∣ , ∣ B p ℓ ∣ } and F ( ℓ ) : = k − M ( ℓ ) F(\ell) := k - M(\ell) F ( ℓ ) := k − M ( ℓ ) . Since the blocks are strictly ordered, at most one node at each depth contains records from both parties. When this common node exists, F ( ℓ ) F(\ell) F ( ℓ ) is the larger of the two counts of records outside it. As each child contains only records from its parent, M ( ℓ ) M(\ell) M ( ℓ ) cannot increase with depth; consequently, F ( ℓ ) F(\ell) F ( ℓ ) is nondecreasing from F ( 0 ) = 0 F(0) = 0 F ( 0 ) = 0 to F ( b ) = k F(b) = k F ( b ) = k . We can therefore use noisy binary search to find adjacent depths L , H = L + 1 L, H = L + 1 L , H = L + 1 whose values of F F F bracket a target level α \alpha α up to the estimation error, with α \alpha α chosen well above that error.
To estimate F ( ℓ ) F(\ell) F ( ℓ ) without incurring the unwanted log m \log m log m cost, we estimate M ( ℓ ) M(\ell) M ( ℓ ) through a small hashed histogram. A direct noisy histogram over all 2 ℓ 2^\ell 2 ℓ prefixes would incur an error that grows with ℓ \ell ℓ , but at most 2 k 2k 2 k prefixes are occupied. The parties therefore use the same public hash into O ( k 2 / β 0 ) O(k^2/\beta_0) O ( k 2 / β 0 ) buckets, which keeps distinct occupied prefixes in different buckets with probability at least 1 − β 0 / 2 1 - \beta_0/2 1 − β 0 /2 . They then add Laplace noise to their bucket counts and take the maximum over buckets of the smaller noisy count. With privacy budget ε 0 \varepsilon_0 ε 0 per party, this estimates M ( ℓ ) M(\ell) M ( ℓ ) , and hence F ( ℓ ) F(\ell) F ( ℓ ) , to error O ( log ( k / β 0 ) / ε 0 ) O(\log(k/\beta_0)/\varepsilon_0) O ( log ( k / β 0 ) / ε 0 ) with failure probability at most β 0 \beta_0 β 0 . The error thus depends logarithmically on k k k and 1 / β 0 1/\beta_0 1/ β 0 , rather than on the full prefix universe.
These estimates locate a useful depth, but we must still determine which block comes first. If the estimated increase from F ( L ) F(L) F ( L ) to F ( H ) F(H) F ( H ) is large, the common node at depth L L L splits enough records between its two children to reveal the order. Otherwise, F ( H ) F(H) F ( H ) is already of order α \alpha α , while the common node at depth H H H still contains most of each block. The records accumulated outside this node then reveal the order: in Match, Alice’s lie to its left and Bob’s to its right, with these directions reversed in Far. In both cases, each party computes a comparison statistic from weighted maxima of counts. These statistics have constant sensitivity even when their maximizing prefixes change, and their difference distinguishes the two orders with a margin of Ω ( α ) \Omega(\alpha) Ω ( α ) . We can therefore add noise without identifying the common prefix itself.
The search uses only T = ⌈ log 2 b ⌉ = O ( log log m ) T=\lceil\log_2 b\rceil=O(\log\log m) T = ⌈ log 2 b ⌉ = O ( log log m ) adaptive queries. We give each query a privacy budget ε 0 = ε / ( 2 T ) \varepsilon_0=\varepsilon/(2T) ε 0 = ε / ( 2 T ) and a failure budget β 0 = β / ( 2 T ) \beta_0=\beta/(2T) β 0 = β / ( 2 T ) , leaving half of each party’s privacy budget and half of the total failure budget for the final comparison. With these allocations, a margin α = O ( ( T / ε ) log ( k T / β ) ) \alpha=O((T/\varepsilon)\log(kT/\beta)) α = O (( T / ε ) log ( k T / β )) suffices to dominate the noise in both stages. Taking k ≥ 4 α k\ge4\alpha k ≥ 4 α makes the final comparisons reliable; since k = ⌊ t / 2 ⌋ k=\lfloor t/2\rfloor k = ⌊ t /2 ⌋ , this condition holds when t t t is a sufficiently large constant multiple of ( T / ε ) log ( T / ( ε β ) ) (T/\varepsilon)\log(T/(\varepsilon\beta)) ( T / ε ) log ( T / ( εβ )) . Thus, at constant failure probability, we obtain an ε \varepsilon ε -private tester with resolution t = O ~ ( log log ( m ) / ε ) t=\widetilde{O}(\log\log(m)/\varepsilon) t = O ( log log ( m ) / ε ) , independent of the original dataset size n n n , provided n ≥ t n\ge t n ≥ t .
Algorithm for Halfspaces: Cutting Plane in the Dual Space
For the class of halfspaces in R d \mathbb{R}^d R d with ∣ X ∣ = m |\mathcal{X}|=m ∣ X ∣ = m , our main idea is to move to the dual space, where each classifier h w h_w h w is represented by a point w ∈ R d w\in\mathbb{R}^d w ∈ R d . Since only the predictions on the public domain matter, the Sauer–Shelah lemma allows us to restrict attention to a finite public set of candidate classifiers W d i s c \mathcal{W}_{\mathrm{disc}} W disc of size at most m d m^d m d .
Let G A k G_A^k G A k and G B k G_B^k G B k denote the candidates that make at most k = ⌊ ( t − 1 ) / ( 2 ( d + 1 ) ) ⌋ k=\lfloor(t-1)/(2(d+1))\rfloor k = ⌊( t − 1 ) / ( 2 ( d + 1 ))⌋ mistakes on Alice’s and Bob’s data, respectively. Unlike the ordered witness blocks for thresholds, these sets of low-error classifiers can be very unstructured. Our key observation is that their convex hulls distinguish the two cases. In
Match, the sets share a consistent classifier w ⋆ w^\star w ⋆ . In Far, even their convex hulls are disjoint:
conv ( G A k ) ∩ conv ( G B k ) = ∅ . \operatorname{conv}(G_A^k) \cap\operatorname{conv}(G_B^k) = \varnothing. conv ( G A k ) ∩ conv ( G B k ) = ∅ .
To see why, suppose a point belongs to both convex hulls. By Carathéodory’s theorem, it is a convex combination of at most d + 1 d + 1 d + 1 low-error classifiers from each party. A convex combination can misclassify a record only if at least one of its constituent classifiers misclassifies that record. Thus, this point would define a classifier with at most 2 ( d + 1 ) k < t 2(d + 1)k < t 2 ( d + 1 ) k < t mistakes on the pooled data, contradicting Far.
This geometric separation suggests an iterative elimination algorithm, inspired by [6 ] . The parties maintain a public set of active candidates. In Far, a hyperplane separates the convex hulls of their remaining low-error classifiers. At least one side of this hyperplane contains at most half of the active candidates. Thus, at least one party has a halfspace cut that retains all of its low-error classifiers while discarding at least half of the candidates.
The next challenge is to choose such a cut privately. Searching directly over all possible halfspace cuts would be costly in terms of privacy. Instead, we use halfspace containers [6 , 16 ] to obtain a small public menu of 2 O ( d ) 2^{O(d)} 2 O ( d ) subsets of the active candidates. Whenever a halfspace contains at most half of the active candidates, one of these containers covers its candidates while containing at most 3 / 4 3/4 3/4 of the active set. We can therefore search over this small menu while still eliminating a constant fraction of the candidates.
Alice and Bob use the exponential mechanism with a low-sensitivity cost to select a container or abstain. With high probability, at least one party selects a shrinking container in each Far round, while every selected container preserves w ⋆ w^\star w ⋆ in Match. Thus, within O ( d log m ) O(d \log m) O ( d log m ) rounds, the active set becomes empty in Far and remains nonempty in Match.
Considering all the factors, we can show this algorithm is an ε \varepsilon ε -private tester with t = O ~ ( d 3 log ( m ) / ε ) t = \widetilde{O}(d^3 \log(m)/\varepsilon) t = O ( d 3 log ( m ) / ε ) . Intuitively, the resolution O ~ ( d 3 log ( m ) / ε ) \widetilde{O}(d^3 \log(m)/\varepsilon) O ( d 3 log ( m ) / ε ) factors as d log m ε ⋅ d ⋅ d \frac{d \log m}{\varepsilon} \cdot d \cdot d ε d l o g m ⋅ d ⋅ d , where d log m ε \frac{d \log m}{\varepsilon} ε d l o g m arises from splitting the privacy budget across O ( d log m ) O(d \log m) O ( d log m ) elimination rounds, the second factor of d d d is the entropy of the containers that we need to pay due to exponential mechanism, and the third factor of d d d comes from Carathéodory theorem when relating the global mistake gap t t t to the local error threshold k = Θ ( t / d ) k = \Theta(t/d) k = Θ ( t / d ) .
Related Work
Our work lies at the intersection of three lines of research: communication-efficient distributed learning, testing whether a good model exists without learning one, and data valuation.
Distributed Communication-Efficient Learning. A line of work studies communication-efficient distributed learning [3 CBC16, 31 ] . Broadly speaking, in the two-party setting, each agent holds a set of labeled examples, and the players communicate to learn a classifier with small error. In much of this work, the algorithms communicate with each other using samples, and the goal is to minimize the number of samples communicated. However, [3 ] also propose and analyze the more general setting in which communication can be arbitrary, and the goal is to minimize the total number of bits communicated, possibly subject to privacy constraints. Our setting can be viewed as a property-testing version of the distributed communication-efficient learning problem.
More closely related to our halfspace algorithm is the work of [6 ] , who study distributed halfspace learning and convex set disjointness over a known finite domain and measure communication in bits. Their goal is to minimize the number of bits exchanged. Our focus is on the smallest resolution t t t for testing whether opt H = 0 \operatorname{opt}_{\mathcal{H}} = 0 opt H = 0 or opt H ≥ t \operatorname{opt}_{\mathcal{H}} \ge t opt H ≥ t under two-party DP. On the technical side, our halfspace algorithm builds on their container approach, using the improved container bounds of [16 ] . We apply this approach to sets of low-error classifiers in the dual space and select containers privately to eliminate candidates while preserving a shared, consistent classifier when one exists.
Testing and Estimating Learnability. A related line of work asks whether one can test if the optimal population error using classifiers in a hypothesis class is small or large without learning a classifier [2 , 32 , 5 , 34 ] . In the active testing setting, the learner has access to a large pool of unlabeled samples and chooses which points to label. The goal is to answer this question using as few label queries as possible. A related question with access to i.i.d. labeled samples is studied in [34 ] . Under distributional assumptions, they show that the optimal population loss can be estimated using fewer samples than are needed to learn an accurate predictor. For example, in isotropic regression with suitable moment bounds, they show a quadratic gap in the dependence on dimension: O ( d ) O(\sqrt{d}) O ( d ) samples suffice to estimate the optimal squared loss to constant accuracy, whereas learning can require Ω ( d ) \Omega(d) Ω ( d ) samples. They also discuss using such estimates for data valuation and comparing separate and pooled datasets to assess whether joint training is promising.
These works share our goal of assessing prediction error without learning a classifier. They focus on using as few labels or samples as possible to assess population error. Our main constraint is privacy: two agents hold labeled datasets and want to determine whether a single classifier in their hypothesis class fits all the pooled records, or whether every classifier makes at least t t t mistakes. The challenge is to achieve a small resolution t t t while protecting individual records throughout the agents’ interaction. We provide upper and lower bounds on t t t that quantify the cost of two-party DP.
Similar to this line of work, our threshold result also distinguishes between private testing and private learning. For fixed fractional empirical testing gap, learning accuracy, and confidence, our tester needs only O ~ ( log log m / ε ) \widetilde{O}(\log\log m/\varepsilon) O ( log log m / ε ) samples per party, whereas distribution-free pure-private threshold learning requires Ω ( log m / ε ) \Omega(\log m/\varepsilon) Ω ( log m / ε ) samples even with a trusted curator.
Data Valuation and Selection. Data valuation supports decisions throughout the machine learning pipeline, from selecting and pricing training data to debugging trained models [48 , 46 , 30 ] . Popular approaches guide these decisions by assigning scores to individual records or entire datasets using Shapley values or leave-one-out contributions [26 , 27 , 25 ] . These methods must, however, account for non-monotonicity, since adding training data can worsen downstream test performance, including when pooling data from heterogeneous sources [39 , 15 ] . Our opt H ( S A , S B ) > t \operatorname{opt}_{\mathcal{H}}(S_A,S_B) > t opt H ( S A , S B ) > t scenario captures a related obstacle when individually realizable datasets admit no shared classifier that fits both perfectly. For data selection, such non-monotonicity can give adaptive methods a strict advantage over fixed score-based methods under equal value-query budgets [40 ] , paralleling the benefit of interaction between agents in private threshold testing noted in Remark 3.4.
Private Data Appraisal and Valuation. Existing private appraisal methods differ in both what they protect and what their assessments establish. Secure-computation methods, for example, estimate learning gains or Shapley values while permitting leakage through the released appraisal [50 , 46 ] . Differential privacy can address such leakage, but its scope varies across methods. Protocols combining homomorphic encryption with label differential privacy leave features public [1 ] , while private KNN-based valuation assumes a trusted server and excludes validation-data privacy [49 ] . Similar asymmetries arise in dataset valuation that protects sellers without providing corresponding guarantees for the buyer [JMMC24]. Protecting both sides therefore remains a challenge, as broader analyses of private valuation also emphasize [20 ] . Beyond these privacy constraints, the assessment itself may not certify compatibility. Similarity-based methods estimate dataset divergence or correlation without establishing whether a shared classifier fits the combined data [23 , 14 JMMC24] . Likewise, protecting fixed valuation scores does not itself ensure that they identify useful data combinations [40 ] . These limitations leave unresolved the worst-case accuracy limits of compatibility testing under two-party record-level differential privacy without a trusted curator or computational assumptions.
Preliminaries
Notations
For a positive integer r r r , let [ r ] : = { 1 , … , r } [r] := \{1,\ldots,r\} [ r ] := { 1 , … , r } . The public data domain is denoted by X ⊆ R d \mathcal{X} \subseteq\mathbb{R}^{d} X ⊆ R d ; when it is finite, we write m : = ∣ X ∣ m := |\mathcal{X}| m := ∣ X ∣ . For S , S ′ ∈ X n S,S' \in\mathcal{X}^{n} S , S ′ ∈ X n , we write S ∼ S ′ S \sim S' S ∼ S ′ when they are neighbors , meaning that one record is replaced; the corresponding replacement distance is denoted by d h a m ( S , S ′ ) d_{\mathrm{ham}}(S,S') d ham ( S , S ′ ) . For two vectors u , v ∈ R d u,v \in\mathbb{R}^{d} u , v ∈ R d , we write ⟨ u , v ⟩ \langle u,v\rangle ⟨ u , v ⟩ for the Euclidean inner product, conv ( G ) \operatorname{conv}(G) conv ( G ) for the convex hull of G ⊆ R d G \subseteq\mathbb{R}^{d} G ⊆ R d . Finally, Lap ( b ) \operatorname{Lap}(b) Lap ( b ) is the mean-zero Laplace distribution with scale b b b .
Two-Party DP and Connection to Information Complexity
In an interactive protocol, privacy must hold not only for the final Match or Far output, but for the entire interaction transcript observed by each party. For a protocol Π \Pi Π and inputs ( S A , S B ) (S_A,S_B) ( S A , S B ) , let View A Π ( S A , S B ) \operatorname{View}^{\Pi}_{A}(S_A,S_B) View A Π ( S A , S B ) denote Alice’s complete view of the execution, consisting of her input S A S_A S A , her private randomness, the public randomness, all exchanged messages, and the common output; define View B Π ( S A , S B ) \operatorname{View}^{\Pi}_{B}(S_A,S_B) View B Π ( S A , S B ) similarly.
Definition 2.1 (Differential privacy for two-party protocols [MМPRTV10]). A two-party protocol Π \Pi Π is ε \varepsilon ε -differentially private if each party’s view is ε \varepsilon ε -differentially private with respect to the other party’s dataset. More precisely, for every fixed S A S_A S A , every pair of neighboring datasets S B , S B ′ S_B,S'_B S B , S B ′ , and every measurable event E E E ,
Pr [ View A Π ( S A , S B ) ∈ E ] ≤ e ε Pr [ View A Π ( S A , S B ′ ) ∈ E ] , (6) \Pr[\operatorname{View}^{\Pi}_{A}(S_A,S_B)\in E] \le e^{\varepsilon}\Pr[\operatorname{View}^{\Pi}_{A}(S_A,S'_B)\in E], \tag*{(6)} Pr [ View A Π ( S A , S B ) ∈ E ] ≤ e ε Pr [ View A Π ( S A , S B ′ ) ∈ E ] , (6)
and, symmetrically, for every fixed S B S_B S B , every pair of neighboring datasets S A , S A ′ S_A, S_A' S A , S A ′ , and every measurable event E E E ,
Pr [ View B Π ( S A , S B ) ∈ E ] ≤ e ε Pr [ View B Π ( S A ′ , S B ) ∈ E ] . \Pr[\operatorname{View}^{\Pi}_B(S_A,S_B) \in E] \le e^{\varepsilon}\Pr[\operatorname{View}^{\Pi}_B(S_A',S_B) \in E]. Pr [ View B Π ( S A , S B ) ∈ E ] ≤ e ε Pr [ View B Π ( S A ′ , S B ) ∈ E ] .
We require differential privacy on the full input domain, including inputs that do not satisfy the promise of the testing problem.
We also make a repeated use of this lemma which is a simple modification of the connection between DP and internal information complexity from [37 ] .
Lemma 2.2. Let A \mathcal{A} A and B \mathcal{B} B be finite sets, and let E A : A → X n E_A:\mathcal{A}\to\mathcal{X}^n E A : A → X n and E B : B → X n E_B:\mathcal{B}\to\mathcal{X}^n E B : B → X n be deterministic maps that encode inputs in A \mathcal{A} A and B \mathcal{B} B as size-n n n datasets over X \mathcal{X} X . Define the encoding diameter of E A E_A E A and E B E_B E B as follows:
Δ A : = max a , a ′ ∈ A d h a m ( E A ( a ) , E A ( a ′ ) ) , Δ B : = max b , b ′ ∈ B d h a m ( E B ( b ) , E B ( b ′ ) ) . \Delta_A := \max_{a,a'\in\mathcal{A}} d_{\mathrm{ham}}(E_A(a),E_A(a')), \qquad\Delta_B := \max_{b,b'\in\mathcal{B}} d_{\mathrm{ham}}(E_B(b),E_B(b')). Δ A := a , a ′ ∈ A max d ham ( E A ( a ) , E A ( a ′ )) , Δ B := b , b ′ ∈ B max d ham ( E B ( b ) , E B ( b ′ )) . Suppose ( A , B ) ∼ μ (A,B)\sim\mu ( A , B ) ∼ μ for a distribution μ \mu μ over A × B \mathcal{A}\times\mathcal{B} A × B , and Alice and Bob execute an ε \varepsilon ε -differentially private protocol Π \Pi Π on the encoded datasets ( E A ( A ) , E B ( B ) ) (E_A(A),E_B(B)) ( E A ( A ) , E B ( B )) . Let T T T denote the resulting public transcript (consisting of the public randomness, all exchanged messages, and the final output). Then the internal information cost of T T T (in bits) satisfies
I ( A ; T ∣ B ) + I ( B ; T ∣ A ) ≤ ε ( Δ A + Δ B ) ln 2 . I(A;T\mid B)+I(B;T\mid A)\le\frac{\varepsilon(\Delta_A+\Delta_B)}{\ln2}. I ( A ; T ∣ B ) + I ( B ; T ∣ A ) ≤ ln 2 ε ( Δ A + Δ B ) .
Threshold in One Dimension
In this section, we focus on the class of thresholds on real line. In particular, define
H t h r = { h θ : R → { ± 1 } ∣ ∃ θ ∈ R s.t. h θ ( x ) = 1 for x ≥ θ and h θ ( x ) = − 1 for x < θ } . (7) \mathcal{H}_{\mathrm{thr}}=\{h_\theta:\mathbb{R}\to\{\pm1\}\mid\exists\theta\in\mathbb{R}\text{ s.t. }h_\theta(x)=1\text{ for }x\ge\theta\text{ and }h_\theta(x)=-1\text{ for }x<\theta\}. \tag*{(7)} H thr = { h θ : R → { ± 1 } ∣ ∃ θ ∈ R s.t. h θ ( x ) = 1 for x ≥ θ and h θ ( x ) = − 1 for x < θ } . (7)
First notice that opt H t h r ( S A , S B ) ≤ n \operatorname{opt}_{\mathcal{H}_{\mathrm{thr}}}(S_A,S_B)\le n opt H thr ( S A , S B ) ≤ n , therefore, the non-vacuous regime for the resolution will be 1 ≤ t ≤ n 1\le t\le n 1 ≤ t ≤ n . We first provide an information-theoretic lower bound on t t t for this class, and then present an algorithm that (nearly) matches the lower bound.
Lower bound for thresholds
The next result presents a fundamental lower bound on t t t for the class of thresholds. The main idea is to embed a binary function with a high information complexity and then use the connection between information complexity and two-view DP from [37 ] to provide a lower bound.
Theorem 3.1 (Lower bound for thresholds). There exist universal constants c > 0 c>0 c > 0 and m 0 ∈ N m_0\in\mathbb{N} m 0 ∈ N such that the following holds. For every m ≥ m 0 m\ge m_0 m ≥ m 0 , every X ⊆ R \mathcal{X}\subseteq\mathbb{R} X ⊆ R with ∣ X ∣ = m |\mathcal{X}|=m ∣ X ∣ = m , every ε > 0 \varepsilon>0 ε > 0 , and every n , t ∈ N n,t\in\mathbb{N} n , t ∈ N such that t ≤ n t\le n t ≤ n , we have that every ε \varepsilon ε -private tester for H t h r \mathcal{H}_{\mathrm{thr}} H thr with resolution t t t for dataset of size n n n (from Definition 1.2 ) requires
t ≥ c log log m ε . t\ge c\frac{\log\log m}{\varepsilon}. t ≥ c ε log log m .
Remark 3.2. Fix ε ∈ ( 0 , 1 ] \varepsilon\in(0,1] ε ∈ ( 0 , 1 ] and n ∈ N n\in\mathbb{N} n ∈ N . Theorem 3.1 implies that no ε \varepsilon ε -private tester for thresholds over R \mathbb{R} R at dataset size n n n can achieve a resolution 1 ≤ t ≤ n 1\le t\le n 1 ≤ t ≤ n . Indeed, restricting such a tester to a finite domain of size m m m would give t ≥ c log log ( m ) / ε t\ge c\log\log(m)/\varepsilon t ≥ c log log ( m ) / ε for arbitrarily large m m m , contradicting the fact that t t t is fixed.
A trusted curator can achieve resolution t = O ( 1 / ε ) t = O(1/\varepsilon) t = O ( 1/ ε ) , independently of the domain size, by adding Laplace noise to the sensitivity-one quantity opt. This resolution is nontrivial provided n ≥ C / ε n \ge C/\varepsilon n ≥ C / ε for a sufficiently large universal constant C C C . Thus, on finite domains of size m m m , the two-party setting incurs an unavoidable multiplicative Ω ( log log m ) \Omega(\log\log m) Ω ( log log m ) overhead relative to the centralized resolution.
Remark 3.3. Under cryptographic assumptions, secure function evaluation can emulate the trusted curator and achieve t = O ( 1 / ε ) t = O(1/\varepsilon) t = O ( 1/ ε ) with computational differential privacy [37 ] . However, Theorem 3.1 shows that information-theoretic view-based privacy requires t = Ω ( log log ( m ) / ε ) t = \Omega(\log\log(m)/\varepsilon) t = Ω ( log log ( m ) / ε ) . Thus, the two models exhibit a separation in the achievable promise gap.
Remark 3.4. The lower bound in Theorem 3.1 allows arbitrary interaction between Alice and Bob. For one-way protocols with bounded worst-case error, however, the standard information-theoretic argument underlying the Greater-Than lower bound [42 Chapter 6] shows that Alice’s message must reveal Ω ( log m ) \Omega(\log m) Ω ( log m ) bits about her input when it is uniformly random. Indeed, given any prefix of Alice’s input, Bob can choose a comparison that predicts her next bit with constant error; Fano’s inequality and the chain rule then give the information lower bound. Applying this argument to the encoding in Theorem 3.1 , group privacy bounds the information revealed by Alice’s message by O ( ε t ) O(\varepsilon t) O ( εt ) , yielding t = Ω ( log ( m ) / ε ) t = \Omega(\log(m)/\varepsilon) t = Ω ( log ( m ) / ε ) . The private-interior-point baseline in § 3.2.3 achieves the matching O ( log ( m ) / ε ) O(\log(m)/\varepsilon) O ( log ( m ) / ε ) resolution. Thus, interaction is necessary to achieve our improved resolution of O ~ ( log log ( m ) / ε ) \widetilde{O}(\log\log(m)/\varepsilon) O ( log log ( m ) / ε ) .
Proof of Theorem 3.1 . Without loss of generality write X = { x 1 , … , x m } \mathcal{X} = \{x_1, \ldots, x_m\} X = { x 1 , … , x m } where x 1 ≤ ⋯ ≤ x m x_1 \le\cdots\le x_m x 1 ≤ ⋯ ≤ x m . Fix an arbitrary integer n ≥ t n \ge t n ≥ t and let ℓ = ⌊ log 2 ( m − 2 ) ⌋ \ell= \lfloor\log_2(m - 2) \rfloor ℓ = ⌊ log 2 ( m − 2 )⌋ . For i , j ∈ { 2 , … , 2 ℓ + 1 } i,j \in\{2,\ldots,2^\ell+1\} i , j ∈ { 2 , … , 2 ℓ + 1 } , define the size-n n n multisets
S A ( i ) = ( x 1 , … , x 1 , x i , … , x i ) , S B ( j ) = ( x j , … , x j , x m , … , x m ) , S_A^{(i)} = (x_1,\ldots,x_1,x_i,\ldots,x_i), \qquad S_B^{(j)} = (x_j,\ldots,x_j,x_m,\ldots,x_m), S A ( i ) = ( x 1 , … , x 1 , x i , … , x i ) , S B ( j ) = ( x j , … , x j , x m , … , x m ) ,
where the first n − t n-t n − t entries of S A ( i ) S_A^{(i)} S A ( i ) are x 1 x_1 x 1 , its last t t t entries are x i x_i x i , the first t t t entries of S B ( j ) S_B^{(j)} S B ( j ) are x j x_j x j , and its last n − t n-t n − t entries are x m x_m x m .
where Alice’s records are negatively labeled and Bob’s records are positively labeled. Then,
opt H t h r ( S A ( i ) , S B ( j ) ) = { 0 , i < j , t , i ≥ j . (8) \operatorname{opt}_{\mathcal{H}_{\mathrm{thr}}}\left(S_A^{(i)},S_B^{(j)}\right) =
\begin{cases}
0, & i < j,\\
t, & i \ge j.
\end{cases} \tag*{(8)} opt H thr ( S A ( i ) , S B ( j ) ) = { 0 , t , i < j , i ≥ j . (8)
There exists a distribution μ ℓ \mu_\ell μ ℓ over { 2 , … , 2 ℓ + 1 } 2 \{2,\ldots,2^\ell+1\}^2 { 2 , … , 2 ℓ + 1 } 2 such that every protocol computing G T ℓ ( i , j ) = 1 { i ≥ j } \mathrm{GT}_\ell(i,j) = \mathbf{1}\{i \ge j\} GT ℓ ( i , j ) = 1 { i ≥ j } with error at most 1 / 10 1/10 1/10 has internal information cost Ω ( log ℓ ) \Omega(\log\ell) Ω ( log ℓ ) [10 , 47 ] . More precisely, let Π \Pi Π be a private tester from Definition 1.2 . Then, given ( I , J ) ∼ μ ℓ (I,J) \sim\mu_\ell ( I , J ) ∼ μ ℓ , run Π \Pi Π on ( S A ( I ) , S B ( J ) ) (S_A^{(I)},S_B^{(J)}) ( S A ( I ) , S B ( J ) ) . By Equation (8) , the resulting protocol computes G T ℓ \mathrm{GT}_\ell GT ℓ with error at most 1 / 10 1/10 1/10 . Therefore, if T T T consists of the shared randomness, exchanged messages, and common output, then I ( I ; T ∣ J ) + I ( J ; T ∣ I ) = Ω ( log ℓ ) I(I;T \mid J) + I(J;T \mid I) = \Omega(\log\ell) I ( I ; T ∣ J ) + I ( J ; T ∣ I ) = Ω ( log ℓ ) by [10 , 47 ] . The padding is independent of i i i and j j j , and hence both encoding diameters are at most 2 t 2t 2 t . By Lemma 2.2 ,
I ( I ; T ∣ J ) + I ( J ; T ∣ I ) ≤ 4 ε t ln 2 . I(I;T \mid J) + I(J;T \mid I) \le\frac{4\varepsilon t}{\ln2}. I ( I ; T ∣ J ) + I ( J ; T ∣ I ) ≤ ln 2 4 εt .
Then, it follows that t = Ω ( log ℓ ε ) = Ω ( log log m ε ) t = \Omega\left(\frac{\log\ell}{\varepsilon}\right) = \Omega\left(\frac{\log\log m}{\varepsilon}\right) t = Ω ( ε l o g ℓ ) = Ω ( ε l o g l o g m ) , as claimed.
Algorithm for Thresholds
In this part, we present our algorithm that achieves the lower bound up to log log log ( m ) \log\log\log(m) log log log ( m ) factor.
Structural Properties and Binary Trie
First, we introduce a notation: For a multiset S S S of size n n n over a totally ordered domain, write its elements, with multiplicity, as s ( 1 ) ≤ ⋯ ≤ s ( n ) s_{(1)} \le\cdots\le s_{(n)} s ( 1 ) ≤ ⋯ ≤ s ( n ) . For k ∈ [ n ] k \in[n] k ∈ [ n ] , define the multisets of the k k k smallest and k k k largest elements by S k ↓ : = { s ( 1 ) , … , s ( k ) } S_k^{\downarrow} := \{s_{(1)},\ldots,s_{(k)}\} S k ↓ := { s ( 1 ) , … , s ( k ) } , and S k ↑ : = { s ( n − k + 1 ) , … , s ( n ) } S_k^{\uparrow} := \{s_{(n-k+1)},\ldots,s_{(n)}\} S k ↑ := { s ( n − k + 1 ) , … , s ( n ) } . Now, we are ready to present a structural property.
Lemma 3.5. Let X = { 0 , 1 } b \mathcal{X} = \{0,1\}^{b} X = { 0 , 1 } b where m = 2 b m = 2^{b} m = 2 b . Define ψ A ( x ) = x ∥ 1 \psi_A(x) = x\mathbin{\|}1 ψ A ( x ) = x ∥ 1 and ψ B ( x ) = x ∥ 0 \psi_B(x) = x\mathbin{\|}0 ψ B ( x ) = x ∥ 0 . In particular, ψ A \psi_A ψ A and ψ B \psi_B ψ B add a bit 1 and 0 after the least significant bit of the input, respectively. Let S A , S B S_A,S_B S A , S B be multisets of size n n n over X \mathcal{X} X , let k ∈ [ n ] k \in[n] k ∈ [ n ] , and write S ~ A = ψ A ( S A ) \widetilde{S}_A = \psi_A(S_A) S A = ψ A ( S A ) and S ~ B = ψ B ( S B ) \widetilde{S}_B = \psi_B(S_B) S B = ψ B ( S B ) . Then opt H t h r ( S ~ A , S ~ B ) = opt H t h r ( S A , S B ) \operatorname{opt}_{\mathcal{H}_{\mathrm{thr}}}(\widetilde{S}_A,\widetilde{S}_B) = \operatorname{opt}_{\mathcal{H}_{\mathrm{thr}}}(S_A,S_B) opt H thr ( S A , S B ) = opt H thr ( S A , S B ) . Moreover:
if opt H t h r ( S A , S B ) = 0 \operatorname{opt}_{\mathcal{H}_{\mathrm{thr}}}(S_A,S_B) = 0 opt H thr ( S A , S B ) = 0 , then max S ~ A , k ↑ < min S ~ B , k ↓ \max\widetilde{S}_{A,k}^{\uparrow} < \min\widetilde{S}_{B,k}^{\downarrow} max S A , k ↑ < min S B , k ↓ ;
if opt H t h r ( S A , S B ) ≥ 2 k − 1 \operatorname{opt}_{\mathcal{H}_{\mathrm{thr}}}(S_A,S_B) \ge2k - 1 opt H thr ( S A , S B ) ≥ 2 k − 1 , then min S ~ A , k ↑ > max S ~ B , k ↓ \min\widetilde{S}_{A,k}^{\uparrow} > \max\widetilde{S}_{B,k}^{\downarrow} min S A , k ↑ > max S B , k ↓ .
Proof. Reading strings as integers, ψ A ( x ) = 2 x + 1 \psi_A(x) = 2x + 1 ψ A ( x ) = 2 x + 1 and ψ B ( x ) = 2 x \psi_B(x) = 2x ψ B ( x ) = 2 x . Thus both maps are strictly increasing, and their images are disjoint by parity.
At an even threshold θ ~ = 2 s \widetilde{\theta} = 2s θ = 2 s , the Alice and Bob errors are, respectively,
{ x ∈ S A : x ≥ s } , { x ∈ S B : x < s } . \{x \in S_A : x \ge s\}, \qquad\{x \in S_B : x < s\}. { x ∈ S A : x ≥ s } , { x ∈ S B : x < s } .
which are exactly the errors at threshold s s s in the original instance. At an odd threshold θ ~ = 2 s + 1 \widetilde{\theta} = 2s + 1 θ = 2 s + 1 , the error multisets are
{ x ∈ S A : x ≥ s } , { x ∈ S B : x ≤ s } . \{x \in S_A : x \ge s\}, \qquad\{x \in S_B : x \le s\}. { x ∈ S A : x ≥ s } , { x ∈ S B : x ≤ s } .
Thus, compared with 2 s 2s 2 s , the odd threshold 2 s + 1 2s + 1 2 s + 1 makes exactly the same errors except that it may additionally misclassify Bob’s records at s s s . It can therefore never be better than 2 s 2s 2 s . It follows that the two optimal errors are equal.
If the original instance is realizable, then max S A < min S B \max S_A < \min S_B max S A < min S B . So every element of S ~ A \widetilde{S}_A S A lies below every element of S ~ B \widetilde{S}_B S B , proving part (i). For part (ii), suppose instead that
a : = min S ~ A , k ↑ ≤ b : = max S ~ B , k ↓ . a := \min\widetilde{S}_{A,k}^{\uparrow} \le b := \max\widetilde{S}_{B,k}^{\downarrow}. a := min S A , k ↑ ≤ b := max S B , k ↓ .
Since a a a is odd and b b b is even, we have a < b a < b a < b . At threshold b b b , every Alice error is strictly larger than a a a , so there are at most k − 1 k - 1 k − 1 Alice errors. Likewise, every Bob error is strictly smaller than b b b , so there are at most k − 1 k - 1 k − 1 Bob errors. Therefore,
opt H t h r ( S ~ A , S ~ B ) ≤ 2 k − 2 , \operatorname{opt}_{\mathcal{H}_{\mathrm{thr}}}(\widetilde{S}_A,\widetilde{S}_B) \le2k - 2, opt H thr ( S A , S B ) ≤ 2 k − 2 ,
contradicting the assumption. This proves part (ii).
The lemma reduces the promise problem to a comparison between two extremal blocks. Fix k = ⌊ t / 2 ⌋ k = \lfloor t/2 \rfloor k = ⌊ t /2 ⌋ . In the Match case, every point in S ~ A , k ↑ \widetilde{S}_{A,k}^{\uparrow} S A , k ↑ lies below every point in S ~ B , k ↓ \widetilde{S}_{B,k}^{\downarrow} S B , k ↓ . In the Far case, their order is reversed: every point in S ~ A , k ↑ \widetilde{S}_{A,k}^{\uparrow} S A , k ↑ lies above every point in S ~ B , k ↓ \widetilde{S}_{B,k}^{\downarrow} S B , k ↓ . In particular, being far from realizable creates not merely one misordered pair, but two blocks of k k k records with opposite order. This introduces a stability that we later exploit: the promised cases can be distinguished using robust order statistics rather than the maximum of S A S_A S A and the minimum of S B S_B S B , which may each depend on a single record.
Remark 3.6. To avoid ties between records owned by different parties, we apply the lifting function ψ A \psi_A ψ A and ψ B \psi_B ψ B from Lemma 3.5 and suppress the tildes. Since this transformation is local, it entails no loss of generality. Henceforth, we assume that the supports of S A , k ↑ S_{A,k}^{\uparrow} S A , k ↑ and S B , k ↓ S_{B,k}^{\downarrow} S B , k ↓ are disjoint, so the preceding order comparisons are strict. Without loss of generality to reduce notation clutter, we assume Alice and Bob only use S A , k ↑ S_{A,k}^{\uparrow} S A , k ↑ and S B , k ↓ S_{B,k}^{\downarrow} S B , k ↓ during the execution of the algorithm. It is easy to see that these subsets also respect the neighboring condition we require for the definition of DP.
Definition 3.7 (Binary Trie and Common Node). Let X = { 0 , 1 } b \mathcal{X} = \{0,1\}^{b} X = { 0 , 1 } b with 2 b = m 2^{b} = m 2 b = m . Let T \mathcal{T} T be the complete binary trie whose nodes are the binary strings v ∈ { 0 , 1 } ≤ b = ⋃ r = 0 b { 0 , 1 } r v \in\{0,1\}^{\le b} = \bigcup_{r=0}^{b}\{0,1\}^{r} v ∈ { 0 , 1 } ≤ b = ⋃ r = 0 b { 0 , 1 } r . The depth of a node is its length ∣ v ∣ |v| ∣ v ∣ : the root is the empty string at depth 0, and the leaves are the depth-b b b strings, which we identify with X \mathcal{X} X . For each ℓ ∈ [ b ] \ell\in[b] ℓ ∈ [ b ] and p ∈ { 0 , 1 } ℓ p \in\{0,1\}^{\ell} p ∈ { 0 , 1 } ℓ define A p ℓ = { x i ∈ S A : prefix ℓ ( x i ) = p } A_{p}^{\ell} = \{x_i \in S_A : \operatorname{prefix}_{\ell}(x_i) = p\} A p ℓ = { x i ∈ S A : prefix ℓ ( x i ) = p } and similarly define B p ℓ = { x i ∈ S B : prefix ℓ ( x i ) = p } B_{p}^{\ell} = \{x_i \in S_B : \operatorname{prefix}_{\ell}(x_i) = p\} B p ℓ = { x i ∈ S B : prefix ℓ ( x i ) = p } . Also, we have the convention that A p 0 = S A A_{p}^{0} = S_A A p 0 = S A and B p 0 = S B B_{p}^{0} = S_B B p 0 = S B . At depth ℓ \ell ℓ a node is called common if it contains both Alice and Bob points.
Next lemma shows an important property of the binary trie node, that we use in the sequel to design an algorithm.
Lemma 3.8. Fix X = { 0 , 1 } b \mathcal{X} = \{0,1\}^{b} X = { 0 , 1 } b and n ∈ N n \in\mathbb{N} n ∈ N . Let S A , S B ⊆ X n S_A, S_B \subseteq\mathcal{X}^{n} S A , S B ⊆ X n be two sequence of length n n n and assume the promise holds. Recall the definition of the trie node from Definition 3.7 . Then, for every given depth ℓ ∈ [ b ] \ell\in[b] ℓ ∈ [ b ] , there exists at most one node that contains both Alice and Bob’s data, i.e., ∣ { p ∈ { 0 , 1 } ℓ : B p ℓ ⋅ A p ℓ ≠ 0 } ∣ ≤ 1 |\{p \in\{0,1\}^{\ell} : B_{p}^{\ell} \cdot A_{p}^{\ell} \ne0\}| \le1 ∣ { p ∈ { 0 , 1 } ℓ : B p ℓ ⋅ A p ℓ = 0 } ∣ ≤ 1 . Also, at depth b b b , we have ∣ { p ∈ { 0 , 1 } b : B p ℓ ⋅ A p ℓ ≠ 0 } ∣ = 0 |\{p \in\{0,1\}^{b} : B_{p}^{\ell} \cdot A_{p}^{\ell} \ne0\}| = 0 ∣ { p ∈ { 0 , 1 } b : B p ℓ ⋅ A p ℓ = 0 } ∣ = 0 .
Proof. Under the promise from Lemma 3.5 and Remark 3.6, either max S A < min S B \max S_A < \min S_B max S A < min S B or max S B , k < min S A \max S_{B,k} < \min S_A max S B , k < min S A . First, we prove the last claim: by Lemma 3.5 (recall also Remark 3.6), we have that under the promise, the block of Alice and Bob’s data are disjoint. For the first claim, we prove by contradiction. Lets assume at depth ℓ \ell ℓ , there exists two nodes that are both common and denoted by p 1 ∈ { 0 , 1 } ℓ p_1 \in\{0,1\}^{\ell} p 1 ∈ { 0 , 1 } ℓ and p 2 ∈ { 0 , 1 } ℓ p_2 \in\{0,1\}^{\ell} p 2 ∈ { 0 , 1 } ℓ such that A p 1 ℓ ⋅ B p 1 ℓ ≠ 0 A_{p_1}^{\ell} \cdot B_{p_1}^{\ell} \ne0 A p 1 ℓ ⋅ B p 1 ℓ = 0 and A p 2 ℓ ⋅ B p 2 ℓ ≠ 0 A_{p_2}^{\ell} \cdot B_{p_2}^{\ell} \ne0 A p 2 ℓ ⋅ B p 2 ℓ = 0 . Assume without loss of generality that p 1 < p 2 p_1 < p_2 p 1 < p 2 in lexicographic order. Every leaf below p 1 p_1 p 1 is smaller than every leaf below p 2 p_2 p 2 . Choose a p 1 ∈ A p 1 ℓ a_{p_1} \in A_{p_1}^{\ell} a p 1 ∈ A p 1 ℓ , b p 1 ∈ B p 1 ℓ b_{p_1} \in B_{p_1}^{\ell} b p 1 ∈ B p 1 ℓ , a p 2 ∈ A p 2 ℓ a_{p_2} \in A_{p_2}^{\ell} a p 2 ∈ A p 2 ℓ , b p 2 ∈ B p 2 ℓ b_{p_2} \in B_{p_2}^{\ell} b p 2 ∈ B p 2 ℓ . It follows that a p 1 < b p 2 a_{p_1} < b_{p_2} a p 1 < b p 2 and b p 1 < a p 2 b_{p_1} < a_{p_2} b p 1 < a p 2 . If max S A < min S B \max S_A < \min S_B max S A < min S B , then a q < b p a_q < b_p a q < b p , contradicting b p < a q b_p < a_q b p < a q . If min S A > max S B \min S_A > \max S_B min S A > max S B , then a p > b q a_p > b_q a p > b q , contradicting a p < b q a_p < b_q a p < b q . □
Example 3.9. To gain some intuition, behind how the binary trie structure, consider Figure 3 where it shows X = { 0 , 1 } 3 \mathcal{X} = \{0,1\}^{3} X = { 0 , 1 } 3 . Here, opt ( S A , S B ) = 0 \operatorname{opt}(S_A,S_B) = 0 opt ( S A , S B ) = 0 . Note that at depth zero, the single node is always common and the last depth doesn’t have any common nodes. ◃
Lemma 3.10 (escaped points). Fix X = { 0 , 1 } b \mathcal{X} = \{0,1\}^{b} X = { 0 , 1 } b and n ∈ N n \in\mathbb{N} n ∈ N . Let S A , S B ⊆ X n S_A, S_B \subseteq\mathcal{X}^{n} S A , S B ⊆ X n be two sequence of length n n n and assume the promise holds. Define the function F : [ b ] → N F : [b] \to\mathbb{N} F : [ b ] → N as F ( ℓ ) = n − max p ∈ { 0 , 1 } ℓ min { A p ℓ , B p ℓ } F(\ell) = n - \max_{p \in\{0,1\}^{\ell}} \min\{A_{p}^{\ell}, B_{p}^{\ell}\} F ( ℓ ) = n − max p ∈ { 0 , 1 } ℓ min { A p ℓ , B p ℓ } . Then F ℓ F_{\ell} F ℓ is a non-decreasing function with F ( 0 ) = 0 F(0) = 0 F ( 0 ) = 0 and F ( b ) = n F(b) = n F ( b ) = n .
Proof. Set M ( ℓ ) : = max p ∈ { 0 , 1 } ℓ min { ∣ A p ℓ ∣ , ∣ B p ℓ ∣ } M(\ell) := \max_{p \in\{0,1\}^{\ell}} \min\{|A_{p}^{\ell}|, |B_{p}^{\ell}|\} M ( ℓ ) := max p ∈ { 0 , 1 } ℓ min { ∣ A p ℓ ∣ , ∣ B p ℓ ∣ } , so that F ( ℓ ) = n − M ( ℓ ) F(\ell) = n - M(\ell) F ( ℓ ) = n − M ( ℓ ) . If q q q is a node at depth ℓ + 1 \ell+ 1 ℓ + 1 and p p p is its parent, then A q ℓ + 1 ⊆ A p ℓ A_{q}^{\ell+1} \subseteq A_{p}^{\ell} A q ℓ + 1 ⊆ A p ℓ and B q ℓ + 1 ⊆ B p ℓ B_{q}^{\ell+1} \subseteq B_{p}^{\ell} B q ℓ + 1 ⊆ B p ℓ . Hence
min { ∣ A q ℓ + 1 ∣ , ∣ B q ℓ + 1 ∣ } ≤ min { ∣ A p ℓ ∣ , ∣ B p ℓ ∣ } ≤ M ( ℓ ) . \min\{|A_{q}^{\ell+1}|, |B_{q}^{\ell+1}|\} \le\min\{|A_{p}^{\ell}|, |B_{p}^{\ell}|\} \le M(\ell). min { ∣ A q ℓ + 1 ∣ , ∣ B q ℓ + 1 ∣ } ≤ min { ∣ A p ℓ ∣ , ∣ B p ℓ ∣ } ≤ M ( ℓ ) .
Maximizing over q q q gives M ( ℓ + 1 ) ≤ M ( ℓ ) M(\ell+ 1) \le M(\ell) M ( ℓ + 1 ) ≤ M ( ℓ ) , and therefore F ( ℓ + 1 ) ≥ F ( ℓ ) F(\ell+ 1) \ge F(\ell) F ( ℓ + 1 ) ≥ F ( ℓ ) . At the root, ∣ A 0 0 ∣ = ∣ B 0 0 ∣ = n |A_{0}^{0}| = |B_{0}^{0}| = n ∣ A 0 0 ∣ = ∣ B 0 0 ∣ = n , so F ( 0 ) = 0 F(0) = 0 F ( 0 ) = 0 . At depth b b b , no leaf is common by Lemma 3.8 ; hence M ( b ) = 0 M(b) = 0 M ( b ) = 0 and F ( b ) = n F(b) = n F ( b ) = n . □
Distributed Private Common-Frequency Estimation
In this section, we describe an algorithm for a problem that we refer to as distributed common heavy hitter . This subroutine later used in our algorithm for the thresholds.
The task is simple: Alice and Bob hold a categorical dataset of size n n n over a finite universe of size m m m . Their goal is to estimate the maximum number of points in a shared bucket in the distributed two-party DP case. We propose an algorithm for this task in Algorithm 1.
Algorithm 1 Private Common-Frequency Estimation: A P C F \mathcal{A}_{\mathrm{PCF}} A PCF
Input: Alice holds S A = ( X 1 , … , X n ) ∈ [ m ] n S_A=(X_1,\ldots,X_n)\in[m]^n S A = ( X 1 , … , X n ) ∈ [ m ] n , Bob holds S A = ( Y 1 , … , Y n ) ∈ [ m ] n S_A=(Y_1,\ldots,Y_n)\in[m]^n S A = ( Y 1 , … , Y n ) ∈ [ m ] n ; parameters ε , β \varepsilon,\beta ε , β .
Output: estimate f ^ \hat{f} f ^ of f = max k ∈ [ m ] min { A k , B k } f=\max_{k\in[m]}\min\{A_k,B_k\} f = max k ∈ [ m ] min { A k , B k } .
R ← ⌈ 4 n 2 / β ⌉ R\leftarrow\lceil4n^2/\beta\rceil R ← ⌈ 4 n 2 / β ⌉ .
if m ≤ R m\le R m ≤ R then
g ← id [ m ] g\leftarrow\operatorname{id}_{[m]} g ← id [ m ] , R ← m R\leftarrow m R ← m . ▷ domain already small: no hashing (id [ m ] \operatorname{id}_{[m]} id [ m ] is the identity map)
else
draw g : [ m ] → [ R ] g:[m]\to[R] g : [ m ] → [ R ] uniformly at random from public coins. ▷ public hash
Alice releases A ~ z ← ∣ { i : g ( X i ) = z } ∣ + Lap ( 2 / ε ) \widetilde{A}_z\leftarrow|\{i:g(X_i)=z\}|+\operatorname{Lap}(2/\varepsilon) A z ← ∣ { i : g ( X i ) = z } ∣ + Lap ( 2/ ε ) for all z ∈ [ R ] z\in[R] z ∈ [ R ] .
Bob releases B ~ z ← ∣ { i : g ( Y i ) = z } ∣ + Lap ( 2 / ε ) \widetilde{B}_z\leftarrow|\{i:g(Y_i)=z\}|+\operatorname{Lap}(2/\varepsilon) B z ← ∣ { i : g ( Y i ) = z } ∣ + Lap ( 2/ ε ) for all z ∈ [ R ] z\in[R] z ∈ [ R ] .
return f ^ ← max z ∈ [ R ] min { A ~ z , B ~ z } \hat{f}\leftarrow\max_{z\in[R]}\min\{\widetilde{A}_z,\widetilde{B}_z\} f ^ ← max z ∈ [ R ] min { A z , B z } .
Theorem 3.11 (Private Common-Frequency Estimation). For every n , m ∈ N n,m\in\mathbb{N} n , m ∈ N , ε > 0 \varepsilon>0 ε > 0 , and β ∈ ( 0 , 1 ) \beta\in(0,1) β ∈ ( 0 , 1 ) , Algorithm 1 is an ε \varepsilon ε -two-party differentially private protocol. Moreover, for every pair of inputs S A , S B ∈ [ m ] n S_A,S_B\in[m]^n S A , S B ∈ [ m ] n , with probability at least 1 − β 1-\beta 1 − β , ∣ f ^ − f ( S A , S B ) ∣ ≤ 2 ε ln ( 4 r β ) \left|\hat{f}-f(S_A,S_B)\right|\le\frac{2}{\varepsilon}\ln\left(\frac{4r}{\beta}\right) f ^ − f ( S A , S B ) ≤ ε 2 ln ( β 4 r ) where r : = min { m , ⌈ 4 n 2 β ⌉ } r:=\min\left\{m,\left\lceil\frac{4n^2}{\beta}\right\rceil\right\} r := min { m , ⌈ β 4 n 2 ⌉ } . In particular,
Pr ( ∣ f ^ − f ( S A , S B ) ∣ ≤ 2 log ( 16 n 2 / β ) ε ) ≥ 1 − β . \Pr\left(\left|\hat{f}-f(S_A,S_B)\right|\le\frac{2\log(16n^2/\beta)}{\varepsilon}\right)\ge1-\beta. Pr ( f ^ − f ( S A , S B ) ≤ ε 2 log ( 16 n 2 / β ) ) ≥ 1 − β .
Proof. For z ∈ [ r ] z\in[r] z ∈ [ r ] , let
A z g : = ∣ { i ∈ [ n ] : g ( X i ) = z } ∣ , B z g : = ∣ { i ∈ [ n ] : g ( Y i ) = z } ∣ . A_z^g:=|\{i\in[n]:g(X_i)=z\}|,\qquad B_z^g:=|\{i\in[n]:g(Y_i)=z\}|. A z g := ∣ { i ∈ [ n ] : g ( X i ) = z } ∣ , B z g := ∣ { i ∈ [ n ] : g ( Y i ) = z } ∣.
We first prove privacy. For every fixed public hash function g g g , changing one record changes the corresponding hashed histogram in ℓ 1 \ell_1 ℓ 1 -distance by at most 2. Thus, by Lemma B.1 , Alice’s release ( A ~ z ) z ∈ [ r ] (\widetilde{A}_z)_{z\in[r]} ( A z ) z ∈ [ r ] is ε \varepsilon ε -DP in S A S_A S A , and Bob’s release ( B ~ z ) z ∈ [ r ] (\widetilde{B}_z)_{z\in[r]} ( B z ) z ∈ [ r ] is ε \varepsilon ε -DP in S B S_B S B . All subsequent computations are post-processing. Therefore, Bob’s view is ε \varepsilon ε -DP in S A S_A S A and Alice’s view is ε \varepsilon ε -DP in S B S_B S B , as required.
It remains to prove accuracy. Let U : = supp ( S A ) ∪ supp ( S B ) \mathcal{U}:=\operatorname{supp}(S_A)\cup\operatorname{supp}(S_B) U := supp ( S A ) ∪ supp ( S B ) , and we have ∣ U ∣ ≤ 2 n |\mathcal{U}|\le2n ∣ U ∣ ≤ 2 n . If r = m r=m r = m , then g g g is the identity. Otherwise, by a union bound,
Pr [ g is not injective on U ] ≤ ( 2 n 2 ) 1 r ≤ β 2 . \Pr[g\text{ is not injective on }\mathcal{U}]\le\binom{2n}{2}\frac{1}{r}\le\frac{\beta}{2}. Pr [ g is not injective on U ] ≤ ( 2 2 n ) r 1 ≤ 2 β .
On the complementary event, every occupied hash bucket contains records from a single element of [ m ] [m] [ m ] . Consequently, max z ∈ [ r ] min { A z g , B z g } = max j ∈ [ m ] min { A j , B j } = f ( S A , S B ) \max_{z\in[r]}\min\{A_z^g,B_z^g\}=\max_{j\in[m]}\min\{A_j,B_j\}=f(S_A,S_B) max z ∈ [ r ] min { A z g , B z g } = max j ∈ [ m ] min { A j , B j } = f ( S A , S B ) . Set τ : = 2 ε ln ( 4 r β ) \tau:=\frac{2}{\varepsilon}\ln\left(\frac{4r}{\beta}\right) τ := ε 2 ln ( β 4 r ) . There are 2 r 2r 2 r independent Laplace variables, each with scale 2 / ε 2/\varepsilon 2/ ε . Hence, by Lemma B.1 ,
Pr [ max z ∈ [ r ] max { ∣ A ~ z − A z g ∣ , ∣ B ~ z − B z g ∣ } > τ ] ≤ 2 r e − ε τ / 2 = β 2 . \Pr\left[\max_{z\in[r]}\max\{|\widetilde{A}_z-A_z^g|,|\widetilde{B}_z-B_z^g|\}>\tau\right]\le2r e^{-\varepsilon\tau/2}=\frac{\beta}{2}. Pr [ z ∈ [ r ] max max { ∣ A z − A z g ∣ , ∣ B z − B z g ∣ } > τ ] ≤ 2 r e − ε τ /2 = 2 β .
On the complementary event, for every z ∈ [ r ] z\in[r] z ∈ [ r ] , ∣ min { A ~ z , B ~ z } − min { A z g , B z g } ∣ ≤ τ \left|\min\{\widetilde{A}_z,\widetilde{B}_z\}-\min\{A_z^g,B_z^g\}\right|\le\tau min { A z , B z } − min { A z g , B z g } ≤ τ . Taking the maximum over z z z therefore gives ∣ f ^ − max z ∈ [ r ] min { A z g , B z g } ∣ ≤ τ \left|\hat{f}-\max_{z\in[r]}\min\{A_z^g,B_z^g\}\right|\le\tau f ^ − max z ∈ [ r ] min { A z g , B z g } ≤ τ . Combining the hashing and noise events proves the stated guarantee. □ \square □
Main Algorithm
In this section building up on various lemma we proved in the previous two sections, we state the main algorithm and its guarantees. Before, doing it, we discuss a simple baseline that shows why simple idea fails
Simple baseline: private interior point. A natural approach is for Alice to privately select a point lying within the range of her witness block and send it to Bob. Bob can then determine whether his witness block lies above or below this point. We show that this idea already solves the promise problem, but at a log ( m ) \log(m) log ( m ) cost that our main algorithm improves to log log ( m ) \log\log(m) log log ( m ) . For θ ∈ X \theta\in\mathcal{X} θ ∈ X , define depth S A ( θ ) : = min { ∣ { x ∈ S A : x ≤ θ } ∣ , ∣ { x ∈ S A : x ≥ θ } ∣ } \operatorname{depth}_{S_A}(\theta) := \min\{\lvert\{x \in S_A : x \le\theta\}\rvert, \lvert\{x \in S_A : x \ge\theta\}\rvert\} depth S A ( θ ) := min {∣{ x ∈ S A : x ≤ θ }∣ , ∣{ x ∈ S A : x ≥ θ }∣} . This score has sensitivity one. Moreover, max θ ∈ X depth S A ( θ ) ≥ ⌈ k / 2 ⌉ \max_{\theta\in\mathcal{X}} \operatorname{depth}_{S_A}(\theta) \ge\lceil k/2\rceil max θ ∈ X depth S A ( θ ) ≥ ⌈ k /2 ⌉ , since every median of S A S_A S A has depth at least ⌈ k / 2 ⌉ \lceil k/2\rceil ⌈ k /2 ⌉ . Alice applies the exponential mechanism with privacy ε \varepsilon ε and score depth S A \operatorname{depth}_{S_A} depth S A , and sends the resulting point θ ^ \hat{\theta} θ ^ to Bob. The exponential-mechanism guarantee implies that, if k ≥ C log ( 2 m / β ) ε k \ge\frac{C\log(2m/\beta)}{\varepsilon} k ≥ ε C l o g ( 2 m / β ) , with probability at least 1 − β / 2 1-\beta/2 1 − β /2 , for a sufficiently large universal constant C C C , then depth S A ( θ ^ ) > 0 \operatorname{depth}_{S_A}(\hat{\theta}) > 0 depth S A ( θ ^ ) > 0 and hence θ ^ ∈ [ min S A , max S A ] \hat{\theta} \in[\min S_A,\max S_A] θ ^ ∈ [ min S A , max S A ] . Then, Bob releases c ~ B : = ∣ { x ∈ S B : x < θ ^ } ∣ + Lap ( 1 / ε ) \widetilde{c}_B := \lvert\{x \in S_B : x < \hat{\theta}\}\rvert+ \operatorname{Lap}(1/\varepsilon) c B := ∣{ x ∈ S B : x < θ ^ }∣ + Lap ( 1/ ε ) and outputs Match if c ~ B < k / 2 \widetilde{c}_B < k/2 c B < k /2 and Far otherwise. It follows that the baseline solves the t t t -promise problem for t ≥ C log ( 2 m / β ) ε t \ge\frac{C\log(2m/\beta)}{\varepsilon} t ≥ ε C l o g ( 2 m / β ) . The dependence on m m m comes entirely from privately selecting a point from an m m m -element domain. As we will see, our main algorithm improves exponentially over this baseline by avoiding this direct selection by searching over a different object.
Overview of the Technical Ideas of Our Algorithm. In this section, we provide an overview of the ideas in Algorithm 2. We organize the ordered domain as a binary trie as defined in Definition 3.7 . The first step is two perform private private binary search in order to find
ℓ ⋆ = min { ℓ ∈ [ b ] : F ( ℓ ) ≥ α } , \ell^\star= \min\{\ell\in[b] : F(\ell) \ge\alpha\}, ℓ ⋆ = min { ℓ ∈ [ b ] : F ( ℓ ) ≥ α } ,
where F : [ b ] → N F : [b] \to\mathbb{N} F : [ b ] → N defined as F ( ℓ ) = n − max p ∈ { 0 , 1 } ℓ min { A p ℓ , B p ℓ } F(\ell) = n - \max_{p \in\{0,1\}^{\ell}} \min\{A_p^\ell, B_p^\ell\} F ( ℓ ) = n − max p ∈ { 0 , 1 } ℓ min { A p ℓ , B p ℓ } . Notice that for every depth ℓ \ell ℓ , { A p ℓ } p ∈ { 0 , 1 } ℓ \{A_p^\ell\}_{p \in\{0,1\}^{\ell}} { A p ℓ } p ∈ { 0 , 1 } ℓ and { B p ℓ } p ∈ { 0 , 1 } ℓ \{B_p^\ell\}_{p \in\{0,1\}^{\ell}} { B p ℓ } p ∈ { 0 , 1 } ℓ can be seen as a histogram. The straightforward implementation of DP histogram estimation incurs an error of O ( ℓ / ε 0 ) O(\ell/\varepsilon_0) O ( ℓ / ε 0 ) which can be O ( log ( m ) / ε 0 ) O(\log(m)/\varepsilon_0) O ( log ( m ) / ε 0 ) . The main observation here is that we only need to estimate max p ∈ { 0 , 1 } ℓ min { A p ℓ , B p ℓ } \max_{p \in\{0,1\}^{\ell}} \min\{A_p^\ell, B_p^\ell\} max p ∈ { 0 , 1 } ℓ min { A p ℓ , B p ℓ } without estimating the whole histogram. To do so, we develop Algorithm 1 where the main idea is to use public random hashing to design a distributed algorithm with two-view DP guarantee such that the error due to DP to estimate max p ∈ { 0 , 1 } ℓ min { A p ℓ , B p ℓ } \max_{p \in\{0,1\}^{\ell}} \min\{A_p^\ell, B_p^\ell\} max p ∈ { 0 , 1 } ℓ min { A p ℓ , B p ℓ } is independent of the domain size.
Once the binary search returns two consecutive depths L L L and H H H , we consider whether the increase from F ( L ) F(L) F ( L ) to F ( H ) F(H) F ( H ) is large or small. For intuition, suppose first that the common nodes at these depths are known. If the increase is large, then many points leave the common node between depths L L L and H H H . Since H = L + 1 H = L + 1 H = L + 1 , this information is concentrated in a single split, and comparing the numbers of Alice and Bob points in its two children reveals their relative order. To gain intuition behind this case, see Figure 5 a. Our algorithm is based on privately finding the orientation of split on Alice and Bob’s side and comparing them.
Figure 5. Intuition behind the two-cases of the second stage of Algorithm 2
If the increase is small, the information is not concentrated in the final split. Instead, because F ( H ) F(H) F ( H ) is already above the noise level, sufficiently many points have accumulated outside the common node by depth H H H , while the common node itself remains heavy. Under the Match promise, Alice’s points outside this node lie to its left and Bob’s points lie to its right; under the Far promise, these directions are reversed. To gain intuition see Figure 5 b. Thus, comparing the total mass on the two sides reveals the correct answer. The actual implementation is more challenging because the common node depends on the private datasets and cannot be released explicitly; our statistics are designed to recover these two comparisons without identifying it.
Algorithm 2 Data Valuation for Threshold: A v a l − t h r \mathcal{A}_{\mathrm{val-thr}} A val − thr
Input: Alice holds S A = ( X 1 , … , X n ) ∈ ( { 0 , 1 } b ) n S_A=(X_1,\ldots,X_n)\in(\{0,1\}^{b})^n S A = ( X 1 , … , X n ) ∈ ({ 0 , 1 } b ) n , Bob holds S B = ( Y 1 , … , Y n ) ∈ ( { 0 , 1 } b ) n S_B=(Y_1,\ldots,Y_n)\in(\{0,1\}^{b})^n S B = ( Y 1 , … , Y n ) ∈ ({ 0 , 1 } b ) n ; parameters ε , β \varepsilon,\beta ε , β .
Output:
T ← ⌈ log 2 b ⌉ T\leftarrow\lceil\log_2 b\rceil T ← ⌈ log 2 b ⌉ , ε 0 ← ε / 2 T \varepsilon_0\leftarrow\varepsilon/2T ε 0 ← ε /2 T , β 0 ← β / 2 T \beta_0\leftarrow\beta/2T β 0 ← β /2 T .
2.
α ← 20 max { 2 ε 0 ln ( 4 ⌈ 4 n 2 / β 0 ⌉ β 0 ) , 28 ε ln 4 β } . \alpha\leftarrow20\max\left\{\frac{2}{\varepsilon_0}\ln\left(\frac{4\lceil4n^2/\beta_0\rceil}{\beta_0}\right),\frac{28}{\varepsilon}\ln\frac{4}{\beta}\right\}. α ← 20 max { ε 0 2 ln ( β 0 4 ⌈ 4 n 2 / β 0 ⌉ ) , ε 28 ln β 4 } .
Set L ← 0 L\leftarrow0 L ← 0 , H ← b H\leftarrow b H ← b , F ^ ( 0 ) ← 0 \widehat{F}(0)\leftarrow0 F ( 0 ) ← 0 , and F ^ ( b ) ← n \widehat{F}(b)\leftarrow n F ( b ) ← n . ▷ Initialization of binary search
while H − L > 1 H-L>1 H − L > 1 do ▷ Binary search step
ℓ ← ⌊ ( L + H ) / 2 ⌋ \ell\leftarrow\lfloor(L+H)/2\rfloor ℓ ← ⌊( L + H ) /2 ⌋ .
Form the prefix datasets
S A ℓ ← ( Prefix ℓ ( X 1 ) , … , Prefix ℓ ( X n ) ) , S B ℓ ← ( Prefix ℓ ( Y 1 ) , … , Prefix ℓ ( Y n ) ) . S_A^\ell\leftarrow(\operatorname{Prefix}_{\ell}(X_1),\ldots,\operatorname{Prefix}_{\ell}(X_n)),\qquad S_B^\ell\leftarrow(\operatorname{Prefix}_{\ell}(Y_1),\ldots,\operatorname{Prefix}_{\ell}(Y_n)). S A ℓ ← ( Prefix ℓ ( X 1 ) , … , Prefix ℓ ( X n )) , S B ℓ ← ( Prefix ℓ ( Y 1 ) , … , Prefix ℓ ( Y n )) .
F ^ ( ℓ ) ← n − A P C F ( S A ℓ , S B ℓ ; ε 0 , β 0 ) \widehat{F}(\ell)\leftarrow n-\mathcal{A}_{\mathrm{PCF}}(S_A^\ell,S_B^\ell;\varepsilon_0,\beta_0) F ( ℓ ) ← n − A PCF ( S A ℓ , S B ℓ ; ε 0 , β 0 ) . ▷ See Algorithm 1
if F ^ ( ℓ ) < α \widehat{F}(\ell)<\alpha F ( ℓ ) < α then
L ← ℓ L\leftarrow\ell L ← ℓ .
else
H ← ℓ H\leftarrow\ell H ← ℓ .
Δ ^ ← F ^ ( H ) − F ^ ( L ) \widehat{\Delta}\leftarrow\widehat{F}(H)-\widehat{F}(L) Δ ← F ( H ) − F ( L ) .
if Δ ^ ≥ α / 2 \widehat{\Delta}\ge\alpha/2 Δ ≥ α /2 then ▷ large-jump case
Alice releases
V ^ A ← max p ∈ { 0 , 1 } L { 6 ∣ A p L ∣ + ∣ A p 0 H ∣ } − max p ∈ { 0 , 1 } L { 6 ∣ A p L ∣ + ∣ A p 1 H ∣ } + Lap ( 28 / ε ) . \widehat{V}_A\leftarrow\max_{p\in\{0,1\}^{L}}\left\{6|A_p^L|+|A_{p0}^H|\right\}-\max_{p\in\{0,1\}^{L}}\left\{6|A_p^L|+|A_{p1}^H|\right\}+\operatorname{Lap}(28/\varepsilon). V A ← p ∈ { 0 , 1 } L max { 6∣ A p L ∣ + ∣ A p 0 H ∣ } − p ∈ { 0 , 1 } L max { 6∣ A p L ∣ + ∣ A p 1 H ∣ } + Lap ( 28/ ε ) .
Bob releases
V ^ B ← max p ∈ { 0 , 1 } L { 6 ∣ B p L ∣ + ∣ B p 0 H ∣ } − max p ∈ { 0 , 1 } L { 6 ∣ B p L ∣ + ∣ B p 1 H ∣ } + Lap ( 28 / ε ) . \widehat{V}_B\leftarrow\max_{p\in\{0,1\}^{L}}\left\{6|B_p^L|+|B_{p0}^H|\right\}-\max_{p\in\{0,1\}^{L}}\left\{6|B_p^L|+|B_{p1}^H|\right\}+\operatorname{Lap}(28/\varepsilon). V B ← p ∈ { 0 , 1 } L max { 6∣ B p L ∣ + ∣ B p 0 H ∣ } − p ∈ { 0 , 1 } L max { 6∣ B p L ∣ + ∣ B p 1 H ∣ } + Lap ( 28/ ε ) .
else ▷ cumulative-escape case
Alice releases
V ^ A ← max p ∈ { 0 , 1 } H { 6 ∣ A p H ∣ + ∑ q ∈ { 0 , 1 } H q ≺ p ∣ A q H ∣ } − max p ∈ { 0 , 1 } H { 6 ∣ A p H ∣ + ∑ q ∈ { 0 , 1 } H q ≻ p ∣ A q H ∣ } + Lap ( 28 / ε ) . \widehat{V}_A\leftarrow\max_{p\in\{0,1\}^{H}}\left\{6|A_p^H|+\sum_{\substack{q\in\{0,1\}^{H}\\q\prec p}}|A_q^H|\right\}-\max_{p\in\{0,1\}^{H}}\left\{6|A_p^H|+\sum_{\substack{q\in\{0,1\}^{H}\\q\succ p}}|A_q^H|\right\}+\operatorname{Lap}(28/\varepsilon). V A ← p ∈ { 0 , 1 } H max ⎩ ⎨ ⎧ 6∣ A p H ∣ + q ∈ { 0 , 1 } H q ≺ p ∑ ∣ A q H ∣ ⎭ ⎬ ⎫ − p ∈ { 0 , 1 } H max ⎩ ⎨ ⎧ 6∣ A p H ∣ + q ∈ { 0 , 1 } H q ≻ p ∑ ∣ A q H ∣ ⎭ ⎬ ⎫ + Lap ( 28/ ε ) .
Bob releases
V ^ B ← max p ∈ { 0 , 1 } H { 6 ∣ B p H ∣ + ∑ q ∈ { 0 , 1 } H q ≺ p ∣ B q H ∣ } − max p ∈ { 0 , 1 } H { 6 ∣ B p H ∣ + ∑ q ∈ { 0 , 1 } H q ≻ p ∣ B q H ∣ } + Lap ( 28 / ε ) . \widehat{V}_B\leftarrow\max_{p\in\{0,1\}^{H}}\left\{6|B_p^H|+\sum_{\substack{q\in\{0,1\}^{H}\\q\prec p}}|B_q^H|\right\}-\max_{p\in\{0,1\}^{H}}\left\{6|B_p^H|+\sum_{\substack{q\in\{0,1\}^{H}\\q\succ p}}|B_q^H|\right\}+\operatorname{Lap}(28/\varepsilon). V B ← p ∈ { 0 , 1 } H max ⎩ ⎨ ⎧ 6∣ B p H ∣ + q ∈ { 0 , 1 } H q ≺ p ∑ ∣ B q H ∣ ⎭ ⎬ ⎫ − p ∈ { 0 , 1 } H max ⎩ ⎨ ⎧ 6∣ B p H ∣ + q ∈ { 0 , 1 } H q ≻ p ∑ ∣ B q H ∣ ⎭ ⎬ ⎫ + Lap ( 28/ ε ) .
return Match if V ^ A ≥ V ^ B \widehat{V}_A\ge\widehat{V}_B V A ≥ V B ; otherwise return Far.
Theorem 3.12 (Upper bound for thresholds). There exists a universal constant C > 0 C > 0 C > 0 such that the following holds. Let X ⊆ R X \subseteq\mathbb{R} X ⊆ R be a finite ordered domain with ∣ X ∣ = m ≥ 4 |X| = m \ge4 ∣ X ∣ = m ≥ 4 , let n ∈ N n \in\mathbb{N} n ∈ N , let ε ∈ ( 0 , 1 ] \varepsilon\in(0,1] ε ∈ ( 0 , 1 ] and β ∈ ( 0 , 1 / 3 ] \beta\in(0,1/3] β ∈ ( 0 , 1/3 ] , and let t ∈ [ n ] t \in[n] t ∈ [ n ] satisfy
t ≥ C log log m ε ( log ( 1 β ) + log ( log log m ε ) ) t \ge C \frac{\log\log m}{\varepsilon}\left(\log\left(\frac{1}{\beta}\right)+\log\left(\frac{\log\log m}{\varepsilon}\right)\right) t ≥ C ε log log m ( log ( β 1 ) + log ( ε log log m ) ) Also, assume n ≥ C log log m ε log ( log log m β ε ) n \ge C \frac{\log\log m}{\varepsilon}\log\left(\frac{\log\log m}{\beta\varepsilon}\right) n ≥ C ε l o g l o g m log ( β ε l o g l o g m ) . Then A v a l − t h r \mathcal{A}_{\mathrm{val-thr}} A val − thr (in Algorithm 2) is an ε \varepsilon ε -private tester for H t h r \mathcal{H}_{\mathrm{thr}} H thr with resolution t t t with failure probability at most β \beta β .
Proof. The proof has three parts: private binary search , and then, after that we have two cases, large-jump case and cumulative-escape case .
Step 1: private binary search. The goal of this step is to approximate
ℓ ⋆ = min { ℓ ∈ [ b ] : F ( ℓ ) ≥ α } . \ell^\star= \min\{\ell\in[b] : F(\ell) \ge\alpha\}. ℓ ⋆ = min { ℓ ∈ [ b ] : F ( ℓ ) ≥ α } .
Define the following bad event
B b s = { ∃ t ≤ T : ∣ F ( ℓ t ) − F ^ ( ℓ t ) ∣ ≥ α / 20 } , (9) \mathcal{B}_{bs} = \{\exists t \le T : |F(\ell_t) - \widehat{F}(\ell_t)| \ge\alpha/20\}, \tag*{(9)} B b s = { ∃ t ≤ T : ∣ F ( ℓ t ) − F ( ℓ t ) ∣ ≥ α /20 } , (9)
where { ℓ t } t ∈ [ T ] \{\ell_t\}_{t \in[T]} { ℓ t } t ∈ [ T ] are itself random variable denoting the indices chosen by the private binary search step. Also, T = O ( log ( b ) ) T = O(\log(b)) T = O ( log ( b )) and deterministic. Then, we write
Pr ( B b s ) ≤ ∑ t = 1 T Pr ( ∣ F ( ℓ t ) − F ^ ( ℓ t ) ∣ ≥ α / 20 ) ≤ ∑ t = 1 T E ℓ t [ Pr ( ∣ F ( ℓ t ) − F ^ ( ℓ t ) ∣ ≥ α / 20 ∣ ℓ t ) ] ≤ T ⋅ β 0 ≤ β . \begin{aligned}
\Pr(\mathcal{B}_{bs}) \le\sum_{t=1}^{T}\Pr\left(|F(\ell_t)-\widehat{F}(\ell_t)|\ge\alpha/20\right) \\
&\le\sum_{t=1}^{T}\mathbb{E}_{\ell_t}\left[\Pr\left(|F(\ell_t)-\widehat{F}(\ell_t)|\ge\alpha/20\mid\ell_t\right)\right] \\
&\le T\cdot\beta_0 \\
&\le\beta.
\end{aligned} Pr ( B b s ) ≤ t = 1 ∑ T Pr ( ∣ F ( ℓ t ) − F ( ℓ t ) ∣ ≥ α /20 ) ≤ t = 1 ∑ T E ℓ t [ Pr ( ∣ F ( ℓ t ) − F ( ℓ t ) ∣ ≥ α /20 ∣ ℓ t ) ] ≤ T ⋅ β 0 ≤ β .
where the first step is due to the union bound, and the second step is by the law of the total probability, then, in the third step, we use the fact that each execution of the A P C F \mathcal{A}_{\mathrm{PCF}} A PCF uses a fresh randomness independent of the history of the algorithm to obtain the result. Notice that the private binary search maintains L , H L, H L , H such that H − L = 1 H-L=1 H − L = 1 and F ^ ( L ) ≤ α ≤ F ^ ( H ) \widehat{F}(L) \le\alpha\le\widehat{F}(H) F ( L ) ≤ α ≤ F ( H ) . Under the event of B b s c \mathcal{B}_{\mathrm{bs}}^{c} B bs c , we have
F ( L ) ≤ F ^ ( L ) + α 20 ≤ 21 α 20 , F ( H ) ≥ F ^ ( H ) − α 20 ≥ 19 α 20 . (10) \begin{aligned}
F(L) &\le\widehat{F}(L)+\frac{\alpha}{20}\le\frac{21\alpha}{20},\\
F(H) &\ge\widehat{F}(H)-\frac{\alpha}{20}\ge\frac{19\alpha}{20}. \tag*{(10)}
\end{aligned} F ( L ) F ( H ) ≤ F ( L ) + 20 α ≤ 20 21 α , ≥ F ( H ) − 20 α ≥ 20 19 α . (10)
Step 2-Case 1: Large Jump. Suppose that Δ ^ ≥ α / 2 \widehat{\Delta}\ge\alpha/2 Δ ≥ α /2 . Throughout the proof, we assume we are under B b s c \mathcal{B}_{\mathrm{bs}}^{c} B bs c in Equation (9) . First, we have Δ ≥ Δ ^ − α 10 ≥ 2 α 5 \Delta\ge\widehat{\Delta}-\frac{\alpha}{10}\ge\frac{2\alpha}{5} Δ ≥ Δ − 10 α ≥ 5 2 α . Since n ≥ 4 α n\ge4\alpha n ≥ 4 α , we have F ( L ) ≤ 21 α 20 ≤ 21 n 80 < n F(L)\le\frac{21\alpha}{20}\le\frac{21n}{80}<n F ( L ) ≤ 20 21 α ≤ 80 21 n < n . Therefore, n − F ( L ) > 0 n-F(L)>0 n − F ( L ) > 0 , so there is a node p ∈ { 0 , 1 } L p\in\{0,1\}^{L} p ∈ { 0 , 1 } L satisfying min { ∣ A p L ∣ , ∣ B p L ∣ } = n − F ( L ) \min\{|A_{p}^{L}|,|B_{p}^{L}|\}=n-F(L) min { ∣ A p L ∣ , ∣ B p L ∣ } = n − F ( L ) . By Lemma 3.8 , this is the unique common node at depth L L L .
We next show that the maximization used in this case are attained at the common node. We prove correctness when optA t h r ( S A , S B ) = 0 \operatorname{optA}_{\mathrm{thr}}(S_A,S_B)=0 optA thr ( S A , S B ) = 0 ; the Far case is symmetric. To show that the argmax is achieved at the common node consider an arbitrary q ≠ p q\ne p q = p where q ∈ { 0 , 1 } L q\in\{0,1\}^{L} q ∈ { 0 , 1 } L . Then, for every j ∈ { 0 , 1 } j\in\{0,1\} j ∈ { 0 , 1 }
6 ∣ A p L ∣ + ∣ A p j L ∣ − ( 6 ∣ A q L ∣ + ∣ A q j L ∣ ) = 6 ( ∣ A p L ∣ − ∣ A q L ∣ ) + ∣ A p j L ∣ − ∣ A q j L ∣ ≥ 6 ( n − F ( L ) − F ( L ) ) − F ( L ) ≥ 6 n − 13 F ( L ) ≥ 6 n − 13 21 α 20 . \begin{aligned}
6|A_p^L|+|A_{pj}^L|-(6|A_q^L|+|A_{qj}^L|)
&=6(|A_p^L|-|A_q^L|)+|A_{pj}^L|-|A_{qj}^L|\\
&\ge6(n-F(L)-F(L))-F(L)\\
&\ge6n-13F(L)\\
&\ge6n-13\frac{21\alpha}{20}.
\end{aligned} 6∣ A p L ∣ + ∣ A p j L ∣ − ( 6∣ A q L ∣ + ∣ A q j L ∣ ) = 6 ( ∣ A p L ∣ − ∣ A q L ∣ ) + ∣ A p j L ∣ − ∣ A q j L ∣ ≥ 6 ( n − F ( L ) − F ( L )) − F ( L ) ≥ 6 n − 13 F ( L ) ≥ 6 n − 13 20 21 α .
where the second step is by ∣ A p L ∣ = n − F ( L ) |A_p^L|=n-F(L) ∣ A p L ∣ = n − F ( L ) , ∣ A q L ∣ ≤ F ( L ) |A_q^L|\le F(L) ∣ A q L ∣ ≤ F ( L ) , ∣ A p j L ∣ ≥ 0 |A_{pj}^L|\ge0 ∣ A p j L ∣ ≥ 0 , and ∣ A q j L ∣ ≤ ∣ A q L ∣ ≤ F ( L ) |A_{qj}^L|\le|A_q^L|\le F(L) ∣ A q j L ∣ ≤ ∣ A q L ∣ ≤ F ( L ) . The last step follows from Equation (10) . It shows that assuming n ≥ 4 α n\ge4\alpha n ≥ 4 α the last term is positive and hence p p p is argmax. Similarly it holds for Bob’s maximization part. Therefore,
max p ∈ { 0 , 1 } L { 6 ∣ A p L ∣ + ∣ A p 0 H ∣ } − max p ∈ { 0 , 1 } L { 6 ∣ A p L ∣ + ∣ A p 1 H ∣ } = ∣ A p 0 H ∣ − ∣ A p 1 H ∣ . \max_{p\in\{0,1\}^{L}}\left\{6|A_p^L|+|A_{p0}^H|\right\}-\max_{p\in\{0,1\}^{L}}\left\{6|A_p^L|+|A_{p1}^H|\right\}=|A_{p0}^H|-|A_{p1}^H|. p ∈ { 0 , 1 } L max { 6∣ A p L ∣ + ∣ A p 0 H ∣ } − p ∈ { 0 , 1 } L max { 6∣ A p L ∣ + ∣ A p 1 H ∣ } = ∣ A p 0 H ∣ − ∣ A p 1 H ∣.
and similarly
max p ∈ { 0 , 1 } L { 6 ∣ B p L ∣ + ∣ B p 0 H ∣ } − max p ∈ { 0 , 1 } L { 6 ∣ B p L ∣ + ∣ B p 1 H ∣ } = ∣ B p 0 H ∣ − ∣ B p 1 H ∣ . \max_{p\in\{0,1\}^{L}}\left\{6|B_p^L|+|B_{p0}^H|\right\}-\max_{p\in\{0,1\}^{L}}\left\{6|B_p^L|+|B_{p1}^H|\right\}=|B_{p0}^H|-|B_{p1}^H|. p ∈ { 0 , 1 } L max { 6∣ B p L ∣ + ∣ B p 0 H ∣ } − p ∈ { 0 , 1 } L max { 6∣ B p L ∣ + ∣ B p 1 H ∣ } = ∣ B p 0 H ∣ − ∣ B p 1 H ∣.
Under the Match promise, every Alice point is smaller than every Bob point. Therefore, Alice cannot have a point in p 1 p1 p 1 while Bob has a point in p 0 p0 p 0 . Equivalently, ∣ A p 1 H ∣ ⋅ ∣ B p 0 H ∣ = 0 |A_{p1}^H|\cdot|B_{p0}^H|=0 ∣ A p 1 H ∣ ⋅ ∣ B p 0 H ∣ = 0 . Suppose first that ∣ A p 1 H ∣ = 0 |A_{p1}^H|=0 ∣ A p 1 H ∣ = 0 ; the case ∣ B p 0 H ∣ = 0 |B_{p0}^H|=0 ∣ B p 0 H ∣ = 0 follows similarly. Since Δ > 0 \Delta>0 Δ > 0 , we must have ∣ A p 0 H ∣ > ∣ B p 0 H ∣ |A_{p0}^H|>|B_{p0}^H| ∣ A p 0 H ∣ > ∣ B p 0 H ∣ . Indeed, otherwise the common frequency would remain ∣ A p 0 H ∣ |A_{p0}^H| ∣ A p 0 H ∣ when passing from depth L L L to depth H H H , contradicting Δ > 0 \Delta>0 Δ > 0 . Therefore,
n − F ( L ) = min { ∣ A p 0 H ∣ , ∣ B p 0 H ∣ + ∣ B p 1 H ∣ } n-F(L)=\min\left\{|A_{p0}^H|,|B_{p0}^H|+|B_{p1}^H|\right\} n − F ( L ) = min { ∣ A p 0 H ∣ , ∣ B p 0 H ∣ + ∣ B p 1 H ∣ }
whereas n − F ( H ) = ∣ B p 0 H ∣ n-F(H)=|B_{p0}^H| n − F ( H ) = ∣ B p 0 H ∣ . It follows that
Δ = F ( H ) − F ( L ) = min { ∣ A p 0 H ∣ − ∣ B p 0 H ∣ , ∣ B p 1 H ∣ } . \begin{aligned}
\Delta&=F(H)-F(L)\\
&=\min\left\{|A_{p0}^H|-|B_{p0}^H|,|B_{p1}^H|\right\}.
\end{aligned} Δ = F ( H ) − F ( L ) = min { ∣ A p 0 H ∣ − ∣ B p 0 H ∣ , ∣ B p 1 H ∣ } .
Consequently, the difference between Alice’s and Bob’s unnoised statistics is
( ∣ A p 0 H ∣ − ∣ A p 1 H ∣ ) − ( ∣ B p 0 H ∣ − ∣ B p 1 H ∣ ) = ∣ A p 0 H ∣ − ∣ B p 0 H ∣ + ∣ B p 1 H ∣ ≥ 2 min { ∣ A p 0 H ∣ − ∣ B p 0 H ∣ , ∣ B p 1 H ∣ } = 2 Δ ≥ 4 α 5 . \begin{aligned}
\left(|A_{p0}^H|-|A_{p1}^H|\right)-\left(|B_{p0}^H|-|B_{p1}^H|\right)
&=|A_{p0}^H|-|B_{p0}^H|+|B_{p1}^H|\\
&\ge2\min\left\{|A_{p0}^H|-|B_{p0}^H|,|B_{p1}^H|\right\}\\
&=2\Delta\ge\frac{4\alpha}{5}.
\end{aligned} ( ∣ A p 0 H ∣ − ∣ A p 1 H ∣ ) − ( ∣ B p 0 H ∣ − ∣ B p 1 H ∣ ) = ∣ A p 0 H ∣ − ∣ B p 0 H ∣ + ∣ B p 1 H ∣ ≥ 2 min { ∣ A p 0 H ∣ − ∣ B p 0 H ∣ , ∣ B p 1 H ∣ } = 2Δ ≥ 5 4 α .
The case ∣ B p 0 H ∣ = 0 |B_{p_0}^{H}|=0 ∣ B p 0 H ∣ = 0 follows symmetrically and gives the same lower bound. Thus, under the Match promise, the unnoised statistic has positive margin at least 4 α / 5 4\alpha/5 4 α /5 . The final noise is smaller than this margin under the corresponding good event, and hence the algorithm outputs Match.
Under the Far promise, the order is reversed, and the same argument gives a negative margin of magnitude at least 4 α / 5 4\alpha/5 4 α /5 . Therefore, the algorithm outputs Far.
For the privacy proof, each maximization has sensitivity at most 7, since changing one record changes the term multiplied by 6 by at most 6 and the remaining count by at most 1. Therefore, the difference of the two maximizations has sensitivity at most 14, and allocating privacy budget ε / 2 \varepsilon/2 ε /2 to its release requires noise Lap ( 14 / ( ε / 2 ) ) = Lap ( 28 / ε ) \operatorname{Lap}(14/(\varepsilon/2))=\operatorname{Lap}(28/\varepsilon) Lap ( 14/ ( ε /2 )) = Lap ( 28/ ε ) .
Step 2-Case 2: Cumulative Escape. Suppose that Δ ^ < α / 2 \widehat{\Delta}<\alpha/2 Δ < α /2 . Under B b s c \mathcal{B}_{\mathrm{bs}}^{c} B bs c , we have Δ ≤ Δ ^ + α / 10 < 3 α / 5 \Delta\le\widehat{\Delta}+\alpha/10<3\alpha/5 Δ ≤ Δ + α /10 < 3 α /5 . Therefore, by Equation (10) ,
F ( H ) = F ( L ) + Δ < 21 α 20 + 3 α 5 = 33 α 20 . F(H)=F(L)+\Delta<\frac{21\alpha}{20}+\frac{3\alpha}{5}=\frac{33\alpha}{20}. F ( H ) = F ( L ) + Δ < 20 21 α + 5 3 α = 20 33 α .
Since n ≥ 4 α n\ge4\alpha n ≥ 4 α , we have F ( H ) < 33 n / 80 < n F(H)<33n/80<n F ( H ) < 33 n /80 < n . Therefore, there is a node p ∈ { 0 , 1 } H p\in\{0,1\}^{H} p ∈ { 0 , 1 } H satisfying min { ∣ A p H ∣ , ∣ B p H ∣ } = n − F ( H ) \min\{|A_{p}^{H}|,|B_{p}^{H}|\}=n-F(H) min { ∣ A p H ∣ , ∣ B p H ∣ } = n − F ( H ) . By Lemma 3.8 , this is the unique common node at depth H H H . We next show that the maximizations used in this case are attained at p p p . As before, we prove correctness under the Match promise; the Far case follows symmetrically. Consider an arbitrary q ∈ { 0 , 1 } H q\in\{0,1\}^{H} q ∈ { 0 , 1 } H with q ≠ p q\ne p q = p . Then
6 ∣ A p H ∣ + ∣ A < p H ∣ − ( 6 ∣ A q H ∣ + ∣ A < q H ∣ ) ≥ 6 ( n − F ( H ) − F ( H ) ) − n = 5 n − 12 F ( H ) > 5 n − 12 33 n 80 = n 20 > 0. \begin{aligned}
6|A_{p}^{H}|+|A_{<p}^{H}|-\left(6|A_{q}^{H}|+|A_{<q}^{H}|\right)
\ge6(n-F(H)-F(H))-n \\
&=5n-12F(H) \\
&>5n-12\frac{33n}{80}=\frac{n}{20}>0.
\end{aligned} 6∣ A p H ∣ + ∣ A < p H ∣ − ( 6∣ A q H ∣ + ∣ A < q H ∣ ) ≥ 6 ( n − F ( H ) − F ( H )) − n = 5 n − 12 F ( H ) > 5 n − 12 80 33 n = 20 n > 0.
Here we used ∣ A p H ∣ ≥ n − F ( H ) |A_{p}^{H}|\ge n-F(H) ∣ A p H ∣ ≥ n − F ( H ) , ∣ A q H ∣ ≤ F ( H ) |A_{q}^{H}|\le F(H) ∣ A q H ∣ ≤ F ( H ) , ∣ A < p H ∣ ≥ 0 |A_{<p}^{H}|\ge0 ∣ A < p H ∣ ≥ 0 , and ∣ A < q H ∣ ≤ n |A_{<q}^{H}|\le n ∣ A < q H ∣ ≤ n . The same argument, with A > p H A_{>p}^{H} A > p H and A > q H A_{>q}^{H} A > q H in place of A < p H A_{<p}^{H} A < p H and A < q H A_{<q}^{H} A < q H , shows that the second maximization is also attained at p p p . The same conclusion holds for Bob. Therefore,
max q ∈ { 0 , 1 } H { 6 ∣ A q H ∣ + ∣ A < q H ∣ } − max q ∈ { 0 , 1 } H { 6 ∣ A q H ∣ + ∣ A > q H ∣ } = ∣ A < p H ∣ − ∣ A > p H ∣ , \max_{q\in\{0,1\}^{H}}\{6|A_{q}^{H}|+|A_{<q}^{H}|\}-\max_{q\in\{0,1\}^{H}}\{6|A_{q}^{H}|+|A_{>q}^{H}|\}=|A_{<p}^{H}|-|A_{>p}^{H}|, q ∈ { 0 , 1 } H max { 6∣ A q H ∣ + ∣ A < q H ∣ } − q ∈ { 0 , 1 } H max { 6∣ A q H ∣ + ∣ A > q H ∣ } = ∣ A < p H ∣ − ∣ A > p H ∣ ,
and similarly,
max q ∈ { 0 , 1 } H { 6 ∣ B q H ∣ + ∣ B < q H ∣ } − max q ∈ { 0 , 1 } H { 6 ∣ B q H ∣ + ∣ B > q H ∣ } = ∣ B < p H ∣ − ∣ B > p H ∣ . \max_{q\in\{0,1\}^{H}}\{6|B_{q}^{H}|+|B_{<q}^{H}|\}-\max_{q\in\{0,1\}^{H}}\{6|B_{q}^{H}|+|B_{>q}^{H}|\}=|B_{<p}^{H}|-|B_{>p}^{H}|. q ∈ { 0 , 1 } H max { 6∣ B q H ∣ + ∣ B < q H ∣ } − q ∈ { 0 , 1 } H max { 6∣ B q H ∣ + ∣ B > q H ∣ } = ∣ B < p H ∣ − ∣ B > p H ∣.
Under the Match promise, every Alice point is smaller than every Bob point. Since p p p contains points from both parties, Alice cannot have a point to the right of p p p , and Bob cannot have a point to the left of p p p . Hence ∣ A > p H ∣ = ∣ B < p H ∣ = 0 |A_{>p}^{H}|=|B_{<p}^{H}|=0 ∣ A > p H ∣ = ∣ B < p H ∣ = 0 . Moreover, min { ∣ A p H ∣ , ∣ B p H ∣ } = n − F ( H ) \min\{|A_{p}^{H}|,|B_{p}^{H}|\}=n-F(H) min { ∣ A p H ∣ , ∣ B p H ∣ } = n − F ( H ) implies
max { n − ∣ A p H ∣ , n − ∣ B p H ∣ } = F ( H ) . \max\{n-|A_{p}^{H}|,n-|B_{p}^{H}|\}=F(H). max { n − ∣ A p H ∣ , n − ∣ B p H ∣ } = F ( H ) .
Consequently, the difference between Alice’s and Bob’s unnoised statistics is
( ∣ A < p H ∣ − ∣ A > p H ∣ ) − ( ∣ B < p H ∣ − ∣ B > p H ∣ ) = ∣ A < p H ∣ + ∣ B > p H ∣ ≥ F ( H ) ≥ 19 α 20 . \begin{aligned}
\left(|A_{<p}^{H}|-|A_{>p}^{H}|\right)-\left(|B_{<p}^{H}|-|B_{>p}^{H}|\right) \\
&=|A_{<p}^{H}|+|B_{>p}^{H}| \\
&\ge F(H)\ge\frac{19\alpha}{20}.
\end{aligned} ( ∣ A < p H ∣ − ∣ A > p H ∣ ) − ( ∣ B < p H ∣ − ∣ B > p H ∣ ) = ∣ A < p H ∣ + ∣ B > p H ∣ ≥ F ( H ) ≥ 20 19 α .
Thus, under the Match promise, the unnoised statistic has positive margin at least 19 α / 20 19\alpha/20 19 α /20 . The final noise is smaller than this margin under the corresponding good event, and hence the algorithm outputs Match.
Under the Far promise, the order is reversed, and the same argument gives a negative margin of magnitude at least 19 α / 20 19\alpha/20 19 α /20 . Therefore, the algorithm outputs Far.
For the privacy proof, each maximization has sensitivity at most 7, since changing one record changes the term multiplied by 6 by at most 6 and the remaining count by at most 1. Therefore, the difference of the two maximizations has sensitivity at most 14, and allocating privacy budget ε / 2 \varepsilon/2 ε /2 to its release requires noise Lap ( 14 / ( ε / 2 ) ) = Lap ( 28 / ε ) \operatorname{Lap}(14/(\varepsilon/2)) = \operatorname{Lap}(28/\varepsilon) Lap ( 14/ ( ε /2 )) = Lap ( 28/ ε ) . □
Corollary 3.13 (Mixed labels for thresholds). There exists a universal constant C > 0 C > 0 C > 0 such that the following holds. Let X ⊆ R \mathcal{X} \subseteq\mathbb{R} X ⊆ R be a finite ordered domain with ∣ X ∣ = m ≥ 4 |\mathcal{X}| = m \ge4 ∣ X ∣ = m ≥ 4 , and suppose that each party P ∈ A , B P \in A, B P ∈ A , B holds a labeled dataset S P ∈ ( X × { ± 1 } ) n S_P \in(\mathcal{X} \times\{\pm1\})^n S P ∈ ( X × { ± 1 } ) n . Define opt H t h r m i x ( S A , S B ) = min θ ∈ R ∑ P ∈ { A , B } ∑ ( x , y ) ∈ S P 1 { h θ ( x ) ≠ y } \operatorname{opt}^{\mathrm{mix}}_{\mathcal{H}_{\mathrm{thr}}}(S_A,S_B) = \min_{\theta\in\mathbb{R}} \sum_{P \in\{A,B\}} \sum_{(x,y) \in S_P} \mathbf{1}\{h_\theta(x) \ne y\} opt H thr mix ( S A , S B ) = min θ ∈ R ∑ P ∈ { A , B } ∑ ( x , y ) ∈ S P 1 { h θ ( x ) = y } . Let ε ∈ ( 0 , 1 ] \varepsilon\in(0,1] ε ∈ ( 0 , 1 ] , β ∈ ( 0 , 1 / 3 ] \beta\in(0,1/3] β ∈ ( 0 , 1/3 ] , and t ∈ [ n ] t \in[n] t ∈ [ n ] . If
t ≥ C log log ( m + 2 ) ε log ( log log ( m + 2 ) β ε ) , t \ge C \frac{\log\log(m+2)}{\varepsilon} \log\left(\frac{\log\log(m+2)}{\beta\varepsilon}\right), t ≥ C ε log log ( m + 2 ) log ( β ε log log ( m + 2 ) ) , then there exists an ε \varepsilon ε -private tester that distinguishes opt H t h r m i x ( S A , S B ) = 0 \operatorname{opt}^{\mathrm{mix}}_{\mathcal{H}_{\mathrm{thr}}}(S_A,S_B) = 0 opt H thr mix ( S A , S B ) = 0 from opt H t h r m i x ( S A , S B ) ≥ t \operatorname{opt}^{\mathrm{mix}}_{\mathcal{H}_{\mathrm{thr}}}(S_A,S_B) \ge t opt H thr mix ( S A , S B ) ≥ t with failure probability at most β \beta β .
Proof. For P ∈ A , B P \in A, B P ∈ A , B , let S P − S_P^- S P − and S P + S_P^+ S P + denote the multisets of negative and positive points held by P P P , respectively. For P , Q ∈ A , B P,Q \in A,B P , Q ∈ A , B , write q P Q : = opt H t h r ( S P − , S Q + ) q_{PQ} := \operatorname{opt}_{\mathcal{H}_{\mathrm{thr}}}(S_P^-,S_Q^+) q P Q := opt H thr ( S P − , S Q + ) . We first observe that
max P , Q ∈ { A , B } q P Q ≤ opt H t h r m i x ( S A , S B ) ≤ ∑ P , Q ∈ { A , B } q P Q . \max_{P,Q\in\{A,B\}} q_{PQ} \le\operatorname{opt}^{\mathrm{mix}}_{\mathcal{H}_{\mathrm{thr}}}(S_A,S_B) \le\sum_{P,Q\in\{A,B\}} q_{PQ}. P , Q ∈ { A , B } max q P Q ≤ opt H thr mix ( S A , S B ) ≤ P , Q ∈ { A , B } ∑ q P Q .
The first inequality holds because each q P Q q_{PQ} q P Q optimizes the threshold error over a subset of the pooled records. For the upper bound, for each pair ( P , Q ) (P,Q) ( P , Q ) , let θ P Q \theta_{PQ} θ P Q achieve q P Q q_{PQ} q P Q and let E P Q E_{PQ} E P Q be the records it misclassifies, so ∣ E P Q ∣ = q P Q |E_{PQ}| = q_{PQ} ∣ E P Q ∣ = q P Q . Set R = ⋃ P , Q E P Q R = \bigcup_{P,Q} E_{PQ} R = ⋃ P , Q E P Q . Consider any negative point x x x and positive point y y y that do not belong to R R R . If x ∈ S P − x \in S_P^- x ∈ S P − and y ∈ S Q + y \in S_Q^+ y ∈ S Q + , then both points are correctly classified by θ P Q \theta_{PQ} θ P Q , and hence x < θ P Q ≤ y x < \theta_{PQ} \le y x < θ P Q ≤ y . Therefore, every surviving negative point is smaller than every surviving positive point, so the records outside R R R are jointly realizable by a threshold. It follows that opt H t h r m i x ( S A , S B ) ≤ ∣ R ∣ ≤ ∑ P , Q q P Q \operatorname{opt}^{\mathrm{mix}}_{\mathcal{H}_{\mathrm{thr}}}(S_A,S_B) \le|R| \le\sum_{P,Q} q_{PQ} opt H thr mix ( S A , S B ) ≤ ∣ R ∣ ≤ ∑ P , Q q P Q .
Let τ = ⌈ t / 4 ⌉ \tau= \lceil t/4\rceil τ = ⌈ t /4 ⌉ . In the Match case, every q P Q q_{PQ} q P Q is zero. In the Far case, the preceding upper bound gives ∑ P , Q q P Q ≥ t \sum_{P,Q} q_{PQ} \ge t ∑ P , Q q P Q ≥ t , and hence some q P Q q_{PQ} q P Q is at least τ \tau τ . We therefore perform four tests and return Far if any of them returns Far.
The cross-party quantities q A B q_{AB} q A B and q B A q_{BA} q B A are tested using Algorithm 2, with Bob playing the negative-holder role in the second execution. To obtain equal-length inputs, fix public points x − < min X x_- < \min\mathcal{X} x − < min X and x + > max X x_+ > \max\mathcal{X} x + > max X . In a cross-party test, the negative holder replaces every positive record by x − x_- x − , while the positive holder replaces every negative record by x + x_+ x + . The resulting datasets have size n n n , and their threshold optimum is exactly the corresponding q P Q q_{PQ} q P Q . Indeed, an optimal threshold may be chosen in ( x − , x + ] (x_-,x_+] ( x − , x + ] , in which case all dummy records are classified correctly. This preprocessing is recordwise and therefore maps neighboring labeled datasets to neighboring padded datasets. Notice that the enlarged domain has size m + 2 m+2 m + 2 .
For a same-party quantity q P P q_{PP} q P P , replacing one labeled record changes the error of every threshold by at most one, and hence q P P q_{PP} q P P has sensitivity one. Party P P P releases q ~ P P : = q P P + Lap ( 4 / ε ) \widetilde q_{PP} := q_{PP} + \operatorname{Lap}(4/\varepsilon) q P P := q P P + Lap ( 4/ ε ) and returns Far precisely when q ~ P P ≥ τ / 2 \widetilde q_{PP} \ge\tau/2 q P P ≥ τ /2 . This release is ε / 4 \varepsilon/4 ε /4 -differentially private and distinguishes q P P = 0 q_{PP}=0 q P P = 0 from q P P ≥ τ q_{PP}\ge\tau q P P ≥ τ with failure probability at most β / 4 \beta/4 β /4 , provided that τ ≥ ( 8 / ε ) log ( 4 / β ) \tau\ge(8/\varepsilon)\log(4/\beta) τ ≥ ( 8/ ε ) log ( 4/ β ) .
We run the two cross-party testers with parameters ( ε / 4 , β / 4 ) (\varepsilon/4, \beta/4) ( ε /4 , β /4 ) and allocate the same privacy and failure budgets to the two same-party tests. By composition, the resulting protocol is ε \varepsilon ε -two-party differentially private, and by a union bound its failure probability is at most β \beta β . After increasing the universal constant C C C , the assumptions of the corollary ensure that τ \tau τ satisfies the conditions of Theorem 3.12 with domain size m + 2 m+2 m + 2 and parameters ( ε / 4 , β / 4 ) (\varepsilon/4, \beta/4) ( ε /4 , β /4 ) . This proves the result. □
Halfspaces in High Dimensions
In this part we focus on the halfspace in high dimensions. More precisely, X ⊆ R d \mathcal{X} \subseteq\mathbb{R}^{d} X ⊆ R d . Also, without loss of generality, we assume Alice and Bob add a dummy dimension to their datapoints and therefore we can represent the class of halfspaces with
H h s = { h w : R d → { ± 1 } ∣ ∃ w ∈ R d s.t. h w ( x ) = sign ( ⟨ w , x ⟩ ) } . (11) \mathcal{H}_{\mathrm{hs}} = \{h_w : \mathbb{R}^{d} \to\{\pm1\} \mid\exists w \in\mathbb{R}^{d}\ \text{s.t. } h_w(x) = \operatorname{sign}(\langle w,x\rangle)\}. \tag*{(11)} H hs = { h w : R d → { ± 1 } ∣ ∃ w ∈ R d s.t. h w ( x ) = sign (⟨ w , x ⟩)} . (11)
Lower bound for half spaces
In this section, we prove an information-theoretic lower bound for H h s \mathcal{H}_{\mathrm{hs}} H hs . The argument for the lower bound is rather technical and it involves many steps. We first provide an overview of the proof and defer the formal proof to Appendix in Section D. In this informal overview, we only focus on the case of large n n n , i.e., n ≥ C m d log ( m / d ) ε n \ge C^{\frac{md\log(m/d)}{\varepsilon}} n ≥ C ε m d l o g ( m / d ) to highlight main ideas. With slight modifications, we can extend the lower bound to all values of n n n as shown in Section E and Theorem E.4 .
Overview of the lower bound proof.
The proof follows the same template as that of Theorem 3.1 . We take a binary predicate with high information complexity and embed it into the promise problem, so that the verdict of the promise problem determines the value of the predicate, and then derive the lower bound from the connection between DP and information complexity from [37 ] . For half spaces, however, constructing an embedding with a sufficiently large gap is substantially more delicate.
Our starting viewpoint is that each of Alice’s and Bob’s data points may be read as a linear constraint on w ∈ R d w \in\mathbb{R}^{d} w ∈ R d , namely ⟨ w , x ⟩ ≥ 0 \langle w,x\rangle\ge0 ⟨ w , x ⟩ ≥ 0 or ⟨ w , x ⟩ < 0 \langle w,x\rangle< 0 ⟨ w , x ⟩ < 0 . Viewed in the dual space, the problem therefore becomes a linear program with finitely many constraints: in the Match case the system is feasible, whereas in the Far case it remains infeasible even after removing any t − 1 t-1 t − 1 of the constraints. Achieving this robustness in the Far case is the main technical difficulty of the proof, and it is what leads us to use error-correcting codes.
Step 1: from inner product in a finite field to LP constraints. Let s s s be an integer determined later and q = 2 s q = 2^{s} q = 2 s . Alice holds a secret u ∈ F q k u \in\mathbb{F}_{q}^{k} u ∈ F q k and Bob holds a secret v ∈ F q k v \in\mathbb{F}_{q}^{k} v ∈ F q k , and they wish to compute
F ( u , v ) = ⨁ i = 1 k Tr F q / F 2 ( u i v i ) ∈ { 0 , 1 } . (12) F(u,v) = \bigoplus_{i=1}^{k} \operatorname{Tr}_{\mathbb{F}_{q}/\mathbb{F}_{2}}(u_i v_i) \in\{0,1\}. \tag*{(12)} F ( u , v ) = i = 1 ⨁ k Tr F q / F 2 ( u i v i ) ∈ { 0 , 1 } . (12)
F F F can be thought of as the trace of an inner product over F q \mathbb{F}_{q} F q . The internal information complexity of computing F F F is Θ ( k log q ) \Theta(k\log q) Θ ( k log q ) ; for completeness we provide a proof in Lemma B.6 . We now build a linear program whose feasibility decides Equation (12) . The main challenge here is that the computation in Equation (12) consists of non-linear components over F q \mathbb{F}_{q} F q while we want to convert it to a feasibility problem in R d \mathbb{R}^d R d . To do so, we embed the q q q elements of F q \mathbb{F}_q F q as the vertices of a convex polygon P q ⊂ R 2 P_q \subset\mathbb{R}^2 P q ⊂ R 2 lying on a parabola (so each symbol uses two real coordinates) and represent each finite-field relation by the convex hull of its satisfying assignments (for details see Figure 6 ). The resulting LP constraints come in two kinds: A public family , known to both parties and independent of u u u and v v v , performs the computation itself: it holds the variables in the right ranges, forms the k k k trace products, and accumulates their parity, with one last row requiring the parity to be zero. A private family encodes Alice’s and Bob’s secrets: each coordinate of u u u and v v v contributes exactly one constraint, which we call a pin (the naming is inspired by our way of converting elements of F q \mathbb{F}_q F q into a linear program in Figure 6 ). Once all 2 k 2k 2 k of these are present, the fixed rows propagate the computation and the program is feasible if and only if F ( u , v ) = 0 F(u,v)=0 F ( u , v ) = 0 . We give the details of constructing such a program in Section D. Reading this back through Definition 1.2, we obtain a Match instance whenever F ( u , v ) = 0 F(u,v)=0 F ( u , v ) = 0 and a Far instance whenever F ( u , v ) = 1 F(u,v)=1 F ( u , v ) = 1 . This feasibility equivalence alone certifies only a Far gap of one. Repeating each linear constraint L L L times guarantees opt H h s ≥ L \operatorname{opt}_{\mathcal{H}_{\mathrm{hs}}} \ge L opt H hs ≥ L , but the encoding diameter of Lemma 2.2 rises to 2 k L 2kL 2 k L . Then, by the connection of the IC and encoding diameter from Lemma 2.2, we obtain only t = Ω ( log q / ε ) t=\Omega(\log q/\varepsilon) t = Ω ( log q / ε ) . The shortcoming of the simple repetition is that the Far gap grows only with the replication L L L , while the diameter grows with k L kL k L . Can we instead design a program that is already robust to deleting D ≥ 1 D \ge1 D ≥ 1 of its private pins (i.e., erasing D D D symbols) before any replication?
Figure 6. Encdoing of the field’s element into R \mathbb{R} R
Step 2: turning code robustness into LP rigidity. What we need is that, as long as the public constraints remain enforced, removing fewer than D D D pins still leaves the final answer intact. We therefore encode u u u and v v v using a public systematic linear code with generator G = [ I k ∣ A ] ∈ F q k × K G=[I_k\mid A]\in\mathbb{F}_q^{k\times K} G = [ I k ∣ A ] ∈ F q k × K where K > k K>k K > k . Let c = u G c=uG c = u G and c ′ = v G c'=vG c ′ = v G ; since the code is systematic, the first k k k coordinates of c c c and c ′ c' c ′ are u u u and v v v themselves, so the trace products of Equation (12) are still formed from these coordinates alone. Three properties are needed from the code and its parity-check representation: constant rate, so that k = Θ ( K ) k=\Theta(K) k = Θ ( K ) ; linear LP-erasure rigidity , meaning the LP tolerates up to D = Θ ( K ) D=\Theta(K) D = Θ ( K ) removed pins; and constant check degree, for a reason that will become clear below.
The key subtlety is that local parity-check constraints only enforce a convex relaxation of the code, which can admit fractional solutions even when the code has large minimum distance. We therefore use a code whose parity-check graph is a constant-degree unique-neighbor expander [21 ] : whenever fewer than D = Θ ( K ) D=\Theta(K) D = Θ ( K ) pins are removed, any nonempty set of unpinned coordinates has an adjacent check touching it in a single coordinate, whose remaining pinned coordinates force that variable back to its true symbol (Lemma C.2 ). Thus the public constraints still uniquely force c c c and c ′ c' c ′ . Two counts govern the size of the construction, and separating them is what makes a large alphabet affordable. First, since each F q \mathbb{F}_q F q symbol is embedded in R 2 \mathbb{R}^2 R 2 , the program uses d L P : = 4 K + 2 k + 1 = Θ ( K ) d_{\mathrm{LP}}:=4K+2k+1=\Theta(K) d LP := 4 K + 2 k + 1 = Θ ( K ) variables (2 K 2K 2 K per codeword, k k k trace, and k + 1 k+1 k + 1 parity variables), independent of q q q , allowing us to choose K = Θ ( d ) K=\Theta(d) K = Θ ( d ) . Second, each distinct LP inequality corresponds to a point in the public data space X \mathcal{X} X of size m m m . Because each check involves only ρ = O ( 1 ) \rho=O(1) ρ = O ( 1 ) symbols, its convex hull in R 2 ρ \mathbb{R}^{2\rho} R 2 ρ is described by q O ( ρ 2 ) = q O ( 1 ) q^{O(\rho^2)}=q^{O(1)} q O ( ρ 2 ) = q O ( 1 ) inequalities; across all O ( K ) O(K) O ( K ) checks and computation gates, this yields at most C K q a CKq^a C K q a distinct records for universal constants C , a C,a C , a (a replication increases multiplicities, not distinct records). The alphabet therefore costs records but never dimensions, which is what will let us take q q q large.
Step 3: producing the Far gap.
Converting each LP inequality into a labeled data point, we repeat each private pin L L L times and each public constraint J J J times. When F ( u , v ) = 0 F(u,v)=0 F ( u , v ) = 0 , all constraints are simultaneously satisfiable. When F ( u , v ) = 1 F(u,v)=1 F ( u , v ) = 1 , removing fewer than min { J , D L } \min\{J,DL\} min { J , D L } constraints leaves at least one copy of every public constraint intact and removes all L L L copies of fewer than D D D private pins; by Step 2, the remaining LP remains infeasible. Hence
F ( u , v ) = 0 ⟹ opt H h s ( S A , S B ) = 0 , F ( u , v ) = 1 ⟹ opt H h s ( S A , S B ) ≥ min { J , D L } . F(u,v)=0 \;\Longrightarrow\; \operatorname{opt}_{\mathcal{H}_{\mathrm{hs}}}(S_A,S_B)=0,\qquad F(u,v)=1 \;\Longrightarrow\; \operatorname{opt}_{\mathcal{H}_{\mathrm{hs}}}(S_A,S_B)\ge\min\{J,DL\}. F ( u , v ) = 0 ⟹ opt H hs ( S A , S B ) = 0 , F ( u , v ) = 1 ⟹ opt H hs ( S A , S B ) ≥ min { J , D L } .
For a hypothetical t t t -tester, take J = ⌈ t ⌉ J=\lceil t\rceil J = ⌈ t ⌉ and L = ⌈ t / D ⌉ L=\lceil t/D\rceil L = ⌈ t / D ⌉ , which guarantees a Far gap of at least t t t . Since only the K K K private pins depend on each party’s secret, the encoding diameter is Δ A , Δ B ≤ K L = O ( t + K ) \Delta_A,\Delta_B\le KL=O(t+K) Δ A , Δ B ≤ K L = O ( t + K ) (as D = Θ ( K ) D=\Theta(K) D = Θ ( K ) ). The information lower bound and Lemma 2.2 then give
Ω ( K log q ) ≤ O ( ε ( t + K ) ) . \Omega(K\log q)\le O(\varepsilon(t+K)). Ω ( K log q ) ≤ O ( ε ( t + K )) .
which forces t = Ω ( K log q / ε ) t=\Omega(K\log q/\varepsilon) t = Ω ( K log q / ε ) for sufficiently small ε \varepsilon ε . Finally, we choose K = Θ ( d ) K=\Theta(d) K = Θ ( d ) and the largest power of two q q q satisfying B K q a log 2 q ≤ m / 8 BKq^a\log_2q\le m/8 B K q a log 2 q ≤ m /8 for a universal constant B B B . This keeps the number of distinct records at most m m m and gives log q = Θ ( log ( m / d ) ) \log q=\Theta(\log(m/d)) log q = Θ ( log ( m / d )) , yielding t = Ω ( d log ( m / d ) ε ) t=\Omega\left(\frac{d\log(m/d)}{\varepsilon}\right) t = Ω ( ε d l o g ( m / d ) ) . Moreover, when t = O ( K log q / ε ) t=O(K\log q/\varepsilon) t = O ( K log q / ε ) , the total number of repeated records is O ( K q a J + K L ) = O ( m d / ε ) O(Kq^aJ+KL)=O(md/\varepsilon) O ( K q a J + K L ) = O ( m d / ε ) , which fits within datasets of size n ≥ C m d / ε n\ge Cmd/\varepsilon n ≥ C m d / ε after padding with input-independent records.
Theorem 4.1 (Lower bound for halfspaces). There are universal constants c , C , C 0 , ε 0 > 0 c,C,C_0,\varepsilon_0>0 c , C , C 0 , ε 0 > 0 and d 0 ∈ N d_0\in\mathbb{N} d 0 ∈ N such that, for every d , m ∈ N d,m\in\mathbb{N} d , m ∈ N with d ≥ d 0 d\ge d_0 d ≥ d 0 and m ≥ C 0 d m\ge C_0d m ≥ C 0 d , there is a public domain X d , m ⊆ R d \mathcal{X}_{d,m}\subseteq\mathbb{R}^d X d , m ⊆ R d with ∣ X d , m ∣ = m |\mathcal{X}_{d,m}|=m ∣ X d , m ∣ = m such that for every 0 < ε ≤ ε 0 0<\varepsilon\le\varepsilon_0 0 < ε ≤ ε 0 , every integer n ≥ C 0 m d log ( m / d ) ε n\ge C_0\frac{md\log(m/d)}{\varepsilon} n ≥ C 0 ε m d l o g ( m / d ) , and every resolution t ∈ [ n ] t\in[n] t ∈ [ n ] , an ε \varepsilon ε -private tester for H h s \mathcal{H}_{\mathrm{hs}} H hs at dataset size n n n over X d , m \mathcal{X}_{d,m} X d , m must satisfy
t ≥ c d log ( m / d ) ε . t\ge c\frac{d\log(m/d)}{\varepsilon}. t ≥ c ε d log ( m / d ) . The domain is chosen independently of n , t , ε n,t,\varepsilon n , t , ε .
Proof. Take the fixed domain from Lemma E.3 , and let R = Θ ( d ) R=\Theta(d) R = Θ ( d ) be its largest block length. Let B , a B,a B , a be the constants in Lemma E.1 . Among powers of two q ≥ 2 q\ge2 q ≥ 2 , choose the largest satisfying
B R q a log 2 q ≤ m / 8. (13) BRq^a\log_2q\le m/8. \tag*{(13)} B R q a log 2 q ≤ m /8. (13)
For C 0 C_0 C 0 sufficiently large, q = 2 q=2 q = 2 is admissible. Every such alphabet is included in X d , m \mathcal{X}_{d,m} X d , m , since q a ≤ m / ( 8 B R ) q^a\le m/(8BR) q a ≤ m / ( 8 B R ) .
Put s = log 2 q ≥ 1 s=\log_2q\ge1 s = log 2 q ≥ 1 . Maximality gives
m 8 B R < ( 2 q ) a ( s + 1 ) ≤ 2 a + 1 q a s . \frac{m}{8BR}<(2q)^a(s+1)\le2^{a+1}q^as. 8 B R m < ( 2 q ) a ( s + 1 ) ≤ 2 a + 1 q a s .
Taking logarithms and using log 2 s ≤ s \log_2s\le s log 2 s ≤ s shows that s = Ω ( log ( m / R ) ) = Ω ( log ( m / d ) ) s=\Omega(\log(m/R))=\Omega(\log(m/d)) s = Ω ( log ( m / R )) = Ω ( log ( m / d )) . The reverse bound follows from (12) and R = Θ ( d ) R=\Theta(d) R = Θ ( d ) , so s = Θ ( log ( m / d ) ) s=\Theta(\log(m/d)) s = Θ ( log ( m / d )) .
By Lemma E.1 , the construction at scale ( R , q ) (R,q) ( R , q ) forces t ≥ b R s / ε t\ge bRs/\varepsilon t ≥ b R s / ε whenever n ε ≥ A R 2 q a s n\varepsilon\ge AR^2q^as n ε ≥ A R 2 q a s , for universal A , b > 0 A,b>0 A , b > 0 . The retuned alphabet satisfies
A R 2 q a s ≤ A 8 B R m ≤ C d m AR^2q^as\le\frac{A}{8B}Rm\le Cdm A R 2 q a s ≤ 8 B A R m ≤ C d m
for a sufficiently large universal C C C . Thus the assumed sample size suffices, with each dataset padded to exactly n n n as in Lemma E.1 . Since R = Θ ( d ) R=\Theta(d) R = Θ ( d ) and s = Θ ( log ( m / d ) ) s=\Theta(\log(m/d)) s = Θ ( log ( m / d )) , we obtain the claimed lower bound. □
Remark 4.2. The surprising behavior of this lower bound compared to the optimal gap for thresholds in Theorems 3.1 and 3.12 is that the dependence on m m m is exponentially larger, i.e., log log ( m ) \log\log(m) log log ( m ) for threshold and log ( m ) \log(m) log ( m ) for half spaces. Also, t t t exhibits (at least) a linear dependence on d d d . ⊳
Algorithm for Half Spaces
Structural Results and Tools
As our lower bound in the Theorem 4.1 shows we need to assume that X = { x 1 , … , x m } ⊂ R d \mathcal{X} = \{x_1, \ldots, x_m\} \subset\mathbb{R}^d X = { x 1 , … , x m } ⊂ R d is a finite set. Let ∣ X ∣ = m |\mathcal{X}| = m ∣ X ∣ = m . Towards developing our algorithm, we mainly focus on the dual space: consider the dual space denoted by W = R d \mathcal{W} = \mathbb{R}^d W = R d where we map each h w ∈ H h s h_w \in\mathcal{H}_{\mathrm{hs}} h w ∈ H hs to w ∈ R d w \in\mathbb{R}^d w ∈ R d .
The cardinality of W \mathcal{W} W is unbounded. However, we only need to consider the projection of X \mathcal{X} X using H h s \mathcal{H}_{\mathrm{hs}} H hs , which means that two w ∼ w ′ w \sim w' w ∼ w ′ are equivalent if and only if the predictions of h w h_w h w and h w ′ h_{w'} h w ′ over X \mathcal{X} X be the same. The next lemma shows that considering this equivalent class , we can reduce W \mathcal{W} W to a discrete set. The proof is an application of Sauer–Shelah lemma [44 , 45 ] .
Lemma 4.3 (Finite dual discretization). Let X = { x 1 , … , x m } ⊆ R d \mathcal{X} = \{x_1, \ldots, x_m\} \subseteq\mathbb{R}^d X = { x 1 , … , x m } ⊆ R d . There exists a finite set W d i s c ⊆ R d \mathcal{W}_{\mathrm{disc}} \subseteq\mathbb{R}^d W disc ⊆ R d such that, for every w ∈ R d w \in\mathbb{R}^d w ∈ R d , some w ′ ∈ W d i s c w' \in\mathcal{W}_{\mathrm{disc}} w ′ ∈ W disc satisfies h w ( x i ) = h w ′ ( x i ) h_w(x_i) = h_{w'}(x_i) h w ( x i ) = h w ′ ( x i ) for every i ∈ [ m ] i \in[m] i ∈ [ m ] . Moreover, ∣ W d i s c ∣ ≤ ( 2 m ) d |\mathcal{W}_{\mathrm{disc}}| \le(2m)^d ∣ W disc ∣ ≤ ( 2 m ) d .
For each w ∈ W d i s c w \in\mathcal{W}_{\mathrm{disc}} w ∈ W disc , define the local error of h h h on Alice and Bob datasets as
f A ( w ) = ∑ x ∈ S A 1 { h w ( x ) ≠ − 1 } f B ( w ) = ∑ x ∈ S B 1 { h w ( x ) ≠ 1 } , f_A(w) = \sum_{x \in S_A} 1\{h_w(x) \ne-1\} \qquad f_B(w) = \sum_{x \in S_B} 1\{h_w(x) \ne1\}, f A ( w ) = x ∈ S A ∑ 1 { h w ( x ) = − 1 } f B ( w ) = x ∈ S B ∑ 1 { h w ( x ) = 1 } ,
and using this notation, we have opt H h s = min w ∈ W d i s c { f A ( w ) + f B ( w ) } \operatorname{opt}_{\mathcal{H}_{\mathrm{hs}}} = \min_{w \in\mathcal{W}_{\mathrm{disc}}} \{f_A(w) + f_B(w)\} opt H hs = min w ∈ W disc { f A ( w ) + f B ( w )} . For every k ∈ [ n ] k \in[n] k ∈ [ n ] , we also define the set of local low-error classifiers on Alice and Bob’s side as
G A k = { w ∈ W d i s c : f A ( w ) ≤ k } G B k = { w ∈ W d i s c : f B ( w ) ≤ k } . G_A^k = \{w \in\mathcal{W}_{\mathrm{disc}} : f_A(w) \le k\} \qquad G_B^k = \{w \in\mathcal{W}_{\mathrm{disc}} : f_B(w) \le k\}. G A k = { w ∈ W disc : f A ( w ) ≤ k } G B k = { w ∈ W disc : f B ( w ) ≤ k } .
Note that G A k G_A^k G A k and G B k G_B^k G B k as sets live in the dual space. Also, since W d i s c \mathcal{W}_{\mathrm{disc}} W disc is a finite set, these sets are generally non-convex in R d \mathbb{R}^d R d . However, when we consider the convex hull of them, we have an important property.
Lemma 4.4. Fix n , d ∈ N n, d \in\mathbb{N} n , d ∈ N , datasets S A , S B ∈ X n S_A, S_B \in\mathcal{X}^n S A , S B ∈ X n , and t ∈ [ n ] t \in[n] t ∈ [ n ] . Suppose that opt H h s ( S A , S B ) ≥ t \operatorname{opt}_{\mathcal{H}_{\mathrm{hs}}}(S_A, S_B) \ge t opt H hs ( S A , S B ) ≥ t . Then, for every k ∈ N 0 k \in\mathbb{N}_0 k ∈ N 0 satisfying k ≤ t − 1 2 ( d + 1 ) k \le\frac{t-1}{2(d+1)} k ≤ 2 ( d + 1 ) t − 1 , we have conv ( G A k ) ∩ conv ( G B k ) = ∅ \operatorname{conv}(G_A^k) \cap\operatorname{conv}(G_B^k) = \varnothing conv ( G A k ) ∩ conv ( G B k ) = ∅ .
Proof. Suppose toward a contradiction that there exists v ∈ conv ( G A k ) ∩ conv ( G B k ) v \in\operatorname{conv}(G_A^k) \cap\operatorname{conv}(G_B^k) v ∈ conv ( G A k ) ∩ conv ( G B k ) . By Carathéodory’s theorem Lemma B.3 , there exist a 1 , … , a r ∈ G A k a_1, \ldots, a_r \in G_A^k a 1 , … , a r ∈ G A k , where r ≤ d + 1 r \le d + 1 r ≤ d + 1 , such that v v v is a convex combination of a 1 , … , a r a_1, \ldots, a_r a 1 , … , a r . If v v v misclassifies a point in S A S_A S A , then at least one a i a_i a i also misclassifies that point. Consequently,
f A ( v ) ≤ ∑ i = 1 r f A ( a i ) ≤ ( d + 1 ) k . f_A(v) \le\sum_{i=1}^{r} f_A(a_i) \le(d+1)k. f A ( v ) ≤ i = 1 ∑ r f A ( a i ) ≤ ( d + 1 ) k .
Applying the same argument to a representation of v v v as a convex combination of at most d + 1 d + 1 d + 1 points from G B k G_B^k G B k gives f B ( v ) ≤ ( d + 1 ) k f_B(v) \le(d + 1)k f B ( v ) ≤ ( d + 1 ) k . Therefore,
f A ( v ) + f B ( v ) ≤ 2 ( d + 1 ) k ≤ t − 1 , f_A(v) + f_B(v) \le2(d+1)k \le t - 1, f A ( v ) + f B ( v ) ≤ 2 ( d + 1 ) k ≤ t − 1 ,
contradicting opt H h s ( S A , S B ) ≥ t \operatorname{opt}_{\mathcal{H}_{\mathrm{hs}}}(S_A, S_B) \ge t opt H hs ( S A , S B ) ≥ t . □
Our algorithm employes a geometric tool that is called container . It is a special case of the cover where we also assume that the cover includes the set. We use the following result from [6 , 16 ] .
Lemma 4.5 (Halfspace containers [6 , 16 ] ). There exists a universal constant c > 0 c > 0 c > 0 such that the following holds. For every finite set S ⊆ R d S \subseteq\mathbb{R}^{d} S ⊆ R d and every η ∈ ( 0 , 1 ) \eta\in(0,1) η ∈ ( 0 , 1 ) , there exists a family C ( S , η ) ⊆ 2 S \mathcal{C}(S,\eta) \subseteq2^{S} C ( S , η ) ⊆ 2 S satisfying
∣ C ( S , η ) ∣ ≤ ( 2 η ) c d . |\mathcal{C}(S,\eta)| \le\left(\frac{2}{\eta}\right)^{cd}. ∣ C ( S , η ) ∣ ≤ ( η 2 ) c d . Moreover, for every affine halfspace H ⊆ R d H \subseteq\mathbb{R}^{d} H ⊆ R d , there exists C ∈ C ( S , η ) C \in\mathcal{C}(S,\eta) C ∈ C ( S , η ) such that
H ∩ S ⊆ C and ∣ C ∖ H ∣ ≤ η ∣ S ∣ . H \cap S \subseteq C \qquad\text{and} \qquad|C \setminus H| \le\eta|S|. H ∩ S ⊆ C and ∣ C ∖ H ∣ ≤ η ∣ S ∣.
Main Algorithm
In this section, we present the algorithm for half spaces. We begin with providing an overview of the technical ideas.
Overview of the algorithm. Our algorithm is a private elimination procedure over the finite dual space W d i s c W_{\mathrm{disc}} W disc . At round r r r , the parties maintain a public set W r W_r W r of candidate halfspaces and attempt to replace it by a substantially smaller set. Ideally, a party would find a halfspace in the dual space that contains all of its low-error candidates but at most half of W r W_r W r . Such a halfspace defines a safe cut, but the family of possible halfspaces is too large to search privately. The container lemma in Lemma 4.5 replaces these halfspace cuts by a small public family: whenever such a halfspace exists, the family contains a set covering it and containing at most 3 ∣ W r ∣ / 4 3|W_r|/4 3∣ W r ∣/4 candidates.
For P ∈ { A , B } P \in\{A,B\} P ∈ { A , B } , let G P k ( W r ) G^{k}_{P}(W_r) G P k ( W r ) denote the candidates in W r W_r W r with local error at most k k k . We assign each container a low-sensitivity cost measuring the quality of the best halfspace candidate it excludes; its cost is zero exactly when it contains all of G P k ( W r ) G^{k}_{P}(W_r) G P k ( W r ) . Alice and Bob can therefore use the exponential mechanism to privately search the public container family for a safe shrinking cut.
The high level goal is as follows: The two promise cases interact with this mechanism in opposite ways, and the stopping rule reads off which occurred: a Match instance keeps W r W_r W r nonempty forever, while a Far instance is driven to W r = ∅ W_r = \varnothing W r = ∅ within log ( ∣ W d i s c ∣ ) \log(|W_{\mathrm{disc}}|) log ( ∣ W disc ∣ ) rounds, thanks to Lemma 4.3 that shows ∣ W d i s c ∣ |W_{\mathrm{disc}}| ∣ W disc ∣ is finite.
Far case. By Lemma 4.4 , conv ( G A k ( W r ) ) \operatorname{conv}(G^{k}_{A}(W_r)) conv ( G A k ( W r )) and conv ( G B k ( W r ) ) \operatorname{conv}(G^{k}_{B}(W_r)) conv ( G B k ( W r )) are disjoint. A separating hyperplane therefore places one party’s low-error region in a halfspace containing at most half of W r W_r W r . By Lemma 4.5 , that party has a zero-cost real container of size at most 3 ∣ W r ∣ / 4 3|W_r|/4 3∣ W r ∣/4 in the public menu. On a high-probability good event that we define later, each party selects a real container, not necessarily the particular zero-cost one. Since every real member of the menu has size at most 3 ∣ W r ∣ / 4 3|W_r|/4 3∣ W r ∣/4 , the active set shrinks by a constant factor. See Figure 4 .
Figure 4. The halfspace promise problem viewed in the dual space. Our algorithm works in the dual space and tests which case holds.
Match case. Here the mechanism above can fail: the two low-error regions may overlap and be scattered across W r W_r W r , so neither is guaranteed to lie in a halfspace holding at most half the candidates. We therefore add a dummy action ⊥ \perp ⊥ to the menu, which proposes no cut and lets a party abstain when no safe container exists; both parties abstaining is the protocol’s signal to output Match. The cost of ⊥ \perp ⊥ is fixed at k / 2 k/2 k /2 , wedged between the cost of a safe container (0) and that of any container excluding a consistent classifier (≥ k + 1 \ge k+1 ≥ k + 1 ). This calibration makes abstention the preferred move exactly when no safe cut exists. Concretely, fix a classifier h ⋆ h^\star h ⋆ consistent with both datasets. On a high probability event, that we define in the proof, each party either selects ⊥ \perp ⊥ or selects a real container that still contains w ⋆ w^\star w ⋆ , since any container excluding w ⋆ w^\star w ⋆ costs at least k + 1 > k / 2 k+1 > k/2 k + 1 > k /2 . If both select ⊥ \perp ⊥ the protocol outputs Match; otherwise w ⋆ w^\star w ⋆ survives into W r + 1 W_{r+1} W r + 1 . If the protocol reaches the log ( ∣ W d i s c ∣ \log(|\mathcal{W}_{\mathrm{disc}}| log ( ∣ W disc ∣ step, then the resulting active set is not empty since it contains w ⋆ w^\star w ⋆ . See Figure 4 .
Algorithm 3 Data Valuation for Half Spaces: A v a l − h s \mathcal{A}_{\mathrm{val-hs}} A val − hs
Input: Alice holds S A ∈ X n S_A \in\mathcal{X}^n S A ∈ X n , Bob holds S B ∈ X n S_B \in\mathcal{X}^n S B ∈ X n ; parameters ε \varepsilon ε , β \beta β .
Output: Far or Match.
W 0 ← W d i s c \mathcal{W}_0 \gets\mathcal{W}_{\mathrm{disc}} W 0 ← W disc
Q ← ⌈ log 4 / 3 ∣ W d i s c ∣ ⌉ + 2 Q \gets\lceil\log_{4/3}|\mathcal{W}_{\mathrm{disc}}|\rceil+ 2 Q ← ⌈ log 4/3 ∣ W disc ∣ ⌉ + 2
α ← 2 Q ε ( ln ( 8 c d + 1 ) + ln 2 Q β ) \alpha\gets\frac{2Q}{\varepsilon}\left(\ln(8^{cd}+1)+\ln\frac{2Q}{\beta}\right) α ← ε 2 Q ( ln ( 8 c d + 1 ) + ln β 2 Q ) ▹ \triangleright ▹ c c c is a universal constant.
k ← ⌊ t − 1 2 ( d + 1 ) ⌋ k \gets\left\lfloor\frac{t-1}{2(d+1)}\right\rfloor k ← ⌊ 2 ( d + 1 ) t − 1 ⌋ ▹ \triangleright ▹ see Lemma 4.4 .
ε 0 ← ε / Q \varepsilon_0 \gets\varepsilon/Q ε 0 ← ε / Q .
for r = 0 , … , Q − 1 r=0,\ldots,Q-1 r = 0 , … , Q − 1 do
Construct container family C ( W r , 1 4 ) \mathcal{C}(\mathcal{W}_r,\frac14) C ( W r , 4 1 ) and define the public menu
R r = { C ∈ C ( W r , 1 4 ) ∣ ∣ C ∣ ≤ 3 4 ∣ W r ∣ } ∪ { ⊥ } \mathcal{R}_r=\left\{C\in\mathcal{C}(\mathcal{W}_r,\frac14)\mid|C|\le\frac34|\mathcal{W}_r|\right\}\cup\{\perp\} R r = { C ∈ C ( W r , 4 1 ) ∣ ∣ C ∣ ≤ 4 3 ∣ W r ∣ } ∪ { ⊥ } .
▹ \triangleright ▹ See Lemma 4.5 .
For P ∈ { A , B } P\in\{A,B\} P ∈ { A , B } and C ∈ W r C\in\mathcal{W}_r C ∈ W r , define q P , W r ( C ; S P ) ← { max h ∈ W r ∖ C ( k + 1 − f P ( h ) ) + , C ≠ ⊥ , k 2 , C = ⊥ . q_{P,\mathcal{W}_r}(C;S_P)\gets\begin{cases} \max_{h\in\mathcal{W}_r\setminus C}(k+1-f_P(h))_+, & C\ne\perp,\\ \frac{k}{2}, & C=\perp. \end{cases} q P , W r ( C ; S P ) ← { max h ∈ W r ∖ C ( k + 1 − f P ( h ) ) + , 2 k , C = ⊥ , C =⊥ .
Alice samples C A , r ∼ EM ( q A , W r , R r , S A , ε 0 ) C_{A,r}\sim\operatorname{EM}(q_{A,\mathcal{W}_r},\mathcal{R}_r,S_A,\varepsilon_0) C A , r ∼ EM ( q A , W r , R r , S A , ε 0 ) and releases it. ▹ \triangleright ▹ See Lemma B.4 .
Bob samples C B , r ∼ EM ( q B , W r , R r , S B , ε 0 ) C_{B,r}\sim\operatorname{EM}(q_{B,\mathcal{W}_r},\mathcal{R}_r,S_B,\varepsilon_0) C B , r ∼ EM ( q B , W r , R r , S B , ε 0 ) and releases it. ▹ \triangleright ▹ See Lemma B.4 .
if C A , r = C B , r = ⊥ C_{A,r}=C_{B,r}=\perp C A , r = C B , r =⊥ then
return Match
if C A , r ≠ ⊥ C_{A,r}\ne\perp C A , r = ⊥ then
W r + 1 ← C A , r \mathcal{W}_{r+1}\gets C_{A,r} W r + 1 ← C A , r
else if C A , r = ⊥ C_{A,r}=\perp C A , r =⊥ and C B , r ≠ ⊥ C_{B,r}\ne\perp C B , r = ⊥ then
W r + 1 ← C B , r \mathcal{W}_{r+1}\gets C_{B,r} W r + 1 ← C B , r
return Far if W Q = ∅ \mathcal{W}_Q=\varnothing W Q = ∅ ; otherwise return Match.
Theorem 4.7. Let c c c be a universal constant. Fix a finite data space X ⊆ R d \mathcal{X}\subseteq\mathbb{R}^d X ⊆ R d with ∣ X ∣ = m |\mathcal{X}|=m ∣ X ∣ = m , and let ε ∈ ( 0 , 1 ] \varepsilon\in(0,1] ε ∈ ( 0 , 1 ] , β ∈ ( 0 , 1 ) \beta\in(0,1) β ∈ ( 0 , 1 ) , and t ∈ [ n ] t\in[n] t ∈ [ n ] . Let
t ≥ c ( d 2 log ( 2 m ) ε ( d + log d log ( 2 m ) β ) ) . t \ge c\left(\frac{d^2\log(2m)}{\varepsilon}\left(d+\log\frac{d\log(2m)}{\beta}\right)\right). t ≥ c ( ε d 2 log ( 2 m ) ( d + log β d log ( 2 m ) ) ) . then A v a l − h s \mathcal{A}_{\mathrm{val-hs}} A val − hs in Algorithm 3 is an ε \varepsilon ε -private tester for the t t t -promise problem over H h s \mathcal{H}_{\mathrm{hs}} H hs with success probability at least 1 − β 1-\beta 1 − β .
Proof. The privacy proof is straightforward: in each round, Alice and Bob use exponential mechanism with ε 0 \varepsilon_0 ε 0 . The maximum number of rounds is Q Q Q , therefore, the composition theorem of pure-DP shows that the output satisfies ε \varepsilon ε two-view DP. Next we turn to the utility proof.
First, we define an event and then the rest of the analysis is based on assuming the event holds. For P ∈ { A , B } P \in\{A,B\} P ∈ { A , B } and r ∈ { 0 , … , Q − 1 } r \in\{0,\ldots,Q-1\} r ∈ { 0 , … , Q − 1 } , lets define an event that shows the protocol reaches round r r r and the exponential mechanism is accurate. More precisely,
B P , r : = { r < T } ∩ { q P , W r ( C P , r ; S P ) − min C ∈ R r q P , W r ( C ; S P ) > α } \mathcal{B}_{P,r}:=\{r<T\}\cap\left\{q_{P,\mathcal{W}_r}(C_{P,r};S_P)-\min_{C\in\mathcal{R}_r}q_{P,\mathcal{W}_r}(C;S_P)>\alpha\right\} B P , r := { r < T } ∩ { q P , W r ( C P , r ; S P ) − C ∈ R r min q P , W r ( C ; S P ) > α }
Then define
G : = ⋂ r = 0 Q − 1 ( B A , r c ∩ B B , r c ) . (14) \mathcal{G}:=\bigcap_{r=0}^{Q-1}\left(\mathcal{B}_{A,r}^{c}\cap\mathcal{B}_{B,r}^{c}\right). \tag*{(14)} G := r = 0 ⋂ Q − 1 ( B A , r c ∩ B B , r c ) . (14)
Thus, on G \mathcal{G} G , every exponential-mechanism call that is actually executed has additive cost error at most α \alpha α . It is easy to see that by the union bound and Lemma B.4 , we have Pr ( G c ) ≤ ∑ r = 0 Q − 1 ( Pr ( B A , r ) + Pr ( B B , r ) ) ≤ 2 Q ⋅ β 2 Q = β \Pr(\mathcal{G}^{c})\le\sum_{r=0}^{Q-1}(\Pr(\mathcal{B}_{A,r})+\Pr(\mathcal{B}_{B,r}))\le2Q\cdot\frac{\beta}{2Q}=\beta Pr ( G c ) ≤ ∑ r = 0 Q − 1 ( Pr ( B A , r ) + Pr ( B B , r )) ≤ 2 Q ⋅ 2 Q β = β . For the rest of the proof, we assume G \mathcal{G} G holds.
Case 1: Far holds. We first assume that opt H h s ( S A , S B ) ≥ t \operatorname{opt}_{\mathcal{H}_{\mathrm{hs}}}(S_A,S_B)\ge t opt H hs ( S A , S B ) ≥ t . Consider any round r r r reached by the protocol with W r ≠ ∅ \mathcal{W}_r\ne\varnothing W r = ∅ . Since k = ⌊ t − 1 2 ( d + 1 ) ⌋ k=\left\lfloor\frac{t-1}{2(d+1)}\right\rfloor k = ⌊ 2 ( d + 1 ) t − 1 ⌋ , Lemma 4.4 gives conv ( G A k ( W r ) ) ∩ conv ( G B k ( W r ) ) = ∅ \operatorname{conv}(G_A^k(\mathcal{W}_r))\cap\operatorname{conv}(G_B^k(\mathcal{W}_r))=\varnothing conv ( G A k ( W r )) ∩ conv ( G B k ( W r )) = ∅ .
If both low-error sets are nonempty, strict separation gives opposite affine halfspaces containing conv ( G A k ( W r ) ) \operatorname{conv}(G_A^k(\mathcal{W}_r)) conv ( G A k ( W r )) and conv ( G B k ( W r ) ) \operatorname{conv}(G_B^k(\mathcal{W}_r)) conv ( G B k ( W r )) , respectively. We may perturb the separating hyperplane so that its boundary avoids the finite set W r \mathcal{W}_r W r . The two halfspaces then partition W r \mathcal{W}_r W r , and one contains at most ∣ W r ∣ / 2 |\mathcal{W}_r|/2 ∣ W r ∣/2 points. Thus, for some P ∈ { A , B } P\in\{A,B\} P ∈ { A , B } , there is an affine halfspace H H H such that G P k ( W r ) ⊆ H ∩ W r G_P^k(\mathcal{W}_r)\subseteq H\cap\mathcal{W}_r G P k ( W r ) ⊆ H ∩ W r and ∣ H ∩ W r ∣ ≤ ∣ W r ∣ / 2 |H\cap\mathcal{W}_r|\le|\mathcal{W}_r|/2 ∣ H ∩ W r ∣ ≤ ∣ W r ∣/2 .
By Lemma 4.5 , there is a container C ⋆ ∈ C ( W r , 1 / 4 ) C^\star\in\mathcal{C}(\mathcal{W}_r,1/4) C ⋆ ∈ C ( W r , 1/4 ) containing H ∩ W r H\cap\mathcal{W}_r H ∩ W r and satisfying ∣ C ⋆ ∣ ≤ 3 ∣ W r ∣ / 4 |C^\star|\le3|\mathcal{W}_r|/4 ∣ C ⋆ ∣ ≤ 3∣ W r ∣/4 . Hence C ⋆ ∈ R r C^\star\in\mathcal{R}_r C ⋆ ∈ R r , G P k ( W r ) ⊆ C ⋆ G_P^k(\mathcal{W}_r)\subseteq C^\star G P k ( W r ) ⊆ C ⋆ , and therefore q P , W r ( C ⋆ ; S P ) = 0 q_{P,\mathcal{W}_r}(C^\star;S_P)=0 q P , W r ( C ⋆ ; S P ) = 0 . On the event G \mathcal{G} G , the container released by party P P P consequently satisfies q P , W r ( C P , r ; S P ) ≤ α q_{P,\mathcal{W}_r}(C_{P,r};S_P)\le\alpha q P , W r ( C P , r ; S P ) ≤ α . Since q P , W r ( ⊥ ; S P ) = k / 2 > α q_{P,\mathcal{W}_r}(\perp;S_P)=k/2>\alpha q P , W r ( ⊥ ; S P ) = k /2 > α , it follows that C P , r ≠ ⊥ C_{P,r}\ne\perp C P , r = ⊥ . Thus, the protocol cannot return Match in this round.
The protocol therefore either returns Far because the active set is empty or replaces W r \mathcal{W}_r W r by a real member of R r \mathcal{R}_r R r . In the latter case, ∣ W r + 1 ∣ ≤ 3 ∣ W r ∣ / 4 |\mathcal{W}_{r+1}|\le3|\mathcal{W}_r|/4 ∣ W r + 1 ∣ ≤ 3∣ W r ∣/4 . Consequently, if the protocol completes all Q Q Q rounds, then ∣ W Q ∣ ≤ ( 3 / 4 ) Q ∣ W 0 ∣ < 1 |\mathcal{W}_Q|\le(3/4)^Q|\mathcal{W}_0|<1 ∣ W Q ∣ ≤ ( 3/4 ) Q ∣ W 0 ∣ < 1 , where the last inequality follows from Q = ⌈ log 4 / 3 ∣ W 0 ∣ ⌉ + 2 Q=\lceil\log_{4/3}|\mathcal{W}_0|\rceil+2 Q = ⌈ log 4/3 ∣ W 0 ∣ ⌉ + 2 . Since W Q \mathcal{W}_Q W Q is finite, we must have W Q = ∅ \mathcal{W}_Q=\varnothing W Q = ∅ , and the protocol returns Far.
Case 2: Match holds. Assume that opt H h s ( S A , S B ) = 0 \operatorname{opt}_{\mathcal{H}_{\mathrm{hs}}}(S_A,S_B)=0 opt H hs ( S A , S B ) = 0 , and fix w ⋆ ∈ W 0 w^\star\in\mathcal{W}_0 w ⋆ ∈ W 0 such that f A ( w ⋆ ) = f B ( w ⋆ ) = 0 f_A(w^\star)=f_B(w^\star)=0 f A ( w ⋆ ) = f B ( w ⋆ ) = 0 . We claim that if the protocol reaches round r r r without having already returned Match, then w ⋆ ∈ W r w^\star\in\mathcal{W}_r w ⋆ ∈ W r . The claim holds initially because W 0 = H h s \mathcal{W}_0=\mathcal{H}_{\mathrm{hs}} W 0 = H hs .
Suppose that w ⋆ ∈ W r w^\star\in\mathcal{W}_r w ⋆ ∈ W r . For each P ∈ { A , B } P\in\{A,B\} P ∈ { A , B } , since ⊥ ∈ R r \perp\in\mathcal{R}_r ⊥∈ R r and q P , W r ( ⊥ ; S P ) = k / 2 q_{P,\mathcal{W}_r}(\perp;S_P)=k/2 q P , W r ( ⊥ ; S P ) = k /2 , the event G \mathcal{G} G implies that q P , W r ( C P , r ; S P ) ≤ k / 2 + α ≤ k − 1 q_{P,\mathcal{W}_r}(C_{P,r};S_P)\le k/2+\alpha\le k-1 q P , W r ( C P , r ; S P ) ≤ k /2 + α ≤ k − 1 , where the last inequality follows from k ≥ 2 α + 2 k\ge2\alpha+2 k ≥ 2 α + 2 . However, every real container C ∈ R r C\in\mathcal{R}_r C ∈ R r excluding w ⋆ w^\star w ⋆ has q P , W r ( C ; S P ) ≥ ( k + 1 − f P ( h ⋆ ) ) + = k + 1 q_{P,\mathcal{W}_r}(C;S_P)\ge(k+1-f_P(h^\star))_+=k+1 q P , W r ( C ; S P ) ≥ ( k + 1 − f P ( h ⋆ ) ) + = k + 1 . Thus, every real container released by either party contains w ⋆ w^\star w ⋆ .
If both parties release ⊥ \perp ⊥ , the protocol correctly returns Match. Otherwise, W r + 1 \mathcal{W}_{r+1} W r + 1 is one of the released real containers and therefore contains w ⋆ w^\star w ⋆ . This proves the claim by induction. Consequently, the active set never becomes empty. If the protocol does not terminate earlier, then w ⋆ ∈ W Q w^\star\in\mathcal{W}_Q w ⋆ ∈ W Q , and the protocol returns Match after the final round.
Combining the two cases, the protocol is correct whenever G \mathcal{G} G holds. Since Pr ( G ) ≥ 1 − β \Pr(\mathcal{G})\ge1-\beta Pr ( G ) ≥ 1 − β , its success probability is at least 1 − β 1-\beta 1 − β .
Corollary 4.8 (Mixed labels for halfspaces). Let c c c be the universal constant in Theorem 4.7 . Fix a finite data space X ⊆ R d \mathcal{X} \subseteq\mathbb{R}^{d} X ⊆ R d with ∣ X ∣ = m |\mathcal{X}| = m ∣ X ∣ = m , and suppose that each party P ∈ { A , B } P \in\{A, B\} P ∈ { A , B } holds a labeled dataset S P ∈ ( X × { ± 1 } ) n S_P \in(\mathcal{X} \times\{\pm1\})^n S P ∈ ( X × { ± 1 } ) n . Define opt H h s m i x ( S A , S B ) : = min w ∈ R d ∑ P ∈ { A , B } ∑ ( x , y ) ∈ S P 1 { h w ( x ) ≠ y } \operatorname{opt}^{\mathrm{mix}}_{\mathcal{H}_{\mathrm{hs}}}(S_A,S_B) := \min_{w\in\mathbb{R}^{d}} \sum_{P\in\{A,B\}} \sum_{(x,y)\in S_P} \mathbb{1}\{h_w(x) \ne y\} opt H hs mix ( S A , S B ) := min w ∈ R d ∑ P ∈ { A , B } ∑ ( x , y ) ∈ S P 1 { h w ( x ) = y } . Let ε ∈ ( 0 , 1 ] \varepsilon\in(0,1] ε ∈ ( 0 , 1 ] , β ∈ ( 0 , 1 ) \beta\in(0,1) β ∈ ( 0 , 1 ) , and t ∈ [ n ] t \in[n] t ∈ [ n ] . If
t ≥ c d 2 log ( 2 m ) ε ( d + log d log ( 2 m ) β ) , t \ge c\frac{d^2\log(2m)}{\varepsilon}\left(d+\log\frac{d\log(2m)}{\beta}\right), t ≥ c ε d 2 log ( 2 m ) ( d + log β d log ( 2 m ) ) , then Algorithm 3, after redefining the local error as f P ( w ) : = ∑ ( x , y ) ∈ S P 1 { h w ( x ) ≠ y } f_P(w) := \sum_{(x,y)\in S_P} \mathbb{1}\{h_w(x) \ne y\} f P ( w ) := ∑ ( x , y ) ∈ S P 1 { h w ( x ) = y } , is an ε \varepsilon ε -private tester that distinguishes opt H h s m i x ( S A , S B ) = 0 \operatorname{opt}^{\mathrm{mix}}_{\mathcal{H}_{\mathrm{hs}}}(S_A,S_B)=0 opt H hs mix ( S A , S B ) = 0 from opt H h s m i x ( S A , S B ) ≥ t \operatorname{opt}^{\mathrm{mix}}_{\mathcal{H}_{\mathrm{hs}}}(S_A,S_B)\ge t opt H hs mix ( S A , S B ) ≥ t with success probability at least 1 − β 1-\beta 1 − β .
Proof. The finite dual discretization is unchanged, since it depends only on the predictions over X \mathcal{X} X . Redefine G P ( W ) : = { w ∈ W : f P ( w ) ≤ k } G_P(\mathcal{W}) := \{w \in\mathcal{W}: f_P(w) \le k\} G P ( W ) := { w ∈ W : f P ( w ) ≤ k } . To verify Lemma 4.4 , suppose that v ∈ conv ( G A ( W ) ) ∩ conv ( G B ( W ) ) v \in\operatorname{conv}(G_A(\mathcal{W})) \cap\operatorname{conv}(G_B(\mathcal{W})) v ∈ conv ( G A ( W )) ∩ conv ( G B ( W )) . By Carathéodory’s theorem, write v = ∑ i = 1 r λ i a i v = \sum_{i=1}^{r}\lambda_i a_i v = ∑ i = 1 r λ i a i for some r ≤ d + 1 r \le d+1 r ≤ d + 1 and a 1 , … , a r ∈ G A ( W ) a_1,\ldots,a_r \in G_A(\mathcal{W}) a 1 , … , a r ∈ G A ( W ) . If h v h_v h v misclassifies a labeled record, then at least one h a i h_{a_i} h a i also misclassifies it. Therefore, f A ( v ) ≤ ∑ i = 1 r f A ( a i ) ≤ ( d + 1 ) k f_A(v) \le\sum_{i=1}^{r} f_A(a_i) \le(d+1)k f A ( v ) ≤ ∑ i = 1 r f A ( a i ) ≤ ( d + 1 ) k . The same argument gives f B ( v ) ≤ ( d + 1 ) k f_B(v) \le(d+1)k f B ( v ) ≤ ( d + 1 ) k , and hence, for k = ⌊ ( t − 1 ) / ( 2 ( d + 1 ) ) ⌋ k = \lfloor(t-1)/(2(d+1)) \rfloor k = ⌊( t − 1 ) / ( 2 ( d + 1 ))⌋ ,
opt H h s m i x ( S A , S B ) ≤ f A ( v ) + f B ( v ) ≤ 2 ( d + 1 ) k ≤ t − 1 , \operatorname{opt}^{\mathrm{mix}}_{\mathcal{H}_{\mathrm{hs}}}(S_A,S_B) \le f_A(v)+f_B(v) \le2(d+1)k \le t-1, opt H hs mix ( S A , S B ) ≤ f A ( v ) + f B ( v ) ≤ 2 ( d + 1 ) k ≤ t − 1 ,
contradicting the Far assumption. Thus, the utility proof of Theorem 4.7 applies unchanged.
Finally, replacing one labeled record changes f P ( w ) f_P(w) f P ( w ) by at most one for every w w w , so q P , W ( C ; S P ) q_{P,\mathcal{W}}(C;S_P) q P , W ( C ; S P ) retains sensitivity one. Therefore, the same exponential-mechanism and composition arguments give ε \varepsilon ε -two-party differential privacy. □
Complementary bounds for small data regime
An interesting aspect of the guarantee in Theorem 4.7 is that t t t does not depend on the dataset size n n n . However, the cost of such a result is that we need n ≥ O ~ ( d 3 log ( m ) ε ) n \ge\widetilde{O}\left(\frac{d^3\log(m)}{\varepsilon}\right) n ≥ O ( ε d 3 l o g ( m ) ) . In this section, we show that when n n n is small, there are simple algorithms that extend the private tester to almost all non-trivial dataset sizes.
A tester with t = n t=n t = n . In this section, we give a simple algorithm to show that, as long as n ≳ 1 / ε 2 n \gtrsim1/\varepsilon^2 n ≳ 1/ ε 2 , we can design a tester that achieves t = n t=n t = n . Note that this is the largest resolution one can ask for: classifier w = 0 w=0 w = 0 labels every point as positive and makes exactly n n n mistakes, and therefore opt H h s ( S A , S B ) ≤ n \operatorname{opt}_{\mathcal{H}_{\mathrm{hs}}}(S_A,S_B)\le n opt H hs ( S A , S B ) ≤ n for all datasets.
The starting point is a simple characterization of when opt H h s ( S A , S B ) = n \operatorname{opt}_{\mathcal{H}_{\mathrm{hs}}}(S_A,S_B)=n opt H hs ( S A , S B ) = n . Note that we assume the homogeneous halfspaces in Equation (11) without a bias term. For x ≠ 0 x\ne0 x = 0 , let R ( x ) = { λ x : λ > 0 } R(x)=\{\lambda x:\lambda>0\} R ( x ) = { λ x : λ > 0 } denote the ray through x x x , and let R = { R ( x ) : x ∈ X } \mathcal{R}=\{R(x):x\in\mathcal{X}\} R = { R ( x ) : x ∈ X } be the set of rays that contain at least one point of the domain. Since the domain is finite, this set is well-defined. Notice that R \mathcal{R} R is public.
For every R ∈ R R\in\mathcal{R} R ∈ R , let a R a_R a R and b R b_R b R denote the number of Alice’s and Bob’s records that lie on R R R , respectively. The main observation is that a homogeneous halfspace assigns the same label to all the points on a ray. Using this observation, it is straightforward to see that
opt H h s ( S A , S B ) = n ⟺ a R = b R for all R ∈ R , \operatorname{opt}_{\mathcal{H}_{\mathrm{hs}}}(S_A,S_B)=n \Longleftrightarrow a_R=b_R\text{ for all }R\in\mathcal{R}, opt H hs ( S A , S B ) = n ⟺ a R = b R for all R ∈ R ,
opt H h s ( S A , S B ) = 0 ⟹ a R b R = 0 for all R ∈ R . (15) \operatorname{opt}_{\mathcal{H}_{\mathrm{hs}}}(S_A,S_B)=0 \Longrightarrow a_Rb_R=0\text{ for all }R\in\mathcal{R}. \tag*{(15)} opt H hs ( S A , S B ) = 0 ⟹ a R b R = 0 for all R ∈ R . (15)
In other words, in the Far case Alice and Bob have exactly the same histogram over the rays, while in the Match case their histograms have disjoint supports. Therefore, at t = n t=n t = n the testing problem reduces to checking whether two histograms over R \mathcal{R} R are identical, and this can be done using shared randomness as we show in Algorithm 4. The next theorem summarize the privacy and utility guarantee of this tester.
Algorithm 4 Testing at Resolution t = n t=n t = n for Half Spaces: A m a x − h s \mathcal{A}_{\mathrm{max-hs}} A max − hs
Input: Alice holds S A = ( X 1 , … , X n ) ∈ X n S_A=(X_1,\ldots,X_n)\in\mathcal{X}^n S A = ( X 1 , … , X n ) ∈ X n , Bob holds S B = ( Y 1 , … , Y n ) ∈ X n S_B=(Y_1,\ldots,Y_n)\in\mathcal{X}^n S B = ( Y 1 , … , Y n ) ∈ X n ; parameter ε \varepsilon ε .
Output: Far or Match.
Let k k k be a large enough constant, and set τ ← n / 2 \tau\leftarrow\sqrt{n}/2 τ ← n /2 .
Draw independent and uniform signs σ j , R ∈ { ± 1 } \sigma_{j,R}\in\{\pm1\} σ j , R ∈ { ± 1 } for j ∈ [ k ] j\in[k] j ∈ [ k ] and R ∈ R R\in\mathcal{R} R ∈ R . ▹ \triangleright ▹ public randomness
Alice computes a R ← ∣ { i ∈ [ n ] : X i ∈ R } ∣ a_R\leftarrow|\{i\in[n]:X_i\in R\}| a R ← ∣ { i ∈ [ n ] : X i ∈ R } ∣ for every R ∈ R R\in\mathcal{R} R ∈ R , and releases
Z A , j ← ∑ R ∈ R σ j , R a R + Lap ( 2 k / ε ) for all j ∈ [ k ] . Z_{A,j}\leftarrow\sum_{R\in\mathcal{R}}\sigma_{j,R}a_R+\operatorname{Lap}(2k/\varepsilon)\qquad\text{for all }j\in[k]. Z A , j ← R ∈ R ∑ σ j , R a R + Lap ( 2 k / ε ) for all j ∈ [ k ] .
Bob computes b R ← ∣ { i ∈ [ n ] : Y i ∈ R } ∣ b_R\leftarrow|\{i\in[n]:Y_i\in R\}| b R ← ∣ { i ∈ [ n ] : Y i ∈ R } ∣ for every R ∈ R R\in\mathcal{R} R ∈ R , and releases
Z B , j ← ∑ R ∈ R σ j , R b R + Lap ( 2 k / ε ) for all j ∈ [ k ] . Z_{B,j}\leftarrow\sum_{R\in\mathcal{R}}\sigma_{j,R}b_R+\operatorname{Lap}(2k/\varepsilon)\qquad\text{for all }j\in[k]. Z B , j ← R ∈ R ∑ σ j , R b R + Lap ( 2 k / ε ) for all j ∈ [ k ] .
if max j ∈ [ k ] ∣ Z A , j − Z B , j ∣ > τ \max_{j\in[k]}|Z_{A,j}-Z_{B,j}|>\tau max j ∈ [ k ] ∣ Z A , j − Z B , j ∣ > τ then
return Match
else
return Far
Theorem 4.9. There exists a universal constant C > 0 C>0 C > 0 such that the following holds. Fix a finite data space X ⊆ R d ∖ { 0 } \mathcal{X}\subseteq\mathbb{R}^d\setminus\{0\} X ⊆ R d ∖ { 0 } and ε > 0 \varepsilon>0 ε > 0 , and let n ≥ C / ε 2 n\ge C/\varepsilon^2 n ≥ C / ε 2 . Then there exists a non-interactive algorithm (presented in Algorithm 4) that is ε \varepsilon ε -private tester with resolution t = n t=n t = n for H h s \mathcal{H}_{\mathrm{hs}} H hs .
In the Match case, consider a line containing a negative record. A classifier realizing the pooled data must place all negative records on this line on one ray and all positive records on the opposite ray. Consequently, multiplying by the labels makes all contributions from this line have the same sign. Since the signs on different lines are independent, the pooled sketch has variance at least N − ≥ n / 2 N_{-} \ge n/2 N − ≥ n /2 . The same moment argument as before therefore gives a sketch of magnitude at least n / 2 \sqrt{n}/2 n /2 with constant probability.
We repeat the sketch a constant number of times. Replacing one labeled record changes each sketch by at most two and the negative-label count by at most one, so Laplace noise of scale O ( 1 / ε ) O(1/\varepsilon) O ( 1/ ε ) suffices for privacy. Combining the count test with the sketches gives an ε \varepsilon ε -private tester whenever n ≥ C / ε 2 n \ge C/\varepsilon^{2} n ≥ C / ε 2 , for 0 < ε ≤ 1 0 < \varepsilon\le1 0 < ε ≤ 1 and a universal constant C C C , independently of d d d and m m m .
A tester using private synthetic data generation
A simple application of the mechanism in [7 ] gives a complementary bound with a better dependence on the dimension. The idea is to have each party privately release a small synthetic dataset that approximates its empirical error for every halfspace. Then, using it, we can approximate opt \operatorname{opt} opt .
Theorem 4.11 (Halfspace testing via the BLR net mechanism). There exists a universal constant C > 0 C > 0 C > 0 such that the following holds. Fix d ≥ 1 d \ge1 d ≥ 1 , m , n ∈ N m,n \in\mathbb{N} m , n ∈ N , and ε ∈ ( 0 , 1 ] \varepsilon\in(0,1] ε ∈ ( 0 , 1 ] . For every public data space X ⊆ R d \mathcal{X} \subseteq\mathbb{R}^{d} X ⊆ R d with ∣ X ∣ = m |\mathcal{X}| = m ∣ X ∣ = m and every t ∈ [ n ] t \in[n] t ∈ [ n ] satisfying
t ≥ C ( n 2 d log ( 2 m ) ε ) 1 / 3 , (16) t \ge C\left(\frac{n^{2}d\log(2m)}{\varepsilon}\right)^{1/3}, \tag*{(16)} t ≥ C ( ε n 2 d log ( 2 m ) ) 1/3 , (16) there exists a non-interactive, randomized, ε \varepsilon ε -two-party DP tester for H h s \mathcal{H}_{\mathrm{hs}} H hs with resolution $t.
Remark 4.12. At first glance, the dependence on d d d , log m \log m log m , and 1 / ε 1/\varepsilon 1/ ε in Theorem 4.11 scales as ( d log ( 2 m ) ε ) 1 / 3 \left(\frac{d\log(2m)}{\varepsilon}\right)^{1/3} ( ε d l o g ( 2 m ) ) 1/3 , which might appear to contradict our lower bound of t = Ω ( d log ( m / d ) ε ) t = \Omega\left(\frac{d\log(m/d)}{\varepsilon}\right) t = Ω ( ε d l o g ( m / d ) ) from Theorem 4.1 . However, there is no contradiction because the bound in Equation (16) also depends on n n n and can be written as a weighted geometric mean of n n n and d log ( 2 m ) ε \frac{d\log(2m)}{\varepsilon} ε d l o g ( 2 m ) :
( n 2 d log ( 2 m ) ε ) 1 / 3 = n 2 / 3 ( d log ( 2 m ) ε ) 1 / 3 . \left(\frac{n^{2}d\log(2m)}{\varepsilon}\right)^{1/3}
= n^{2/3}\left(\frac{d\log(2m)}{\varepsilon}\right)^{1/3}. ( ε n 2 d log ( 2 m ) ) 1/3 = n 2/3 ( ε d log ( 2 m ) ) 1/3 .
Recall that any non-trivial tester requires t ≤ n t \le n t ≤ n (since opt H h s ( S A , S B ) ≤ n \operatorname{opt}_{\mathcal{H}_{\mathrm{hs}}}(S_A,S_B) \le n opt H hs ( S A , S B ) ≤ n holds trivially for all datasets of size n n n ). For the right-hand side of Equation (16) to be at most n n n , we must have n ≳ d log ( 2 m ) ε n \gtrsim\frac{d\log(2m)}{\varepsilon} n ≳ ε d l o g ( 2 m ) . In this regime, the geometric mean always lies between d log ( 2 m ) ε \frac{d\log(2m)}{\varepsilon} ε d l o g ( 2 m ) and n n n , namely
d log ( 2 m ) ε ≲ ( n 2 d log ( 2 m ) ε ) 1 / 3 ≤ n . \frac{d\log(2m)}{\varepsilon} \lesssim\left(\frac{n^{2}d\log(2m)}{\varepsilon}\right)^{1/3} \le n. ε d log ( 2 m ) ≲ ( ε n 2 d log ( 2 m ) ) 1/3 ≤ n .
Conversely, when n ≪ d log ( 2 m ) ε n \ll\frac{d\log(2m)}{\varepsilon} n ≪ ε d l o g ( 2 m ) , the right-hand side of Equation (16) exceeds n n n , so the range of admissible t ∈ [ n ] t \in[n] t ∈ [ n ] in Theorem 4.11 is empty. Thus, whenever Theorem 4.11 is non-vacuous, its resolution t t t is always consistent with Theorem 4.1 .
Acknowledgment
Mahdi Haghifam would like to thank Alireza Fallah and Adam Smith for discussions on this project.
AI Disclosure
GPT-5 and GPT-6 were used to explore ideas for proving lower bound for halfspaces in Theorem 4.1 . Authors also used AI tools for generating the figures in the paper as well as polishing, editing, and proof-reading.
On the in-population error induced by OPT
The joint empirical loss opt H \operatorname{opt}_{\mathcal{H}} opt H can be directly linked to an in-distribution loss. Suppose the agents want to minimize the loss with respect to a target distribution P ∈ Δ ( X × { ± 1 } ) P \in\Delta(\mathcal{X} \times\{ \pm1 \}) P ∈ Δ ( X × { ± 1 }) . We define the minimal population error on this distribution for a hypothesis class H \mathcal{H} H as follows:
R H ∗ ( P ) : = inf h ∈ H Pr ( X , Y ) ∼ P [ h ( X ) ≠ Y ] . (17) R_{\mathcal{H}}^*(P) := \inf_{h \in\mathcal{H}} \Pr_{(X,Y) \sim P}[h(X) \ne Y]. \tag*{(17)} R H ∗ ( P ) := h ∈ H inf ( X , Y ) ∼ P Pr [ h ( X ) = Y ] . (17)
Clearly, if the agents require target population error at most τ ∈ ( 0 , 1 ) \tau\in(0,1) τ ∈ ( 0 , 1 ) , then R H ∗ ( P ) > τ R_{\mathcal{H}}^*(P) > \tau R H ∗ ( P ) > τ implies that no shared classifier in H \mathcal{H} H , including the Empirical Risk Minimizer (ERM) on their joint data, can satisfy them. In the following remark, we highlight how distinguishing between the conditions on opt H \operatorname{opt}_{\mathcal{H}} opt H in (2) allows us to detect a large R H ∗ ( P ) R_{\mathcal{H}}^*(P) R H ∗ ( P ) , and vice-versa under distributional assumptions on S A S_A S A , S B S_B S B .
We go back to our running examples and apply Remark A.1 to demonstrate how, in real applications, a large joint empirical loss opt H ( S A , S B ) \operatorname{opt}_{\mathcal{H}}(S_A,S_B) opt H ( S A , S B ) certifies that R H ∗ > τ R_{\mathcal{H}}^*>\tau R H ∗ > τ with high probability.
Example 3 (Learning from disparate label sets). Remember that Alice’s labels are all − 1 -1 − 1 , and Bob’s are all + 1 +1 + 1 , but their data otherwise follow the target distribution conditioned on the label. Formally, let P A : = P ⋅ ∣ y = − 1 P_A := P_{\cdot\mid y=-1} P A := P ⋅ ∣ y = − 1 and P B : = P ⋅ ∣ y = 1 P_B := P_{\cdot\mid y=1} P B := P ⋅ ∣ y = 1 , and suppose S A ∼ P A n S_A \sim P_A^n S A ∼ P A n and S B ∼ P B n S_B \sim P_B^n S B ∼ P B n independently. If both labels have equal probability under P P P , then 1 2 ( P A + P B ) = P \frac{1}{2}(P_A+P_B)=P 2 1 ( P A + P B ) = P . By Remark A.1, with probability at least 1 − δ 1-\delta 1 − δ , opt H ( S A , S B ) = 0 \operatorname{opt}_{\mathcal{H}}(S_A,S_B)=0 opt H ( S A , S B ) = 0 implies R H ∗ ( P ) ≤ η n R_{\mathcal{H}}^*(P)\le\eta_n R H ∗ ( P ) ≤ η n , whereas opt H ( S A , S B ) > t \operatorname{opt}_{\mathcal{H}}(S_A,S_B)>t opt H ( S A , S B ) > t implies R H ∗ ( P ) > t / ( 2 n ) − η n R_{\mathcal{H}}^*(P)>t/(2n)-\eta_n R H ∗ ( P ) > t / ( 2 n ) − η n . Thus, the joint empirical loss allows the agents to conclude whether a shared classifier in H \mathcal{H} H can achieve low target error.
Example 4 (Identifying conflicting labeling practices). Remember that the agents encounter the same distribution of emails but may assign different labels to the same email. Let D D D denote their common feature distribution and f A , f B ∈ H f_A,f_B \in\mathcal{H} f A , f B ∈ H their deterministic labeling functions, so each task is individually realizable. Let P A P_A P A and P B P_B P B be the distributions of ( X , f A ( X ) ) (X,f_A(X)) ( X , f A ( X )) and ( X , f B ( X ) ) (X,f_B(X)) ( X , f B ( X )) , respectively, for X ∼ D X \sim D X ∼ D . We take the target distribution to be P = 1 2 ( P A + P B ) P=\frac{1}{2}(P_A+P_B) P = 2 1 ( P A + P B ) , a symmetric choice that gives the two agents’ test errors equal weight. Suppose S A ∼ P A n S_A \sim P_A^n S A ∼ P A n and S B ∼ P B n S_B \sim P_B^n S B ∼ P B n independently. By Remark A.1, with probability at least 1 − δ 1-\delta 1 − δ , opt H ( S A , S B ) = 0 \operatorname{opt}_{\mathcal{H}}(S_A,S_B)=0 opt H ( S A , S B ) = 0 implies that a shared classifier in H \mathcal{H} H can achieve test error at most η n \eta_n η n under P P P , whereas opt H ( S A , S B ) > 2 n ( τ + η n ) \operatorname{opt}_{\mathcal{H}}(S_A,S_B)>2n(\tau+\eta_n) opt H ( S A , S B ) > 2 n ( τ + η n ) rules out any shared classifier in H \mathcal{H} H with test error at most τ \tau τ under P P P .
Helper Lemma
Lemma B.1 (Laplace mechanism [17 ] ). Let f : X n → R d f:\mathcal{X}^n \to\mathbb{R}^d f : X n → R d be a vector-valued function with ℓ 1 \ell_1 ℓ 1 -sensitivity at most Δ \Delta Δ , meaning ∥ f ( X ) − f ( X ′ ) ∥ 1 ≤ Δ \lVert f(X)-f(X')\rVert_1 \le\Delta ∥ f ( X ) − f ( X ′ ) ∥ 1 ≤ Δ for all neighboring X , X ′ X,X' X , X ′ , i.e., two X , X ′ ∈ X n X,X' \in\mathcal{X}^n X , X ′ ∈ X n that differ only in a single element. The mechanism f ~ ( X ) = f ( X ) + Z \widetilde{f}(X)=f(X)+Z f ( X ) = f ( X ) + Z , where Z ∼ ( Lap ( Δ / ε ) ) ⊗ d Z\sim(\operatorname{Lap}(\Delta/\varepsilon))^{\otimes d} Z ∼ ( Lap ( Δ/ ε ) ) ⊗ d , is ε \varepsilon ε -DP. Here Lap ( b ) \operatorname{Lap}(b) Lap ( b ) denotes the mean-zero Laplace distribution with density ( 2 b ) − 1 e − ∣ z ∣ / b (2b)^{-1}e^{-|z|/b} ( 2 b ) − 1 e − ∣ z ∣/ b . Moreover, for every X ∈ X n X\in\mathcal{X}^n X ∈ X n and β ∈ ( 0 , 1 ] \beta\in(0,1] β ∈ ( 0 , 1 ] , we have Pr [ ∥ f ~ ( X ) − f ( X ) ∥ ≥ Δ ε log ( 1 / β ) ] ≤ β \Pr\left[\lVert\widetilde{f}(X)-f(X)\rVert\ge\frac{\Delta}{\varepsilon}\log(1/\beta)\right]\le\beta Pr [ ∥ f ( X ) − f ( X )∥ ≥ ε Δ log ( 1/ β ) ] ≤ β .
Lemma B.2 (A constant-degree expander [41 ] ). There are universal constants δ > 0 \delta>0 δ > 0 and K 0 ∈ N K_0\in\mathbb{N} K 0 ∈ N such that, for every even K ≥ K 0 K\ge K_0 K ≥ K 0 , there is a simple bipartite graph with K K K variable nodes of degree four and K / 2 K/2 K /2 check nodes of degree eight satisfying
∣ Γ ( T ) ∣ > 2 ∣ T ∣ whenever ∅ ≠ T ⊆ [ K ] and ∣ T ∣ < δ K . |\Gamma(T)|>2|T|\qquad\text{whenever }\varnothing\ne T\subseteq[K]\text{ and }|T|<\delta K. ∣Γ ( T ) ∣ > 2∣ T ∣ whenever ∅ = T ⊆ [ K ] and ∣ T ∣ < δ K . Here Γ ( T ) \Gamma(T) Γ ( T ) is the set of checks adjacent to T T T . Consequently, every such T T T has a check adjacent to exactly one of its variables.
Lemma B.3 (Carathéodory’s theorem [11 ] ). Let X ⊆ R d \mathcal{X}\subseteq\mathbb{R}^d X ⊆ R d . Every point x ∈ conv ( X ) x\in\operatorname{conv}(\mathcal{X}) x ∈ conv ( X ) can be written as a convex combination of at most d + 1 d+1 d + 1 points of X \mathcal{X} X . That is for every x ∈ conv ( X ) x\in\operatorname{conv}(\mathcal{X}) x ∈ conv ( X ) , there exist n ≤ d + 1 n\le d+1 n ≤ d + 1 , points x 1 , … , x n ∈ X x_1,\ldots,x_n\in\mathcal{X} x 1 , … , x n ∈ X , and coefficients λ 1 , … , λ n ≥ 0 \lambda_1,\ldots,\lambda_n\ge0 λ 1 , … , λ n ≥ 0 with ∑ i = 1 n λ i = 1 \sum_{i=1}^n\lambda_i=1 ∑ i = 1 n λ i = 1 such that x = ∑ i = 1 n λ i x i x=\sum_{i=1}^n\lambda_i x_i x = ∑ i = 1 n λ i x i .
Lemma B.4 (Exponential mechanism [38 ] ). Let R \mathcal{R} R be a finite output space, and let q : X ⋆ × R → R q:\mathcal{X}^{\star}\times\mathcal{R}\to\mathbb{R} q : X ⋆ × R → R be a cost function with sensitivity at most Δ > 0 \Delta>0 Δ > 0 ; that is, for every pair of neighboring datasets S , S ′ S,S' S , S ′ and every r ∈ R r\in\mathcal{R} r ∈ R , ∣ q ( S , r ) − q ( S ′ , r ) ∣ ≤ Δ |q(S,r)-q(S',r)|\le\Delta ∣ q ( S , r ) − q ( S ′ , r ) ∣ ≤ Δ . The exponential mechanism outputs r ∈ R r\in\mathcal{R} r ∈ R with probability
Pr [ EM ( S ) = r ] = exp ( − ε q ( S , r ) 2 Δ ) ∑ r ′ ∈ R exp ( − ε q ( S , r ′ ) 2 Δ ) . \Pr[\operatorname{EM}(S)=r]=\frac{\exp\left(-\frac{\varepsilon q(S,r)}{2\Delta}\right)}{\sum_{r'\in\mathcal{R}}\exp\left(-\frac{\varepsilon q(S,r')}{2\Delta}\right)}. Pr [ EM ( S ) = r ] = ∑ r ′ ∈ R exp ( − 2Δ εq ( S , r ′ ) ) exp ( − 2Δ εq ( S , r ) ) . Then EM \operatorname{EM} EM is ε \varepsilon ε -DP. Moreover, for every β ∈ ( 0 , 1 ) \beta\in(0,1) β ∈ ( 0 , 1 ) , with probability at least 1 − β 1-\beta 1 − β ,
q ( S , M exp ( S ) ) ≤ min r ∈ R q ( S , r ) + 2 Δ ε ( ln ∣ R ∣ + ln 1 β ) . q(S,M_{\exp}(S))\le\min_{r\in\mathcal{R}}q(S,r)+\frac{2\Delta}{\varepsilon}\left(\ln|\mathcal{R}|+\ln\frac{1}{\beta}\right). q ( S , M e x p ( S )) ≤ r ∈ R min q ( S , r ) + ε 2Δ ( ln ∣ R ∣ + ln β 1 ) .
Lemma B.5 (From LP constraints to halfspace datasets). Fix p , M ∈ N p,M\in\mathbb{N} p , M ∈ N . Let R = { a i T x ≤ b i } i ∈ [ M ] \mathcal{R}=\{a_i^{\mathsf T}x\le b_i\}_{i\in[M]} R = { a i T x ≤ b i } i ∈ [ M ] be a finite set of linear constraints on x ∈ R p x\in\mathbb{R}^p x ∈ R p . There exists a constant η > 0 \eta>0 η > 0 , depending only on R \mathcal{R} R , with the following property. For each i ∈ [ M ] i\in[M] i ∈ [ M ] , define the points
x i − : = ( a i , − b i − η ) ∈ R p + 1 and x i + : = ( − a i , b i ) ∈ R p + 1 , x_i^-:=(a_i,-b_i-\eta)\in\mathbb{R}^{p+1}\qquad\text{and}\qquad x_i^+:=(-a_i,b_i)\in\mathbb{R}^{p+1}, x i − := ( a i , − b i − η ) ∈ R p + 1 and x i + := ( − a i , b i ) ∈ R p + 1 , together with x 0 − : = − e p + 1 x_0^-:=-e_{p+1} x 0 − := − e p + 1 and x 0 + : = e p + 1 x_0^+:=e_{p+1} x 0 + := e p + 1 in R p + 1 \mathbb{R}^{p+1} R p + 1 . Then for any subsets I A , I B ⊆ [ M ] I_A,I_B\subseteq[M] I A , I B ⊆ [ M ] , if we form the datasets
S A : = { x i − : i ∈ I A } ∪ { x 0 − } and S B : = { x i + : i ∈ I B } ∪ { x 0 + } , S_A:=\{x_i^-:i\in I_A\}\cup\{x_0^-\}\qquad\text{and}\qquad S_B:=\{x_i^+:i\in I_B\}\cup\{x_0^+\}, S A := { x i − : i ∈ I A } ∪ { x 0 − } and S B := { x i + : i ∈ I B } ∪ { x 0 + } , the linear system { a i T x ≤ b i : i ∈ I A ∪ I B } \{a_i^{\mathsf T}x\le b_i:i\in I_A\cup I_B\} { a i T x ≤ b i : i ∈ I A ∪ I B } is feasible in R p \mathbb{R}^p R p if and only if opt H h s ( S A , S B ) = 0 \operatorname{opt}_{\mathcal{H}_{\mathrm{hs}}}(S_A,S_B)=0 opt H hs ( S A , S B ) = 0 (that is, there exists h w ∈ H h s h_w\in\mathcal{H}_{\mathrm{hs}} h w ∈ H hs such that h w ( x ) = − 1 h_w(x)=-1 h w ( x ) = − 1 for all x ∈ S A x\in S_A x ∈ S A and h w ( x ) = + 1 h_w(x)=+1 h w ( x ) = + 1 for all x ∈ S B x\in S_B x ∈ S B ).
Proof. We first choose η > 0 \eta>0 η > 0 depending only on R \mathcal{R} R . For each subset T ⊆ [ M ] T\subseteq[M] T ⊆ [ M ] such that { a i T x ≤ b i : i ∈ T } \{a_i^{\mathsf T}x\le b_i:i\in T\} { a i T x ≤ b i : i ∈ T } is infeasible, Farkas’ lemma gives a vector y T ∈ R ≥ 0 T y_T\in\mathbb{R}_{\ge0}^T y T ∈ R ≥ 0 T with ∑ i ∈ T ( y T ) i = 1 \sum_{i\in T}(y_T)_i=1 ∑ i ∈ T ( y T ) i = 1 , ∑ i ∈ T ( y T ) i a i = 0 \sum_{i\in T}(y_T)_ia_i=0 ∑ i ∈ T ( y T ) i a i = 0 , and y T T b i > 0 y_T^{\mathsf T}b_i>0 y T T b i > 0 . Since [ M ] [M] [ M ] has only finitely many subsets, we can choose η > 0 \eta>0 η > 0 smaller than min { y T T b i : T ⊆ [ M ] is infeasible } \min\{y_T^{\mathsf T}b_i:T\subseteq[M]\text{ is infeasible}\} min { y T T b i : T ⊆ [ M ] is infeasible } (or η = 1 \eta=1 η = 1 if every subset is feasible). Now fix any I A , I B ⊆ [ M ] I_A,I_B\subseteq[M] I A , I B ⊆ [ M ] and let S : = I A ∪ I B S:=I_A\cup I_B S := I A ∪ I B . Suppose first that there exists x ∈ R p x\in\mathbb{R}^p x ∈ R p satisfying a i T x ≤ b i a_i^{\mathsf T}x\le b_i a i T x ≤ b i for all i ∈ S i\in S i ∈ S , and consider h w ∈ H h s h_w\in\mathcal{H}_{\mathrm{hs}} h w ∈ H hs with w = ( x , 1 ) ∈ R p + 1 w=(x,1)\in\mathbb{R}^{p+1} w = ( x , 1 ) ∈ R p + 1 . Then ⟨ w , x 0 − ⟩ = − 1 < 0 \langle w,x_0^-\rangle=-1<0 ⟨ w , x 0 − ⟩ = − 1 < 0 , ⟨ w , x 0 + ⟩ = 1 ≥ 0 \langle w,x_0^+\rangle=1\ge0 ⟨ w , x 0 + ⟩ = 1 ≥ 0 , and for every i ∈ I A i\in I_A i ∈ I A and j ∈ I B j\in I_B j ∈ I B ,
⟨ w , x i − ⟩ = a i T x − b i − η ≤ − η < 0 , ⟨ w , x j + ⟩ = − a j T x + b j ≥ 0. (18) \langle w,x_i^-\rangle=a_i^{\mathsf T}x-b_i-\eta\le-\eta<0,\qquad\langle w,x_j^+\rangle=-a_j^{\mathsf T}x+b_j\ge0. \tag*{(18)} ⟨ w , x i − ⟩ = a i T x − b i − η ≤ − η < 0 , ⟨ w , x j + ⟩ = − a j T x + b j ≥ 0. (18)
Hence h w ( x ) = − 1 h_w(x)=-1 h w ( x ) = − 1 for all x ∈ S A x\in S_A x ∈ S A and h w ( x ) = + 1 h_w(x)=+1 h w ( x ) = + 1 for all x ∈ S B x\in S_B x ∈ S B , so opt H h s ( S A , S B ) = 0 \operatorname{opt}_{\mathcal{H}_{\mathrm{hs}}}(S_A,S_B)=0 opt H hs ( S A , S B ) = 0 . Conversely, suppose opt H h s ( S A , S B ) = 0 \operatorname{opt}_{\mathcal{H}_{\mathrm{hs}}}(S_A,S_B)=0 opt H hs ( S A , S B ) = 0 , and let w = ( v , λ ) ∈ R p × R w=(v,\lambda)\in\mathbb{R}^p\times\mathbb{R} w = ( v , λ ) ∈ R p × R be such that h w ( x ) = − 1 h_w(x)=-1 h w ( x ) = − 1 for all x ∈ S A x\in S_A x ∈ S A and h w ( x ) = + 1 h_w(x)=+1 h w ( x ) = + 1 for all x ∈ S B x\in S_B x ∈ S B . Since x 0 − ∈ S A x_0^-\in S_A x 0 − ∈ S A , we have ⟨ w , x 0 − ⟩ = − λ < 0 \langle w,x_0^-\rangle=-\lambda<0 ⟨ w , x 0 − ⟩ = − λ < 0 , so λ > 0 \lambda>0 λ > 0 . Setting x ~ : = v / λ ∈ R p \tilde{x}:=v/\lambda\in\mathbb{R}^p x ~ := v / λ ∈ R p , the conditions ⟨ w , x i − ⟩ < 0 \langle w,x_i^-\rangle<0 ⟨ w , x i − ⟩ < 0 for i ∈ I A i\in I_A i ∈ I A and ⟨ w , x j + ⟩ ≥ 0 \langle w,x_j^+\rangle\ge0 ⟨ w , x j + ⟩ ≥ 0 for j ∈ I B j\in I_B j ∈ I B divide by λ > 0 \lambda>0 λ > 0 to yield
a i T x ~ < b i + η ( ∀ i ∈ I A ) and a j T x ~ ≤ b j ( ∀ j ∈ I B ) . a_i^{\mathsf{T}}\tilde{x}<b_i+\eta\quad(\forall i\in I_A)
\qquad\text{and}\qquad
a_j^{\mathsf{T}}\tilde{x}\le b_j\quad(\forall j\in I_B). a i T x ~ < b i + η ( ∀ i ∈ I A ) and a j T x ~ ≤ b j ( ∀ j ∈ I B ) .
In particular, a i T x ~ ≤ b i + η a_i^{\mathsf{T}}\tilde{x}\le b_i+\eta a i T x ~ ≤ b i + η for all i ∈ S i\in S i ∈ S . By our choice of η \eta η , the system { a i T x ≤ b i : i ∈ S } \{a_i^{\mathsf{T}}x\le b_i:i\in S\} { a i T x ≤ b i : i ∈ S } cannot be infeasible, since otherwise
0 = ∑ i ∈ S ( y S ) i a i T x ~ ≤ ∑ i ∈ S ( y S ) i ( b i + η ) = − γ S + η < 0 , 0=\sum_{i\in S}(y_S)_i a_i^{\mathsf{T}}\tilde{x}\le\sum_{i\in S}(y_S)_i(b_i+\eta)=-\gamma_S+\eta<0, 0 = i ∈ S ∑ ( y S ) i a i T x ~ ≤ i ∈ S ∑ ( y S ) i ( b i + η ) = − γ S + η < 0 ,
a contradiction. Thus { a i T x ≤ b i : i ∈ I A ∪ I B } \{a_i^{\mathsf{T}}x\le b_i:i\in I_A\cup I_B\} { a i T x ≤ b i : i ∈ I A ∪ I B } is feasible in R p \mathbb{R}^p R p .
Lemma B.6 (Information complexity of the trace inner product). Let q = 2 s q=2^s q = 2 s , and let μ \mu μ be the uniform product distribution over F q k × F q k \mathbb{F}_q^k\times\mathbb{F}_q^k F q k × F q k . For F ( u , v ) = ⨁ i = 1 k Tr F q / F 2 ( u i v i ) F(u,v)=\bigoplus_{i=1}^k\operatorname{Tr}_{\mathbb{F}_q/\mathbb{F}_2}(u_iv_i) F ( u , v ) = ⨁ i = 1 k Tr F q / F 2 ( u i v i ) , we have IC μ , 1 / 10 ( F ) = Ω ( k log q ) \operatorname{IC}_{\mu,1/10}(F)=\Omega(k\log q) IC μ , 1/10 ( F ) = Ω ( k log q ) .
Proof. The trace pairing ( a , b ) ↦ Tr F q / F 2 ( a b ) (a,b)\mapsto\operatorname{Tr}_{\mathbb{F}_q/\mathbb{F}_2}(ab) ( a , b ) ↦ Tr F q / F 2 ( ab ) is nondegenerate. Therefore, there exist dual F 2 \mathbb{F}_2 F 2 -bases { α 1 , … , α s } \{\alpha_1,\ldots,\alpha_s\} { α 1 , … , α s } and { β 1 , … , β s } \{\beta_1,\ldots,\beta_s\} { β 1 , … , β s } of F q \mathbb{F}_q F q satisfying
Tr F q / F 2 ( α r β ℓ ) = 1 { r = ℓ } . \operatorname{Tr}_{\mathbb{F}_q/\mathbb{F}_2}(\alpha_r\beta_\ell)=1\{r=\ell\}. Tr F q / F 2 ( α r β ℓ ) = 1 { r = ℓ } .
Write u i = ∑ r = 1 s x i , r α r u_i=\sum_{r=1}^s x_{i,r}\alpha_r u i = ∑ r = 1 s x i , r α r , v i = ∑ r = 1 s y i , r β r v_i=\sum_{r=1}^s y_{i,r}\beta_r v i = ∑ r = 1 s y i , r β r where x i , r , y i , r ∈ F 2 x_{i,r},y_{i,r}\in\mathbb{F}_2 x i , r , y i , r ∈ F 2 . Then
Tr F q / F 2 ( u i v i ) = ⨁ r = 1 s x i , r y i , r , \operatorname{Tr}_{\mathbb{F}_q/\mathbb{F}_2}(u_iv_i)=\bigoplus_{r=1}^s x_{i,r}y_{i,r}, Tr F q / F 2 ( u i v i ) = r = 1 ⨁ s x i , r y i , r ,
and consequently F ( u , v ) = ⨁ i = 1 k ⨁ r = 1 s x i , r y i , r = IP k s ( x , y ) F(u,v)=\bigoplus_{i=1}^k\bigoplus_{r=1}^s x_{i,r}y_{i,r}=\operatorname{IP}_{ks}(x,y) F ( u , v ) = ⨁ i = 1 k ⨁ r = 1 s x i , r y i , r = IP k s ( x , y ) . The changes of basis are bijections, so under μ \mu μ , the corresponding x , y ∈ { 0 , 1 } k s x,y\in\{0,1\}^{ks} x , y ∈ { 0 , 1 } k s are independent and uniform. Moreover, any protocol for F F F induces a protocol for IP k s \operatorname{IP}_{ks} IP k s with the same error and the same internal information cost. It is folklore that binary inner product on N N N independent uniform bits has constant-error internal information complexity Ω ( N ) \Omega(N) Ω ( N ) . Therefore, IC μ , 1 / 10 ( F ) = Ω ( k s ) = Ω ( k log q ) \operatorname{IC}_{\mu,1/10}(F)=\Omega(ks)=\Omega(k\log q) IC μ , 1/10 ( F ) = Ω ( k s ) = Ω ( k log q ) .
Embedding a codeword into an LP
In this section, we develop the machinery that turns a pair of codewords into an instance of the promise problem. Recall from § 4.1.1 that each data point imposes one linear constraint on w w w , so that opt H h s = 0 \operatorname{opt}_{\mathcal{H}_{\mathrm{hs}}}=0 opt H hs = 0 means the resulting system is feasible, while opt H h s ≥ t \operatorname{opt}_{\mathcal{H}_{\mathrm{hs}}}\ge t opt H hs ≥ t means it remains infeasible even after removing any t − 1 t-1 t − 1 constraints. Our goal is therefore a linear program whose feasibility encodes the value of a hard function, and in which each private symbol contributes exactly one inequality.
Our representation is based on the fundamental-polytope approach from the literature on LP decoding of linear codes [21 , 19 ] . In that approach, each row of a parity-check matrix is represented by the convex hull of its satisfying local assignments, and the resulting local polytopes are intersected to obtain an LP relaxation of the code. We adapt this machinery in two ways. First, we represent every symbol of F q \mathbb{F}_q F q as an exposed vertex of a two-dimensional polygon, rather than using the standard indicator representation. Second, we interpret the symbol-selecting inequalities as private data records and the parity-check inequalities as public records.
The function we want to compute. Fix the public parity-check matrix H ∈ F q M × K H \in\mathbb{F}_q^{M \times K} H ∈ F q M × K with M = K / 2 M = K/2 M = K /2 , retaining all its rows, and let C H = { c ∈ F q K : H c T = 0 } C_H = \{c \in\mathbb{F}_q^K : Hc^{\mathsf{T}} = 0\} C H = { c ∈ F q K : H c T = 0 } have dimension k k k . We assume that the code is systematic, i.e., the first k k k coordinates carry the message and the remaining K − k K-k K − k are determined by the checks. Alice holds a codeword c ∈ C H c \in C_H c ∈ C H , Bob holds a codeword c ′ ∈ C H c' \in C_H c ′ ∈ C H , and they wish to compute
F ( c , c ′ ) = ⨁ i ∈ [ k ] Tr ( c i c i ′ ) ∈ { 0 , 1 } , F(c,c') = \bigoplus_{i \in[k]} \operatorname{Tr}(c_i c'_i) \in\{0,1\}, F ( c , c ′ ) = i ∈ [ k ] ⨁ Tr ( c i c i ′ ) ∈ { 0 , 1 } ,
where Tr \operatorname{Tr} Tr denotes the trace from F q \mathbb{F}_q F q to F 2 \mathbb{F}_2 F 2 . The computation has three parts: each party’s input must be a genuine codeword, that is, satisfy every parity check of H H H ; for every message coordinate i ∈ [ k ] i \in[k] i ∈ [ k ] , the bit Tr ( c i c i ′ ) \operatorname{Tr}(c_i c'_i) Tr ( c i c i ′ ) must be computed from the pair ( c i , c i ′ ) (c_i,c'_i) ( c i , c i ′ ) ; and these k k k bits must be combined by parity. We then plant a public condition on the output, namely that the resulting parity equals zero. Feasibility of the whole system is then equivalent to F ( c , c ′ ) = 0 F(c,c') = 0 F ( c , c ′ ) = 0 .
Encoding a field element. Fix an enumeration F q = { α 0 , … , α q − 1 } \mathbb{F}_q = \{\alpha_0,\ldots,\alpha_{q-1}\} F q = { α 0 , … , α q − 1 } and set e ( α j ) : = ( j , j 2 ) ∈ R 2 e(\alpha_j) := (j,j^2) \in\mathbb{R}^2 e ( α j ) := ( j , j 2 ) ∈ R 2 . To represent a field-valued variable we introduce a pair of real variables z i = ( z i ( 1 ) , z i ( 2 ) ) z_i = (z_i^{(1)},z_i^{(2)}) z i = ( z i ( 1 ) , z i ( 2 ) ) and impose the public constraint z i ∈ P q z_i \in P_q z i ∈ P q , where P q P_q P q is the convex hull of the q q q points e ( α 0 ) , … , e ( α q − 1 ) e(\alpha_0),\ldots,e(\alpha_{q-1}) e ( α 0 ) , … , e ( α q − 1 ) . Since these points lie on a parabola, they are in convex position, and P q P_q P q is described by
0 ≤ z i ( 1 ) ≤ q − 1 , z i ( 2 ) ≤ ( q − 1 ) z i ( 1 ) , z i ( 2 ) ≥ ( 2 j + 1 ) z i ( 1 ) − j ( j + 1 ) for j = 0 , … , q − 2. 0 \le z_i^{(1)} \le q-1,\qquad z_i^{(2)} \le(q-1)z_i^{(1)},\qquad z_i^{(2)} \ge(2j+1)z_i^{(1)} - j(j+1)\ \text{ for } j=0,\ldots,q-2. 0 ≤ z i ( 1 ) ≤ q − 1 , z i ( 2 ) ≤ ( q − 1 ) z i ( 1 ) , z i ( 2 ) ≥ ( 2 j + 1 ) z i ( 1 ) − j ( j + 1 ) for j = 0 , … , q − 2.
See Figure 6 . This constraint says only that z i z_i z i is some mixture of field elements. The private input supplies one further inequality that selects a particular vertex.
Lemma C.1 (Pinning a field element). Let ℓ j ( u , v ) : = 2 j u − v \ell_j(u,v) := 2ju-v ℓ j ( u , v ) := 2 j u − v . Then for every s ∈ { 0 , … , q − 1 } s \in\{0,\ldots,q-1\} s ∈ { 0 , … , q − 1 } ,
ℓ j ( e ( α s ) ) = 2 j s − s 2 = j 2 − ( s − j ) 2 ≤ j 2 , \ell_j(e(\alpha_s)) = 2js-s^2 = j^2-(s-j)^2 \le j^2, ℓ j ( e ( α s )) = 2 j s − s 2 = j 2 − ( s − j ) 2 ≤ j 2 , with equality if and only if s = j s=j s = j . Consequently, the constraints z i ∈ P q z_i \in P_q z i ∈ P q and ℓ j ( z i ) ≥ j 2 \ell_j(z_i) \ge j^2 ℓ j ( z i ) ≥ j 2 together force z i = e ( α j ) z_i=e(\alpha_j) z i = e ( α j ) .
Proof. The displayed identity is immediate. Since z i ∈ P q z_i \in P_q z i ∈ P q , we may write z i = ∑ s λ s e ( α s ) z_i = \sum_s \lambda_s e(\alpha_s) z i = ∑ s λ s e ( α s ) with λ s ≥ 0 \lambda_s \ge0 λ s ≥ 0 and ∑ s λ s = 1 \sum_s \lambda_s = 1 ∑ s λ s = 1 . As ℓ j \ell_j ℓ j is linear,
ℓ j ( z i ) = ∑ s λ s ℓ j ( e ( α s ) ) = j 2 − ∑ s λ s ( s − j ) 2 . \ell_j(z_i) = \sum_s \lambda_s \ell_j(e(\alpha_s)) = j^2-\sum_s \lambda_s(s-j)^2. ℓ j ( z i ) = s ∑ λ s ℓ j ( e ( α s )) = j 2 − s ∑ λ s ( s − j ) 2 .
The constraint ℓ j ( z i ) ≥ j 2 \ell_j(z_i) \ge j^2 ℓ j ( z i ) ≥ j 2 therefore forces ∑ s λ s ( s − j ) 2 ≤ 0 \sum_s \lambda_s(s-j)^2 \le0 ∑ s λ s ( s − j ) 2 ≤ 0 . Every term is nonnegative, so λ s = 0 \lambda_s=0 λ s = 0 for all s ≠ j s \ne j s = j , whence λ j = 1 \lambda_j=1 λ j = 1 and z i = e ( α j ) z_i=e(\alpha_j) z i = e ( α j ) . □
We call ℓ j ( z i ) ≥ j 2 \ell_j(z_i) \ge j^2 ℓ j ( z i ) ≥ j 2 the pin for α j \alpha_j α j ; see Figure 6 . The inequalities describing P q P_q P q are public and identical for every input, and only the choice among the q q q candidate pins depends on the data. Thus one private inequality specifies one of q q q field elements, which is exactly the accounting we need.
Imposing a public relation. The pins determine each symbol individually, but we also need to impose relations among several variables, such as a parity check or the trace of a product. Such relations are not linear, so we cannot write them directly. The following device converts any finite relation into linear inequalities.
Let R \mathcal{R} R be a public finite set of tuples ( a 1 , … , a r , b 1 , … , b r ) ∈ F q r × { 0 , 1 } r (a_1,\ldots,a_r,b_1,\ldots,b_r) \in\mathbb{F}_q^r \times\{0,1\}^r ( a 1 , … , a r , b 1 , … , b r ) ∈ F q r × { 0 , 1 } r , listing the assignments we wish to allow. Replace each field symbol a i a_i a i by the point e ( a i ) e(a_i) e ( a i ) , we describe the construction above, and leave the bits unchanged. Then, let Q R Q_R Q R be the convex hull of the resulting points in R 2 r + r ′ \mathbb{R}^{2r+r'} R 2 r + r ′ . These points are rational, so Q R Q_R Q R is a rational polytope and admits a finite description by linear inequalities, which we add to the linear program as public rows. If the polytope is lower-dimensional, this description includes equations for its affine hull, each written as two inequalities. This also covers q = 2 q=2 q = 2 . In the linear program this reads
( z 1 , … , z r , β 1 , … , β r ′ ) ∈ Q R , (z_1,\ldots,z_r,\beta_1,\ldots,\beta_{r'}) \in Q_R, ( z 1 , … , z r , β 1 , … , β r ′ ) ∈ Q R ,
where the z i z_i z i are symbol variables and the β j \beta_j β j are real variables playing the role of bits. By itself this constraint is weak, as it says only that the tuple is some mixture of allowed assignments. The point is that pinning some of the variables collapses the mixture.
Lemma C.2 (Forced relation). Fix nonnegative integers r , r ′ r,r' r , r ′ with r + r ′ > 0 r+r'>0 r + r ′ > 0 , and use the convention [ 0 ] = ∅ [0]=\emptyset [ 0 ] = ∅ . Let R ⊆ F q r × { 0 , 1 } r ′ R\subseteq\mathbb{F}_q^r\times\{0,1\}^{r'} R ⊆ F q r × { 0 , 1 } r ′ be a finite relation and let Q R Q_R Q R be as above. Fix index sets S ⊆ [ r ] S\subseteq[r] S ⊆ [ r ] and S ′ ⊆ [ r ′ ] S'\subseteq[r'] S ′ ⊆ [ r ′ ] , together with symbols a i ∈ F q a_i\in\mathbb{F}_q a i ∈ F q for i ∈ S i\in S i ∈ S and bits b j ∈ { 0 , 1 } b_j\in\{0,1\} b j ∈ { 0 , 1 } for j ∈ S ′ j\in S' j ∈ S ′ , and suppose that R R R contains exactly one tuple T ⋆ T^\star T ⋆ whose i i i -th symbol equals a i a_i a i for every i ∈ S i\in S i ∈ S and whose j j j th bit equals b j b_j b j for every j ∈ S ′ j\in S' j ∈ S ′ . If a point ( z , β ) ∈ Q R (z,\beta)\in Q_R ( z , β ) ∈ Q R satisfies z i = e ( a i ) z_i=e(a_i) z i = e ( a i ) for every i ∈ S i\in S i ∈ S and β j = b j \beta_j=b_j β j = b j for every j ∈ S ′ j\in S' j ∈ S ′ , then ( z , β ) (z,\beta) ( z , β ) equals the point generated by T ⋆ T^\star T ⋆ ; in particular every remaining coordinate is forced to the value it takes in T ⋆ T^\star T ⋆ .
Proof. We want to show that in any convex decomposition of ( z , β ) (z,\beta) ( z , β ) over the generating points, all the weight sits on the single generator coming from T ⋆ T^\star T ⋆ .
Write ( z , β ) = ∑ T ∈ R λ T v T (z,\beta)=\sum_{T\in R}\lambda_Tv_T ( z , β ) = ∑ T ∈ R λ T v T with λ T ≥ 0 \lambda_T\geq0 λ T ≥ 0 and ∑ T λ T = 1 \sum_T\lambda_T=1 ∑ T λ T = 1 , where v T v_T v T denotes the point generated by the tuple T T T . Define
L : = ∑ i ∈ S ℓ a i + ∑ j ∈ S ′ σ b j , σ 0 ( β ) : = − β , σ 1 ( β ) : = β , L:=\sum_{i\in S}\ell_{a_i}+\sum_{j\in S'}\sigma_{b_j},\qquad\sigma_0(\beta):=-\beta,\quad\sigma_1(\beta):=\beta, L := i ∈ S ∑ ℓ a i + j ∈ S ′ ∑ σ b j , σ 0 ( β ) := − β , σ 1 ( β ) := β ,
where ℓ α j \ell_{\alpha_j} ℓ α j denotes ℓ j \ell_j ℓ j , ℓ a i \ell_{a_i} ℓ a i acts on the i i i th symbol block and σ b j \sigma_{b_j} σ b j on the $jth bit coordinate. By Lemma C.1 , ℓ a i \ell_{a_i} ℓ a i is maximized over { e ( α ) : α ∈ F q } \{e(\alpha):\alpha\in\mathbb{F}_q\} { e ( α ) : α ∈ F q } exactly at e ( a i ) e(a_i) e ( a i ) , and clearly σ b j \sigma_{b_j} σ b j is maximized over { 0 , 1 } \{0,1\} { 0 , 1 } exactly at b j b_j b j . Writing M M M for the resulting maximum, we therefore have L ( v T ) ≤ M L(v_T)\leq M L ( v T ) ≤ M for every T ∈ R T\in R T ∈ R , with equality if and only if T T T agrees with the fixed assignment on S ∪ S ′ S\cup S' S ∪ S ′ . On the other hand L ( z , β ) = M L(z,\beta)=M L ( z , β ) = M by hypothesis, so linearity gives
∑ T λ T ( M − L ( v T ) ) = 0. \sum_T\lambda_T\bigl(M-L(v_T)\bigr)=0. T ∑ λ T ( M − L ( v T ) ) = 0.
Every summand is nonnegative, so λ T = 0 \lambda_T = 0 λ T = 0 whenever L ( v T ) < M L(v_T) < M L ( v T ) < M ; that is, only tuples agreeing with the fixed assignment receive positive weight. By hypothesis T ⋆ T^\star T ⋆ is the only such tuple, so λ T ⋆ = 1 \lambda_{T^\star} = 1 λ T ⋆ = 1 and ( z , β ) = v T ⋆ (z,\beta)=v_{T^\star} ( z , β ) = v T ⋆ . □
The four families of constraints. We can now assemble the linear program. Every part of the computation becomes an instance of the device above, and the private inputs enter through the pins alone.
Symbols. For every i ∈ [ K ] i \in[K] i ∈ [ K ] we introduce z i ∈ R 2 z_i \in\mathbb{R}^2 z i ∈ R 2 with z i ∈ P q z_i \in P_q z i ∈ P q , and Alice’s private input contributes the pin forcing z ˉ i = e ( c i ) \bar{z}_i=e(c_i) z ˉ i = e ( c i ) . Bob contributes an identical family for a second collection z 1 ′ , … , z K ′ z'_1,\ldots,z'_K z 1 ′ , … , z K ′ .
Parity checks. For every row a a a of H H H , we take R R R to be the set of assignments to the coordinates in the support of a a a that satisfy the check, and impose membership in the corresponding polytope. Applying this to both collections guarantees that pinned inputs are codewords.
Trace products. For every i ∈ [ k ] i \in[k] i ∈ [ k ] we introduce a real variable r i r_i r i and take R = { ( a , b , Tr ( a b ) ) : a , b ∈ F q } R=\{(a,b,\operatorname{Tr}(ab)):a,b\in\mathbb{F}_q\} R = {( a , b , Tr ( ab )) : a , b ∈ F q } . Once z i z_i z i and z i ′ z'_i z i ′ are pinned, Lemma C.2 forces r i = Tr ( c i c i ′ ) ∈ { 0 , 1 } r_i=\operatorname{Tr}(c_i c'_i)\in\{0,1\} r i = Tr ( c i c i ′ ) ∈ { 0 , 1 } .
Parity of the bits. We introduce p 0 , … , p k p_0,\ldots,p_k p 0 , … , p k , impose the public row p 0 = 0 p_0=0 p 0 = 0 , and for every j ∈ [ k ] j\in[k] j ∈ [ k ] take R R R to be the four even-parity binary triples, so that the constraint reads ( p j − 1 , r j , p j ) ∈ Q ⊕ (p_{j-1},r_j,p_j)\in Q_{\oplus} ( p j − 1 , r j , p j ) ∈ Q ⊕ . Once p j − 1 p_{j-1} p j − 1 and r j r_j r j are bits, Lemma C.2 forces p j = p j − 1 ⊕ r j p_j=p_{j-1}\oplus r_j p j = p j − 1 ⊕ r j .
Finally we plant the public row p k = 0 p_k=0 p k = 0 . Notice that these families are not independent: the pins force the symbols, the symbols force the trace bits, and the trace bits together with the running parities force the next parity. Each gadget produces integral values only because the previous one already has, so the constraints must be read as a chain rather than as a set.
Size of the embedding.
Lemma C.3 (Dimension and number of rows). The linear program above uses 4 K + 2 k + 1 4K+2k+1 4 K + 2 k + 1 real variables. If H H H has M M M rows of maximum support size ρ \rho ρ , the public rows together with all candidate private pins admit a rational description using at most
O ( K q ) + M q O ( ρ 2 ) + O ( k q 4 + k ) O(Kq)+Mq^{O(\rho^2)}+O(kq^4+k) O ( K q ) + M q O ( ρ 2 ) + O ( k q 4 + k ) inequalities. For the chosen ( 4 , 8 ) (4,8) ( 4 , 8 ) -biregular graph, M = K / 2 M=K/2 M = K /2 and ρ = 8 \rho=8 ρ = 8 , so both the public-row count and the full catalogue size are at most B K q a BKq^a B K q a for universal B , a B,a B , a .
Proof. The symbol blocks contribute 4 K 4K 4 K variables, the trace bits contribute k k k , and the running parities contribute k + 1 k+1 k + 1 .
For the rows, use a rational facet description in each polytope’s affine hull, together with equations defining that affine hull. The Upper Bound Theorem [36 ] bounds the number of facets of a polytope with N N N vertices and intrinsic dimension h ≥ 1 h\geq1 h ≥ 1 by O h ( N ⌊ h / 2 ⌋ ) O_h(N^{\lfloor h/2\rfloor}) O h ( N ⌊ h /2 ⌋ ) . In ambient dimension D D D , at most D D D affine-hull equations add at most 2 D 2D 2 D inequalities; a zero-dimensional polytope needs only these equations. All local ambient dimensions here are bounded by max { 2 ρ , 5 } \max\{2\rho,5\} max { 2 ρ , 5 } .
Symbol constraints and pins. Each of the 2 K 2K 2 K symbol blocks uses O ( q ) O(q) O ( q ) public inequalities for P q P_q P q , including the affine-hull equation when q = 2 q=2 q = 2 . Each block has q q q possible pins, although only one pin per block is selected for a dataset. Thus the public symbol rows and all 2 K q 2Kq 2 K q candidate pins use O ( K q ) O(Kq) O ( K q ) catalogue rows.
Parity checks. Fix a check with r a r_a r a nonzero entries. Its valid local assignments are the solutions of one linear equation in r a r_a r a unknowns over F q \mathbb{F}_q F q : any r a − 1 r_a-1 r a − 1 of the symbols may be chosen freely, and the remaining one is then determined, since its coefficient is nonzero. There are therefore q r a − 1 q^{r_a-1} q r a − 1 valid assignments, so the associated polytope has at most q r a − 1 q^{r_a-1} q r a − 1 vertices. Each symbol is represented by a point of R 2 \mathbb{R}^2 R 2 , so this polytope lies in dimension at most 2 r a 2r_a 2 r a . Applying the Upper Bound Theorem with N = q r a − 1 N=q^{r_a-1} N = q r a − 1 and D = 2 r a D=2r_a D = 2 r a bounds the facet count by O ( q r a ( r a − 1 ) ) O(q^{r_a(r_a-1)}) O ( q r a ( r a − 1 ) ) ; at most 4 r a 4r_a 4 r a inequalities describe the affine hull. Sum over all M M M checks for both words, including redundant checks, to obtain M q O ( ρ 2 ) Mq^{O(\rho^2)} M q O ( ρ 2 ) rows.
Trace products. The trace polytope is the convex hull of the q 2 q^2 q 2 points ( e ( a ) , e ( b ) , Tr ( a b ) ) (e(a),e(b),\operatorname{Tr}(ab)) ( e ( a ) , e ( b ) , Tr ( ab )) for a , b ∈ F q a,b\in\mathbb{F}_q a , b ∈ F q , which lie in R 5 \mathbb{R}^5 R 5 . The Upper Bound Theorem with N = q 2 N=q^2 N = q 2 and D = 5 D=5 D = 5 gives O ( q 4 ) = q O ( 1 ) O(q^4)=q^{O(1)} O ( q 4 ) = q O ( 1 ) facets, and there is one such polytope for each of the k k k message coordinates. Including at most ten inequalities for each affine hull, these contribute O ( k q 4 ) O(kq^4) O ( k q 4 ) public rows.
Parity accumulation. The polytope Q ⊕ Q_{\oplus} Q ⊕ is the convex hull of four points in R 3 \mathbb{R}^3 R 3 and therefore has a constant-size description; there is one for each of the k k k steps of the chain, and the endpoint equations p 0 = 0 p_0=0 p 0 = 0 and p k = 0 p_k=0 p k = 0 contribute four inequalities. Together these give O ( k ) O(k) O ( k ) inequalities, which is absorbed into the last term.
With ρ = 8 \rho=8 ρ = 8 , one may take a = 56 a=56 a = 56 , increasing a universal multiplicative constant to cover every family. All descriptions are rational and fixed before the inputs are specified. The conversion in Lemma B.5 adds one homogenizing coordinate and two anchor types; these are counted in Lemma D.3. Repetitions affect the dataset lengths, but not this count of domain points. □
Proof of the lower bound for half spaces
Step 1: construction. Fix an even K ≥ K 0 K\ge K_0 K ≥ K 0 , where K 0 K_0 K 0 is the universal minimum in Lemma B.2, increased so that δ K 0 ≥ 2 \delta K_0\ge2 δ K 0 ≥ 2 . Let q = 2 s q=2^s q = 2 s with s ≥ 1 s\ge1 s ≥ 1 . For each such K K K , fix once and for all a graph supplied by that lemma. This is a ( 4 , 8 ) (4,8) ( 4 , 8 ) -biregular graph on K K K variable nodes and M = K / 2 M=K/2 M = K /2 check nodes satisfying, for a universal constant δ > 0 \delta>0 δ > 0 ,
∣ Γ ( T ) ∣ > 2 ∣ T ∣ for every T ⊆ [ K ] with 0 < ∣ T ∣ < δ K . (19) |\Gamma(T)|>2|T| \qquad\text{for every } T\subseteq[K]\text{ with }0<|T|<\delta K. \tag*{(19)} ∣Γ ( T ) ∣ > 2∣ T ∣ for every T ⊆ [ K ] with 0 < ∣ T ∣ < δ K . (19)
where Γ ( a ) ⊆ [ K ] \Gamma(a)\subseteq[K] Γ ( a ) ⊆ [ K ] denotes the support of check a a a and Γ ( T ) \Gamma(T) Γ ( T ) the set of checks meeting T T T . The same chosen graph is used for every alphabet and every dataset size at this block length. Let H ∈ { 0 , 1 } M × K H\in\{0,1\}^{M\times K} H ∈ { 0 , 1 } M × K be its incidence matrix, viewed over F q \mathbb{F}_q F q , and define
C H : = { c ∈ F q K : H c T = 0 } , k : = dim F q ( C H ) . C_H:=\{c\in\mathbb{F}_q^K:Hc^{\mathsf T}=0\},\qquad k:=\dim_{\mathbb{F}_q}(C_H). C H := { c ∈ F q K : H c T = 0 } , k := dim F q ( C H ) .
Since rank ( H ) ≤ M = K / 2 \operatorname{rank}(H)\le M=K/2 rank ( H ) ≤ M = K /2 , we have k ≥ K / 2 k\ge K/2 k ≥ K /2 . After relabeling the coordinates, fix a systematic generator G = [ I k ∣ A ] ∈ F q k × K G=[I_k\mid A]\in\mathbb{F}_q^{k\times K} G = [ I k ∣ A ] ∈ F q k × K .
Alice holds u ∈ F q k u \in\mathbb{F}_q^k u ∈ F q k and Bob holds v ∈ F q k v \in\mathbb{F}_q^k v ∈ F q k . Set c = u G c = uG c = u G and c ′ = v G c' = vG c ′ = v G , and define
F ( u , v ) = ⨁ i = 1 k Tr F q / F 2 ( u i v i ) . (20) F(u,v) = \bigoplus_{i=1}^{k} \operatorname{Tr}_{\mathbb{F}_q/\mathbb{F}_2}(u_i v_i). \tag*{(20)} F ( u , v ) = i = 1 ⨁ k Tr F q / F 2 ( u i v i ) . (20)
Since G G G is systematic, F ( u , v ) = ⨁ i = 1 k Tr F q / F 2 ( c i c i ′ ) F(u,v) = \bigoplus_{i=1}^{k} \operatorname{Tr}_{\mathbb{F}_q/\mathbb{F}_2}(c_i c'_i) F ( u , v ) = ⨁ i = 1 k Tr F q / F 2 ( c i c i ′ ) , so the trace products may be formed from the first k k k coordinates of the codewords. We also write F ( c , c ′ ) F(c,c') F ( c , c ′ ) for this value.
By Section C, the pair ( c , c ′ ) (c,c') ( c , c ′ ) determines a rational linear program Λ ( c , c ′ ) \Lambda(c,c') Λ ( c , c ′ ) with 4 K + 2 k + O ( 1 ) 4K+2k+O(1) 4 K + 2 k + O ( 1 ) variables and at most C K q a CKq^a C K q a possible distinct rows, for universal constants C , a C,a C , a . Its only input-dependent rows are the K K K pins determined by c c c and the K K K pins determined by c ′ c' c ′ . We retain all M M M parity checks, including redundant ones, since the rigidity proof uses the graph structure rather than a basis. Write P H P_H P H for the set of z ∈ ( R 2 ) K z \in(\mathbb{R}^2)^K z ∈ ( R 2 ) K satisfying every public symbol and parity-check row, so that P H ⊆ P q K P_H \subseteq P_q^K P H ⊆ P q K and e ( c ) ∈ P H e(c) \in P_H e ( c ) ∈ P H for every c ∈ C H c \in C_H c ∈ C H ; note that P H P_H P H is only a relaxation of C H C_H C H and may contain further points.
Step 2: Correctness and the gap. Let Λ ( c , c ′ ) \Lambda(c,c') Λ ( c , c ′ ) denote the linear program consisting of all public rows together with the 2 K 2K 2 K private pins determined by c c c and c ′ c' c ′ .
Lemma D.1 (Feasibility decides F F F ). For every c , c ′ ∈ C H c,c' \in C_H c , c ′ ∈ C H , the system Λ ( c , c ′ ) \Lambda(c,c') Λ ( c , c ′ ) is feasible if and only if F ( c , c ′ ) = 0 F(c,c')=0 F ( c , c ′ ) = 0 .
Proof. Suppose F ( c , c ′ ) = 0 F(c,c')=0 F ( c , c ′ ) = 0 . Set z i = e ( c i ) z_i=e(c_i) z i = e ( c i ) and z i ′ = e ( c i ′ ) z'_i=e(c'_i) z i ′ = e ( c i ′ ) for i ∈ [ K ] i \in[K] i ∈ [ K ] , r i = Tr ( c i c i ′ ) r_i=\operatorname{Tr}(c_i c'_i) r i = Tr ( c i c i ′ ) for i ∈ [ k ] i \in[k] i ∈ [ k ] , and p j = ⨁ i ≤ j r i p_j=\bigoplus_{i \le j} r_i p j = ⨁ i ≤ j r i . Each pin is satisfied by Lemma C.1 ; each parity-check polytope is satisfied because H c T = H ( c ′ ) T = 0 Hc^{\mathsf{T}}=H(c')^{\mathsf{T}}=0 H c T = H ( c ′ ) T = 0 , so the corresponding local assignment is valid and its embedding is one of the generators; the trace and parity polytopes are satisfied for the same reason; and p k = F ( c , c ′ ) = 0 p_k=F(c,c')=0 p k = F ( c , c ′ ) = 0 satisfies the planted row.
Conversely, let ( z , z ′ , r , p ) (z,z',r,p) ( z , z ′ , r , p ) be a solution of Λ ( c , c ′ ) \Lambda(c,c') Λ ( c , c ′ ) . By Lemma C.1 the pins force z i = e ( c i ) z_i=e(c_i) z i = e ( c i ) and z i ′ = e ( c i ′ ) z'_i=e(c'_i) z i ′ = e ( c i ′ ) for every i i i . For each i ∈ [ k ] i \in[k] i ∈ [ k ] the trace relation assigns a unique bit to the pair ( c i , c i ′ ) (c_i,c'_i) ( c i , c i ′ ) , so Lemma C.2 forces r i = Tr ( c i c i ′ ) r_i=\operatorname{Tr}(c_i c'_i) r i = Tr ( c i c i ′ ) . Since p 0 = 0 p_0=0 p 0 = 0 and each r i r_i r i is a bit, induction on j j j with Lemma C.2 applied to Q ⊕ Q_{\oplus} Q ⊕ forces p j = ⨁ i ≤ j r i p_j=\bigoplus_{i \le j}r_i p j = ⨁ i ≤ j r i . In particular p k = F ( c , c ′ ) p_k=F(c,c') p k = F ( c , c ′ ) , and the planted row gives F ( c , c ′ ) = 0 F(c,c')=0 F ( c , c ′ ) = 0 . □
The feasibility equivalence by itself gives no quantitative guarantee after pins are removed. The next lemma supplies that guarantee: the codeword remains forced after a linear number of erasures, even inside the fractional relaxation P H P_H P H .
Lemma D.2 (LP-erasure rigidity). Let c ∈ C H c \in C_H c ∈ C H , let S ⊆ [ K ] S \subseteq[K] S ⊆ [ K ] with ∣ S ∣ < δ K |S|<\delta K ∣ S ∣ < δ K , and let z ∈ P H z \in P_H z ∈ P H satisfy z i = e ( c i ) z_i=e(c_i) z i = e ( c i ) for every i ∉ S i \notin S i ∈ / S . Then z = e ( c ) z=e(c) z = e ( c ) .
Proof. Let T : = { i ∈ [ K ] : z i ≠ e ( c i ) } T:=\{i \in[K]: z_i \ne e(c_i)\} T := { i ∈ [ K ] : z i = e ( c i )} and suppose T ≠ ∅ T \ne\varnothing T = ∅ . Since T ⊆ S T \subseteq S T ⊆ S we have 0 < ∣ T ∣ < δ K 0<|T|<\delta K 0 < ∣ T ∣ < δ K , so Equation (19) gives ∣ Γ ( T ) ∣ > 2 ∣ T ∣ |\Gamma(T)|>2|T| ∣Γ ( T ) ∣ > 2∣ T ∣ . Every coordinate lies in exactly four checks, so T T T is incident to 4 ∣ T ∣ 4|T| 4∣ T ∣ edges; if every check in Γ ( T ) \Gamma(T) Γ ( T ) met T T T at least twice, then 4 ∣ T ∣ ≥ 2 ∣ Γ ( T ) ∣ 4|T| \ge2|\Gamma(T)| 4∣ T ∣ ≥ 2∣Γ ( T ) ∣ , contradicting expansion. Hence some check a a a has Γ ( a ) ∩ T = { i ∗ } \Gamma(a) \cap T=\{i^*\} Γ ( a ) ∩ T = { i ∗ } for a single i ∗ i^* i ∗ .
Every j ∈ Γ ( a ) ∖ { i ∗ } j \in\Gamma(a)\setminus\{i^*\} j ∈ Γ ( a ) ∖ { i ∗ } lies outside T T T , so z j = e ( c j ) z_j=e(c_j) z j = e ( c j ) . As H a , i ∗ ≠ 0 H_{a,i^*}\ne0 H a , i ∗ = 0 , the relation of check a a a assigns a unique symbol to coordinate i ∗ i^* i ∗ given the other seven, so Lemma C.2 forces z i ∗ = e ( c i ∗ ) z_{i^*}=e(c_{i^*}) z i ∗ = e ( c i ∗ ) , contradicting i ∗ ∈ T i^* \in T i ∗ ∈ T . Hence T = ∅ T=\varnothing T = ∅ . □
Step 3: from the LP to fixed-size datasets. We now convert the linear program Λ ( c , c ′ ) \Lambda(c,c') Λ ( c , c ′ ) into size-n n n datasets for Alice and Bob via Lemma B.5 . Let R K , q \mathcal{R}_{K,q} R K , q be the finite set of all linear constraints $a_i^{\mathsf{T}}x \le b_i appearing in the public polytopes (with each equality written as two inequalities), together with all q q q candidate pins at each of Alice’s and Bob’s K K K symbol positions. Applying Lemma B.5 to R K , q \mathcal{R}_{K,q} R K , q yields a constant η K , q > 0 \eta_{K,q} > 0 η K , q > 0 and the corresponding points x i − , x i + ∈ R p + 1 x_i^{-}, x_i^{+} \in\mathbb{R}^{p+1} x i − , x i + ∈ R p + 1 , along with the anchors x 0 − = − e p + 1 x_0^{-} = -e_{p+1} x 0 − = − e p + 1 and x 0 + = e p + 1 x_0^{+} = e_{p+1} x 0 + = e p + 1 . Let X K , q ⊂ R p + 1 \mathcal{X}_{K,q} \subset\mathbb{R}^{p+1} X K , q ⊂ R p + 1 denote the resulting finite set of points; note that X K , q \mathcal{X}_{K,q} X K , q depends only on ( K , q ) (K,q) ( K , q ) and is independent of the inputs ( u , v ) (u,v) ( u , v ) and the testing parameters.
Let R p u b R_{\mathrm{pub}} R pub be the number of public LP rows. For positive integers J , L J,L J , L , Alice’s encoder E A J , L ( u ) E_{A}^{J,L}(u) E A J , L ( u ) consists of L L L copies of each of her K K K selected pin records and J J J copies of x 0 − x_0^{-} x 0 − . Bob’s encoder E B J , L ( v ) E_{B}^{J,L}(v) E B J , L ( v ) consists of L L L copies of each of his K K K selected pin records, J J J copies of each public row record, and J J J copies of x 0 + x_0^{+} x 0 + . In particular, their lengths are
N A = K L + J , N B = K L + ( R p u b + 1 ) J . (21) N_A = KL + J, \qquad N_B = KL + (R_{\mathrm{pub}} + 1)J. \tag*{(21)} N A = K L + J , N B = K L + ( R pub + 1 ) J . (21)
For these intermediate samples, opt denotes the same pooled-error sum, allowing unequal lengths; only the padded size-n n n inputs below are passed to the tester. Put D = ⌊ δ K ⌋ ≥ δ K / 2 D = \lfloor\delta K \rfloor\ge\delta K/2 D = ⌊ δ K ⌋ ≥ δ K /2 .
Lemma D.3 (Gap, diameter, and encoding resources). There are universal constants B , a ≥ 1 B, a \ge1 B , a ≥ 1 such that, for every even K ≥ K 0 K \ge K_0 K ≥ K 0 and every q = 2 s ≥ 2 q = 2^s \ge2 q = 2 s ≥ 2 , the preceding construction has a fixed domain X K , q \mathcal{X}_{K,q} X K , q in dimension at most 6 K + 2 6K + 2 6 K + 2 with ∣ X K , q ∣ ≤ B K q a |\mathcal{X}_{K,q}| \le BKq^a ∣ X K , q ∣ ≤ B K q a and R p u b + 1 ≤ B K q a R_{\mathrm{pub}} + 1 \le BKq^a R pub + 1 ≤ B K q a . For every J , L ∈ N J, L \in\mathbb{N} J , L ∈ N and every pair of secrets,
F ( u , v ) = 0 ⟹ opt H h s = 0 , F ( u , v ) = 1 ⟹ opt H h s ≥ min { J , D L } . F(u,v) = 0 \implies\operatorname{opt}_{\mathcal{H}_{hs}} = 0, \qquad F(u,v) = 1 \implies\operatorname{opt}_{\mathcal{H}_{hs}} \ge\min\{J,DL\}. F ( u , v ) = 0 ⟹ opt H h s = 0 , F ( u , v ) = 1 ⟹ opt H h s ≥ min { J , D L } . Each party’s encoder has length at most B K q a J + K L BKq^aJ + KL B K q a J + K L and replacement diameter at most K L KL K L . Whenever both lengths are at most n n n , the encoders can be padded to exactly n n n records per party with the same promise and diameter guarantees. The domain depends only on ( K , q ) (K,q) ( K , q ) .
Proof. The LP has p = 4 K + 2 k + 1 ≤ 6 K + 1 p = 4K + 2k + 1 \le6K + 1 p = 4 K + 2 k + 1 ≤ 6 K + 1 real variables. Lemma C.3 bounds its public rows and all candidate pins by O ( K q a ) O(Kq^a) O ( K q a ) for a fixed universal a a a . Lemma B.5 adds one coordinate and two anchor types. Taking B B B large enough proves the domain and row counts, and (21) gives the dataset lengths.
If F ( u , v ) = 0 F(u,v) = 0 F ( u , v ) = 0 , Lemma D.1 gives an exact feasible LP witness. Its homogenization classifies every record, including the anchors, correctly by Lemma B.5 .
Suppose F ( u , v ) = 1 F(u,v) = 1 F ( u , v ) = 1 and a halfspace makes fewer than min { J , D L } \min\{J,DL\} min { J , D L } errors. No public row or anchor can be misclassified, because each appears J J J times. Let S A , S B ⊆ [ K ] S_A, S_B \subseteq[K] S A , S B ⊆ [ K ] be the positions of misclassified pin records. Each such record appears L L L times, so ∣ S A ∣ , ∣ S B ∣ < D |S_A|, |S_B| < D ∣ S A ∣ , ∣ S B ∣ < D . Keep every public row and precisely the pins outside S A , S B S_A, S_B S A , S B . The halfspace is consistent with all these records and the normalization anchor. By Lemma B.5 , the corresponding original closed LP subsystem has an exact witness ( z ⋆ , z ′ ⋆ , r ⋆ , p ⋆ ) (z^\star,z'^\star,r^\star,p^\star) ( z ⋆ , z ′ ⋆ , r ⋆ , p ⋆ ) . The retained original pins give z i ⋆ = e ( c i ) z_i^\star= e(c_i) z i ⋆ = e ( c i ) for i ∉ S A i \notin S_A i ∈ / S A and z i ′ ⋆ = e ( c i ′ ) z_i'^\star= e(c_i') z i ′ ⋆ = e ( c i ′ ) for i ∉ S B i \notin S_B i ∈ / S B . Both symbol vectors satisfy all original public rows, so Lemma D.2 forces z ⋆ = e ( c ) z^\star= e(c) z ⋆ = e ( c ) and z ′ ⋆ = e ( c ′ ) z'^\star= e(c') z ′ ⋆ = e ( c ′ ) . The trace and parity relations now force p k ⋆ = F ( u , v ) = 1 p_k^\star= F(u,v) = 1 p k ⋆ = F ( u , v ) = 1 , contradicting the retained public condition p k ⋆ = 0 p_k^\star= 0 p k ⋆ = 0 .
For the diameter, fix an ordering with one block of L L L positions for each pin and fixed positions for the public rows and anchors. Changing a party’s secret replaces at most K L KL K L records; all other positions are identical. This bounds the replacement distance also when the datasets are regarded as multisets.
Finally, append n − N A n - N_A n − N A copies of x 0 − x_0^{-} x 0 − to Alice and n − N B n - N_B n − N B copies of x 0 + x_0^{+} x 0 + to Bob. These multiplicities depend only on the public parameters. Every Match witness already satisfies the anchors, while adding records cannot decrease the Far optimum. The fixed padding changes neither replacement diameter. Group privacy applies on the full size-n n n input space, so the padded encoders can be used directly in Lemma 2.2 . □ \square □
With J = ⌈ t ⌉ J = \lceil t \rceil J = ⌈ t ⌉ and L = ⌈ J / D ⌉ L = \lceil J/D \rceil L = ⌈ J / D ⌉ , the Far gap is at least t t t , the diameter is O ( t + K ) O(t + K) O ( t + K ) , and the sample cost is O ( K q a ( t + 1 ) + t + K ) O(Kq^a(t + 1) + t + K) O ( K q a ( t + 1 ) + t + K ) per party.
Sample-size-dependent lower bounds
The proof of Theorem 4.1 gives a lower bound of Ω ( d log ( m / d ) / ε ) \Omega(d\log(m/d)/\varepsilon) Ω ( d log ( m / d ) / ε ) whenever n ≳ m d / ε n \gtrsim md/\varepsilon n ≳ m d / ε . To obtain bounds for smaller datasets, we return to the encoders in Section D. Recall that Bob repeats each of its O ( K q a ) O(Kq^a) O ( K q a ) public records t t t times, where a a a is a universal constant. These repetitions contribute O ( K q a t ) O(Kq^at) O ( K q a t ) records, so the choice of block length and alphabet must also account for the dataset size. For smaller datasets, we shorten the block length or reduce the alphabet. This makes the encoders shorter, at the price of a weaker lower bound of order K log q / ε K\log q/\varepsilon K log q / ε .
We call a pair ( r , q ) (r,q) ( r , q ) a scale , where r r r is an even block length and q q q is a power of two. At scale ( r , q ) (r,q) ( r , q ) , we prove that
t ≳ r log q ε provided that n ε ≳ r 2 q a log q . t \gtrsim\frac{r\log q}{\varepsilon}
\qquad\text{provided that}\qquad
n\varepsilon\gtrsim r^2q^a\log q. t ≳ ε r log q provided that n ε ≳ r 2 q a log q .
Indeed, when t t t is below the claimed lower bound, the encoders use O ( r 2 q a log q / ε ) O(r^2q^a\log q/\varepsilon) O ( r 2 q a log q / ε ) records, which fit into datasets of size n n n under this condition. For n ε ≤ d 2 n\varepsilon\le d^2 n ε ≤ d 2 , with n ε n\varepsilon n ε above a universal constant, we use the binary alphabet and choose r = Θ ( n ε ) r=\Theta(\sqrt{n\varepsilon}) r = Θ ( n ε ) . This gives t ≳ n / ε t\gtrsim\sqrt{n/\varepsilon} t ≳ n / ε . For n ε > d 2 n\varepsilon>d^2 n ε > d 2 , we use r = Θ ( d ) r=\Theta(d) r = Θ ( d ) and choose the largest alphabet allowed by the sample and domain sizes. This gives
t ≳ d ε log ( 2 + min { m d , n ε d 2 } ) . t\gtrsim\frac{d}{\varepsilon}\log\left(2+\min\left\{\frac{m}{d},\frac{n\varepsilon}{d^2}\right\}\right). t ≳ ε d log ( 2 + min { d m , d 2 n ε } ) .
In particular, the bound reaches Ω ( d log ( m / d ) / ε ) \Omega(d\log(m/d)/\varepsilon) Ω ( d log ( m / d ) / ε ) once n ε ≳ m d n\varepsilon\gtrsim md n ε ≳ m d .
We also want the bounds for different sample sizes to hold on the same public domain. We therefore choose a domain X d , m \mathcal{X}_{d,m} X d , m containing all the scales needed for the two regimes, before n , t , ε n,t,\varepsilon n , t , ε are specified. Each scale ( r , q ) (r,q) ( r , q ) uses O ( r q a ) O(rq^a) O ( r q a ) points. By taking geometrically spaced block lengths and alphabets, we can include these scales while keeping the total number of points at most m m m . All logarithms in this section are to base two.
Lemma E.1 (Lower bound at a single scale). There are universal constants A , B , a ≥ 1 A,B,a\ge1 A , B , a ≥ 1 , b > 0 b>0 b > 0 , ε 0 ∈ ( 0 , 1 ] \varepsilon_0\in(0,1] ε 0 ∈ ( 0 , 1 ] , and an even integer r 0 ≥ 2 r_0\ge2 r 0 ≥ 2 such that the following holds. For every even r ≥ r 0 r\ge r_0 r ≥ r 0 and every q = 2 s q=2^s q = 2 s with s ∈ N s\in\mathbb{N} s ∈ N , there is a public domain X r , q ⊆ R 6 r + 2 \mathcal{X}_{r,q}\subseteq\mathbb{R}^{6r+2} X r , q ⊆ R 6 r + 2 that depends only on ( r , q ) (r,q) ( r , q ) and has ∣ X r , q ∣ ≤ B r q a |\mathcal{X}_{r,q}|\le Brq^a ∣ X r , q ∣ ≤ B r q a , such that for every 0 < ε ≤ ε 0 0<\varepsilon\le\varepsilon_0 0 < ε ≤ ε 0 and every n ∈ N n\in\mathbb{N} n ∈ N satisfying
n ε ≥ A r 2 q a log q (22) n\varepsilon\ge Ar^2q^a\log q \tag*{(22)} n ε ≥ A r 2 q a log q (22) every ε \varepsilon ε -private tester for H h s \mathcal{H}_{\mathrm{hs}} H hs over X r , q \mathcal{X}_{r,q} X r , q with resolution t t t at dataset size n n n satisfies
t ≥ b r log q ε . (23) t\ge b\frac{r\log q}{\varepsilon}. \tag*{(23)} t ≥ b ε r log q . (23)
Proof. Let K 0 K_0 K 0 be the minimum block length used in Section D. Choose an even r 0 ≥ K 0 r_0\ge K_0 r 0 ≥ K 0 large enough that δ r 0 ≥ 2 \delta r_0\ge2 δ r 0 ≥ 2 , where δ \delta δ is the expansion constant of Lemma B.2 . For every even r ≥ r 0 r\ge r_0 r ≥ r 0 , we use the domain X K , q \mathcal{X}_{K,q} X K , q of Lemma D.3 with K = r K=r K = r , padding its coordinates with zeros if necessary. The bounds on its dimension and size follow from that lemma, and the domain depends only on ( r , q ) (r,q) ( r , q ) . Recall also that D = ⌊ δ r ⌋ ≥ δ r / 2 D=\lfloor\delta r\rfloor\ge\delta r/2 D = ⌊ δ r ⌋ ≥ δ r /2 .
Let 0 < ε ≤ ε 0 0<\varepsilon\le\varepsilon_0 0 < ε ≤ ε 0 and let n n n satisfy (22) , where ε 0 ≤ 1 \varepsilon_0\le1 ε 0 ≤ 1 will be fixed at the end of the proof. Suppose toward a contradiction that there is an ε \varepsilon ε -private tester with resolution t < b r log q / ε t<br\log q/\varepsilon t < b r log q / ε , where b ≤ 1 b\le1 b ≤ 1 will also be fixed at the end. Since t t t is an integer, we apply Lemma D.3 with J = t J=t J = t and L = ⌈ t / D ⌉ L=\lceil t/D\rceil L = ⌈ t / D ⌉ . These choices give D L ≥ t DL\ge t D L ≥ t . Thus the encodings of two secrets u , v u,v u , v satisfy opt H h s = 0 \operatorname{opt}_{\mathcal{H}_{\mathrm{hs}}}=0 opt H hs = 0 when F ( u , v ) = 0 F(u,v)=0 F ( u , v ) = 0 , and opt H h s ≥ 1 \operatorname{opt}_{\mathcal{H}_{\mathrm{hs}}}\ge1 opt H hs ≥ 1 when F ( u , v ) = 1 F(u,v)=1 F ( u , v ) = 1 .
First, we check that the encoders fit into n n n records. Each encoder has length at most B r q a t + r L Brq^at+rL B r q a t + r L , including the public records and anchors. Since r L ≤ r t / D + r ≤ 2 t / δ + r rL\le rt/D+r\le2t/\delta+r r L ≤ r t / D + r ≤ 2 t / δ + r , we obtain
B r q a t + r L ≤ ( B + 2 δ + 1 ) r 2 q a log q ε ≤ n . (24) Brq^at+rL
\le\left(B+\frac{2}{\delta}+1\right)\frac{r^2q^a\log q}{\varepsilon}
\le n. \tag*{(24)} B r q a t + r L ≤ ( B + δ 2 + 1 ) ε r 2 q a log q ≤ n . (24)
The first inequality uses t < r log q / ε t < r \log q/\varepsilon t < r log q / ε , r r r , q a q^{a} q a , log q ≥ 1 \log q \ge1 log q ≥ 1 , and ε ≤ 1 \varepsilon\le1 ε ≤ 1 . The second follows from (22) , provided that A ≥ B + 2 / δ + 1 A \ge B + 2/\delta+ 1 A ≥ B + 2/ δ + 1 . By Lemma D.3 , we can therefore pad both encoders to exactly n n n records without changing their promise or encoding diameters.
Next, let k ≥ r / 2 k \ge r/2 k ≥ r /2 be the dimension of the code, and let ( u , v ) (u,v) ( u , v ) be uniform on F q k × F q k \mathbb{F}_{q}^{k} \times\mathbb{F}_{q}^{k} F q k × F q k . Running the tester on the padded encodings, and reading Match as 0 and Far as 1, gives a protocol computing F ( u , v ) F(u,v) F ( u , v ) with error at most 1 / 10 1/10 1/10 . By Lemma B.6 , its internal information cost is at least c I r log q c_I r \log q c I r log q for a universal c I > 0 c_I > 0 c I > 0 .
On the other hand, changing one party’s secret replaces at most r L rL r L of its records. Hence
Δ A + Δ B ≤ 2 r L ≤ 4 t δ + 2 r ≤ C Δ ( t + r ) (25) \Delta_A + \Delta_B \le2rL \le\frac{4t}{\delta} + 2r \le C_\Delta(t+r) \tag*{(25)} Δ A + Δ B ≤ 2 r L ≤ δ 4 t + 2 r ≤ C Δ ( t + r ) (25)
for a universal C Δ C_\Delta C Δ . Applying Lemma 2.2 to the same protocol gives
c I r log q ≤ C I ε ( t + r ) , C I : = C Δ ln 2 . c_I r \log q \le C_I\varepsilon(t+r), \qquad C_I := \frac{C_\Delta}{\ln2}. c I r log q ≤ C I ε ( t + r ) , C I := ln 2 C Δ .
We now fix
ε 0 : = min { 1 , c I 2 C I } , b : = min { 1 , c I 4 C I } . \varepsilon_0 := \min\left\{1,\frac{c_I}{2C_I}\right\}, \qquad b := \min\left\{1,\frac{c_I}{4C_I}\right\}. ε 0 := min { 1 , 2 C I c I } , b := min { 1 , 4 C I c I } .
Since ε ≤ ε 0 \varepsilon\le\varepsilon_0 ε ≤ ε 0 and log q ≥ 1 \log q \ge1 log q ≥ 1 , the term C I ε r C_I\varepsilon r C I ε r is at most c I r log q / 2 c_I r\log q/2 c I r log q /2 . Rearranging therefore gives
t ≥ c I 2 C I r log q ε > b r log q ε , t \ge\frac{c_I}{2C_I}\frac{r\log q}{\varepsilon} > b\frac{r\log q}{\varepsilon}, t ≥ 2 C I c I ε r log q > b ε r log q ,
contradicting the assumption on t t t . □
Remark E.2 (Larger domains). Lemma E.1 also applies to testers over any domain X ⊆ R d \mathcal{X} \subseteq\mathbb{R}^{d} X ⊆ R d , with d ≥ 6 r + 2 d \ge6r+2 d ≥ 6 r + 2 , containing a copy of X r , q \mathcal{X}_{r,q} X r , q padded with zeros. Indeed, restricting a tester to datasets supported on this copy preserves privacy and its completeness and soundness guarantees. Padding with zeros also preserves opt H s \operatorname{opt}_{\mathcal{H}_{\mathrm{s}}} opt H s : a normal w ∈ R d w \in\mathbb{R}^{d} w ∈ R d makes the same predictions on the copy as its first 6 r + 2 6r+2 6 r + 2 coordinates, and every normal in R 6 r + 2 \mathbb{R}^{6r+2} R 6 r + 2 extends to R d \mathbb{R}^{d} R d by adding zero coordinates. ◁
We next construct a single domain containing the scales needed for both regimes.
Lemma E.3 (A single domain for all scales). Let A A A , B B B , a a a , r 0 r_0 r 0 be the constants of Lemma E.1 . There are universal constants κ > 0 \kappa> 0 κ > 0 , C 0 ≥ 1 C_0 \ge1 C 0 ≥ 1 , and d 0 ∈ N d_0 \in\mathbb{N} d 0 ∈ N such that the following holds for every d ≥ d 0 d \ge d_0 d ≥ d 0 and m ≥ C 0 d m \ge C_0d m ≥ C 0 d . There are an integer h ≥ 0 h \ge0 h ≥ 0 and a public domain X d , m ⊆ R d \mathcal{X}_{d,m} \subseteq\mathbb{R}^{d} X d , m ⊆ R d with ∣ X d , m ∣ = m |\mathcal{X}_{d,m}| = m ∣ X d , m ∣ = m , depending only on ( d , m ) (d,m) ( d , m ) , such that R : = r 0 2 h R := r_0 2^h R := r 0 2 h satisfies
κ d 2 < R ≤ κ d , 6 R + 2 ≤ d , A 2 a R 2 ≤ d 2 16 , \frac{\kappa d}{2} < R \le\kappa d, \qquad6R+2 \le d, \qquad A2^aR^2 \le\frac{d^2}{16}, 2 κ d < R ≤ κ d , 6 R + 2 ≤ d , A 2 a R 2 ≤ 16 d 2 , and X d , m \mathcal{X}_{d,m} X d , m contains a copy of X r , q \mathcal{X}_{r,q} X r , q padded with zeros for every scale ( r , q ) (r,q) ( r , q ) in
{ ( r 0 2 j , 2 ) : 0 ≤ j ≤ h } ∪ { ( R , 2 s ) : s ≥ 1 , 2 a s ≤ m 8 B R } . (26) \left\{(r_0 2^j,2):0 \le j \le h\right\} \cup\left\{(R,2^s):s \ge1,\ 2^{as} \le\frac{m}{8BR}\right\}. \tag*{(26)} { ( r 0 2 j , 2 ) : 0 ≤ j ≤ h } ∪ { ( R , 2 s ) : s ≥ 1 , 2 a s ≤ 8 B R m } . (26)
Proof. Fix
κ : = min { 1 12 , 1 4 2 A a } , C 0 : = 8 B 2 a . \kappa:= \min\left\{\frac{1}{12},\frac{1}{4\sqrt{2}A^a}\right\}, \qquad C_0 := 8B2^a. κ := min { 12 1 , 4 2 A a 1 } , C 0 := 8 B 2 a .
Let R R R be the largest element of { r 0 2 j : j ≥ 0 } \{r_0 2^j:j \ge0\} { r 0 2 j : j ≥ 0 } that is at most κ d \kappa d κ d . Choose an integer d 0 ≥ max { 4 , r 0 / κ } d_0 \ge\max\{4,r_0/\kappa\} d 0 ≥ max { 4 , r 0 / κ } , so that r 0 ≤ κ d r_0 \le\kappa d r 0 ≤ κ d and this maximum exists. Maximality gives 2 R > κ d 2R > \kappa d 2 R > κ d . The remaining bounds on R R R follow from the choice of κ \kappa κ :
6 R + 2 ≤ d / 2 + 2 ≤ d , A 2 a R 2 ≤ A 2 a κ 2 d 2 ≤ d 2 / 16. (27) 6R+2 \le d/2+2 \le d, \qquad A2^aR^2 \le A2^a\kappa^2d^2 \le d^2/16. \tag*{(27)} 6 R + 2 ≤ d /2 + 2 ≤ d , A 2 a R 2 ≤ A 2 a κ 2 d 2 ≤ d 2 /16. (27)
For each scale in (26) , fix one construction of X r , q \mathcal{X}_{r,q} X r , q , including its graph, inequality descriptions, and the constant η \eta η of Lemma B.5 . Pad each domain with zeros to embed it in R d \mathbb{R}^{d} R d . All these embeddings fit because 6 r + 2 ≤ 6 R + 2 ≤ d 6r+2 \le6R+2 \le d 6 r + 2 ≤ 6 R + 2 ≤ d .
We count the points contributed by the two families separately. The binary scales ( r 0 2 j , 2 ) (r_0 2^j,2) ( r 0 2 j , 2 ) use at most
∑ j = 0 h B 2 a r 0 2 j < 2 B 2 a R \sum_{j=0}^{h} B2^{a}r_0 2^j < 2B2^{a}R j = 0 ∑ h B 2 a r 0 2 j < 2 B 2 a R
points. Since m ≥ C 0 d ≥ 8 B 2 a R m \ge C_0d \ge8B2^{a}R m ≥ C 0 d ≥ 8 B 2 a R , this is less than m / 4 m/4 m /4 .
For the alphabet scales ( R , 2 s ) (R,2^s) ( R , 2 s ) , let Q Q Q be the largest power of two satisfying Q a ≤ m / ( 8 B R ) Q^a \le m/(8BR) Q a ≤ m / ( 8 B R ) . The inequality m ≥ 8 B 2 a R m \ge8B2^aR m ≥ 8 B 2 a R ensures that Q ≥ 2 Q \ge2 Q ≥ 2 exists. These scales use at most
B R ∑ s = 1 log Q 2 a s ≤ B R Q a 1 − 2 − a ≤ 2 B R Q a ≤ m 4 BR\sum_{s=1}^{\log Q}2^{as} \le\frac{BRQ^a}{1-2^{-a}} \le2BRQ^a \le\frac{m}{4} B R s = 1 ∑ l o g Q 2 a s ≤ 1 − 2 − a B R Q a ≤ 2 B R Q a ≤ 4 m
points, where we used a ≥ 1 a \ge1 a ≥ 1 .
The union therefore has fewer than m / 2 m/2 m /2 points. We add distinct unused points until the domain has exactly m m m points. None of the encoded datasets uses these additional points. All choices have been made using only ( d , m ) (d,m) ( d , m ) , before the testing parameters are specified. □
We now choose a scale for each sample size.
Theorem E.4 (Lower bounds as a function of the sample size). There are universal constants c > 0 c>0 c > 0 , N 0 ≥ 1 N_0\ge1 N 0 ≥ 1 , C 0 ≥ 1 C_0\ge1 C 0 ≥ 1 , d 0 ∈ N d_0\in\mathbb{N} d 0 ∈ N , and ε 0 > 0 \varepsilon_0>0 ε 0 > 0 such that the following holds. Let d ≥ d 0 d\ge d_0 d ≥ d 0 and m ≥ C 0 d m\ge C_0d m ≥ C 0 d , and let X d , m \mathcal{X}_{d,m} X d , m be the domain of Lemma E.3 . For every 0 < ε ≤ ε 0 0<\varepsilon\le\varepsilon_0 0 < ε ≤ ε 0 and every n ∈ N n\in\mathbb{N} n ∈ N with n ε ≥ N 0 n\varepsilon\ge N_0 n ε ≥ N 0 , every ε \varepsilon ε -private tester for H h s \mathcal{H}_{\mathrm{hs}} H hs over X d , m \mathcal{X}_{d,m} X d , m with resolution t t t at dataset size n n n satisfies
t ≥ c { n / ε , N 0 ≤ n ε ≤ d 2 , d ε log ( 2 + min { m d , n ε d 2 } ) , n ε > d 2 . (28) t \ge c
\begin{cases}
\sqrt{n/\varepsilon}, & N_0 \le n\varepsilon\le d^2,\\
\frac{d}{\varepsilon}\log\left(2+\min\left\{\frac{m}{d},\frac{n\varepsilon}{d^2}\right\}\right), & n\varepsilon>d^2.
\end{cases} \tag*{(28)} t ≥ c { n / ε , ε d log ( 2 + min { d m , d 2 n ε } ) , N 0 ≤ n ε ≤ d 2 , n ε > d 2 . (28)
Proof. Use the constants and scales of Lemmas E.1 and E.3 , and set N : = n ε N:=n\varepsilon N := n ε and N 0 : = A 2 a 0 2 N_0:=A^2a_0^2 N 0 := A 2 a 0 2 . Enlarge d 0 d_0 d 0 if necessary so that d 2 ≥ N 0 d^2\ge N_0 d 2 ≥ N 0 . Every scale used below belongs to (26) . Thus Lemma E.1 applies to testers over X d , m \mathcal{X}_{d,m} X d , m by Remark E.2.
Case 1: N 0 ≤ N ≤ d 2 N_0\le N\le d^2 N 0 ≤ N ≤ d 2 . We use the binary alphabet and choose r r r to be the largest element of { r 0 2 j : 0 ≤ j ≤ h } \{r_0 2^j:0\le j\le h\} { r 0 2 j : 0 ≤ j ≤ h } satisfying A 2 a 2 r 2 ≤ N A^2a^2r^2\le N A 2 a 2 r 2 ≤ N . Such an r r r exists because A 2 a 0 2 = N 0 ≤ N A^2a_0^2=N_0\le N A 2 a 0 2 = N 0 ≤ N .
To obtain the claimed bound, we need r = Ω ( N ) r=\Omega(\sqrt{N}) r = Ω ( N ) . If r < R r<R r < R , the next scale 2 r 2r 2 r violates the constraint, so
A 2 a ( 2 r ) 2 > N ⟹ r > N 2 A 2 a . A2^a(2r)^2>N \quad\Longrightarrow\quad r>\frac{\sqrt{N}}{2\sqrt{A2^a}}. A 2 a ( 2 r ) 2 > N ⟹ r > 2 A 2 a N .
If r = R r=R r = R , then r > κ d / 2 ≥ κ N / 2 r>\kappa d/2\ge\kappa\sqrt{N}/2 r > κ d /2 ≥ κ N /2 , since N ≤ d 2 N\le d^2 N ≤ d 2 . Thus r ≥ c r N r\ge c_r\sqrt{N} r ≥ c r N for c r : = min { 1 / ( 2 A 2 a ) , κ / 2 } c_r:=\min\{1/(2\sqrt{A2^a}),\kappa/2\} c r := min { 1/ ( 2 A 2 a ) , κ /2 } .
For q = 2 q=2 q = 2 , the inequality N ≥ A 2 a r 2 N\ge A2^ar^2 N ≥ A 2 a r 2 is precisely the sample-size condition in Lemma E.1 . Applying that lemma gives
t ≥ b r ε ≥ b c r N ε = b c r n ε ε = b c r n ε . (29) t\ge b\frac{r}{\varepsilon}\ge bc_r\frac{\sqrt{N}}{\varepsilon}=bc_r\frac{\sqrt{n\varepsilon}}{\varepsilon}=bc_r\sqrt{\frac{n}{\varepsilon}}. \tag*{(29)} t ≥ b ε r ≥ b c r ε N = b c r ε n ε = b c r ε n . (29)
Case 2: N > d 2 N > d^2 N > d 2 . We use the largest block length R R R and choose the alphabet to satisfy both the domain-size and sample-size constraints. Define
Z : = min { m 8 B R , N A R 2 } , X : = min { m d , N d 2 } . Z := \min\left\{\frac{m}{8BR}, \frac{N}{AR^2}\right\}, \qquad X := \min\left\{\frac{m}{d}, \frac{N}{d^2}\right\}. Z := min { 8 B R m , A R 2 N } , X := min { d m , d 2 N } .
and let q = 2 s q = 2^s q = 2 s be the largest power of two satisfying
q a log q ≤ Z . (30) q^a \log q \le Z. \tag*{(30)} q a log q ≤ Z . (30)
The choice q = 2 q = 2 q = 2 is admissible. Indeed, m / ( 8 B R ) ≥ 2 a m/(8BR) \ge2^a m / ( 8 B R ) ≥ 2 a because m ≥ C 0 d ≥ 8 B 2 a R m \ge C_0d \ge8B2^aR m ≥ C 0 d ≥ 8 B 2 a R , while
N A R 2 > 16 ⋅ 2 a \frac{N}{AR^2} > 16 \cdot2^a A R 2 N > 16 ⋅ 2 a
follows from A 2 a R 2 ≤ d 2 / 16 < N / 16 A2^aR^2 \le d^2/16 < N/16 A 2 a R 2 ≤ d 2 /16 < N /16 .
Since log q ≥ 1 \log q \ge1 log q ≥ 1 , our choice gives
q a ≤ q a log q ≤ Z ≤ m 8 B R . q^a \le q^a \log q \le Z \le\frac{m}{8BR}. q a ≤ q a log q ≤ Z ≤ 8 B R m .
Thus ( R , q ) (R,q) ( R , q ) is one of the scales included in X d , m \mathcal{X}_{d,m} X d , m . We also have N ≥ A R 2 q a log q N \ge AR^2q^a\log q N ≥ A R 2 q a log q , so Lemma E.1 gives
t ≥ b R log q ε . t \ge b\frac{R\log q}{\varepsilon}. t ≥ b ε R log q .
It remains to show that log q \log q log q is large enough. By maximality, 2 q 2q 2 q violates (25), and therefore Z < ( 2 q ) a log ( 2 q ) = 2 a ( s + 1 ) ( s + 1 ) Z < (2q)^a\log(2q) = 2^{a(s+1)}(s+1) Z < ( 2 q ) a log ( 2 q ) = 2 a ( s + 1 ) ( s + 1 ) . Using Z ≥ 2 Z \ge2 Z ≥ 2 and log ( s + 1 ) ≤ s \log(s+1) \le s log ( s + 1 ) ≤ s , we obtain log ( 2 + Z ) ≤ 1 + log Z < 1 + a + ( a + 1 ) s ≤ 2 ( a + 1 ) s \log(2+Z) \le1+\log Z < 1+a+(a+1)s \le2(a+1)s log ( 2 + Z ) ≤ 1 + log Z < 1 + a + ( a + 1 ) s ≤ 2 ( a + 1 ) s . Also, R ≤ κ d R \le\kappa d R ≤ κ d implies Z ≥ c Z X Z \ge c_ZX Z ≥ c Z X for a universal c Z > 0 c_Z > 0 c Z > 0 , and hence log ( 2 + Z ) ≥ c ′ log ( 2 + X ) \log(2+Z) \ge c'\log(2+X) log ( 2 + Z ) ≥ c ′ log ( 2 + X ) for a universal c ′ > 0 c' > 0 c ′ > 0 . Combining these inequalities gives log q ≥ c q log ( 2 + X ) \log q \ge c_q\log(2+X) log q ≥ c q log ( 2 + X ) , where c q : = c ′ / ( 2 ( a + 1 ) ) c_q := c'/(2(a+1)) c q := c ′ / ( 2 ( a + 1 )) .
Finally, R > κ d / 2 R > \kappa d/2 R > κ d /2 , so
t ≥ b R log q ε ≥ b κ c q 2 d ε log ( 2 + X ) . t \ge b\frac{R\log q}{\varepsilon} \ge\frac{b\kappa c_q}{2}\frac{d}{\varepsilon}\log(2+X). t ≥ b ε R log q ≥ 2 bκ c q ε d log ( 2 + X ) .
This gives the claimed bound for N > d 2 N > d^2 N > d 2 . Taking c : = min { b c R , b κ c q / 2 } c := \min\{bc_R,b\kappa c_q/2\} c := min { b c R , bκ c q /2 } completes the proof. □
The two expressions agree up to universal constants when n ε ≍ d 2 n\varepsilon\asymp d^2 n ε ≍ d 2 , where both give Ω ( d / ε ) \Omega(d/\varepsilon) Ω ( d / ε ) . The theorem includes the resolution t = n t=n t = n . As with the large-sample result, the sample-size conditions are sufficient and are not asserted to be sharp.
Additional Algorithms for Small-dataset Regime
Proof of Theorem 4.9 . We write opt = opt H h s ( S A , S B ) \operatorname{opt} = \operatorname{opt}_{\mathcal{H}_{\mathrm{hs}}}(S_A,S_B) opt = opt H hs ( S A , S B ) . We first prove privacy and then turn to the utility proof.
We first prove the privacy guarantee. Fix the public signs. Replacing one record of Alice moves one unit of count from one ray to another, so it changes each ∑ R σ j , R a R \sum_R \sigma_{j,R}a_R ∑ R σ j , R a R by at most two. Therefore, the vector of the k k k sums has ℓ 1 \ell_1 ℓ 1 -sensitivity at most 2 k 2k 2 k , and by Lemma B.1 and composition property of DP [17 ] , Alice’s release is ε \varepsilon ε -DP. The same argument works for Bob. Since the public signs are independent of the datasets and the output is a post-processing of the two releases, the protocol satisfies ε \varepsilon ε -two-party DP. To show this algorithm achieves the claimed resolution, for j ∈ [ k ] j \in[k] j ∈ [ k ] , let D j = ∑ R σ j , R ( a R − b R ) D_j = \sum_R \sigma_{j,R}(a_R-b_R) D j = ∑ R σ j , R ( a R − b R ) be the difference of the two sums before adding the noise. Since k k k is a constant, the tail bound of the Laplace distribution and a union bound over the 2 k 2k 2 k noise terms show that, with probability at least 0.95, all of them are O ( 1 / ε ) O(1/\varepsilon) O ( 1/ ε ) in absolute value. In the rest of the proof we assume that this event holds. Then, by choosing C C C large enough, every noise term is at most n / 8 \sqrt{n}/8 n /8 , and hence ∣ Z A , j − Z B , j − D j ∣ ≤ n / 4 \lvert Z_{A,j}-Z_{B,j}-D_j\rvert\le\sqrt{n}/4 ∣ Z A , j − Z B , j − D j ∣ ≤ n /4 for every j j j .
Under the Far promise, Equation (15) gives a R = b R a_R=b_R a R = b R for every R R R . Thus D j = 0 D_j=0 D j = 0 and ∣ Z A , j − Z B , j ∣ ≤ n / 4 < τ \lvert Z_{A,j}-Z_{B,j}\rvert\le\sqrt{n}/4 < \tau ∣ Z A , j − Z B , j ∣ ≤ n /4 < τ for every j j j , and the algorithm outputs Far.
Under the Match promise, Equation (15) gives a R b R = 0 a_Rb_R=0 a R b R = 0 for every R R R . Let V = ∑ R ( a R − b R ) 2 V=\sum_R(a_R-b_R)^2 V = ∑ R ( a R − b R ) 2 . Since the counts are integers with disjoint supports,
V = ∑ R ( a R 2 + b R 2 ) ≥ ∑ R ( a R + b R ) = 2 n . V=\sum_R(a_R^2+b_R^2)\ge\sum_R(a_R+b_R)=2n. V = R ∑ ( a R 2 + b R 2 ) ≥ R ∑ ( a R + b R ) = 2 n .
The signs are independent, so E [ D j 2 ] = V \mathbb{E}[D_j^2]=V E [ D j 2 ] = V and E [ D j 4 ] ≤ 3 V 2 \mathbb{E}[D_j^4]\le3V^2 E [ D j 4 ] ≤ 3 V 2 . By the Paley–Zygmund inequality,
Pr [ D j 2 ≥ V / 2 ] ≥ ( E [ D j 2 ] ) 2 4 E [ D j 4 ] ≥ 1 12 , \Pr[D_j^2\ge V/2]\ge\frac{(\mathbb{E}[D_j^2])^2}{4\mathbb{E}[D_j^4]}\ge\frac{1}{12}, Pr [ D j 2 ≥ V /2 ] ≥ 4 E [ D j 4 ] ( E [ D j 2 ] ) 2 ≥ 12 1 ,
and in particular ∣ D j ∣ ≥ n \lvert D_j\rvert\ge\sqrt{n} ∣ D j ∣ ≥ n with probability at least 1 / 12 1/12 1/12 . The k k k repetitions are independent, so if k k k is large enough, with probability at least 0.95 there exists j ∈ [ k ] j\in[k] j ∈ [ k ] with ∣ D j ∣ ≥ n \lvert D_j\rvert\ge\sqrt{n} ∣ D j ∣ ≥ n . For this j j j , we have ∣ Z A , j − Z B , j ∣ ≥ n − n / 4 > τ \lvert Z_{A,j}-Z_{B,j}\rvert\ge\sqrt{n}-\sqrt{n}/4>\tau ∣ Z A , j − Z B , j ∣ ≥ n − n /4 > τ , and the algorithm outputs Match.
In both cases, the failure probability is at most 0.1, which completes the proof. □
Proof of Theorem 4.11. First of all, by Sauer–Shelah lemma (see Lemma 4.3 ), we can restrict H h s \mathcal{H}_{\mathrm{hs}} H hs to a finite discretization H 0 ⊆ H h s \mathcal{H}_0\subseteq\mathcal{H}_{\mathrm{hs}} H 0 ⊆ H hs since X \mathcal{X} X is public and each party can find the projection of the hypothesis class on X \mathcal{X} X .
For any finite multiset D D D over X \mathcal{X} X , define the local error at Alice and Bob as
f A ( h ; D ) : = ∑ x ∈ D 1 { h ( x ) ≠ − 1 } , f B ( h ; D ) : = ∑ x ∈ D 1 { h ( x ) ≠ + 1 } . f_A(h;D):=\sum_{x\in D}\mathbf{1}\{h(x)\ne-1\},\qquad f_B(h;D):=\sum_{x\in D}\mathbf{1}\{h(x)\ne+1\}. f A ( h ; D ) := x ∈ D ∑ 1 { h ( x ) = − 1 } , f B ( h ; D ) := x ∈ D ∑ 1 { h ( x ) = + 1 } .
Using this notation, we can rewrite the optimal error as
opt H h s ( S A , S B ) = min h ∈ H 0 ( f A ( h ; S A ) + f B ( h ; S B ) ) . \operatorname{opt}_{\mathcal{H}_{\mathrm{hs}}}(S_A,S_B)=\min_{h\in\mathcal{H}_0}\left(f_A(h;S_A)+f_B(h;S_B)\right). opt H hs ( S A , S B ) = h ∈ H 0 min ( f A ( h ; S A ) + f B ( h ; S B ) ) .
Then, we show how we can use the mechanism of [7 ] to perform the test. Fix a sufficiently large universal constant c 0 > 0 c_0>0 c 0 > 0 , and set
s : = ⌈ c 0 n 2 d t 2 ⌉ and D s : = X s . s:=\left\lceil\frac{c_0n^2d}{t^2}\right\rceil\qquad\text{and}\qquad\mathcal{D}_s:=\mathcal{X}^s. s := ⌈ t 2 c 0 n 2 d ⌉ and D s := X s .
The family D s \mathcal{D}_s D s is public and has cardinality m s m^s m s . We claim that, for every party P ∈ { A , B } P\in\{A,B\} P ∈ { A , B } and every S P ∈ X n S_P\in\mathcal{X}^n S P ∈ X n , some D P ⋆ ∈ D s D_P^\star\in\mathcal{D}_s D P ⋆ ∈ D s satisfies
max h ∈ H 0 ∣ f P ( h ; S P ) − n s f P ( h ; D P ⋆ ) ∣ ≤ t 16 . \max_{h\in\mathcal{H}_0}\left|f_P(h;S_P)-\frac{n}{s}f_P(h;D_P^\star)\right|\le\frac{t}{16}. h ∈ H 0 max f P ( h ; S P ) − s n f P ( h ; D P ⋆ ) ≤ 16 t .
Indeed, view S P S_P S P as defining an empirical distribution Unif ( S P ) \operatorname{Unif}(S_P) Unif ( S P ) over X \mathcal{X} X . Sample s s s records independently and uniformly from S P S_P S P , with replacement, and let D ∈ D s D\in\mathcal{D}_s D ∈ D s be the resulting multiset. Notice that 1 n f P ( h ; S P ) \frac{1}{n}f_P(h;S_P) n 1 f P ( h ; S P ) is the population error of h h h under Unif ( S P ) \operatorname{Unif}(S_P) Unif ( S P ) and 1 s f P ( h ; D ) \frac{1}{s}f_P(h;D) s 1 f P ( h ; D ) is its empirical average over the s s s draws. Since the class of halfspaces H h s \mathcal{H}_{\mathrm{hs}} H hs in R d \mathbb{R}^d R d has VC dimension d d d , standard VC uniform convergence guarantees that for a sufficiently large universal constant c 0 > 0 c_0 > 0 c 0 > 0 , a sample of size s = ⌈ c 0 d / α 2 ⌉ s = \lceil c_0d/\alpha^2\rceil s = ⌈ c 0 d / α 2 ⌉ with α = t / ( 16 n ) \alpha= t/(16n) α = t / ( 16 n ) satisfies
Pr [ max h ∈ H 0 ∣ 1 n f P ( h ; S P ) − 1 s f P ( h ; D ) ∣ > t 16 n ] ≤ 1 2 . \Pr\left[\max_{h\in\mathcal{H}_0}\left|\frac{1}{n}f_P(h;S_P)-\frac{1}{s}f_P(h;D)\right|>\frac{t}{16n}\right]\leq\frac{1}{2}. Pr [ h ∈ H 0 max n 1 f P ( h ; S P ) − s 1 f P ( h ; D ) > 16 n t ] ≤ 2 1 .
Multiplying both sides inside the probability by n n n shows that such a D P ⋆ ∈ D s D_P^\star\in\mathcal{D}_s D P ⋆ ∈ D s exists.
Then, we explain the protocol. Each party P P P independently applies the exponential mechanism with privacy parameter ϵ \epsilon ϵ to the public family D s \mathcal{D}_s D s , using the cost
q P ( S P , D ) : = max h ∈ H 0 ∣ f P ( h ; S P ) − n s f P ( h ; D ) ∣ . q_P(S_P,D):=\max_{h\in\mathcal{H}_0}\left|f_P(h;S_P)-\frac{n}{s}f_P(h;D)\right|. q P ( S P , D ) := h ∈ H 0 max f P ( h ; S P ) − s n f P ( h ; D ) .
Let D P D_P D P be its released synthetic dataset. After exchanging their releases, the parties compute
opt ^ : = n s min h ∈ H 0 ( f A ( h ; D A ) + f B ( h ; D B ) ) . \widehat{\operatorname{opt}}:=\frac{n}{s}\min_{h\in\mathcal{H}_0}\left(f_A(h;D_A)+f_B(h;D_B)\right). opt := s n h ∈ H 0 min ( f A ( h ; D A ) + f B ( h ; D B ) ) .
They output Match if opt ^ ≤ t / 2 \widehat{\operatorname{opt}}\leq t/2 opt ≤ t /2 , and Far otherwise.
The privacy proof is straightforward: For every fixed candidate D D D , changing one input record changes f P ( h ; S P ) f_P(h;S_P) f P ( h ; S P ) by at most one for every h h h . Therefore, q P ( S P , D ) q_P(S_P,D) q P ( S P , D ) has sensitivity at most one, and each release is ϵ \epsilon ϵ -DP by Lemma B.4 . The public parameters n n n and s s s remain fixed, so the same sensitivity bound holds on the full domain of finite input multisets.
Then, we prove the utility guarantee. By Lemma B.4 , for each party P P P , with probability at least 19 / 20 19/20 19/20 ,
q P ( S P , D P ) ≤ min D ∈ D s q P ( S P , D ) + 2 ϵ ( log ∣ D s ∣ + log 20 ) ≤ t 16 + 2 ϵ ( s log m + log 20 ) ≤ t 8 . \begin{aligned}
q_P(S_P,D_P)\leq\min_{D\in\mathcal{D}_s}q_P(S_P,D)+\frac{2}{\epsilon}(\log|\mathcal{D}_s|+\log20) \\
&\leq\frac{t}{16}+\frac{2}{\epsilon}(s\log m+\log20) \\
&\leq\frac{t}{8}.
\end{aligned} q P ( S P , D P ) ≤ D ∈ D s min q P ( S P , D ) + ϵ 2 ( log ∣ D s ∣ + log 20 ) ≤ 16 t + ϵ 2 ( s log m + log 20 ) ≤ 8 t .
The last inequality follows from Equation (16) , for a sufficiently large universal constant C C C , using s = O ( n 2 d / t 2 ) s=O(n^2d/t^2) s = O ( n 2 d / t 2 ) and t ≤ n t\leq n t ≤ n .
By a union bound, both releases satisfy this guarantee with probability at least 0.9 0.9 0.9 . On this event, the total error of every halfspace is approximated within t / 4 t/4 t /4 . Taking the minimum over h h h therefore gives
∣ opt ^ − opt H h s ( S A , S B ) ∣ ≤ t 4 . \left|\widehat{\operatorname{opt}}-\operatorname{opt}_{\mathcal{H}_{\mathrm{hs}}}(S_A,S_B)\right|\leq\frac{t}{4}. opt − opt H hs ( S A , S B ) ≤ 4 t .
If the pooled data are realizable, then opt ^ ≤ t / 4 \widehat{\operatorname{opt}}\leq t/4 opt ≤ t /4 , so the protocol outputs Match. If opt H h s ( S A , S B ) ≥ t \operatorname{opt}_{\mathcal{H}_{\mathrm{hs}}}(S_A,S_B)\geq t opt H hs ( S A , S B ) ≥ t , then opt ^ ≥ 3 t / 4 \widehat{\operatorname{opt}}\geq3t/4 opt ≥ 3 t /4 , so it outputs Far. This proves completeness and soundness. □