betaFeedback

WorksComputer Science

Differential rank preservation manifold learning

Abstract

Manifold learning methods like t-SNE and UMAP preserve local neighborhoods, but they do not preserve the ranking of the neighbors inside them. Two embeddings can return the same neighbor set while disagreeing on which point is closest. This loss of order degrades downstream applications that value the order in the embedding. We propose Smooth Rank t-SNE (SR-tSNE), which adds a differentiable rank term to the standard t-SNE loss. To enable efficient training, this term uses a sigmoid rank approximation to explicitly force a point's true high-dimensional neighbors to outrank unrelated intruders and retain their original order. We also introduce Ordinal DLP (O-DLP), an extension of the Degree of Locality Preservation (DLP) metric that credits a neighbor only when it returns at its original rank. Across eight benchmarks spanning image, text, speech, remote-sensing and single-cell data, SR-tSNE outperforms 12 baselines in rank preservation and neighborhood overlap. Ablation studies demonstrate our rank term alone boosts O-DLP by 5% to 28%, a gain not achievable using standard pairwise loss. This advantage transfers to downstream applications. SR-tSNE achieves the best label-transfer performance across seven vision datasets. For cell-type discovery across six single-cell datasets, it ranks first in 17 of 18 dataset-metric combinations, whether clustering resolution is selected with or without cell-type labels. In intraday trading, SR-tSNE achieves higher returns, lower risk, and lower turnover than the baselines. To our knowledge, SR-tSNE is the first t-SNE-family method with explicit order preservation.

Introduction

Manifold learning preserves which points are neighbors in the embedding, not the order among them. Methods such as t-SNE [1], UMAP [2], PHATE [3] and TriMap [4] embed data from a high-dimensional input space while seeking to preserve neighborhood relationships. However, what they preserve is the membership of that neighborhood; the order ranking inside it is left free. An embedding can recover exactly the right neighbors and still shuffle them, putting the nearest neighbor at rank 5 and the second-nearest at rank 1. This can make downstream analyses that rely on neighbor order in the embedding less effective.

The challenge. Many downstream tasks rely on neighbor order as well as set membership. In vision-based retrieval and label transfer using the nearest neighbor, the top-ranked neighbor directly determines the predicted label; scrambling positions can introduce labeling errors. In single-cell clustering, community detection builds graphs from top-kk lists; rank errors that change these lists’ membership can alter graph connectivity and split homogeneous cell populations into artificial clusters. In algorithmic trading, marker selection algorithms compute rank-correlation metrics between input and embedding spaces to flag trade signals; ordinal shifts can alter entry timing. Preserving both neighborhood membership and neighbor order allows the embedding to more faithfully represent the local structure of the input data.

The loss of neighbor order in manifold learning stems directly from how standard losses are formulated and how embedding performance is evaluated. Both current optimization objectives and downstream metrics are entirely order-agnostic. For example, t-SNE minimizes the KL divergence based purely on pairwise similarities, never tracking where a neighbor lands. Consequently, two embeddings can achieve nearly identical KL loss while ordering their neighbors completely differently. Similarly, the Degree of Locality Preservation (DLP) [5], a manifold explainability metric, treats each neighborhood as an unordered set, assigning identical credit whether a true neighbor sits at rank 1 or rank kk.

Furthermore, many manifold learning methods focus exclusively on neighborhood membership, scalability, or global structure [2, 3, 6, 7, 8], deliberately sacrificing ordinal fidelity. The crowding problem further bounds how much neighborhood structure two dimensions can hold [9], and order is precisely what these methods give up. While methods like UMAP, PHATE, and LocalMAP improve local preservation or scalability, none require a retained neighbor to maintain its exact rank in the embedding. Practitioners are thus forced to interpret visualizations for structural relations that the underlying embedding principles never guarantee [10]. Although NeRV [11, 12] incorporates retrieval-oriented trade-offs into manifold learning, its precision–recall criteria still evaluate set membership rather than intra-neighborhood order.

SR-tSNE. To address this challenge, we introduce Smooth Rank t-SNE (SR-tSNE), a method designed to preserve both neighborhood membership and exact neighbor order. SR-tSNE augments the standard t-SNE loss with a single differentiable rank term. For each data point, this term optimizes two criteria at once. It encourages true input-space neighbors to rank ahead of embedding-space intruders, the points outside the corresponding input-space neighborhood. It also penalizes any reshuffling among the true neighbors themselves. Using a sigmoid formulation, discrete ranks are approximated by differentiable values, allowing direct optimization via standard gradient descent while a logarithmic discount prioritizes the nearest ranks. A single regularization weight, λ\lambda, governs the balance between neighborhood preservation and rank preservation.

O-DLP. To evaluate how well SR-tSNE preserves neighbor order, we propose Ordinal Degree of Locality Preservation (O-DLP). While standard DLP checks only whether input-space neighbors remain neighbors in the embedding, regardless of their order, O-DLP credits a retained neighbor only when it appears at its exact original rank. This enables O-DLP to distinguish an embedding that faithfully preserves the original neighbor order from one that merely recovers the same neighbors in a shuffled order.

Empirical evaluations demonstrate that SR-tSNE more effectively preserves neighbor order and improves downstream performance. Benchmarked against 12 baseline methods across eight diverse datasets: spanning computer vision, text, speech, remote sensing, and single-cell transcriptomics, SR-tSNE consistently achieves the highest O-DLP and neighborhood overlap scores on every dataset. These gains also extend to neighborhood preservation across scales, as measured by the area under the RNXR_{NX} curve. SR-tSNE achieves the highest scores under both local and global evaluation regimes on MNIST and CIFAR-10. In downstream applications, SR-tSNE outperforms competing methods in image retrieval, label transfer, and single-cell cell-type discovery, while also delivering higher risk-adjusted returns in a rank-sensitive algorithmic trading pipeline.

Our contributions. (1) Method: SR-tSNE augments t-SNE with end-to-end differentiable rank regularization to penalize intruders and preserve exact neighbor order. (2) Theory: We prove zero loss at perfect order, exponential soft-to-exact rank convergence, strict order preference where t-SNE/UMAP lack it, and an upper bound on order deficit. (3) Metric: O-DLP extends DLP from neighborhood membership to exact-rank fidelity. (4) Benchmarks: Across eight datasets, SR-tSNE outperforms 12 baselines in O-DLP and neighborhood overlap, with higher RNXR_{NX} AUC. (5) Utility: We demonstrate gains in label transfer, single-cell cell-type discovery, and algorithmic trading; the regularizer extends to any differentiable neighborhood-embedding objective.

Neighbor-preserving manifold learning methods keep local affinities or neighborhood membership, not order. t-SNE [1], UMAP [2] and PHATE [3] optimize different similarity objectives. Laplacian Eigenmaps [6], LLE [13] and PaCMAP [14] instead preserve local graph or distance relationships. NeRV [12] incorporates a retrieval-oriented trade-off into a manifold learning objective and is the closest such objective to ours. All of them preserve local relationships. None of them asks a neighbor to come back to its own place in the list. SR-tSNE keeps t-SNE as it is and adds one term for that place.

Recent refinements modify neighborhood selection rather than internal ordering. LocalMAP [7] dynamically adapts neighbor graphs, FUnc-SNE [15] drops the fixed neighborhood size, and preprocessing techniques denoise the manifold prior to embedding [16]. While these methods adjust which candidate points are embedded, they leave the order within the neighborhood unconstrained. DREAMS [8] regularizes t-SNE by trading local fidelity for global PCA alignment; in contrast, SR-tSNE regularizes t-SNE specifically to enforce fine-grained local neighbor order.

Rank- and comparison-based methods are the closest conceptual relatives, but they address order incompletely. Triplet-based approaches such as TriMap [4] and t-STE [17] sample isolated relative comparisons, leaving the exact rank order of an entire neighborhood free. Conversely, purely ordinal methods like RankVisu [18] and local ordinal embedding [19] rely solely on discrete rank relations, entirely discarding the continuous distance densities that preserve cluster geometry. SR-tSNE bridges this gap: it retains t-SNE’s continuous similarity modeling while adding a smooth, listwise penalty that jointly aligns the full input-space neighbor ranking and expels embedding-space intruders.

Existing evaluation metrics assess rank preservation at varying granularities, but stop short of requiring exact positional fidelity. The co-ranking matrix [20] provides the overarching framework, while summary metrics such as QNXQ_{NX} and RNXR_{NX} [20, 21] evaluate overall neighborhood shifts and serve as standard embedding benchmarks [22]. At the local level, DLP [5] measures unordered neighborhood overlap, and nDCG provides a graded score that tolerates position shifts. O-DLP complements these metrics by enforcing the strictest criterion: a retained neighbor earns credit if and only if it recovers its exact original rank.

Methodology

SR-tSNE Overview. SR-tSNE augments the standard t-SNE objective with a differentiable regularizer that explicitly enforces neighbor order. Let X∈Rn×dX \in\mathbb{R}^{n \times d} denote the input-space data and Y∈Rn×2Y \in\mathbb{R}^{n \times2} the target low-dimensional embedding coordinates. Standard t-SNE [1] minimizes the Kullback–Leibler divergence KL⁡(P∥Q)\operatorname{KL}(P \Vert Q). Here P=[pij]P = [p_{ij}] holds symmetrized Gaussian affinities computed from input-space distances, with per-point bandwidths σi\sigma_i calibrated to a target perplexity. Q=[qij]Q = [q_{ij}] holds Student-tt probabilities computed from embedding-space distances, with qij∝(1+∥yi−yj∥2)−1q_{ij} \propto(1 + \lVert y_i - y_j \rVert^2)^{-1}.

The overall objective of SR-tSNE is formulated as:

Ltotal(Y)=KL⁡(P∥Q)⏟neighbor similarity+λ⋅Lrank(Y)⏟rank loss(1)\mathcal{L}_{\mathrm{total}}(Y) = \underbrace{\operatorname{KL}(P \Vert Q)}_{\text{neighbor similarity}} + \lambda\cdot\underbrace{\mathcal{L}_{\mathrm{rank}}(Y)}_{\text{rank loss}} \tag*{(1)}

where Lrank\mathcal{L}_{\mathrm{rank}} is the smooth rank-regularization term detailed in § 3.1, and λ≥0\lambda\ge0 balances neighborhood affinity against exact ordinal preservation. When λ=0\lambda= 0, the objective reduces to standard t-SNE. By optimizing Ltotal\mathcal{L}_{\mathrm{total}}, SR-tSNE encourages embeddings that preserve both local neighborhood membership and the fine-grained ordinal ranking of true neighbors.

Smooth Rank Loss

Standard t-SNE minimizes KL divergence between pairwise similarity distributions, leaving the internal rank order of neighbors unconstrained. To resolve this, SR-tSNE introduces a differentiable rank loss Lrank\mathcal{L}_{\mathrm{rank}} that enforces two explicit constraints. First, true input-space neighbors must outrank any false intruders drifting into the embedding neighborhood. Second, true neighbors must retain their exact original order, with nearest neighbors prioritized via a logarithmic discount (Figure 1).

Mechanism of the smooth rank regularization on three MNIST queries

Figure 1. Mechanism of the smooth rank regularization on three MNIST queries (k=6k = 6). Solid discs show true input-space neighbors positioned in the embedding space and labeled by original rank: teal preserves exact rank, while coral is displaced. Dashed circles mark embedding-space intruders (from outside the top-30). Lrank\mathcal{L}_{\mathrm{rank}} simultaneously repels intruders and restores neighbor order, improving exact-rank preservation (O-DLP) by 19% over t-SNE on the full 70,000-sample MNIST dataset (Table 4); the gain over the equal-budget control is +17% (Table 2).

1. Candidate Pool Construction. For each point ii, we define its candidate pool by combining two disjoint groups:

  • True Neighbors (Ni\mathcal{N}_i): The ordered sequence of kk nearest neighbors in the input space, (Ni(1),…,Ni(k)\mathcal{N}_i(1), \ldots, \mathcal{N}_i(k)). This list is computed once on XX and fixed. The true input-space rank of candidate j≤kj \le k is simply ri,jin=jr^{\mathrm{in}}_{i,j} = j.

  • Intruders (Ii\mathcal{I}_i): The mm closest embedding points to yiy_i that do not belong to Ni∪{i}\mathcal{N}_i \cup\{i\}. These points represent embedding-space artifacts (impostors [23]) and are updated every TrT_r iterations.

Concatenating both sets forms a candidate pool Ci=Ni⊕IiC_i = \mathcal{N}_i \oplus\mathcal{I}_i of size k+mk + m. For each candidate j∈{1,…,k+m}j \in\{1, \ldots, k + m\}, we compute its squared Euclidean distance to yiy_i in the embedding space: di,j2=∥yi−yCi(j)∥22d_{i,j}^{2} = \lVert y_i - y_{C_i(j)} \rVert_2^2.

Differentiable Soft Rank

Sorting distances directly yields discrete, non-differentiable ranks with zero gradient almost everywhere. To enable backpropagation, we approximate the discrete indicator 1[di,l2<di,j2]\mathbb{1}[d_{i,l}^{2} < d_{i,j}^{2}] using a continuous logistic sigmoid σ(z)=(1+e−z)−1\sigma(z) = (1 + e^{-z})^{-1} with temperature τ>0\tau> 0 [24]. The soft embedding rank of candidate jj is computed by summing the smoothed pairwise comparisons:

ri,jemb=1+∑l=1l≠jk+mσ(di,j2−di,l2τ).(2)r_{i,j}^{\mathrm{emb}} = 1 + \sum_{\substack{l=1 \\ l \ne j}}^{k+m} \sigma\left(\frac{d_{i,j}^{2} - d_{i,l}^{2}}{\tau}\right). \tag*{(2)}

Intuitively, ri,jembr_{i,j}^{\mathrm{emb}} represents 1 plus the effective number of candidates placed closer to yiy_i than candidate jj. As τ→0+\tau\to0^{+}, ri,jembr_{i,j}^{\mathrm{emb}} converges to the exact integer rank.

Loss via Discounted Gain

Only the kk true neighbors earn positive relevance; intruders carry zero gain and act solely as rank obstructions. We assign an input-space relevance score of k−j+1k-j+1 to the jj-th true neighbor. The actual discounted gain GiG_i in the embedding space is:

Gi=∑j=1kk−j+1log⁡2(1+ri,jemb).(3)G_i = \sum_{j=1}^{k} \frac{k-j+1}{\log_2(1+r_{i,j}^{\mathrm{emb}})}. \tag*{(3)}

The maximum possible gain G⋆G^{\star} occurs under perfect preservation, where every true neighbor recovers its exact original rank (ri,jemb=jr_{i,j}^{\mathrm{emb}}=j) ahead of all intruders:

G⋆=∑j=1kk−j+1log⁡2(1+j).(4)G^{\star} = \sum_{j=1}^{k} \frac{k-j+1}{\log_2(1+j)}. \tag*{(4)}

The total rank loss is the average normalized deficit across all nn data points:

Lrank(Y)=1−1n∑i=1nGiG⋆.(5)\mathcal{L}_{\mathrm{rank}}(Y) = 1 - \frac{1}{n}\sum_{i=1}^{n}\frac{G_i}{G^{\star}}. \tag*{(5)}

Here, Lrank(Y)∈[0,1]\mathcal{L}_{\mathrm{rank}}(Y) \in[0,1]. It reaches zero if and only if all kk true neighbors occupy ranks 1,…,k1,\ldots,k in their exact input order without any intruder interference. When m=0m=0, this formulation smoothly reduces to a differentiable Spearman rank correlation (Appendix A.8).

Ordinal Degree of Locality Preservation (O-DLP)

We refer to Lrank(Y)\mathcal{L}_{\mathrm{rank}}(Y) in Eq. (5) as the continuous (soft) rank loss Lranksoft(Y)\mathcal{L}^{\mathrm{soft}}_{\mathrm{rank}}(Y) when evaluated with the soft ranks ri,jembr^{\mathrm{emb}}_{i,j}, and denote its per-point contribution as Lrank,isoft=1−Gi/G⋆\mathcal{L}^{\mathrm{soft}}_{\mathrm{rank},i}=1-G_i/G^{\star}.

Standard Degree of Locality Preservation (DLP) [5] evaluates unordered neighborhood overlap. Specifically, let Axi,kA_{x_i,k} denote the set of kk-nearest neighbors of xix_i in the input space, and let Byi,kB_{y_i,k} denote the corresponding kk-nearest neighbors of yiy_i in the embedding space. DLP measures the average set retention across all nn points:

DLP⁡(k)=1nk∑i=1n∣Axi,k∩Byi,k∣.(6)\operatorname{DLP}(k)=\frac{1}{nk}\sum_{i=1}^{n}\left|A_{x_i,k}\cap B_{y_i,k}\right|. \tag*{(6)}

Because DLP treats neighborhoods strictly as unordered sets, a neighbor drifting from rank 1 to rank kk incurs zero penalty as long as it remains within Byi,kB_{y_i,k}.

O-DLP. O-DLP extends DLP to evaluate exact positional alignment. Let πiin(r)\pi_i^{\mathrm{in}}(r) and πiemb(r)\pi_i^{\mathrm{emb}}(r) denote the index of the rr-th nearest neighbor of point ii in the input space and embedding space, respectively. O-DLP measures the proportion of exact rank matches across all points:

O-DLP⁡(k)=1nk∑i=1n∑r=1k1[πiin(r)=πiemb(r)],(7)\operatorname{O\text{-}DLP}(k)=\frac{1}{nk}\sum_{i=1}^{n}\sum_{r=1}^{k}\mathbf{1}\left[\pi_i^{\mathrm{in}}(r)=\pi_i^{\mathrm{emb}}(r)\right], \tag*{(7)}

where 1[⋅]\mathbf{1}[\cdot] is the indicator function. A score of 1 signifies that all kk-neighborhoods preserve identical rank order across both spaces, whereas 0 indicates that no neighbor reappears at its original position. Because rank-preserved neighbors necessarily belong to the intersection Axi,k∩Byi,kA_{x_i,k}\cap B_{y_i,k}, O-DLP forms a strict lower bound on DLP:

O-DLP⁡(k)≤DLP⁡(k),(8)\operatorname{O\text{-}DLP}(k)\leq\operatorname{DLP}(k), \tag*{(8)}

with equality holding if and only if every retained neighbor maintains its exact rank.

Graded evaluation via nDCG. While O-DLP establishes the strictest standard by requiring exact positional alignment, we also report Normalized Discounted Cumulative Gain (nDCG) [24], the graded form of the quantity the rank term optimizes, which credits a neighbor by the position it reaches rather than only at its exact rank. Crucially, while soft metrics like nDCG can remain deceptively high under local rank inversions, O-DLP exposes whether individual positions are truly preserved. Evaluating both metrics ensures that SR-tSNE improves both coarse neighborhood placement (captured by nDCG) and fine-grained ordinal sequences (captured by O-DLP). Downstream steps such as clustering rest on both parts of a neighborhood. They need a high degree of locality preservation, and they need the order inside it. Eq. (8) bounds one by the other, and Theorem 4(i) separates O-DLP into membership times order. SR-tSNE leads on both.

SR-tSNE Optimization Details

SR-tSNE optimizes the total loss Ltotal(Y)=KL⁡(P∥Q)+λLrank(Y)\mathcal{L}_{\mathrm{total}}(Y) = \operatorname{KL}(P\|Q) + \lambda\mathcal{L}_{\mathrm{rank}}(Y) in two stages: a deterministic warm start followed by joint divergence and refinement.

  1. Two-stage optimization. A standard Barnes–Hut t-SNE [25] warm start (e.g., PCA initialization, perplexity 30, learning rate max⁡(n/12,50)\max(n/12, 50), early exaggeration 12) runs for 1,000 iterations to establish a coarse global layout. Joint refinement then runs for another T=1,000T = 1{,}000 iterations without early exaggeration. Throughout refinement, we set k=10k = 10 neighbors and m=10m = 10 intruders. Every Tr=50T_r = 50 iterations, intruder sets Ii\mathcal{I}_i are refreshed by identifying the mm points nearest to yiy_i in the embedding outside Ni∪{i}\mathcal{N}_i \cup\{i\}.

  1. Temperature annealing. To ensure broad gradient exploration early and strict rank agreement late, the smoothing temperature τ\tau decreases linearly over iterations t∈[1,T]t \in[1,T]:

τ(t)=τstart−t−1T−1(τstart−τend),(9)\tau^{(t)} = \tau_{\mathrm{start}} - \frac{t-1}{T-1}\left(\tau_{\mathrm{start}}-\tau_{\mathrm{end}}\right), \tag*{(9)}

falling from τstart=5.0\tau_{\mathrm{start}} = 5.0 to τend=0.05\tau_{\mathrm{end}} = 0.05. Per Theorem 2, τend=0.05\tau_{\mathrm{end}} = 0.05 guarantees that the soft rank matches the discrete rank to within O(e−γi/0.05)O(e^{-\gamma_i/0.05}), where γi>0\gamma_i > 0 is the smallest gap between the squared embedding distances of two distinct candidates of point ii.

3. Gradient update and re-centering. Coordinates are updated using SGD with Polyak momentum (μ=0.8\mu= 0.8, η=10\eta= 10):

V(t+1)=μV(t)+∇YLtotal(Y(t)),(10)V^{(t+1)} = \mu V^{(t)} + \nabla_Y \mathcal{L}_{\mathrm{total}}(Y^{(t)}), \tag*{(10)}
Y~(t+1)=Y(t)−ηV(t+1).(11)\widetilde{Y}^{(t+1)} = Y^{(t)} - \eta V^{(t+1)}. \tag*{(11)}

Because neither term penalizes translation, we re-center coordinates after each step to remove translational drift:

Y(t+1)=Y~(t+1)−1n∑i=1ny~i(t+1).(12)Y^{(t+1)} = \widetilde{Y}^{(t+1)} - \frac{1}{n}\sum_{i=1}^{n}\widetilde{y}_i^{(t+1)}. \tag*{(12)}

4. Memory complexity and design rationale. For large datasets (n≥50,000n \ge50{,}000), we store the sparse matrix PP with kP=90k_P = 90 affinities per row. The partition function Z=∑a≠b(1+∥ya−yb∥2)−1Z = \sum_{a \ne b}(1 + \lVert y_a-y_b\rVert^2)^{-1} is computed in row chunks of c=10,000c = 10{,}000 using gradient checkpointing, lowering peak memory from O(n2)O(n^2) to O(cn)O(cn). Together, this two-stage schedule ensures that the warm start preserves coarse neighborhood membership, while the annealed rank refinement directly recovers fine-grained neighbor order and expels intruders.

Theoretical Guarantees

Standard objectives like t-SNE and UMAP preserve neighbor order only when high-dimensional affinities strictly differ; when affinities tie, they become indifferent to the ordering. SR-tSNE resolves this by explicitly penalizing rank inversions through its rank loss. This section formalizes this distinction through four guarantees. The rank loss controls O-DLP and vanishes exactly at perfect order (Theorem 1), and the soft rank used in optimization converges exponentially to the discrete rank (Theorem 2). Every inverted neighbor pair is strictly penalized, where t-SNE and UMAP are indifferent (Theorem 3), and the loss bounds the order deficit among retained neighbors relative to an order-blind baseline (Theorem 4).

Preliminaries. For each point ii, the candidate pool Ci=Ni⊕IiC_i = N_i \oplus I_i of size K=k+mK = k + m combines its kk true input-space neighbors NiN_i with mm embedding intruders IiI_i (the mm non-neighbors closest to yiy_i). We sort their embedding distances ∥yi−yCi(j)∥2\lVert y_i-y_{\mathcal{C}_i(j)}\rVert^2 from nearest to farthest to obtain the discrete rank πi−1(j)∈{1,…,K}\pi_i^{-1}(j) \in\{1,\ldots,K\}. Evaluating the discounted gain with these integer ranks rather than the soft values ri,jembr_{i,j}^{\mathrm{emb}} yields the discrete gain Gihard=∑j=1kk−j+1log⁡2(1+πi−1(j))G_i^{\mathrm{hard}} = \sum_{j=1}^{k} \frac{k-j+1}{\log_2(1+\pi_i^{-1}(j))} and the corresponding discrete rank loss:

Lrankhard(Y)=1−1n∑i=1nGihardG⋆=1−1nG⋆∑i=1n∑j=1kk−j+1log⁡2(1+πi−1(j))(13)\mathcal{L}_{\mathrm{rank}}^{\mathrm{hard}}(Y)=1-\frac{1}{n}\sum_{i=1}^{n}\frac{G_i^{\mathrm{hard}}}{G^\star}=1-\frac{1}{nG^\star}\sum_{i=1}^{n}\sum_{j=1}^{k}\frac{k-j+1}{\log_2(1+\pi_i^{-1}(j))} \tag*{(13)}

where G⋆=∑j=1kk−j+1log⁡2(1+j)G^\star=\sum_{j=1}^{k}\frac{k-j+1}{\log_2(1+j)} is the ideal gain, and ΔK=disc⁡(K−1)−disc⁡(K)>0\Delta_K=\operatorname{disc}(K-1)-\operatorname{disc}(K)>0 represents the smallest discrete step of the position discount disc⁡(r)=1/log⁡2(1+r)\operatorname{disc}(r)=1/\log_2(1+r).

All proofs are in Appendix A.14.

Theorem 1 (The rank loss controls O-DLP). Let m≥1m \ge1 and let k′k', the neighborhood size at which the metrics are read, satisfy k′≤min⁡(k,m)k' \le\min(k,m). Then

1−O-DLP(k′)≤2G⋆k′ΔKLrankhard(Y).(14)1-O\text{-}DLP(k')\leq\frac{2G^\star}{k'\Delta_K}\mathcal{L}_{\mathrm{rank}}^{\mathrm{hard}}(Y). \tag*{(14)}

Moreover Lrankhard(Y)=0\mathcal{L}_{\mathrm{rank}}^{\mathrm{hard}}(Y)=0 if and only if O-DLP(k)=1O\text{-}DLP(k)=1, in which case O-DLP(k′)=1O\text{-}DLP(k')=1 for every k′≤kk'\leq k.

Theorem 1 establishes two key results. First, it provides a quantitative surrogate error bound: when convergence is incomplete (Lrankhard>0\mathcal{L}_{\mathrm{rank}}^{\mathrm{hard}}>0), the discrete ordinal error 1−O-DLP(k′)1-O\text{-}DLP(k') is strictly upper-bounded by the loss. The prefactor 2G⋆k′ΔK\frac{2G^\star}{k'\Delta_K} represents the worst-case penalty where every inversion occurs at the smallest discount step ΔK\Delta_K (1.6×1031.6\times10^3 when k=m=k′=10k=m=k'=10). Crucially, this equivalence requires intruders (m≥1m\geq1). Without them (m=0m=0), false points could insert themselves between true neighbors without increasing the loss, since the pairwise order among true neighbors remains unchanged. Assigning target ranks behind all true neighbors to these mm intruders guarantees that any intervening intruder is penalized, thereby locking both neighborhood membership and order.

Theorem 2 (Continuous-to-discrete convergence). Let γi=min⁡a≠b∈Ci∥yi−ya∥2−∥yi−yb∥2>0\gamma_i=\min_{a\ne b\in\mathcal{C}_i}\lVert y_i-y_a\rVert^2-\lVert y_i-y_b\rVert^2>0 be the minimum squared distance gap between distinct candidates of point ii. For any candidate jj, as the sigmoid smoothing temperature τ→0+\tau\to0^+, the continuous soft rank ri,jembr_{i,j}^{\mathrm{emb}} converges exponentially to the true discrete rank πi−1(j)\pi_i^{-1}(j):

∣ri,jemb−πi−1(j)∣≤(K−1)e−γi/τ(15)\left|r_{i,j}^{\mathrm{emb}}-\pi_i^{-1}(j)\right|\leq(K-1)e^{-\gamma_i/\tau} \tag*{(15)}

and the per-point difference between the continuous soft loss Lrank,isoft\mathcal{L}^{\mathrm{soft}}_{\mathrm{rank},i} and the discrete hard loss Lrank,ihard\mathcal{L}^{\mathrm{hard}}_{\mathrm{rank},i} satisfies:

∣Lrank,isoft−Lrank,ihard∣≤k(k+1)(K−1)4ln⁡2G⋆e−γi/τ.(16)\left|\mathcal{L}^{\mathrm{soft}}_{\mathrm{rank},i}-\mathcal{L}^{\mathrm{hard}}_{\mathrm{rank},i}\right| \le\frac{k(k+1)(K-1)}{4\ln2G^\star}e^{-\gamma_i/\tau}. \tag*{(16)}

The gap closes exponentially in the margin over the temperature. The schedule of Section 3.3 ends at τ=0.05\tau= 0.05; a margin of 0.5 in squared distance then leaves a factor e−10≈4.5×10−5e^{-10} \approx4.5 \times10^{-5}. This is why the loss the optimizer minimizes and the metric the tables report agree at the end of training.

The third result isolates what each objective intrinsically demands of neighbor order. By treating each loss purely as a function of squared distances D=(da,b2)D = (d_{a,b}^{2}), we examine an idealized row swap D↦D′:=Di:j↔lD \mapsto D^{\prime} := D^{i:j\leftrightarrow l} that exchanges di,j2d_{i,j}^{2} and di,l2d_{i,l}^{2} and leaves every other entry unchanged. Whether this configuration is geometrically embeddable in 2D is deliberately set aside; this isolates objective preference from dimensionality constraints (empirically evaluated in Section 4.1). Crucially, t-SNE and UMAP penalize order inversions only in proportion to affinity gaps, vanishing when affinities tie. In contrast, SR-tSNE strictly enforces the correct rank order under all conditions.

Theorem 3 (Pairwise order preference). Fix a point ii and two of its true neighbors j<lj < l, so that their input-space distances satisfy di,jH<di,lHd_{i,j}^{\mathrm{H}} < d_{i,l}^{\mathrm{H}}, where di,jH=∥xi−xNi(j)∥2d_{i,j}^{\mathrm{H}} = \lVert x_i - x_{N_i(j)} \rVert_2. Suppose the embedding inverts them, di,l2<di,j2d_{i,l}^{2} < d_{i,j}^{2}, and let D′D^{\prime} be the row swap that corrects the inversion. Then:

(a) For the t-SNE loss with symmetrized affinities pp,

KL(D)−KL(D′)=(pij−pil)[ln⁡(1+di,j2)−ln⁡(1+di,l2)].(17)\mathrm{KL}(D)-\mathrm{KL}(D^{\prime}) = (p_{ij}-p_{il})\left[\ln(1+d_{i,j}^{2})-\ln(1+d_{i,l}^{2})\right]. \tag*{(17)}

The bracket is positive. The loss therefore prefers the correct order when pij>pilp_{ij} > p_{il}, is indifferent when pij=pilp_{ij} = p_{il}, and prefers the inversion when pij<pilp_{ij} < p_{il}.

(b) For the UMAP cross-entropy with fuzzy-union weights vv and low-dimensional kernel w(x)=(1+axb)−1w(x) = (1+ax^{b})^{-1}, in which a>0a > 0 and b>0b > 0 are UMAP’s fitted kernel constants,

CE(D)−CE(D′)=(vij−vil)b[ln⁡di,j2−ln⁡di,l2],(18)\mathrm{CE}(D)-\mathrm{CE}(D^{\prime}) = (v_{ij}-v_{il})b\left[\ln d_{i,j}^{2}-\ln d_{i,l}^{2}\right], \tag*{(18)}

with the same three cases in the sign of vij−vilv_{ij}-v_{il}.

(c) For the rank loss, with r<r′r < r^{\prime} the two positions the pair occupies,

n[Lrankhard(D)−Lrankhard(D′)]≥(l−j)[disc⁡(r)−disc⁡(r′)]G⋆≥ΔKG⋆>0.(19)n\left[\mathcal{L}^{\mathrm{hard}}_{\mathrm{rank}}(D)-\mathcal{L}^{\mathrm{hard}}_{\mathrm{rank}}(D^{\prime})\right] \ge\frac{(l-j)[\operatorname{disc}(r)-\operatorname{disc}(r^{\prime})]}{G^\star} \ge\frac{\Delta_K}{G^\star} > 0. \tag*{(19)}

The preference is strict, and it does not depend on PP, on vv, or on the embedding distances.

Parts (a) and (b) show that t-SNE and UMAP enforce neighbor order strictly through pairwise affinity differences. Consequently, this ordering pressure vanishes, or backfires, under two common scenarios (Lemma S2): UMAP’s fuzzy-union affinities saturate to one for close neighbors, while t-SNE’s symmetrization frequently contradicts true input distances. The latter is remarkably prevalent: on MNIST, 32.9% of adjacent pairs (j,j+1)(j, j+1) in a point’s top-30 list exhibit pi,j+1≥pi,jp_{i,j+1} \ge p_{i,j}, affecting 98.5% of points (19.7% and 91.3% on CIFAR-10). For every such pair, the standard KL divergence is indifferent to the true order or actively favors an inversion. Part (c) highlights the core theoretical guarantee of SR-tSNE: regardless of input affinities, every neighbor inversion incurs a strict penalty of at least ΔK/(nG⋆)\Delta_K/(nG^\star). Corollary S1 ensures that above a finite weight λ⋆\lambda^\star that does not grow with nn, the total loss strictly penalizes all inversions.

Our final theoretical result links the rank loss back to the evaluation metric by decomposing exact-rank preservation into neighborhood membership and internal ordering. As introduced in § 3.4, let O-DLPi_i = DLPi⋅Ai_i \cdot A_i, where DLPi_i measures membership retention and AiA_i represents the fraction of retained neighbors preserving their exact ranks. The theorem below establishes an analytical order-blind baseline for shuffled neighborhoods and proves that the SR-tSNE rank loss directly bounds the ordering degradation among retained neighbors (DLP − O-DLP).

Theorem 4 (Order beyond membership). Fix an embedding and a neighborhood size k′k', and let Ai(k′)A_i(k') denote the fraction of retained neighbors of point ii that preserve their exact original ranks.

(i) Decomposition. O-DLP decomposes into membership times order:

O-DLP(k′)=1n∑i=1nDLPi(k′) Ai(k′),(20)\mathrm{O\text{-}DLP}(k') = \frac{1}{n}\sum_{i=1}^{n}\mathrm{DLP}_i(k')\,A_i(k'), \tag*{(20)}

and its normalized ratio ω(k′):=k′ O-DLP(k′)/DLP(k′)∈[0,k′]\omega(k') := k'\,\mathrm{O\text{-}DLP}(k')/\mathrm{DLP}(k') \in[0,k'] represents the DLPi_i-weighted mean of k′Ai(k′)k'A_i(k').

(ii) Order-blind baseline. If the k′k' embedded neighbors of each point are randomly shuffled, then E[Ai]=1/k′\mathbb{E}[A_i] = 1/k' and Var⁡(k′ O-DLPi)≤DLPi\operatorname{Var}(k'\,\mathrm{O\text{-}DLP}_i) \le\mathrm{DLP}_i. Assuming independent arrangements across points, O-DLP(k′)(k') = DLP(k′)/k′(k')/k' within a standard devi-

ation of k′−1DLP⁡(k′)/nk'^{-1}\sqrt{\operatorname{DLP}(k')/n}, which corresponds to ω(k′)=1\omega(k')=1 within a standard deviation of:

σω=1nDLP⁡(k′)(21)\sigma_{\omega}=\frac{1}{\sqrt{n\operatorname{DLP}(k')}} \tag*{(21)}

(iii) Order deficit bound. For k′≤min⁡(k,m)k' \le\min(k,m), the gap between membership retention and exact-rank preservation is strictly bounded by the discrete rank loss:

DLP⁡(k′)−O-DLP⁡(k′)≤2G∗k′ΔKLrankhard(Y).(22)\operatorname{DLP}(k')-\operatorname{O\text{-}DLP}(k') \le\frac{2G^*}{k'\Delta_K}\mathcal{L}^{\mathrm{hard}}_{\mathrm{rank}}(Y). \tag*{(22)}

Part (ii) is the order-blind baseline: an embedding that keeps the right neighbors and shuffles them scores ω=1\omega=1, and at the benchmark sizes of § 4 Eq. (21) places it within 0.03 for every method that keeps at least a fiftieth of its neighbors, and within 0.07 for all of them. Perfect order is ω=k′\omega=k'. Part (iii) refines Theorem 1. The rank loss bounds not only the distance of O-DLP from one but also the gap between O-DLP and DLP. Driving the loss down therefore tightens the bound on the order of the neighbors that are kept, not only on how many are kept.

Remark. The guarantees above characterize the intrinsic properties of the objectives. Specifically, Theorem 1 establishes the theoretical trajectory of O-DLP as the rank loss is minimized. In practice, § 4.1 empirically evaluates how closely gradient-based optimization approaches these theoretical limits, while the equal-budget λ=0\lambda=0 control in Appendix A.9 confirms that the observed order gains stem directly from the rank regularization term itself.

Experiments

Data and preprocessing. The core benchmark spans eight datasets across five distinct domains: computer vision, audio, natural language, tabular remote sensing, and single-cell transcriptomics (Table 1). For feature representations, MNIST and Fashion-MNIST are evaluated on raw pixels and Covertype retains its 54 native attributes. CIFAR-10 uses 512-dimensional pooled features from an ImageNet-pretrained ResNet-18, AG News mean-pooled bert-base-uncased embeddings (768-d), and Speech Commands mean-pooled wav2vec 2.0 features (768-d).

DatasetDomainnnppClasses
MNISTVision70 00078410
CIFAR-10Vision60 00051210
Fashion-MNISTVision70 00078410
Speech CommandsSpeech59 38776835
Renal-CortexSingle-cell50 02736 03035
Brain-CortexSingle-cell50 21258 23235
CovertypeRemote sense59 999547
AG NewsNLP60 0007684

Table 1. Overview of the eight benchmark datasets. Sources and data types are listed in Table A.11 of Appendix A.2.

For the two single-cell transcriptomics datasets (Renal-Cortex and Brain-Cortex), raw counts first undergo standard library-size normalization and log⁡(1+x)\log(1+x) transformation, followed by the selection of the 2,000 most highly variable genes. These are then reduced via PCA to 50 principal components, providing an identical 50-dimensional input to all methods. Across our benchmark tables, pp denotes the nominal input dimensionality (or the raw gene count for single-cell data prior to PCA).

Peer methods in comparison. We benchmark SR-tSNE against twelve peer methods. Seven are established: t-SNE, UMAP, PHATE, TriMap, PaCMAP, ISOMAP [26] and a 2-layer autoencoder. Four are recent: ParamPaCMAP [27], SUDE [28], LocalMAP [7] and DREAMS [8]. The twelfth is the retrieval-oriented objective NeRV [12]. All methods receive identical inputs and precomputed neighbor graphs, running at their library defaults or official implementations (exact ISOMAP uses k=10k=10 without landmark approximation; NeRV runs through the original dredviz implementation). Eleven of the twelve run on all eight benchmarks at full size. NeRV, whose implementation is built for sizes below 50,000, is compared at matched scale on 5,000-point subsamples of all eight benchmarks (Table A.3 of Appendix A.2). Every paired test therefore runs over all eight datasets.

SR-tSNE setting. SR-tSNE uses the smooth rank loss with k=10k=10 and m=10m=10 throughout. The weight λ\lambda follows dataset size. It is λ=50\lambda=50 for n≥50,000n \ge50{,}000 and λ=5\lambda=5 below that threshold. The same rule carries over to the seven vision datasets of § 4.2.2.

Evaluation metrics. The evaluation combines order, membership and co-ranking criteria. O-DLP is the main criterion for order, DLP scores neighborhood membership, and nDCG grades the returned list by position, all at k=10k=10. We also report the area under the RNXR_{NX} curve on a 20k subsample of MNIST and CIFAR-10, read off the co-ranking matrix and independent of the training objective. All four take the high-dimensional neighbor ranking as reference; the clustering criteria of § 4.2.3 are read against a curated annotation.

The paired tests certify the sign of the lead, and the tables give its size. Every test is a two-sided Wilcoxon signed-rank over datasets, one observation per dataset, so the smallest attainable pp is 0.0078 with eight datasets, 0.0156 with seven and 0.031 with six. A pp at that floor certifies that SR-tSNE leads on every dataset of the pair. Table A.1 of the appendix gives the median and range of the per-dataset differences for every baseline, and we read the two together; the sign counts carry the conclusion and do not depend on a multiplicity correction.

Neighbor Order Preservation

SR-tSNE achieves best exact-rank fidelity. Across all eight benchmarks, SR-tSNE records the highest O-DLP (Table 2, Figure 2), outperforming t-SNE by +6% to +28% (median +18%, Table 4). Nonparametric Wilcoxon signed-rank tests confirm this advantage across all benchmarks (p=0.0078p = 0.0078 against each of the eleven full-scale baselines). On 5,000-point subsamples, SR-tSNE surpasses NeRV by factors of 2.9×2.9\times to 9.2×9.2\times (Table A.3). Absolute O-DLP values run between 0.06 and 0.19, the scale set by a criterion that credits a neighbor only at its exact position. Theorem 1 bounds the ordinal error that remains by the rank loss. This validates Theorems 1 and 2: driving the smooth surrogate down under temperature annealing turns continuous gradient descent into discrete rank optimization.

Datasett-SNEDREAMSλ=0\lambda=0 controlSR-tSNEGain (%)
MNIST.0781.0409.0793.0927+17
CIFAR-10.0545.0254.0548.0597+9
Fashion-MNIST.0672.0497.0672.0792+18
Speech Commands.0809.0533.0818.0858+5
Renal-Cortex.0578.0336.0578.0741+28
Brain-Cortex.0615.0241.0616.0767+25
Covertype.1496.1114.1497.1913+28
AG News.0810.0516.0811.0905+12
Mean.0788.0488.0792.0938+18

Table 2. O-DLP at k=10k=10 for t-SNE, the strongest published baseline (DREAMS), the equal-budget λ=0\lambda=0 control and SR-tSNE, with the mean over datasets and the gain of the rank term over the control. All twelve methods are in Table 4 below.

Figure 2. O-DLP versus neighborhood size kk on the eight benchmarks of Table 1. Each panel carries every method run on that dataset, so the curves match the entries of Appendix Table 4.

Rank optimization simultaneously reinforces neighborhood membership. Sorting true neighbors ahead of mined intruders eliminates false low-dimensional neighbors. Consequently, SR-tSNE achieves the highest DLP on all eight benchmarks (p=0.0078p = 0.0078, Figure A.3). Of the twelve methods, SR-tSNE is the one that holds the highest DLP and the highest O-DLP together, on every benchmark (Tables 4 and 5). A downstream step reads both, so leading both is what carries the gain forward. In neighborhood sweeps (Figure 2), SR-tSNE retains top rank across all 32 dataset–kk settings for O-DLP, and leads DLP at k∈{5,10}k \in\{5, 10\}. This fidelity generalizes across multi-scale metrics. SR-tSNE secures the highest nDCG on all eight datasets (Table A.4). It also takes the top RNXR_{NX} AUC on MNIST and CIFAR-10, in both the local regime (K≤100K \le100) and the full-range one (Table A.10); that criterion measures neighbor agreement across scales, normalized against random chance.

Ablation isolates the rank term as the sole active driver. An equal-budget control (λ=0\lambda= 0), sharing the identical warm start, SGD momentum, and iteration budget, matches baseline t-SNE to within 2% across all benchmarks (Table 2). Activating Lrank\mathcal{L}_{\mathrm{rank}} delivers the entire performance margin, boosting O-DLP by +5% to +28% and DLP by +1% to +4% (p=0.0078p = 0.0078). This improvement is strictly positive across all 30 metric differences over ten paired runs, two datasets by five perturbed starts (Appendix A.9) and all 35 trading names against the control (p=5.8×10−11p = 5.8 \times10^{-11}). Initializing from a 2,000-iteration t-SNE run further amplifies these gains (Table A.18). This confirms Theorem 1 empirically: directly optimizing the rank objective governs the discrete ordinal error, driving O-DLP upward while standard affinity loss leaves it unconstrained.

Order is preserved beyond pure membership. Table 3 reads order against the order-blind baseline of Theorem 4(ii), where ω=1\omega= 1 is a neighborhood kept and shuffled and ω=10\omega= 10 is perfect order. Nine of the eleven peer methods run at full size sit between 1.07 and 1.32, so the order inside their neighborhoods stays close to that baseline. This is the room the rank term is built to recover, and SR-tSNE holds the highest ω\omega on all eight benchmarks. The gain is in the ordering rather than in the membership: from t-SNE to SR-tSNE, O-DLP rises 3.2 to 21.4 times faster than DLP. Read as order recovered above the baseline, ω−1\omega- 1, SR-tSNE stands 27% above t-SNE and 25% above the equal-budget control, on every benchmark. The control shares the warm start, the optimizer, the budget and the seed, so that margin is the rank term’s.

MethodMNCFFMSCRCBCCTAGMean
TriMap1.041.001.061.161.071.051.101.091.07
PaCMAP1.101.021.081.131.071.051.251.081.10
LocalMAP1.101.121.091.091.111.081.301.111.13
Autoencoder1.071.051.071.461.031.041.321.101.14
ParamPaCMAP1.021.011.021.641.031.041.321.101.15
UMAP1.171.151.111.191.131.131.381.141.17
PHATE1.081.101.121.561.131.111.161.171.18
SUDE1.181.161.151.711.041.041.831.371.31
ISOMAP1.120.981.082.661.030.991.371.371.32
DREAMS1.441.801.672.021.361.191.911.771.64
t-SNE2.012.612.002.631.781.792.352.302.18
λ=0\lambda=0 control2.052.632.002.661.781.792.352.302.19
SR-tSNE2.312.782.272.762.202.162.962.542.50

Table 3. Order beyond membership: ω(10)=10⋅O-DLP(10)/DLP(10)\omega(10) = 10 \cdot\mathrm{O\text{-}DLP}(10)/\mathrm{DLP}(10), computed from Tables 4 and 5. ω=1\omega= 1 is the order-blind baseline of Theorem 4(ii), a neighborhood kept and shuffled; ω=10\omega= 10 is perfect order. At these sizes the baseline is exact to within 0.03 by Eq. (21) for every method with DLP ≥0.02\ge0.02, and to within 0.07 for all of them. The eleven peer methods that run at full size are scored on all eight benchmarks; NeRV is compared at matched scale in Table A.3. Rows are sorted by the mean. Columns: MN MNIST, CF CIFAR-10, FM Fashion-MNIST, SC Speech Commands, RC Renal-Cortex, BC Brain-Cortex, CT Covertype, AG AG News.

Position-wise hit rates show this gain concentrates where downstream retrieval reads (Figure 3, Table A.2). At rank position one, SR-tSNE retrieves the true nearest neighbor 1.06× to 1.41× more often than t-SNE. Examples are 0.366 → 0.443 on MNIST and 0.268 → 0.377 on Renal-Cortex. Finally, on a held-out donor split of the Human Lung Cell Atlas [29], SR-tSNE preserves its lead, exceeding t-SNE by +30% in O-DLP (Appendix A.§ 5). These results empirically validate Theorems 3 and 4: the objective suppresses pairwise inversions at the head of the list and strictly minimizes the order deficit beyond membership alone.

Figure 3. Where the recovered order sits. Exact-hit rate at the first five positions: the fraction of points whose rr-th input-space neighbor returns at position rr in the embedding, on four benchmarks (log scale). This is the head of the list, where the position discount places its weight and where retrieval, label transfer and the kk-nearest-neighbor graph read. SR-tSNE lifts every one of these positions on each of the four benchmarks shown. The equal-budget λ=0\lambda=0 control tracks t-SNE, so the lift is the rank term’s. All ten positions on all eight benchmarks are in Table A.2.

Table 4: O-DLP at k=10k=10, the main criterion for order, with λ=0\lambda=0 the equal-budget control: the same warm start and refinement schedule with the rank term switched off. Bold marks the best method per dataset. The last column is the two-sided Wilcoxon signed-rank test of SR-tSNE against that method, paired over the datasets where both are available, bold where p<0.05p < 0.05 and “–” where no test is available. †NeRV runs on a 5,000-point subsample of each benchmark, the largest size its implementation reaches (Table A.3); its row is scored on that subsample and its test is paired with SR-tSNE on the same subsamples.

MethodMNISTCIFARFMNISTSpeechRenalBrainCoverAGNewspp
t-SNE.0781.0545.0672.0809.0578.0615.1496.0810.0078
UMAP.0178.0043.0120.0063.0158.0158.0533.0146.0078
PHATE.0052.0024.0098.0065.0083.0104.0270.0089.0078
TriMap.0097.0025.0080.0038.0132.0124.0306.0088.0078
PaCMAP.0112.0024.0089.0037.0129.0123.0383.0094.0078
ISOMAP.0021.0004.0021.0029.0026.0027.0163.0014.0078
Autoencoder.0071.0024.0056.0059.0067.0070.0255.0059.0078
ParamPaCMAP.0077.0017.0061.0047.0098.0094.0325.0053.0078
SUDE.0054.0025.0070.0061.0076.0065.0512.0080.0078
LocalMAP.0154.0042.0130.0045.0164.0164.0495.0153.0078
DREAMS.0409.0254.0497.0533.0336.0241.1114.0516.0078
NeRV†.0164.0078.0385.0243.0394.0374.0463.0194.0078
SR-tSNE (λ=0\lambda=0).0793.0548.0672.0818.0578.0616.1497.0811.0078
SR-tSNE (ours).0927.0597.0792.0858.0741.0767.1913.0905–

Table 4.

Table 5: DLP at k=10k=10 across all eight datasets, scoring neighborhood membership rather than order, with λ=0\lambda=0 the equal-budget control. Bold marks the best method per dataset. The last column is the two-sided Wilcoxon signed-rank test of SR-tSNE against that method, paired over all eight datasets, bold where p<0.05p < 0.05 and “–” for SR-tSNE itself. †NeRV as in Table 4: scored and paired on the 5,000-point subsamples of Table A.3.

MethodMNISTCIFARFMNISTSpeechRenalBrainCoverAGNewspp
t-SNE.3880.2087.3368.3078.3243.3434.6369.3529.0078
UMAP.1526.0375.1085.0528.1398.1401.3874.1286.0078
PHATE.0481.0218.0872.0418.0734.0938.2331.0760.0078
TriMap.0930.0251.0756.0328.1233.1181.2770.0810.0078
PaCMAP.1018.0235.0827.0326.1205.1176.3063.0872.0078
ISOMAP.0187.0041.0195.0109.0252.0274.1188.0102.0078
Autoencoder.0664.0228.0525.0403.0651.0673.1925.0535.0078
ParamPaCMAP.0757.0168.0596.0287.0949.0908.2455.0481.0078
SUDE.0457.0215.0608.0357.0732.0623.2802.0584.0078
LocalMAP.1401.0376.1193.0412.1474.1513.3800.1376.0078
DREAMS.2839.1412.2983.2634.2477.2028.5832.2915.0078
NeRV†.1354.0761.3153.2023.3284.3103.3466.1669.0078
SR-tSNE (λ=0\lambda=0).3873.2085.3368.3073.3243.3434.6368.3529.0078
SR-tSNE (ours).4009.2149.3493.3110.3375.3545.6452.3568–

Table 5.

Applications

We evaluate three downstream tasks that read the embedding neighbor list from the top down: marker selection in algorithmic trading, retrieval and label transfer in vision, and cell-type discovery in single-cell transcriptomics. Because every embedding is routed through the identical downstream pipeline, reported metrics reflect purely paired differences between dimensional reduction methods.

Algorithmic Trading

Experimental setup. We benchmark SR-tSNE within the trading-map pipeline of [30]. Each 5-minute bar contains eight market features (open, high, low, close, volume, average price, notional value, and time differential), with a binary direction flag indicating the trade side (sign(close − open)). Bars are embedded into 2D, and DBSCAN+K (Appendix A.§ 4) selects trade markers using fixed clustering parameters (minPts = 30, ε\varepsilon set to the 25th percentile of the minPts-distance distribution). We evaluate 35 liquid US equities across seven sectors (5 stocks per group, Table A.12) from 2026-04-14 to 2026-07-09 (≈\approx 4,680 bars per equity; λ=5\lambda= 5, k=m=10k = m = 10). Both SR-tSNE and the equal-budget λ=0\lambda= 0 control refine the identical GPU t-SNE warm start (RAPIDS cuML), isolating the embedding objective as the sole experimental variable. Returns are simulated under realistic friction: 0, 0.5, 1, and 2 basis points (bps) per round trip. Figure 4 follows one equity end to end, from the markers each embedding produces to the positions they open and close.

End-to-end trading on IDCC at 2 bps

Figure 4. End-to-end trading on IDCC at 2 bps (5-minute bars; identical DBSCAN+K hyperparameters). (a, c) Generated markers and (b, d) executed trades. The gold band highlights SR-tSNE’s two strongest round trips (+12.5% vs. +2.6% for t-SNE); gold circles mark trade executions (dashed where t-SNE places no matching side). Over the quarter, SR-tSNE delivers +28.0% return at Sharpe 2.61 versus −7.5% at Sharpe −0.81 for t-SNE.

Theoretical alignment. The marker score reads 1−ρi1-\rho_i, the Spearman rank disagreement between a bar’s input-space and embedded neighbor order. Proposition S2 proves that the per-bar rank loss Lrank,i\mathcal{L}_{\mathrm{rank},i} strictly upper-bounds this quantity. Consequently, SR-tSNE directly minimizes the scoring metric, whereas standard t-SNE remains indifferent whenever affinities tie (Theorem 3(a)).

Superior net returns and risk-adjusted alpha. Under 2 bps transaction costs, SR-tSNE achieves the highest net return on 21 of the 35 equities, with a median return of +13.8% versus +3.6% for t-SNE (Figure A.1). Against a cost-adjusted buy-and-hold baseline, SR-tSNE outperforms on 25 of 35 names (p=0.0017p = 0.0017, Wilcoxon signed-rank test). Beyond raw return, SR-tSNE significantly enhances capital efficiency and drawdown control (Table A.13): median annualized Sharpe ratio rises from 0.63 to 2.32, median maximum drawdown moderates from −11.26% to −10.22%, and total executions decrease from 337 to 266 round trips.

Consistent edge over modern manifold baselines. At 2 bps the median net return of SR-tSNE is +13.8%, against +2.6% for UMAP, +2.8% for TriMap, +4.3% for LocalMAP and +5.6% for DREAMS (Figure 5). Every one of those four paired comparisons favors SR-tSNE (p=3.3×10−7p = 3.3 \times10^{-7}, 2.9×10−62.9 \times10^{-6}, 0.0014 and 0.0010). SR-tSNE takes the top return on 31, 30, 28 and 27 of the 35 names, respectively, and doubles the median of the strongest competitor.

Cross-method comparison across 35 equities

Figure 5. Cross-method comparison across 35 equities at 2 bps under identical DBSCAN+K scoring: (a) net return (%), (b) annualized Sharpe ratio, (c) maximum drawdown (%), and (d) round trips. Boxes indicate interquartile ranges; dots represent individual equities; top numbers denote medians; bottom numbers in (a)–(c) report two-sided paired Wilcoxon pp-values versus SR-tSNE.

Robustness to trading frictions. Because preserving rank reduces false-positive cluster transitions, SR-tSNE trades sparingly and keeps most of its gross return once costs are charged. Raising the cost from 0 to 2 bps, SR-tSNE retains +13.8% of its +18.2%, giving up 4.4 points with round trips steady at 269→266269 \to266; TriMap retains +2.8% of +13.0% with round trips at 1,008→5901{,}008 \to590, and UMAP +2.6% of +9.1% with 542 → 380. A method that trades less pays the spread less often, so the ranking of the arms at the highest cost level follows turnover as well as gross return.

Higher post-marker drift. Evaluating unconditional forward returns confirms that SR-tSNE flags structurally superior inflection points (Figure A.2). It leads t-SNE on 25 of 35 names at 30 minutes (p=0.0003p = 0.0003) and on 21 of 35 at 1 hour (p=0.007p = 0.007). At 2 hours it leads on 25 of 35, with a median forward edge of +0.063% against +0.027% (p=0.005p = 0.005).

Retrieval and Label Transfer in Vision

Exact neighbor alignment at the head of the list. Because downstream retrieval is inspected from the top down, whether the item in position one is truly the nearest neighbor determines downstream utility. Figure 6 inspects the first three retrieved positions across three representative queries each on MNIST, CIFAR-10, and Fashion-MNIST. Three queries on each dataset give nine inspected positions per dataset. SR-tSNE returns all nine at their exact input-space rank on each of the three (9/9, 9/9, 9/9). The equal-budget λ=0\lambda= 0 control returns 7/9 on MNIST, 2/9 on CIFAR-10, and 1/9 on Fashion-MNIST (baseline t-SNE scores 6/9, 2/9, and 1/9, while DREAMS attains 4/9, 1/9, and 0/9). Crucially, ordinal misplacements in the baselines frequently trigger category errors: on the CIFAR-10 dog query, all seven comparative baselines retrieve incorrect classes (predominantly cats), and on Fashion-MNIST Shirt, all seven return a T-shirt. SR-tSNE eliminates these top-rank misclassifications.

Retrieval comparison on MNIST, CIFAR-10, and Fashion-MNIST

Figure 6. Retrieval comparison on (a) MNIST, (b) CIFAR-10, and (c) Fashion-MNIST across the top three embedded positions. Thumbnail badges indicate high-dimensional ground-truth ranks within the query’s top-30; frame colors encode rank displacement (dark green: exact match; pale green: offset by one; orange: offset by ≥ 2; red: outside top-30). Class labels turn red upon class mismatch. Queries illustrate exact-rank preservation where comparative methods introduce ordinal and semantic errors.

Population-level exact-hit counts. This top-of-list advantage generalizes across the entire query distribution. Evaluating all 70,000 MNIST queries, we measure the mean number of the first three embedded neighbors returned at their exact input-space rank. SR-tSNE averages 0.71 exact hits out of three, substantially exceeding the λ=0\lambda= 0 control (0.58), t-SNE (0.57), and DREAMS (0.26), while UMAP, LocalMAP, PaCMAP, and TriMap remain below 0.10.

Transductive label transfer on held-out splits. We next test whether geometric order

preservation improves semantic generalization. Transductive label transfer runs across seven vision benchmarks, spanning 9,298 to 70,000 samples and 10 to 200 classes. Four are raw-pixel sets and three are ResNet-18 feature sets (Table A.5). Embeddings are computed on the full dataset, with a random 20% held out exclusively from the retrieval index as queries. We evaluate precision at rank one (P1\mathrm{P}_{1}) and Mean Reciprocal Rank (MRR) based on the first occurrence of the query class among the ten nearest reference points.

Consistent gains in semantic precision. SR-tSNE achieves strictly positive improvements in both P1\mathrm{P}_{1} and MRR across all seven datasets against all five baselines (p=0.0156p = 0.0156, two-sided Wilcoxon signed-rank test; Table A.20). Relative to t-SNE (the strongest baseline, with median P1=0.8035\mathrm{P}_{1} = 0.8035 and MRR =0.8537= 0.8537), SR-tSNE wins 7/7 evaluations, while the λ=0\lambda= 0 control reproduces t-SNE within 0.0004. The performance margin over the control is most pronounced on highly confusable datasets, adding +0.0016 to +0.0164 in P1\mathrm{P}_{1}, led by EMNIST-Balanced (47 mixed alphanumeric classes) and Fashion-MNIST (Table A.6).

Single-Cell Type Discovery

Cell-type resolution hinges directly on embedded neighbor fidelity. Graph-based clustering reads the embedding’s top-kk list, so exact order governs how cell types resolve. In single-cell transcriptomics, unsupervised cell-type discovery forms the standard endpoint: a community-detection algorithm partitions the kk-nearest-neighbor graph built on 2D coordinates into biological subpopulations. Graph edges exist solely where one cell falls within another’s kk nearest embedded neighbors. Consequently, preserving the ground-truth top-kk ordering prevents intra-type fragmentation, directly evaluating the topological fidelity optimized by our rank loss.

Experimental protocol and oracle-envelope evaluation. We benchmark across six annotated single-cell datasets spanning 5,002 to 50,212 cells and 14 to 35 curated cell types (Table A.7), all preprocessed to 50 principal components. A kk-nearest-neighbor graph (k=15k = 15) is constructed from the 2D coordinates and clustered via Leiden across eight resolutions in [0.1, 2.0]. To isolate geometric quality from resolution choice, each embedding is evaluated at its empirical upper-envelope resolution that maximizes the target criterion against annotations (identical oracle protocol across all methods).

Embeddings are evaluated against curated cell-type annotations using three complementary external criteria. The Adjusted Rand Index (ARI) scores chance-corrected pairwise accuracy and the Adjusted Mutual Information (AMI) information retention across imbalanced populations. The Fowlkes–Mallows Index (FMI) scores geometric precision and recall.

Systematic gains across biological benchmarks. SR-tSNE achieves the highest score in 17 of 18 dataset–criterion comparisons (Table 6). Against t-SNE, PHATE, TriMap, LocalMAP, and DREAMS, SR-tSNE wins on all six datasets across all three metrics (6/6, p=0.031p = 0.031, two-sided Wilcoxon signed-rank test). Against UMAP, it wins 6/6 in ARI and FMI (p=0.031p = 0.031) and 5/6 in AMI (p=0.062p = 0.062). The margin peaks on Trachea, exceeding UMAP by +0.145 in ARI and TriMap by +0.223. These gains do not depend on oracle selection: fixing the Leiden resolution globally at 1.0 or 0.5 preserves SR-tSNE’s top ARI across all six datasets (p=0.031p = 0.031; median gains over t-SNE: +23% and +19%, Table A.9).

Datasett-SNEPHATEUMAPTriMapLocalMAPDREAMSSR-tSNE
ARI ↑\uparrow
Trachea.7841.5082.6771.5988.5982.6001.8221
Liver-MacParland.5776.5347.5815.5842.5066.5629.6012
Kidney.7179.4276.6688.6874.6855.6943.7499
Kidney-Park.2069.2247.2143.2214.1792.1685.2961
Brain-Cortex.1141.1618.1640.1607.1444.1515.2070
Renal-Cortex.3063.1528.3752.3652.3443.3360.3848
Wins6/66/66/66/66/66/6
pp.031.031.031.031.031.031
AMI ↑\uparrow
Trachea.8212.7019.7854.7654.7683.7581.8296
Liver-MacParland.8045.7341.8073.7803.7694.7963.8169
Kidney.7698.6181.7666.7535.7624.7486.7734
Kidney-Park.6004.5593.6037.5983.5930.5753.6360
Brain-Cortex.5683.5794.6163.6148.5977.6037.6477
Renal-Cortex.6797.4094.6939.6723.6851.6789.6887
Wins6/66/65/66/66/66/6
pp.031.031.062.031.031.031
FMI ↑\uparrow
Trachea.8153.5781.7231.6569.6611.6640.8463
Liver-MacParland.6087.5722.6124.6148.5437.5956.6374
Kidney.7459.4830.7020.7186.7188.7251.7757
Kidney-Park.4228.4396.4307.4373.3937.3806.5094
Brain-Cortex.2997.3568.3615.3574.3386.3471.4081
Renal-Cortex.3929.2206.4454.4308.4182.4200.4496
Wins6/66/66/66/66/66/6
pp.031.031.031.031.031.031

Table 6. Clustering performance across six annotated single-cell datasets under the Leiden upper-envelope protocol. Bold indicates the top method per cell. Sign counts and two-sided Wilcoxon signed-rank pp-values evaluate SR-tSNE against each baseline across all datasets (p<0.05p < 0.05 bolded).

Suppression of artificial cluster fragmentation. Standard embeddings over-split homogeneous cell types into spurious sub-clusters (Figure 7). Baseline t-SNE produces 12.2 excess clusters on average (up to +26 on Brain-Cortex), while DREAMS and LocalMAP over-split by +19.2 and +9.7, respectively. In contrast, SR-tSNE holds total cluster error to 3.2 (+0.2 excess, −3.0 short), the range occupied by the closest peer methods, and leads them on 17 of the 18 dataset–criterion cells of biological alignment (Table 6). The λ=0\lambda= 0 control over-splits by +8.3, proving that the rank loss drives this consolidation. By enforcing correct internal rank order, SR-tSNE maintains cluster connectivity. Each annotated type is partitioned into fewer disconnected components and retains more cells in the primary component (0.725 against 0.690 on Renal-Cortex), while the cross-type edge rate stays within 0.6% of baseline (Table A.8).

Cluster count discrepancies relative to curated ground-truth annotations across six single-cell datasets

Figure 7. Cluster count discrepancies relative to curated ground-truth annotations across six single-cell datasets. (a) Signed deviation per dataset (zero denotes the ground truth). (b) Mean absolute error decomposed into over-clustering (excess clusters, solid bars) and under-clustering (omitted clusters, hatched bars). SR-tSNE minimizes spurious cluster fragmentation.

Parameter Sensitivity and Stability

Robust hyperparameter defaults across scales. SR-tSNE relies on three intuitive hyperparameters: neighbor count kk, intruder pool size mm, and regularization weight λ\lambda. We fix k=m=10k = m = 10 across all experiments to align candidate pools with standard retrieval and scoring windows. The loss weight follows a simple dataset size rule: λ=50\lambda= 50 for large datasets (≥50,000\ge50{,}000 points) and λ=5\lambda= 5 for smaller regimes, calibrated once on MNIST and a market-equity pool and frozen thereafter. Performance is robust across scales: SR-tSNE consistently outperforms t-SNE across neighborhood sweeps (k∈{5,10,15,20}k \in\{5, 10, 15, 20\}, Figure 2) and across all scales in RNXR_{NX} AUC (Table A.10). Perturbing initial coordinates across five seeds with 1% Gaussian jitter yields strictly positive gains in all 30 pairwise trials (Table A.17), verifying stability to initialization.

Listwise ranking vs. pairwise alternatives. The listwise form of the term is what delivers the gain. Under identical warm starts, budget and tuning, the listwise term holds the highest O-DLP of the three arms on all eight benchmarks, above both the warm start and a pairwise logistic surrogate over the same candidate pool (Table A.19). It provides +5% to +28% relative gains from that same start, so the discounted listwise penalty is what aligns the whole top-kk list.

Negligible refinement overhead. Evaluating wall-clock time on a dedicated NVIDIA A6000 demonstrates that computing the soft rank loss on candidate pools (CiC_i, K=20K = 20) introduces only 1.8%–2.1% overhead relative to an equal-budget λ=0\lambda= 0 control (26.16 vs. 25.69 minutes on full MNIST, Table A.21). The embedding cost remains dominated by the global t-SNE gradient computation, which admits standard O(nlog⁡n)O(n \log n) Barnes–Hut or FFT acceleration as detailed in § 5.

Refining a longer warm start raises the gain further: from a 2,000-iteration t-SNE run, SR-tSNE leads that run on all eight benchmarks and exceeds its own 1,000-iteration-start result (Table A.18).

Discussion

Unified mechanism across downstream tasks. Our three applications evaluate the identical topological property from complementary perspectives. The trading scorer measures Spearman rank agreement. Vision retrieval evaluates the exact position of retrieved items. Leiden clustering reads the persistent edges within local top-kk graphs. By directly penalizing ordinal violations, the smooth rank loss acts specifically on the exact mechanism that governs downstream performance across these distinct domains.

Decoupling order fidelity from neighborhood membership. The rank objective acts on two complementary aspects of a neighborhood: the permutation order among true neighbors and the exclusion of intruders. Table 3 separates the two through the normalized order index ω\omega, and Theorem 3(b) predicts where the affinity-based baselines sit on it: their ordering pressure follows an affinity gap that can close. The larger share of the O-DLP margin comes from restored internal order, governed by the discounted listwise gain, and a further share from membership, enforced by the intruder penalty (Figure A.3). In single-cell discovery this dual control keeps the Leiden cluster count close to the curated one (Figure 7).

Robustness across initialization schemes. The benefits of rank refinement are invariant to the choice of warm start. Refining CPU-based Barnes–Hut t-SNE yields +5% to +28% gains in O-DLP across image and tabular benchmarks. Concurrently, refining GPU-accelerated cuML t-SNE in the algorithmic trading pipeline translates directly into higher net returns, superior Sharpe ratios, and lower turnover, an advantage that reproduces against t-SNE, the equal-budget control, UMAP and TriMap when initialized from CPU t-SNE (Table S2 of the Supplementary Material). Furthermore, on 5,000-point subsamples, SR-tSNE outperforms NeRV, the closest neighbor-retrieval objective, by factors of 2.9×2.9\times to 9.2×9.2\times in O-DLP (Table A.3), establishing that explicit listwise rank bounds surpass soft information-theoretic divergences.

Extensions and future directions. Three extensions follow directly from this framework: (1) Sub-quadratic scaling: The smooth rank loss scales strictly linearly as O(n(k+m))O(n(k+m)), evaluating only the local candidate pool CiC_i. The computational bottleneck resides entirely in the exact t-SNE partition function (O(n2)O(n^2)) and exhaustive intruder search. Replacing the exact denominator with FFT-accelerated interpolation (FIt-SNE) and adopting approximate nearest-neighbor search brings overall complexity to O(nlog⁡n)O(n\log n) without modifying the rank term. (2) Continuous regularization schedules: While our piecewise size rule (λ∈{5,50}\lambda\in\{5,50\}) proves robust, dynamically scheduling λ(n)\lambda(n) via continuous validation-based tuning could further optimize convergence trajectories. (3) Parametric extensions: Formulating the soft rank loss as an objective for training deep neural encoders will enable instantaneous out-of-sample inference via a single forward pass, bypassing iterative gradient refinement for large-scale streaming pipelines.

Conclusion

Summary of contributions. We introduced SR-tSNE, which adds a smooth listwise rank loss to t-SNE, along with Ordinal Degree of Locality Preservation (O-DLP). Our core contributions are two-fold: (1) Metric formulation and theoretical control: We formulated O-DLP to credit a neighbor only when it returns at its exact high-dimensional rank under a position discount, establishing that the rank loss provably controls the discrete metric deficit (Theorem 1) with exponential surrogate convergence (Theorem 2). (2) A rank-preserving embedding objective: By ranking each point’s high-dimensional neighbors against mined intruders under a position discount, one simple term forces the embedding to place the right neighbors in the right order with negligible computational overhead (∼2%\sim2\%).

Empirical validation across downstream tasks. On eight benchmarks against twelve methods, SR-tSNE attains the highest exact-rank preservation (O-DLP), the highest neighborhood overlap (DLP) and the highest nDCG, on all eight. An equal-budget λ=0\lambda=0 control confirms the rank term as the sole active driver. This gain directly reaches tasks that read neighbor lists from the top down: (1) Algorithmic trading: It delivers the best return of six embeddings on 21 of 35 names with higher Sharpe ratios, shallower drawdowns, and fewer trades; (2) Computer vision: In label transfer across seven vision datasets, it leads on every one in precision at rank one and in mean reciprocal rank; (3) Single-cell discovery: In clustering across six single-cell datasets, it leads on 17 of 18 dataset–criterion cells and prevents spurious cluster over-splitting in Leiden community detection.

Strengths and overall impact. The method’s primary strengths are its economy and its exact match to the criterion: one term and one weight turn t-SNE into an embedding that keeps neighbor order, and O-DLP measures the exact quantity the term is built to control. The empirical evidence spans eight benchmarks, twelve methods, three applications, one held-out corpus, and an equal-budget control. Neighbor order is a fundamental property an embedding can and should be asked for, O-DLP measures whether it was delivered, and delivering it pays wherever a downstream step reads the list from the top. Beyond non-linear manifold learning, establishing that smooth listwise surrogates can provably govern discrete ordinal metrics provides a generalizable blueprint for top-kk representation learning across information retrieval, bioinformatics, and recommendation systems.

Data and code availability. All benchmark datasets are publicly available from their original sources. The implementation of SR-tSNE and its criteria is released at https://github.com/hank08819/SR-tSNE, where embeddings and preprocessing scripts are being added.

Funding

This work is partially supported by NSF 2229138 and the McCollum Endowed Chair startup fund.

Declaration of competing interest

The authors declare that they have no known competing interests.

CRediT authorship contribution statement

Henry Han: Conceptualization, Methodology, Software, Supervision, Writing. Luyu Niu: Methodology, Software, Data curation. Diane Li: Investigation. Jiacun Wang: Investigation

Declaration of generative AI and AI-assisted technologies in the manuscript preparation process

AI is used for english polish.

Supplementary Material

Frequently Asked Questions

This section collects the questions most often raised about SR-tSNE. Every answer states what the paper establishes and names the section, table or figure that carries the evidence.

Q1. What is SR-tSNE? SR-tSNE is t-SNE with one added term, the smooth rank loss of Section 3.1. For each point the loss reads its kk nearest neighbors in the input space, in order. It then asks the embedding to return the same neighbors in the same order. The neighbor that is nearest in the input space should be nearest in the embedding, the second should be second, and so on. The loss also mines intruders: points that lie close to a point in the embedding while belonging to no part of its input-space neighbor list. It pushes every intruder behind every true neighbor. A single weight λ\lambda sets how much the term counts against the t-SNE objective, and Section 3 states the two together.

Q2. What gap does SR-tSNE close? Neighbor-embedding methods preserve who belongs to a neighborhood. The order inside that neighborhood is the part SR-tSNE adds. Two embeddings can hold the same neighbors and still rank them differently, and SR-tSNE fixes both the membership and the order.

Q3. Why does neighbor order matter? Three common pipelines read the neighbor list from the top down. Retrieval and label transfer act on the item in position one. Community detection builds its edges from top-kk lists. The trading scorer of Section 4.2.1 reads a Spearman agreement between the two spaces. Each of them improves when the order improves.

Q4. What is O-DLP? O-DLP credits a neighbor when it returns at its exact input-space rank, Eq. (7). DLP credits the same neighbor for being present at any position. O-DLP is therefore the strict reading of one neighborhood, and Eq. (8) relates the two. A third criterion, nDCG, grades the returned list by position and gives partial credit for a near miss. All three are read at k=10k = 10.

Q5. What is novel here? Three elements are new. The first is an objective in the t-SNE family that asks for exact neighbor order. The second is O-DLP, a criterion for that property. The third is a set of guarantees that tie the optimized loss to the reported metric, Section 3.4.

Q6. What are the contributions? The method adds end-to-end differentiable rank regularization to t-SNE. The theory proves zero loss at perfect order, exponential soft-to-exact convergence, a strict order preference, and an upper bound on the order deficit. The metric extends DLP from membership to exact rank. The benchmarks cover eight datasets and twelve peer methods. The applications cover label transfer, cell-type discovery and intraday trading.

Q7. How does SR-tSNE relate to NeRV, RankVisu and triplet methods? NeRV optimizes a retrieval trade-off defined on neighborhood membership. RankVisu and local ordinal embedding work from discrete rank relations. Triplet methods such as TriMap sample relative comparisons. SR-tSNE keeps t-SNE’s continuous similarities and adds one listwise term. Listwise means the term scores a point’s whole top-kk list at once, rather than one pair at a time. It therefore combines both sources of information. Table A.3 places it beside NeRV at matched scale, where SR-tSNE leads by 2.9× to 9.2× in O-DLP.

Q8. Why does the candidate pool carry intruders? The loss scores one candidate pool per point: the kk true neighbors, followed by the mm mined intruders. Intruders give the loss something to rank the true neighbors against. With m≥1m \ge1, an outside point that moves between two true neighbors raises the loss, which is what locks membership and order together in Theorem 1. The m=0m = 0 case is worked out in Appendix A.8, where the term becomes a differentiable Spearman correlation.

Q9. Why a listwise penalty? The listwise form of Q7 is what delivers the gain. The warm start it improves on is the plain t-SNE run that SR-tSNE refines. From that warm start the listwise term lifts O-DLP by +5% to +28%. On the same candidates, start and budget, the listwise term lifts O-DLP above a pairwise logistic surrogate on all eight benchmarks, Table A.19. Aligning the whole top-kk list at once is therefore the active ingredient.

Q10. Why a soft rank with an annealed temperature? The sigmoid count of Eq. (2) supplies a gradient everywhere, so the discrete ordering becomes trainable. Annealing τ\tau from 5.0 to 0.05 keeps early gradients broad and late ranks sharp. Theorem 2 closes the gap to the discrete rank exponentially in the margin. The loss that is optimized and the metric that is reported therefore agree at the end of training.

Q11. How are kk, mm and λ\lambda chosen? Every experiment in this paper uses k=m=10k = m = 10, the window that retrieval and graph construction read. The weight follows one size rule: λ=50\lambda= 50 at n≥50,000n \ge50{,}000 and λ=5\lambda= 5 below it. The rule was calibrated once and then carried unchanged through every benchmark, application and appendix study, Appendix A.7.

Q12. Does SR-tSNE use labels? SR-tSNE is fully unsupervised. The neighbor list and the intruder pool come from input-space distances alone. Labels and curated annotations enter at evaluation time, where they give an independent reading of the geometry.

Q13. What do the four guarantees give a reader? Theorem 1 bounds the exact-rank deficit by the loss, so minimizing the loss controls the reported metric. Theorem 2 makes the optimized surrogate agree with the discrete rank exponentially fast. Theorem 3 gives a strict preference for the correct order at every inversion. Theorem 4 bounds the order deficit among the neighbors that are kept. The prefactor of Theorem 1 is uniform: it is fixed by kk, mm and the discount, and it holds for every embedding and every dataset. Proofs are in Appendix A.14.

Q14. What does the rank term add to an affinity objective? Theorem 3(a)–(b) shows that the order pressure of t-SNE and UMAP follows an affinity gap. Theorem 3(c) shows that the rank term charges at least Δk/(nG∗)\Delta_k/(nG^*) for every inversion, whatever the affinities are. The room this opens is measurable. On MNIST, 32.9% of adjacent pairs in a top-30 list carry pi,j+1≥pi,jp_{i,j+1} \ge p_{i,j}. At least one such pair occurs for 98.5% of points.

Q15. Is the evaluation independent of the loss? Yes, on four counts. The RNXR_{NX} AUC is read off the co-ranking matrix, so it is independent of the objective. SR-tSNE leads it on MNIST and CIFAR-10, in the local regime and in the global one, Table A.10. Cell types are scored against curated annotations. Label transfer is scored against class labels, and trading in net return. All four agree with the O-DLP ordering.

Q16. What do the absolute O-DLP values tell us? O-DLP counts exact rank matches, so its scale is set by how strict that test is. At k=10k = 10, an O-DLP of 0.093 on MNIST means that about one neighbor in ten comes back at its own position, averaged over the ten positions. SR-tSNE holds the highest value on all eight benchmarks. Table 3 reads the same quantity against the order-blind baseline, and that is the share the downstream tasks convert into higher precision at rank one, cleaner Leiden clusters and higher net return.

Q17. What does the λ=0\lambda= 0 control establish? It shares the warm start, the optimizer, the iteration budget and the seed with SR-tSNE. The rank term is therefore the single difference between the two arms. The control lands within 2% of t-SNE, and the rank term adds +5% to +28% in O-DLP on top of it, Table 2. The margin is therefore attributable to the term itself.

Q18. What happens when the warm start is run longer? The rank term adds order on top of a t-SNE run of any length. Table A.18 continues the warm start to 2,000 iterations on all eight benchmarks. Refined from that longer run, SR-tSNE leads it on all eight benchmarks, by a median of +10%. It also stands +6% above SR-tSNE refined from the 1,000-iteration start.

Q19. What does the paired test certify? Each test is a two-sided Wilcoxon signed-rank over datasets, with one observation per dataset. Eight datasets place the smallest attainable value at 0.0078, and a pp at that floor certifies a win on every dataset of the pair. Table A.1 gives the size of each win, and the two are read together. The sign counts are descriptive, so the conclusion they carry stands under any multiplicity correction.

Q20. How does the result behave across warm starts? It holds from a 1,000-iteration start on all eight benchmarks and from a 2,000-iteration start on all eight, where SR-tSNE leads that longer run by a median of +10%, Table A.18. It holds for a CPU Barnes–Hut start in the benchmarks and for a GPU cuML start in the trading pipeline, § 5.

Q21. How stable is the result across initializations? Five seeds perturb the initial coordinates with 1% Gaussian jitter. All 30 metric differences come out positive, Appendix A.9.

Q22. Why is NeRV compared on 5,000 points? The comparison is run at matched scale, on the range the original drevdivz implementation is built for. Every method is re-run on the same 5,000-point subsample of all eight benchmarks and scored there, Table A.3. SR-tSNE holds the best O-DLP and DLP on every subsample, and leads NeRV by 2.9× to 9.2× in O-DLP.

Q23. Is every peer method scored on every benchmark? Yes. All twelve are scored on all eight benchmarks, so the tables carry no gaps. Eleven of them run at full size. NeRV is scored at matched scale, on the 5,000-point subsamples of Table A.3, the largest size its implementation reaches. Every paired test therefore runs over all eight datasets.

Q24. What does the trading study deliver? It is a paired comparison of embeddings inside one fixed pipeline. The features, the DBSCAN+K settings, the execution rule and the four cost levels are identical for every arm, so the embedding objective is the single variable. At 2 bps SR-tSNE takes the best net return on 21 of 35 names, with a median of +13.8% against +3.6% for t-SNE, a median Sharpe ratio of 2.32 against 0.63, and fewer round trips, Table A.13. Against a cost-adjusted buy-and-hold baseline it leads on 25 of 35 names, p=0.0017p = 0.0017.

Q25. How does the single-cell result behave under the resolution choice? The upper-envelope protocol is applied identically to every method. The conclusion also carries over to a fixed resolution. The Adjusted Rand Index scores chance-corrected agreement with the curated cell-type labels. Setting the Leiden resolution globally at 1.0 and at 0.5 keeps SR-tSNE first on all six datasets. The median gains over t-SNE are +23% and +19%, Table A.9. The cluster counts of Figure 7 agree, with SR-tSNE holding the total deviation to 3.2 clusters.

Q26. How is the label-transfer protocol set up? Embeddings are computed on the full dataset, and a random 20% is then withheld from the retrieval index to serve as queries. This transductive protocol is applied identically to every method. The two criteria are precision at rank one (P1P_{1}) and the mean reciprocal rank of the first correct neighbor (MRR). SR-tSNE leads on all seven datasets in both. A donor-split corpus held out from all tuning is reported in Appendix A.§ 5, where SR-tSNE leads t-SNE by +30% in O-DLP.

Q27. Where in the list does the gain sit? It sits at the head, where the downstream steps read. At position one SR-tSNE returns the true nearest neighbor 1.06× to 1.41× more often than t-SNE, for example 0.367 → 0.443 on MNIST and 0.268 → 0.377 on Renal-Cortex, Figure 3 and Table A.2. Over all 70,000 MNIST queries it returns 0.71 of the first three neighbors at their exact rank, against 0.57 for t-SNE.

Q28. What does the rank term cost? It reads a candidate pool of K=k+m=20K = k + m = 20 per point, so it costs O(n(k+m))O(n(k + m)). That overhead is 1.8% on full MNIST, 26.16 against 25.69 minutes on one NVIDIA A6000, and 2.1% on CIFAR-10, Table A.21.

Q29. How does SR-tSNE scale? The rank term is linear in nn by construction. Barnes–Hut or FFT interpolation for the t-SNE partition function, together with approximate nearest-neighbor search for the intruder pool, brings the whole refinement to O(nlog⁡n)O(n \log n) without changing the rank term, § 5.

Q30. Can the rank term be attached to other embeddings, and where is the code? The term reads pairwise distances inside the embedding alone, so any differentiable neighbor-embedding objective can carry it with the same candidate pool and the same discount. The implementation and the criteria are released at https://github.com/hank08819/SR-tSNE. Every setting is listed in Appendix A.7, and the notation is collected in Table A.15.

Additional figures and tables

BaselinennΔ\DeltaO-DLP MedianΔ\DeltaO-DLP RangeΔ\DeltaDLP MedianΔ\DeltaDLP Range
t-SNE8+0.0133[+0.0049, +0.0417]+0.0097[+0.0032, +0.0132]
UMAP8+0.0711[+0.0554, +0.1380]+0.2345[+0.1774, +0.2582]
PHATE8+0.0743[+0.0573, +0.1643]+0.2666[+0.1931, +0.4121]
TriMap8+0.0765[+0.0572, +0.1607]+0.2747[+0.1898, +0.3682]
PaCMAP8+0.0757[+0.0573, +0.1530]+0.2681[+0.1914, +0.3389]
ISOMAP8+0.0800[+0.0593, +0.1750]+0.3285[+0.2108, +0.5264]
Autoencoder8+0.0768[+0.0573, +0.1658]+0.2920[+0.1921, +0.4527]
ParamPaCMAP8+0.0771[+0.0580, +0.1588]+0.2860[+0.1981, +0.3997]
SUDE8+0.0760[+0.0572, +0.1401]+0.2903[+0.1934, +0.3650]
LocalMAP8+0.0707[+0.0555, +0.1418]+0.2246[+0.1773, +0.2698]
DREAMS8+0.0397[+0.0295, +0.0799]+0.0695[+0.0476, +0.1517]
SR-tSNE (λ=0\lambda=0)8+0.0127[+0.0040, +0.0416]+0.0097[+0.0037, +0.0136]

Table A.1. Table A.1: Effect sizes behind the paired tests: the difference SR-tSNE minus baseline in O-DLP and DLP at k=10k=10, median and range over the datasets the baseline reaches (nn). Computed from Tables 4 and 5.

Table A.2 reads the gain position by position. SR-tSNE leads at the head of every one of the eight lists, and the mean over the ten positions is O-DLP(10).

Table A.3 places every method on one common footing, a 5,000-point subsample.

SR-tSNE holds the best O-DLP and the best DLP on all eight subsamples.

At this size SR-tSNE holds the best O-DLP and DLP on every subsample, and NeRV reaches 11–34% of SR-tSNE’s O-DLP. SR-tSNE leads t-SNE on every subsample, by +0% to +23% in O-DLP and +1% to +2% in DLP, so the gain of the rank term does not depend on the size of the benchmark.

DatasetMethod12345678910
MNISTt-SNE.366.125.075.051.040.032.028.024.020.019
λ=0\lambda=0.378.127.075.051.040.032.027.024.020.019
SR-tSNE.443.170.094.059.044.032.026.022.019.017
CIFARt-SNE.350.079.037.022.015.012.009.008.007.006
λ=0\lambda=0.353.080.036.022.016.012.009.008.007.006
SR-tSNE.385.094.041.023.016.012.008.007.006.005
F-MNISTt-SNE.326.106.064.042.033.027.022.019.017.016
λ=0\lambda=0.325.106.064.042.033.027.022.020.017.016
SR-tSNE.394.142.076.051.036.026.021.018.016.013
Speecht-SNE.455.135.066.042.030.022.018.014.012.011
λ=0\lambda=0.464.136.066.042.029.022.018.014.012.011
SR-tSNE.482.159.078.045.028.020.015.012.011.008
Renalt-SNE.268.087.056.039.031.023.021.019.017.017
λ=0\lambda=0.268.087.056.039.030.023.021.019.018.016
SR-tSNE.377.131.071.044.031.023.021.015.015.014
Braint-SNE.280.098.058.042.033.028.024.020.017.016
λ=0\lambda=0.280.097.058.041.033.028.024.020.018.016
SR-tSNE.382.136.075.047.034.025.022.018.015.014
Cover.t-SNE.485.257.177.130.105.089.076.066.059.053
λ=0\lambda=0.485.257.177.130.105.089.076.066.059.053
SR-tSNE.586.347.253.192.149.116.090.071.059.049
AG Newst-SNE.406.134.074.051.037.030.025.020.017.015
λ=0\lambda=0.406.135.074.051.037.030.025.020.018.015
SR-tSNE.447.170.092.058.039.029.022.018.016.013

Table A.2. Exact-hit rate by position: the fraction of points whose rr-th high-dimensional neighbor returns at position rr in the embedding, r=1,…,10r=1,\ldots,10, for t-SNE, the equal-budget λ=0\lambda=0 control and SR-tSNE on the eight benchmarks. The mean over the ten positions is O-DLP(10).

t-SNEUMAPTriMapPaCMAPNeRVλ=0\lambda=0SR-tSNE
MNISTO-DLP.1228.0454.0300.0309.0164.1233.1294
DLP.4601.3400.2585.2657.1354.4566.4665
CIFAR-10O-DLP.0694.0150.0105.0095.0078.0695.0718
DLP.2512.1262.1025.0869.0761.2482.2549
Fashion-MNISTO-DLP.1031.0370.0297.0290.0385.1036.1137
DLP.4394.3045.2592.2542.3153.4387.4456
Speech CommandsO-DLP.0997.0224.0151.0132.0243.0997.0998
DLP.3511.1833.1351.1284.2023.3495.3562
Renal-CortexO-DLP.0956.0465.0384.0378.0394.0959.1175
DLP.5028.3797.3407.3383.3284.5013.5121
Brain-CortexO-DLP.1043.0487.0383.0367.0374.1051.1227
DLP.5133.3730.3373.3297.3103.5126.5230
CovertypeO-DLP.1750.0729.0522.0462.0463.1769.2124
DLP.6764.5176.4532.3984.3466.6772.6872
AG NewsO-DLP.1022.0328.0224.0206.0194.1035.1089
DLP.3861.2568.2055.1921.1669.3831.3885

Table A.3. Matched-scale comparison on a 5,000-point subsample (seed 42) of each benchmark: every method is re-run on the same subsample and scored on its own high-dimensional neighbor lists at k=10k=10. NeRV runs through the original dredviz implementation at its defaults (λNeRV=0.1\lambda_{\mathrm{NeRV}}=0.1, seed 42). SR-tSNE uses the recipe of § 3.5 at the size-rule weight λ=5\lambda=5; λ=0\lambda=0 is the equal-budget control. Bold marks the best method per row.

MethodMNISTCIFARFMNISTSpeechRenalBrainCoverAGNews
t-SNE.5079.3333.4448.4652.4147.4380.7471.4821
UMAP.1625.0404.1137.0580.1449.1448.4248.1349
PHATE.0495.0232.0909.0499.0772.0976.2379.0809
TriMap.0928.0247.0745.0335.1196.1141.2776.0806
PaCMAP.1025.0230.0825.0330.1182.1143.3237.0869
ISOMAP.0184.0039.0190.0158.0245.0267.1382.0112
Autoencoder.0653.0218.0502.0439.0609.0632.2120.0527
ParamPaCMAP.0731.0156.0567.0323.0901.0855.2696.0464
SUDE.0495.0241.0651.0449.0741.0631.3459.0686
LocalMAP.1428.0379.1207.0420.1460.1494.4037.1409
DREAMS.3360.1947.3731.3670.2834.2195.6703.3710
NeRV†.1387.0748.3295.2123.3308.3143.3728.1670
SR-tSNE (λ=0\lambda=0).5091.3335.4447.4655.4146.4379.7471.4821
SR-tSNE (ours).5689.3682.5025.4993.4890.5121.8013.5260

Table 4 (PDF p. 43). nDCG10_{10} across all methods. The ranked list is the embedding’s own ten nearest neighbors. Relevance is graded by high-dimensional rank and intruders score zero. Bold marks the best per dataset. †NeRV is scored on the 5,000-point subsamples of Table A.3, the largest size its implementation reaches.

Table A.5 lists the seven vision datasets of the label-transfer protocol and the weight the size rule selects for each.

DatasetData typennddClassesQueriesλ\lambda
MNISTGrayscale digits70 0007841014 00050
Fashion-MNISTGrayscale clothing70 0007841014 00050
CIFAR-10Color, ResNet-1860 0005121012 00050
Tiny-ImageNet [31]Color, ResNet-1870 00051220014 00050
EMNIST-Balanced [32]Grayscale characters69 9837844713 99650
CIFAR-100 [33]Color, ResNet-1860 00051210012 00050
USPS [34]Grayscale digits9 298256101 8595

Table 5 (PDF p. 43). The seven vision datasets of the label-transfer protocol. Queries is the held-out fifth used to probe the retrieval index; the remaining points form the index. dd is the input dimensionality, Classes the number of classes, and λ\lambda follows the size rule of Section 4. Sets larger than 70,000 points are stratified-subsampled to that size.

Table A.6 summarizes label transfer over the seven datasets. SR-tSNE leads every baseline on both criteria, at the sign-count floor of the test.

Table A.7 lists the six annotated single-cell datasets and the curated type counts the clustering is scored against.

Table A.8 looks inside the graph that Leiden reads. Under SR-tSNE each annotated type stays in fewer pieces and keeps more of its cells in the main piece.

EmbeddingMedianSR-tSNE wins on (pp)MedianSR-tSNE wins on (pp)
P1↑\mathrm{P}_1 \uparrowMRR↑\mathrm{MRR} \uparrow
UMAP.71597/7 (.0156).79497/7 (.0156)
TriMap.67227/7 (.0156).76117/7 (.0156)
t-SNE.80357/7 (.0156).85377/7 (.0156)
LocalMAP.72677/7 (.0156).79857/7 (.0156)
DREAMS.77077/7 (.0156).83117/7 (.0156)
SR-tSNE.8053—.8547—

Table A.6. Label transfer summarized over the seven vision datasets. Each row is one embedding, Median is its median score over the seven, and SR-tSNE wins on gives the sign count and the two-sided paired Wilcoxon pp of SR-tSNE against that embedding. Bold marks the best median in a column and p<0.05p < 0.05. Per-dataset scores are in Table A.20.

DatasetTissueSourceCellsGenesSparsityTypes
TracheaTracheaHuman Cell Landscape9 94924 99097.1%14
Liver-MacParlandLiverMacParland et al., human liver8 44420 00793.4%20
KidneyKidneyHuman Cell Landscape9 15325 60197.5%18
Kidney-ParkKidneyPark et al., mouse kidney5 00216 27293.7%16
Brain-CortexCerebral cortexPrecentral gyrus dissection50 21258 23290.5%35
Renal-CortexRenal cortexRenal cortex benchmarking atlas50 02736 03095.6%35

Table A.7. The six annotated single-cell datasets of Table 6, where Cells and Genes are the dimensions of the raw count matrix, Sparsity the fraction of its entries that are zero and Types the number of curated cell types. The two atlases above 50 000 cells are subsampled as described in § 4.2.3, and their three columns describe the subsampled matrix. All six are then reduced to the same 50 principal components as in § 4.

Datasett-SNEλ=0\lambda=0SR-tSNEUMAPt-SNEλ=0\lambda=0SR-tSNEUMAPt-SNEλ=0\lambda=0SR-tSNE
\multicolumn{4}{c}{Pieces per type ↓\downarrow}\multicolumn{4}{c}{Largest share ↑\uparrow}\multicolumn{3}{c}{Cross-type edges}
Trachea16.5016.2115.7119.360.9260.9280.9340.8930.0780.0780.079
Liver-MacParland10.4010.229.7910.200.8850.8850.8940.8860.1690.1690.171
Kidney16.1115.9715.5116.000.9290.9300.9350.9340.1230.1230.125
Kidney-Park4.444.444.414.310.8680.8680.8690.8550.0290.0290.031
Brain-Cortex10.069.999.8110.460.6720.6810.7050.6740.0200.0210.021
Renal-Cortex47.4047.4645.9347.260.6900.6920.7250.7170.1580.1590.164

Table A.8. The 15-nearest-neighbor graph of the embedding that Leiden reads (symmetrized, unweighted), on the six datasets of Table 6. Pieces is the number of connected components of the subgraph induced by an annotated cell type, averaged over types; largest share is the fraction of a type held in its largest component, averaged over types; cross-type edges is the fraction of graph edges joining two types. λ=0\lambda=0 and SR-tSNE are means over the five warm starts. Bold marks the best of the four per row.

Table A.9 repeats the clustering with the resolution fixed in advance, so no annotation enters the choice. SR-tSNE keeps the top ARI on all six datasets at both settings.

Datasett-SNEPHATEUMAPTriMapLocalMAPDREAMSλ=0\lambda=0SR-tSNE
Resolution 1.0, ARI ↑\uparrow
Trachea.2343.2100.2156.2201.2082.2013.2311.2674
Liver-MacParland.3333.2988.3150.3448.2893.3098.3270.3835
Kidney.3131.2243.2706.2774.2630.2496.3029.3332
Kidney-Park.0663.0627.0603.0675.0580.0604.0673.0865
Brain-Cortex.0333.0373.0407.0430.0414.0379.0348.0565
Renal-Cortex.1374.0683.1433.1523.1491.1335.1384.1885
SR-tSNE wins on6/66/66/66/66/66/66/6
two-sided pp.031.031.031.031.031.031.031
Mean AMI over the six (6/6 best).5975.5243.5955.5978.5940.5868.5970.6204
Mean FMI over the six (6/6 best).3135.2644.3039.3118.2974.2933.3110.3474
Mean clusters found (truth 23.0)8468746378907952
Resolution 0.5, ARI ↑\uparrow
Trachea.3626.2984.3057.3423.2985.2759.3448.3934
Liver-MacParland.4456.4256.3841.3933.3779.3731.4476.5398
Kidney.4125.2886.3559.3781.3487.3297.4189.4829
Kidney-Park.1062.0934.1049.0994.0829.0996.1010.1209
Brain-Cortex.0519.0546.0662.0680.0599.0601.0509.0830
Renal-Cortex.1881.0942.2037.2031.2025.1744.1886.2468
SR-tSNE wins on6/66/66/66/66/66/66/6
two-sided pp.031.031.031.031.031.031.031
Mean AMI over the six (6/6 best).6325.5517.6306.6305.6260.6187.6323.6585
Mean FMI over the six (6/6 best).3838.3219.3654.3708.3549.3479.3797.4278
Mean clusters found (truth 23.0)6145524458685837

Table A.9. Leiden at a resolution fixed in advance (1.0, then 0.5) for every method and dataset, so no annotation enters the choice; each row scores one partition. ARI per dataset with the sign count and two-sided Wilcoxon pp of SR-tSNE against each method over the six datasets (bold where p<0.05p < 0.05), then AMI and FMI averaged over the six datasets and the mean number of clusters found (the annotations average 23 types). λ=0\lambda=0 and SR-tSNE are means over five warm starts. Bold marks the best per row.

Table A.10 scores a criterion the training objective never sees. SR-tSNE takes the top AUC in both the local and the global regime, on both datasets.

Table A.11 gives the source and the data type of each benchmark alongside its dimensions.

Figure A.1 follows the capital of all 35 names through the quarter, one row per sector group.

Figure A.2 scores the markers themselves, by the move that follows them, so the result does not depend on the execution rule.

MethodAUC (full range, 1/K1/K) MNISTAUC (full range, 1/K1/K) CIFARAUC (K≤100K \le100) MNISTAUC (K≤100K \le100) CIFAR
t-SNE.3327.2252.3627.1909
UMAP.2524.1557.2089.0580
PHATE.1541.1268.0725.0339
TriMap.2211.1457.1453.0464
PaCMAP.2269.1440.1594.0439
ISOMAP.1463.1099.0392.0118
Autoencoder.2393.1489.1285.0481
ParamPaCMAP.2188.1443.1500.0422
SUDE.1392.1166.0595.0325
LocalMAP.2432.1518.1962.0610
DREAMS.2952.1939.2971.1424
SR-tSNE (λ=0\lambda=0).3329.2253.3631.1910
SR-tSNE (ours).3370.2278.3718.1966

Table A.10. RNXR_{NX} AUC on a 20k subsample of MNIST and CIFAR-10. The criterion is read off the co-ranking matrix and is independent of the training objective. All methods are scored on the same 20k subsample (seed 42) from their stored full-size embeddings; λ=0\lambda=0 is the equal-budget control of Table 2. Bold marks the best method per column.

DatasetDomainData TypennddClasses
MNIST [35]Computer VisionGrayscale images (handwritten digits)70 00078410
CIFAR-10 [33]Computer VisionColor images (CNN features)60 00051210
Fashion-MNIST [36]Computer VisionGrayscale images (clothing articles)70 00078410
Speech Commands [37]Speech / AudioSpoken keywords (wav2vec 2.0 features [38])59 38776835
Renal-CortexSingle-cell genomicsscRNA-seq, renal cortex benchmarking atlas50 02736 03035
Brain-CortexSingle-cell genomicsPrecentral gyrus dissection50 21258 23235
Covertype [39]Remote sensingCartographic and soil attributes (tabular)59 999547
AG News [40]Natural languageNews headlines (BERT embeddings [41])60 0007684

Table A.11. Overview of the eight benchmark datasets and their data types. Here nn is the number of points, dd the input dimensionality and Classes the number of annotated labels.

Image not extracted

Figure A.3. Figure A.3 sweeps neighborhood membership across kk. SR-tSNE holds the top curve at k∈{5,10}k \in\{5, 10\} on every benchmark.

Cumulative return of the 35 names at 2 bps, one row per group

Figure A.1. Cumulative return of the 35 names at 2 bps, one row per group. Buy and hold is charged the same round-trip cost.

Per-name trading results

Tables A.12 and A.13 give the per-name results behind § 4.2.1: net return at 0 and 2 bps, and the annualized Sharpe ratio, maximum drawdown and number of round trips at 2 bps, for the 35 equities in seven groups of five. Every arm runs through the identical DBSCAN+K scorer of Appendix A.§ 4 with the same per-stock marker percentile, so only the embedding differs between columns. Figure 5 summarizes both tables at 2 bps.

What the markers are worth

Figure A.2. What the markers are worth. The edge of an embedding on one name is the mean move of the close over the next 24 bars after its buy markers minus the mean move after its sell markers. a gives the two halves separately, each a median over the 35 names of Table A.12; b tracks the median edge at three holding horizons, with the paired Wilcoxon p against t-SNE beside each; c gives the per-name difference against t-SNE at two hours.

The DBSCAN+K Marker Scorer

The DBSCAN+K marker scorer is set out here. DBSCAN [42] is run on the two-dimensional embedding, and the points it labels as noise are the candidates. Each candidate ii carries two components. The geometric one is gi=di/δˉc(i)g_i = d_i/\bar{\delta}_{c(i)}, where c(i)c(i) is the cluster nearest to ii and did_i the distance from ii to that cluster. The scale δˉc\bar{\delta}_c is the mean pairwise distance inside cluster cc, estimated on a 50-point sample. A point sitting far from a tight cluster scores high.

The order component reads rank agreement. Let ρi\rho_i be the Spearman correlation [43] between the ranks of the ten high-dimensional nearest neighbors of ii and the ranks of those same points by embedding distance. A candidate whose neighborhood survives the projection has ρi\rho_i near one and is unlikely to be a genuine anomaly, so the component is 1−ρi1-\rho_i. Both are min–max normalized across the candidate set and added with equal weight,

DLP versus neighborhood size $k$ on eight benchmarks

Figure A.3 (PDF p. 49). DLP versus neighborhood size kk on the same eight benchmarks, with the legend at lower right. Each panel carries the methods for which a sweep across kk was stored; the four methods added at k=10k=10 in Table 5 are scored at that single size and are not swept here. SR-tSNE holds the top curve at k∈{5,10}k \in\{5, 10\} on every benchmark.

si=gi~+(1−ρi~),(23)s_i = \widetilde{g_i} + \left(1-\widetilde{\rho_i}\right), \tag*{(23)}

where the tilde denotes that normalization. The highest-scoring candidates in each direction are retained as markers, long giving a buy and short a sell. We refer to this scorer as DBSCAN+K.

Held-out Validation

The configuration rule and the gains in order hold on held-out data. We add the core Human Lung Cell Atlas [29], a tissue absent from the eight benchmarks, subsampled to 60 000 cells. Twenty donors are held out as queries. Gene selection, scaling and

GroupStock0 bps t-SNEUMAPTriMapLocalMAPDREAMSSR-tSNE2 bps t-SNEUMAPTriMapLocalMAPDREAMSSR-tSNE
TechnologyFICO+29.6+45.5+38.8+62.2+39.0+44.0+26.8+33.2+25.5+56.6+34.1+37.3
PAYC+49.6+55.5+60.4+47.8+62.4+56.6+44.6+43.6+42.0+33.9+54.8+52.8
DLB−10.7−2.8+3.9+7.2−5.3−8.9−12.1−9.6−6.7+1.5−9.0−10.3
RMBS−2.8−3.6+16.6+34.7+3.7+38.0−6.1−10.1+6.1+24.8−1.0+32.8
IDCC−6.2+13.3+33.9+1.3+19.9+29.8−7.5+9.1+22.2−5.1+14.6+28.0
Health careUNH+25.4+40.5+35.1+42.8+42.7+34.2+23.6+35.7+29.9+34.4+39.8+31.5
MRNA+0.9+0.9+17.9+27.8+41.6+3.1−3.8−1.5+12.7+22.1+38.4+1.1
ELAN+11.8−8.7−12.0+14.5−6.0+34.9+6.4−11.1−18.0+4.7−11.5+32.1
BMRN+9.8+30.1+26.7+17.9+9.4+18.2+4.8+21.6+10.7+6.1+3.7+13.8
XRAY+15.5+15.2+15.6+12.5+20.6+22.7+13.4+10.4+2.8+9.4+16.3+17.9
FinancialsSCHW+1.0+4.1+7.6+4.5−1.7+5.0−1.8+1.2+2.2+0.3−4.2+1.8
TROW+18.9+23.0+21.0+15.2+20.8+23.1+13.0+18.1+12.9+7.6+15.4+18.6
AMP+10.1+6.3+12.8+1.8+8.8+16.3+8.4+2.6+0.1−3.9+5.6+12.8
IVZ+2.3+14.6+5.9+10.6+11.9+20.0−2.2+8.6−0.0+1.7+8.7+14.7
MCY+20.5+29.5+30.9+26.1+31.8+26.2+15.6+21.2+19.9+18.3+27.3+21.3
IndustrialsUBER+5.0+9.1−5.7+9.2−0.2+16.0+0.2+1.6−9.0+2.2−2.8+11.0
RTX+1.7−0.5−1.9+6.5+3.6+6.0−0.8−4.6−5.6+0.0+1.8+4.2
DD+4.7+2.2+0.2−1.5−2.6+11.2+2.3−0.8−3.7−4.2−6.7+8.3
ATI+2.1+0.5+10.2+17.3+3.5+25.7−1.2−5.4−2.5+11.3−2.3+22.4
ARCB+16.5+27.2+69.1+15.7+39.6+49.4+14.9+22.3+59.4+11.1+35.5+45.6
ConsumerABNB+2.4+1.7−1.2+3.6+9.4+12.9−0.1−2.5−6.0+0.1+6.8+9.8
DHI+12.4+8.9+19.2+2.5+6.8+14.4+6.8+0.9+8.1−3.3+1.8+11.4
GIS+9.2+6.5+5.4+0.6+12.4+15.4+7.4−1.6+1.9−2.3+5.6+13.4
HRB+35.9+19.0+18.5+17.5+17.6+38.6+31.1+12.7+13.2+11.1+14.9+36.5
CAVA−1.3−9.7−17.5−6.8−2.1+14.0−2.9−13.9−26.4−9.9−8.2+11.2
Energy, REITs & utilitiesOKE+0.3+10.7+15.1+9.7+6.7+14.1−3.5+7.2+8.7+3.6+1.9+10.7
CCI+0.8−3.7−0.5−3.4+14.6+3.8−3.0−6.5−9.6−7.9+12.8−1.0
PBF+36.9+44.5+23.9+65.2+61.3+42.8+31.5+39.8+9.4+60.3+56.5+38.2
EVRG+5.6+0.1+13.0+7.4+6.3+6.8+0.6−3.4+7.0+1.6+3.6+3.6
OHI+6.5+11.1+4.6+14.9+5.0+10.4+1.8+6.8−2.5+11.4+1.2+7.7
Large capsINTC+96.9+67.9+47.9+63.3+82.5+104.8+84.7+56.6+32.4+53.2+74.8+94.6
UAL+8.1+11.9+21.2+7.2+11.2+25.4+3.6+4.1+6.0+4.3+9.2+22.8
AVGO+14.2+14.3+7.0+6.5+1.6+19.2+8.0+9.8+0.6+3.3−0.4+17.0
GOOGL+9.4+0.6+6.1+22.6+9.1+11.7+4.7−4.0−0.4+18.9+6.5+8.9
NFLX−0.8−2.8−13.6−3.3−8.8+25−4.2−9.6−23.5−8.0−11.7−2.6
Median+8.1+9.1+13.0+10.6+9.4+18.2+3.6+2.6+2.8+4.3+5.6+13.8

Table A.12. Returns (%) on the 35 names, five per group across seven groups. Bold marks the best of the six per name.

GroupStockSharpe t-SNEUMAPTriMapLocalMAPDREAMSSR-tSNEMax drawdown (%) t-SNEUMAPTriMapLocalMAPDREAMSSR-tSNERound trips t-SNEUMAPTriMapLocalMAPDREAMSSR-tSNE
TechnologyFICO+2.90+3.19+2.52+5.14+3.05+3.42−10.28−14.97−14.69−7.54−15.71−11.422168801008352357478
PAYC+4.26+4.12+4.25+3.49+5.13+5.16−13.62−7.87−15.87−11.14−8.74−9.493377951214990483246
DLB−1.60−1.29−0.82+0.39−1.16−1.34−16.41−15.66−12.90−8.75−15.59−17.011297181007546390147
RMBS−0.05−0.26+0.72+1.92+0.32+2.08−32.79−33.84−31.67−24.45−31.17−30.38343658942761465381
IDCC−0.81−1.05+2.01+0.43+1.47+2.61−15.21−11.61−17.91−16.83−13.32−10.39128380543649449138
Health careUNH+4.52+4.78+4.18+4.59+5.26+4.22−3.28−7.06−3.56−3.34−5.59−3.92151341388609205161
MRNA−0.10+0.17+1.22+1.74+2.85+0.35−31.22−25.84−16.78−20.65−19.11−24.45478242458454224193
ELAN+0.97−1.31−2.54+0.77−1.66+3.56−21.52−23.52−22.74−15.07−23.11−10.93497223696299191209
BMRN+0.92+4.30+1.97+1.18+0.76+2.34−9.17−6.50−29.90−4.40−9.83−7.9338667513541022542366
XRAY+1.85+1.42+0.51+1.36+1.96+2.32−13.53−12.79−14.05−10.16−10.96−11.311804291170280360400
FinancialsSCHW−0.36+0.35+0.57+0.16−0.86+0.53−13.21−10.95−16.59−11.69−13.07−8.31282287517415265318
TROW+3.00+4.05+2.86+1.77+3.39+4.26−4.08−5.90−6.72−3.58−3.92−3.63499254388482458318
AMP+1.72+0.67+0.12−0.80+1.34+2.94−7.88−11.19−10.94−8.44−6.47−5.151553431198420262305
IVZ−0.26+1.47+0.11+0.41+1.56+2.30−11.26−13.43−14.63−14.00−9.51−6.24456350556817281453
MCY+2.79+3.93+3.57+3.47+4.56+3.54−5.98−5.57−7.76−8.65−4.33−6.08413664821641341397
IndustrialsUBER+0.15+0.38−1.30+0.48−0.32+1.88−7.66−10.76−14.53−8.42−13.69−11.11462490356668262440
RTX−0.09−1.00−1.21−0.10−0.54+1.06−10.14−12.06−14.08−15.94−8.34−7.43220416360630174174
DD+0.52+0.01+0.49−0.56+1.11+1.42−13.06−10.89−12.46−16.55−18.31−12.90227291402277254265
ATI+0.03−0.57−0.17+1.60−0.15+2.93−12.77−13.78−11.89−11.55−16.55−8.323292541220524580266
ARCB+2.07+2.64+5.74+1.33+3.99+5.20−16.55−17.40−9.87−12.63−10.55−7.47132389590408293257
ConsumerABNB+0.11+0.29−0.84+0.13+1.21+1.68−9.41−16.44−13.87−11.79−10.11−10.22244270501316240276
DHI+1.06+0.29+1.42−0.41+0.43+1.62−9.05−8.30−10.99−12.12−9.79−13.83513757402469474265
GIS+1.41−0.26+0.47−0.33+1.17+2.44−8.22−11.66−11.08−8.87−9.85−4.80167639338290628176
HRB+3.32+0.64+1.65+0.35+1.77+2.41−10.53−11.23−11.42−11.77−10.45−7.73362542460564540156
CAVA−0.14−1.45−3.13−0.94−0.76+1.29−16.79−26.21−34.18−18.44−28.67−20.431712771139343647249
Energy, REITs & utilitiesOKE−0.59−1.46−1.79+0.83+0.51+2.11−10.01−19.38−7.82−10.78−11.11−8.18308266473567456301
CCI+0.53+1.18+1.78+1.83+2.04+2.00−16.38−16.16−16.73−20.98−11.35−12.06376302895465159467
PBF+2.63+3.25+1.05+4.84+3.90+2.95−16.37−19.64−23.06−12.10−14.93−16.124013311246503302327
EVRG+0.25−0.91+2.05+0.57+1.06+1.08−6.17−7.68−5.35−4.55−5.93−5.93481242392541252310
OHI+0.58+1.85−0.61+2.98+0.41+1.96−6.55−7.07−10.97−8.76−10.65−5.70378389343309313252
Large capsINTC+3.61+2.69+2.27+3.53+3.29+3.71−14.74−17.66−17.06−16.30−15.51−17.336402941106538420510
UAL+0.63+0.62+0.83+0.64+1.25+2.46−11.88−15.78−13.37−19.03−15.90−12.942507251340274178213
AVGO+1.16+1.48+0.28+0.54+0.19+2.34−11.03−7.62−22.23−16.73−22.44−8.19548400617299199185
GOOGL+1.01−0.62+0.09+2.85+1.25+1.37−7.45−12.69−15.93−9.06−12.87−16.26172463409308246255
NFLX−0.72−2.02−3.47−1.65−1.52−0.40−16.14−17.34−20.71−16.10−22.38−16.253502621215500323510
Median+0.63+0.62+0.57+0.77+1.21+2.32−11.26−12.06−13.87−11.79−11.35−10.22337380590476302266

Table A.13. Risk and activity for the same 35 names at 2 bps: annualized Sharpe ratio, maximum drawdown (%) and number of round trips. Bold marks the best of the six per name; for round trips no method is marked.

the principal components are fitted on reference cells only. Queries are projected with the fitted transform. The corpus then runs the pipeline of § 4 unchanged. At n=60 000n = 60\,000 the size rule selects λ=50\lambda=50.

SR-tSNE leads on the held-out corpus as well. Table A.14 reports the outcome.

SR-tSNE is ahead of t-SNE and UMAP on O-DLP, DLP and nDCG on the query cells, and ahead of t-SNE by +30% in O-DLP.

MethodO-DLP10_{10}DLP10_{10}nDCG10_{10}
UMAP.0183.1636.1718
t-SNE.0654.3754.4664
SR-tSNE (λ=0\lambda=0).0674.3748.4684
SR-tSNE (ours).0847.3931.5447

Table A.14. A held-out dataset: the Human Lung Cell Atlas core, split by donor into 48 000 reference and 12 000 query cells, scored on the query cells. The size rule of Section 4 selects λ=50\lambda=50. Bold marks the best method per column.

Implementation Details

Table A.15 collects the notation of the smooth rank loss, its optimization and the analysis of Section 3.4, with default values in parentheses. The optimizer of Section 3.3 is SGD with Polyak momentum,

v(t+1)=μ⋅v(t)+∇YLtotal(Y(t)),(24)v^{(t+1)} = \mu\cdot v^{(t)} + \nabla_Y \mathcal{L}_{\mathrm{total}}(Y^{(t)}), \tag*{(24)}
Y(t+1)=Y(t)−η⋅v(t+1),(25)Y^{(t+1)} = Y^{(t)} - \eta\cdot v^{(t+1)}, \tag*{(25)}

with η=10\eta= 10, μ=0.8\mu= 0.8 and T=1000T = 1000.

For large nn we store PP sparsely, keeping the top kP=90k_P=90 entries per row. The partition function ZZ is accumulated in row chunks of c=10,000c=10{,}000 with gradient checkpointing, which cuts peak memory from O(n2)O(n^2) to O(c n)O(c\,n); the refinement stage remains O(n2)O(n^2) in time. All gradients come from PyTorch autograd. Every Tr=50T_r = 50 iterations we recompute each point’s intruder set with the same chunked scan, skipping the point itself and its high-dimensional neighbor list. We use m=10m = 10 intruders throughout.

SymbolDescription
NiN_iHigh-dim kk-NN index list of point ii (precomputed, fixed)
Ni(j)N_i(j)Index of the jj-th high-dim nearest neighbor of point ii
IiI_i, CiC_iIntruder set of point ii; candidate pool Ni⊕IiN_i \oplus I_i
mm, TrT_rIntruder count (10) and mining interval (50 iterations)
σ(⋅)\sigma(\cdot)Sigmoid (logistic) function
τ\tau, τstart\tau_{\mathrm{start}}, τend\tau_{\mathrm{end}}Temperature parameter and its annealing endpoints
di,jd_{i,j}Embedding-space distance from yiy_i to yCi(j)y_{C_i(j)}
ri,jinr_{i,j}^{\mathrm{in}}Input-space rank of candidate jj (=j=j by construction)
ri,jembr_{i,j}^{\mathrm{emb}}Soft embedding rank of candidate jj with respect to point ii, Eq. (2)
ρi\rho_i, ρs\rho_sPer-point and mean Spearman correlation
GiG_i, GihardG_i^{\mathrm{hard}}, G⋆G^\starPer-point discounted gain read with the soft ranks, with the integer ranks, and its ideal value, Eqs. (3) and (13)
LrankL_{\mathrm{rank}}Rank loss, Eq. (5); Eq. (A.6) when m=0m=0
LtotalL_{\mathrm{total}}Total loss, KL⁡(P∥Q)+λLrank\operatorname{KL}(P\Vert Q) + \lambda L_{\mathrm{rank}}
λ\lambdaRegularization weight
ZZPartition function ∑j≠i(1+∥yi−yj∥2)−1\sum_{j \ne i}(1+\lVert y_i-y_j\rVert^2)^{-1}
kk, kPk_P, ccNeighbor count (10), sparsity (90), chunk size (10,000)
η\eta, μ\mu, TTLearning rate (10), momentum (0.8), iterations (1000)
k′k'Neighborhood size at which the metrics are read (10)
KK, disc⁡(r)\operatorname{disc}(r), ΔK\Delta_Kk+mk+m; discount 1/log⁡2(1+r)1/\log_2(1+r); its smallest step
πi,rin\pi_{i,r}^{\mathrm{in}}, πi,remb\pi_{i,r}^{\mathrm{emb}}Index of the rr-th nearest neighbor of ii in the input and embedding space
πiemb(j)\pi_i^{\mathrm{emb}}(j)Discrete embedding rank of candidate jj within CiC_i
LrankhardL_{\mathrm{rank}}^{\mathrm{hard}}Rank loss evaluated with the discrete ranks, Eq. (13)
Ai(k′)A_i(k')Fraction of the retained neighbors of ii that sit at their own rank
γi\gamma_iSmallest gap between two candidate squared embedding distances of ii
di,jHd_{i,j}^{H}Input-space distance from xix_i to its jj-th input-space neighbor
DD, D′D'Matrix of squared embedding distances and its single row swap
DLPi\mathrm{DLP}_i, O-DLPi\mathrm{O\text{-}DLP}_iPer-point membership and exact-rank agreement at size k′k'
RiR_i, AiA_iNeighbors kept among the k′k' nearest; their fraction at own rank
ω\omegak′k' O-DLP/DLP: 1 for a shuffle, k′k' for perfect order
deg⁡H(j)\deg_H(j)In-degree of jj in the high-dimensional kk-NN graph

Table A.15. Notation for the smooth rank loss, its optimization and the analysis of Section 3.4. High-dimensional quantities are fixed before training, while low-dimensional ones are recomputed from the current embedding. Default values are given in parentheses.

Hyperparameters

Table A.16 collects every setting of the two-stage pipeline, so that a run can be reproduced from this page alone. Three of them belong to the rank term and the rest are the scikit-learn Barnes–Hut defaults of the warm start.

How the three were chosen, and when. The neighbor count kk, the intruder count mm and the weight λ\lambda were fixed once and then carried unchanged through every benchmark, application and appendix study of this paper. We read O-DLP on MNIST over the grid k,m∈{10,30}k,m \in\{10,30\} and λ∈{5,50,100,200}\lambda\in\{5,50,100,200\}, which selected k=m=10k=m=10 and the large-set weight λ=50\lambda=50; the small-set weight λ=5\lambda=5 was selected on the 200-stock pool from which the 35 trading names of Section 4.2.1 are drawn. No dataset outside those two entered the choice, and no per-dataset tuning was performed afterwards. Setting k=m=10k=m=10 also makes the list the loss ranks the same list that O-DLP, nDCG and the trading scorer read, so the objective and the criteria see one neighborhood.

What the settings mean. The warm start is a standard Barnes–Hut t-SNE: PCA initialization, perplexity 30, learning rate max⁡(n/12,50)\max(n/12, 50), early exaggeration 12 and 1,000 iterations. The refinement runs for another T=1,000T=1{,}000 iterations at η=10\eta=10 with Polyak momentum μ=0.8\mu=0.8 and no early exaggeration. The value of η\eta puts the KL gradient and the rank gradient on the same scale. A pure-KL run under this setting therefore stays close to its warm start, which is why the λ=0\lambda=0 control of Table 4 reproduces t-SNE. The temperature falls linearly from τstart=5.0\tau_{\mathrm{start}}=5.0 to τend=0.05\tau_{\mathrm{end}}=0.05, a range in which Theorem 2 puts the soft rank within O(e−γi/0.05)O(e^{-\gamma i/0.05}) of the discrete rank at the end of training. Intruders are re-mined every Tr=50T_r=50 iterations. For n≥50,000n\geq50{,}000 the affinity matrix keeps kP=90k_P=90 entries per row and the partition function is accumulated in row chunks of c=10,000c=10{,}000.

Sensitivity and reproducibility. The behavior of the three settings away from their defaults is reported in § 4.3: the neighborhood sweep of Figure 2 (k∈{5,10,15,20}k\in\{5,10,15,20\}), the λ=0\lambda=0 control of Table 4, the m=0m=0 case of Appendix A.8 and the five perturbed warm starts of Appendix A.9. All runs use seed 42 and one NVIDIA A6000; the peer methods run at their published settings, described in § 4. The implementation that produces these settings is released with the paper.

The m=0m=0 Special Case

With m=0m=0 no intruders are used. The candidate pool is then the neighbor list itself, Ci=NiC_i=N_i. Nothing outside NiN_i enters the objective, so the loss no longer defends membership. It scores only the order inside the neighborhood. In this case the position discount of Eqs. (3)–(5) can be replaced by the Spearman rank correlation between the two rank sequences. For point ii this is the Pearson correlation of the centered rank vectors. The usual closed form for Spearman’s ρ\rho assumes distinct integer ranks. Soft ranks do not satisfy that assumption, so we use the general form here. Since ri,jin=jr^{\mathrm{in}}_{i,j}=j, the high-dimensional mean is the constant rˉin=(k+1)/2\bar r^{\mathrm{in}}=(k+1)/2. The low-dimensional mean is rˉiremb=1k∑j=1kri,jremb\bar r^{\mathrm{remb}}_i=\frac{1}{k}\sum_{j=1}^{k}r^{\mathrm{remb}}_{i,j}. The per-point Spearman correlation is:

StageParameterValue
Warm start (scikit-learn Barnes–Hut)InitializationPCA
Warm start (scikit-learn Barnes–Hut)Perplexity30
Warm start (scikit-learn Barnes–Hut)Learning ratemax⁡(n/12,50)\max(n/12, 50)
Warm start (scikit-learn Barnes–Hut)Early exaggeration12
Warm start (scikit-learn Barnes–Hut)Iterations1 000
Refinement (PyTorch, exact KL)Learning rate η\eta10
Refinement (PyTorch, exact KL)Momentum μ\mu0.8
Refinement (PyTorch, exact KL)Iterations TT1 000
Refinement (PyTorch, exact KL)Early exaggerationdisabled
Refinement (PyTorch, exact KL)τstart→τend\tau_{\mathrm{start}} \to\tau_{\mathrm{end}}5.0 →\to 0.05 (linear)
Refinement (PyTorch, exact KL)Sparsity kPk_P90
Refinement (PyTorch, exact KL)Chunk size cc10 000
Rank lossLoss (all datasets)smooth rank, m=10m = 10, Tr=50T_r = 50
Rank lossλ\lambda, large datasets (n≥50,000n \ge50{,}000)50
Rank lossλ\lambda, other datasets5
Rank lossNeighbor count kk10

Table A.16. Full hyperparameter settings for the SR-tSNE two-stage pipeline. The warm start and the refinement stage are held fixed across every dataset. Only λ\lambda follows dataset size, at 50 above 50 000 points and 5 below.

ρi=∑j=1k(j−k+12)(ri,jremb−rˉiremb)∑j=1k(j−k+12)2∑j=1k(ri,jremb−rˉiremb)2+ε(26)\rho_i = \frac{\sum_{j=1}^{k}\left(j-\frac{k+1}{2}\right)\left(r^{\mathrm{remb}}_{i,j}-\bar r^{\mathrm{remb}}_i\right)} {\sqrt{\sum_{j=1}^{k}\left(j-\frac{k+1}{2}\right)^2}\sqrt{\sum_{j=1}^{k}\left(r^{\mathrm{remb}}_{i,j}-\bar r^{\mathrm{remb}}_i\right)^2+\varepsilon}} \tag*{(26)}

where ε=10−8\varepsilon=10^{-8} prevents division by zero. In the denominator, the factor built from the high-dimensional ranks, ∑j(j−(k+1)/2)2\sqrt{\sum_j(j-(k+1)/2)^2}, is a positive constant set by kk alone. The gradient therefore flows entirely through ri,jrembr^{\mathrm{remb}}_{i,j}.

The mean Spearman correlation over all nn points is:

ρs=1n∑i=1nρi(27)\rho_s=\frac{1}{n}\sum_{i=1}^{n}\rho_i \tag*{(27)}

and the rank loss is then:

Lrank(Y)=1−ρs=1n∑i=1n(1−min⁡(max⁡(ρi,−1),1)),(28)\mathcal{L}_{\mathrm{rank}}(Y) = 1 - \rho_s = \frac{1}{n}\sum_{i=1}^{n}\left(1-\min\left(\max(\rho_i,-1),1\right)\right), \tag*{(28)}

This lies in [0,2][0,2]. It is zero only when every point’s high-dimensional neighbor order is reproduced exactly in the embedding. The min⁡(max⁡(⋅,−1),1)\min(\max(\cdot,-1),1) projection guards against floating-point round-off in ρi\rho_i.

Multi-seed Stability

Paired seed statistics. Table A.17 reports the paired advantage of SR-tSNE over the λ=0\lambda=0 control across five warm starts perturbed with 1% Gaussian jitter. Every mean is positive, every one of the 30 metric differences over the ten paired runs (two datasets, five perturbed starts) is positive, and standard deviations are at most 0.0005. The jitter moves each point by about the spacing of its nearest neighbors, so the perturbed start carries much less local order. On MNIST its O-DLP after the λ=0\lambda=0 refinement is .056, against .078 for the unperturbed t-SNE. The rank term brings the same start to .092, the level of the unperturbed SR-tSNE run of Table 2. The paired advantage is therefore larger than the gap in that table, and it measures the term’s ability to recover order as well as to add it.

DatasetΔnDCG10\Delta\mathrm{nDCG}_{10}ΔO-DLP10\Delta\mathrm{O\text{-}DLP}_{10}ΔDLP10\Delta\mathrm{DLP}_{10}
MNIST+.1507 ±\pm .0005+.0361 ±\pm .0004+.0594 ±\pm .0002
CIFAR-10+.0795 ±\pm .0003+.0161 ±\pm .0003+.0221 ±\pm .0002

Table A.17. Paired advantage of SR-tSNE over the equal-budget control, mean ±\pm std over five warm starts perturbed with 1% Gaussian jitter. Both methods share each seed’s start, so every comparison is paired. All 30 metric differences over the ten paired runs are positive.

Warm-start length

The gain does not depend on stopping the warm start at 1,000 iterations. Table A.18 continues it to 2,000 and refines from there. SR-tSNE leads that longer run on all eight benchmarks, by a median of +10%, and stands +6% above its own 1,000-iteration-start result.

The form of the rank term

The listwise form is what carries the gain. Table A.19 replaces it with a pairwise logistic surrogate on the same candidate pool, the same warm start and the same budget. SR-tSNE stands above that surrogate on all eight benchmarks.

DatasetStart: 1 000 iterations t-SNEStart: 1 000 iterations SR-tSNEStart: 2 000 iterations t-SNEStart: 2 000 iterations λ=0\lambda=0Start: 2 000 iterations SR-tSNEGain (%)
MNIST.0781.0927.0890.0901.0979+10
CIFAR-10.0545.0597.0624.0626.0641+3
Fashion-MNIST.0672.0792.0768.0768.0843+10
Speech Commands.0809.0858.0893.0900.0900+1
Renal-Cortex.0578.0741.0669.0668.0790+18
Brain-Cortex.0615.0767.0697.0696.0808+16
Covertype.1496.1913.1679.1679.1998+19
AG News.0810.0905.0916.0915.0970+6

Table A.18. Two warm starts. Left: the paper’s t-SNE run of 1 000 iterations and SR-tSNE refined from it. Right: a t-SNE run continued to 2 000 iterations, the λ=0\lambda=0 control and SR-tSNE refined from that longer run, and the gain of SR-tSNE over the longer run. O-DLP at k=10k=10.

DatasetO-DLP t-SNEO-DLP PairwiseO-DLP SR-tSNEDLP t-SNEDLP PairwiseDLP SR-tSNEnDCG t-SNEnDCG PairwisenDCG SR-tSNE
MNIST.0781.0764.0927.3880.3849.4009.5079.5013.5689
CIFAR-10.0545.0528.0597.2087.2070.2149.3333.3275.3682
Fashion-MNIST.0672.0646.0792.3368.3339.3493.4448.4369.5025
Speech Commands.0809.0791.0858.3078.3065.3110.4652.4605.4993
Renal-Cortex.0578.0543.0741.3243.3198.3375.4147.4030.4890
Brain-Cortex.0615.0582.0767.3434.3397.3545.4380.4269.5121
Covertype.1496.1470.1913.6369.6372.6452.7471.7444.8013
AG News.0810.0782.0905.3529.3516.3568.4821.4761.5260

Table A.19. The form of the rank term. From the same scikit-learn warm start, with the same 10 neighbors and 10 mined intruders, optimizer and budget, the refinement runs with a pairwise logistic ordinal loss at the weight selected on MNIST (λ=0.05\lambda= 0.05; larger weights lower every criterion) and with the listwise discounted gain of SR-tSNE. All criteria at k=10k=10.

Per-dataset label-transfer results

Table A.20 gives the per-dataset scores behind the summary of Table A.6. A random fifth of each dataset is removed from the retrieval index and used as queries; the embedding is fitted on all points, so the protocol is transductive and only the index is reduced. Each query is scored from its ten nearest reference points in the two-dimensional embedding: P₁ asks whether position one already carries the query’s class, and MRR takes the reciprocal of the first position that does. Both criteria therefore read a neighbor’s position, not merely its presence. The last column, Δ\Delta, is SR-tSNE minus the equal-budget λ=0\lambda=0 control, so the control’s own score on each row is the SR-tSNE entry minus Δ\Delta. The two rows closing each block give the sign count and the two-sided Wilcoxon signed-rank test over the seven datasets.

Table A.20: Label transfer across seven vision datasets, one paired observation each, with Δ\Delta taken against the equal-budget control (λ=0\lambda=0), whose own score on each row is SR-tSNE minus Δ\Delta. Every query is scored from its ten nearest reference points in the two-dimensional embedding, on the same held-out fifth of each dataset. Bold marks the best score in a row. The two rows below each block give the sign count and the two-sided Wilcoxon signed-rank test over the seven datasets, where bold marks p<0.05p < 0.05.

Datasetnnλ\lambdaUMAPTriMapt-SNELocalMAPDREAMSSR-tSNEΔ\Delta
P1P_1: is the first neighbor returned of the query’s class
MNIST70 00050.8811.8351.9288.9164.9019.9354+0.0063
Fashion-MNIST70 00050.7159.6877.8296.7471.8173.8382+0.0086
CIFAR-1060 00050.7218.6722.8035.7267.7707.8053+0.0016
Tiny-ImageNet70 00050.2479.1689.4561.2706.4044.4584+0.0023
EMNIST-Balanced69 98350.4608.3174.6728.5692.6090.6888+0.0164
CIFAR-10060 00050.3280.2438.5466.3356.4842.5522+0.0058
USPS9 2985.9301.9091.9602.9381.9478.9634+0.0032
SR-tSNE wins on7/77/77/77/77/7
two-sided pp.0156.0156.0156.0156.0156
MRR: reciprocal rank of the first correct neighbor
MNIST70 00050.9151.8835.9459.9394.9299.9507+0.0046
Fashion-MNIST70 00050.8038.7812.8802.8262.8721.8858+0.0057
CIFAR-1060 00050.7949.7611.8537.7985.8311.8547+0.0009
Tiny-ImageNet70 00050.3401.2632.5291.3651.4868.5313+0.0021
EMNIST-Balanced69 98350.5891.4581.7522.6796.7079.7619+0.0099
CIFAR-10060 00050.4321.3521.6215.4420.5749.6247+0.0034
USPS9 2985.9503.9325.9677.9524.9594.9693+0.0017
SR-tSNE wins on7/77/77/77/77/7
two-sided pp.0156.0156.0156.0156.0156

Table A.20.

Runtime

Table A.21 times embedding construction alone, on one NVIDIA A6000 with exclusive access; neighbor search and the evaluation of the criteria are excluded, and every method is timed on the same host. The last two rows differ by the rank term only, so their difference is the cost of the term itself: 1.8% of the refinement stage on MNIST and 2.1% on CIFAR-10, which is where the cost of the method sits. The refinement in this table evaluates the exact partition function, and § 5 gives the O(nlog⁡n)O(n \log n) route that keeps the rank term unchanged.

Stage / methodMNISTCIFAR-10
UMAP0.400.06
PaCMAP0.530.21
TriMap0.630.41
t-SNE (warm start)3.33.1
+ refinement, λ=0\lambda=025.6918.93
+ refinement, SR-tSNE26.1619.33

Table A.21. Wall-clock runtime in minutes on one NVIDIA A6000, timing only embedding construction. The two refinement methods differ only in the rank term.

Proofs

This section proves the four theorems of § 3.6, together with the two lemmas, the two propositions and the corollary they rest on. Results labelled with an S, namely Lemmas S1 and S2, Propositions S1 and S2 and Corollary S1, are stated and proved here. Every other number refers to the main text.

Fix a point ii. Its candidate pool Ci=Ni⊕IiC_i = \mathcal{N}_i \oplus\mathcal{I}_i has length K=k+mK = k + m. Candidate j≤kj \leq k is the jj-th true neighbor and carries relevance rel⁡j=k−j+1\operatorname{rel}_j = k - j + 1; candidates j>kj > k are intruders and carry rel⁡j=0\operatorname{rel}_j = 0. Write RiR_i for the retained neighbors of ii, the true neighbors that the embedding returns among its k′k' nearest points, and pos⁡i(j)\operatorname{pos}_i(j) for the position at which the embedding returns the jj-th true neighbor. The soft and discrete gains of point ii are GisoftG_i^{\mathrm{soft}} and GihardG_i^{\mathrm{hard}}, the first read with the soft ranks of Eq. (2) and the second with the integer ranks. The embedding orders CiC_i by distance to yiy_i; call that ordering πi\pi_i, which orders the candidate pool and is distinct from the neighbor maps πiin\pi_i^{\mathrm{in}} and πiemb\pi_i^{\mathrm{emb}} of Eq. (7), so that πi(r)\pi_i(r) is the candidate at position rr and (πiin)−1(j)(\pi_i^{\mathrm{in}})^{-1}(j) the position of candidate jj. Ties among intruders are immaterial because their relevance is equal; ties elsewhere are broken arbitrarily. The hard per-point gain and loss are

Gihard(πi)=∑r=1Krel⁡πi(r)disc⁡(r),ℓi=1−Gihard(πi)G⋆,Lrankhard=1n∑iℓi,(29)G_i^{\mathrm{hard}}(\pi_i) = \sum_{r=1}^{K} \operatorname{rel}_{\pi_i(r)} \operatorname{disc}(r), \qquad\ell_i = 1 - \frac{G_i^{\mathrm{hard}}(\pi_i)}{G^\star}, \qquad\mathcal{L}_{\mathrm{rank}}^{\mathrm{hard}} = \frac{1}{n}\sum_i \ell_i, \tag*{(29)}

with disc⁡(r)=1/log⁡2(1+r)\operatorname{disc}(r) = 1 / \log_2(1+r) and G⋆G^\star as in Eq. (4). This GihardG_i^{\mathrm{hard}} is the discrete gain of Eq. (13) re-indexed by position; intruders carry relevance zero, so the two sums agree.

Two facts about the discount are used repeatedly: it is strictly decreasing, and its steps disc⁡(r)−disc⁡(r+1)\operatorname{disc}(r)-\operatorname{disc}(r+1) are strictly decreasing in rr, because 1/log⁡2(1+r)1 / \log_2(1+r) is convex. Hence disc⁡(r)−disc⁡(r′)≥ΔK\operatorname{disc}(r)-\operatorname{disc}(r') \ge\Delta_K for all 1≤r<r′≤K1 \le r < r' \le K.

A sorting lemma. Call a pair of candidates (a,b)(a,b) a strict inversion of πi\pi_i if rel⁡a>rel⁡b\operatorname{rel}_a > \operatorname{rel}_b and πi−1(a)>πi−1(b)\pi_i^{-1}(a) > \pi_i^{-1}(b). Let inv⁡(πi)\operatorname{inv}(\pi_i) be their number. The ideal ordering, neighbors 1,…,k1,\ldots,k in order followed by the intruders, has none.

This lemma says that every wrongly ordered pair costs the gain at least one smallest step of the discount.

Lemma S1. G⋆−Gihard(πi)≥inv⁡(πi)ΔKG^\star- G_i^{\mathrm{hard}}(\pi_i) \ge\operatorname{inv}(\pi_i)\Delta_K.

Proof. If inv⁡(πi)>0\operatorname{inv}(\pi_i)>0 there is an adjacent position pair (r,r+1)(r,r+1) with rel⁡πi(r)<rel⁡πi(r+1)\operatorname{rel}_{\pi_i(r)} < \operatorname{rel}_{\pi_i(r+1)}; otherwise relevance would be non-increasing along the list and no strict inversion could exist. Swapping the two candidates raises the gain by

(rel⁡πi(r+1)−rel⁡πi(r))(disc⁡(r)−disc⁡(r+1))≥1⋅ΔK,(30)\left(\operatorname{rel}_{\pi_i(r+1)}-\operatorname{rel}_{\pi_i(r)}\right)\left(\operatorname{disc}(r)-\operatorname{disc}(r+1)\right) \ge1\cdot\Delta_K, \tag*{(30)}

since relevances are integers, and it lowers inv⁡\operatorname{inv} by exactly one: the swapped pair stops being an inversion and no other pair changes its relative order. Repeating until no strict inversion remains takes exactly inv⁡(πi)\operatorname{inv}(\pi_i) swaps and ends at an ordering whose gain is G⋆G^\star, because relevance is then non-increasing along the list and, intruders being interchangeable, that ordering is the ideal one. Summing the gains gives the claim. □\square

Proof of Theorem 1. Let k′≤min⁡(k,m)k' \le\min(k,m) and let TiT_i be the k′k' points nearest to yiy_i among all n−1n-1 others. At most k′k' of them are non-neighbors, and any non-neighbor in TiT_i is one of the k′k' nearest non-neighbors, hence one of the m≥k′m \ge k' nearest, hence in Ii\mathcal{I}_i. So Ti⊂CiT_i \subset C_i, and the position of each element of TiT_i in πi\pi_i equals its position among all n−1n-1 points. O-DLP at k′k' therefore reads the first k′k' positions of πi\pi_i:

1−O-DLP⁡i(k′)=1k′#{r≤k′:πi(r)≠r}=:miss⁡ik′.(31)1-\operatorname{O\text{-}DLP}_i(k') = \frac{1}{k'}\#\{r \le k' : \pi_i(r) \ne r\} =: \frac{\operatorname{miss}_i}{k'}. \tag*{(31)}

Call a candidate displaced if it does not sit where the ideal ordering puts it, that is, a neighbor jj with πi−1(j)≠j\pi_i^{-1}(j) \ne j or an intruder at a position ≤k\le k. Every miss is a position holding a displaced candidate, so miss⁡i≤#displaced⁡\operatorname{miss}_i \le\#\operatorname{displaced}. Each swap in the proof of Lemma S1 moves two candidates and the process ends with none displaced, so #displaced⁡≤2inv⁡(πi)\#\operatorname{displaced} \le2\operatorname{inv}(\pi_i). With Lemma S1,

1−O-DLP⁡i(k′)≤2inv⁡(πi)k′≤2(G⋆−Gihard)k′ΔK=2G⋆k′ΔKℓi.(32)1-\operatorname{O\text{-}DLP}_i(k') \le\frac{2\operatorname{inv}(\pi_i)}{k'} \le\frac{2(G^\star-G_i^{\mathrm{hard}})}{k'\Delta_K} = \frac{2G^\star}{k'\Delta_K}\ell_i. \tag*{(32)}

Averaging over ii gives Eq. (14).

For the equivalence: if Lrankhard=0\mathcal{L}_{\mathrm{rank}}^{\mathrm{hard}}=0 then every ℓi=0\ell_i=0, so by Lemma S1 every πi\pi_i has no strict inversion, so the kk neighbors of ii occupy positions 1,…,k1,\ldots,k in order and every intruder comes after them. A non-candidate is farther from yiy_i than every intruder, hence farther than every neighbor. So the kk nearest points to yiy_i are its kk neighbors in high-dimensional order, and O-DLP⁡i(k′)=1\operatorname{O\text{-}DLP}_i(k')=1 for every k′≤kk'\le k. Conversely, if O-DLP⁡(k)=1\operatorname{O\text{-}DLP}(k)=1 then for every ii the kk nearest points are the neighbors in order, so every intruder, being a non-neighbor, is farther than all of them; πi\pi_i is the ideal ordering and ℓi=0\ell_i=0. □

Proof of Theorem 2. (2) writes the soft rank of candidate jj as 1+∑ℓ≠jσ((di,j2−di,ℓ2)/τ)1+\sum_{\ell\ne j}\sigma((d_{i,j}^2-d_{i,\ell}^2)/\tau), and the hard rank as 1+∑ℓ≠j1[di,ℓ2<di,j2]1+\sum_{\ell\ne j}\mathbf{1}[d_{i,\ell}^2<d_{i,j}^2]. Each summand differs from its indicator by σ(−∣di,j2−di,ℓ2∣/τ)≤e−γi/τ\sigma(-|d_{i,j}^2-d_{i,\ell}^2|/\tau)\le e^{-\gamma_i/\tau}, and there are K−1K-1 summands. That is the first claim. For the second, disc⁡\operatorname{disc} has derivative −ln⁡2/((1+r)ln⁡2(1+r))-\ln2/((1+r)\ln^2(1+r)), whose absolute value on [1,K][1,K] is largest at r=1r=1, where it equals 1/(2ln⁡2)1/(2\ln2). Hence

∣Gisoft−Gihard∣≤∑j=1krel⁡j12ln⁡2(K−1)e−γi/τ=k(k+1)2⋅K−12ln⁡2e−γi/τ,(33)\left|G_i^{\mathrm{soft}}-G_i^{\mathrm{hard}}\right| \le\sum_{j=1}^{k}\operatorname{rel}_j\frac{1}{2\ln2}(K-1)e^{-\gamma_i/\tau} = \frac{k(k+1)}{2}\cdot\frac{K-1}{2\ln2}e^{-\gamma_i/\tau}, \tag*{(33)}

and dividing by G⋆G^\star gives the bound on ∣ℓisoft−ℓihard∣|\ell_i^{\mathrm{soft}}-\ell_i^{\mathrm{hard}}|. □

Proof of Theorem 3. Under the row relaxation each objective is evaluated by its formula on the array DD. The row swap changes only the two entries di,j2d_{i,j}^2 and di,ℓ2d_{i,\ell}^2, so only the terms that read those entries change.

(a) Up to a constant in YY, KL=∑a≠bpabln⁡(1+da,b2)+ln⁡Z\mathrm{KL}=\sum_{a\ne b}p_{ab}\ln(1+d_{a,b}^2)+\ln Z with Z=∑a≠b(1+da,b2)−1Z=\sum_{a\ne b}(1+d_{a,b}^2)^{-1} (Appendix A.6). ZZ is a symmetric function of the entries of DD and is unchanged by the swap. Of the first sum, only the terms (i,j)(i,j) and (i,l)(i,l) change, and

KL⁡(D)−KL⁡(D′)=pijln⁡(1+di,j2)+pilln⁡(1+di,l2)−pijln⁡(1+di,l2)−pilln⁡(1+di,j2)=(pij−pil)[ln⁡(1+di,j2)−ln⁡(1+di,l2)].(34)\begin{aligned} \operatorname{KL}(D)-\operatorname{KL}(D') &= p_{ij}\ln(1+d_{i,j}^{2})+p_{il}\ln(1+d_{i,l}^{2})\\ &\quad-p_{ij}\ln(1+d_{i,l}^{2})-p_{il}\ln(1+d_{i,j}^{2})\\ &=(p_{ij}-p_{il})[\ln(1+d_{i,j}^{2})-\ln(1+d_{i,l}^{2})]. \tag*{(34)} \end{aligned}

The inversion di,l2<di,j2d_{i,l}^{2}<d_{i,j}^{2} makes the bracket positive.

(b) The UMAP objective is CE⁡=−∑a≠b[vabln⁡wab+(1−vab)ln⁡(1−wab)]\operatorname{CE}=-\sum_{a\ne b}[v_{ab}\ln w_{ab}+(1-v_{ab})\ln(1-w_{ab})] with wab=w(da,b2)w_{ab}=w(d_{a,b}^{2}). Write each term as −vablogit⁡(wab)−ln⁡(1−wab)-v_{ab}\operatorname{logit}(w_{ab})-\ln(1-w_{ab}). The second part depends on DD only through the multiset of the two swapped entries and cancels in the difference.

The first part gives (vij−vil)[logit⁡(wil)−logit⁡(wij)](v_{ij}-v_{il})[\operatorname{logit}(w_{il})-\operatorname{logit}(w_{ij})], and for w(x)=(1+axb)−1w(x)=(1+ax^{b})^{-1} one has logit⁡(w(x))=−ln⁡a−bln⁡x\operatorname{logit}(w(x))=-\ln a-b\ln x, so the bracket equals b[ln⁡di,j2−ln⁡di,l2]>0b[\ln d_{i,j}^{2}-\ln d_{i,l}^{2}]>0.

(c) The row swap changes only ℓi\ell_i, and in πi\pi_i it exchanges the candidates at positions r<r′r<r', where ll sits at rr and jj at r′r'. The gain changes by (rel⁡j−rel⁡l)[disc⁡(r)−disc⁡(r′)]=(l−j)[disc⁡(r)−disc⁡(r′)]≥ΔK(\operatorname{rel}_j-\operatorname{rel}_l)[\operatorname{disc}(r)-\operatorname{disc}(r')] = (l-j)[\operatorname{disc}(r)-\operatorname{disc}(r')] \ge\Delta_K, and n[Lrankhard(D)−Lrankhard(D′)]=ℓi(D)−ℓi(D′)n[\mathcal{L}^{\mathrm{hard}}_{\mathrm{rank}}(D)-\mathcal{L}^{\mathrm{hard}}_{\mathrm{rank}}(D')]=\ell_i(D)-\ell_i(D') equals that change divided by G⋆G^\star.

Here pj∣i=exp⁡(−∥xi−xj∥22/2σi2)/Zip_{j|i}=\exp(-\lVert x_i-x_j\rVert_2^2/2\sigma_i^2)/Z_i is the conditional affinity of jj to ii, with σi\sigma_i the perplexity-calibrated bandwidth of Section 3 and ZiZ_i the normalizer that makes p⋅∣ip_{\cdot|i} sum to one.

This lemma says that the affinities t-SNE and UMAP read can tie, or run against the distances, so the request in parts (a) and (b) can vanish or point the wrong way.

Lemma S2 (When the affinities do not follow the distances). (a) In t-SNE, pij<pilp_{ij}<p_{il} with di,jH<di,lHd_{i,j}^{H}<d_{i,l}^{H} if and only if

pi∣l−pi∣j>pj∣i−pl∣i>0(35)p_{i|l}-p_{i|j}>p_{j|i}-p_{l|i}>0 \tag*{(35)}

This occurs when jj sits in a region dense enough that its bandwidth σj\sigma_j makes pi∣jp_{i|j} small, while ll sits in a sparse region with a large σl\sigma_l. The symmetrization pij=(pj∣i+pi∣j)/2np_{ij}=(p_{j|i}+p_{i|j})/2n does not preserve the order of the conditionals p⋅∣ip_{\cdot|i}. (b) In UMAP, vj∣i=1v_{j|i}=1 for the nearest neighbor jj of every point ii, and the fuzzy union gives vij=1v_{ij}=1 whenever either conditional equals one. If jj is the nearest neighbor of ii and ii is the nearest neighbor of ll, then vij=vil=1v_{ij}=v_{il}=1 and the cross-entropy is indifferent to the order of jj and ll around $i.

Proof of Lemma S2. (a) pij<pilp_{ij}<p_{il} reads pj∣i+pi∣j<pl∣i+pi∣lp_{j|i}+p_{i|j}<p_{l|i}+p_{i|l}, which rearranges to the displayed inequality. Its right-hand side is positive because p⋅∣ip_{\cdot|i} is a decreasing function of di,⋅Hd^H_{i,\cdot} at the common bandwidth σi\sigma_i. The left-hand side is exp⁡(−dl,iH2/2σl2)/Zl−exp⁡(−dj,iH2/2σj2)/Zj\exp(-d^{H2}_{l,i}/2\sigma_l^2)/Z_l-\exp(-d^{H2}_{j,i}/2\sigma_j^2)/Z_j, which can be made positive by taking σj\sigma_j small relative to dj,iHd^H_{j,i} so that the second term vanishes, while σl\sigma_l is large relative to dl,iHd^H_{l,i} and ZlZ_l is moderate. Perplexity calibration sets σj\sigma_j small exactly when jj has many close neighbors. (b) UMAP sets vj∣i=exp⁡(−(di,jH−ϱi)/σi)v_{j|i}=\exp(-(d^H_{i,j}-\varrho_i)/\sigma_i) with the local-connectivity offset ϱi=di,jH\varrho_i=d^H_{i,j} for the nearest neighbor jj, so vj∣i=1v_{j|i}=1. The fuzzy union vab=vb∣a+va∣b−vb∣ava∣bv_{ab}=v_{b|a}+v_{a|b}-v_{b|a}v_{a|b} equals one whenever either conditional does. Under the stated hypothesis vj∣i=1v_{j|i}=1 and vi∣l=1v_{i|l}=1, so vij=vil=1v_{ij}=v_{il}=1, and Eq. (18) vanishes. □

Remark (where a loss stops asking for order). Suppose an objective is invariant under every permutation of the distances from ii to its k′k' returned points, and the optimizer’s output inherits that symmetry. Then the returned arrangement is exchangeable, part (ii) applies, and E[ωi]=1\mathbb{E}[\omega_i]=1. Theorem 3(a)–(b) say when the invariance holds for t-SNE and UMAP: exactly when the affinities of the returned points tie, because the row-swap difference then vanishes for every transposition and hence for every permutation. The remark describes the limit in which the two losses stop asking for order; § 4.1 measures where each method sits relative to that baseline.

Proof of Theorem 4. (i) By definition O-DLP⁡i(k)=#{j∈Ri:pos⁡i(j)=j}/k=(∣Ri∣/k)⋅Ai=DLP⁡iAi\operatorname{O\text{-}DLP}_i(k)=\#\{j\in R_i:\operatorname{pos}_i(j)=j\}/k=(|R_i|/k)\cdot A_i=\operatorname{DLP}_iA_i, since only retained neighbors can be hits. Averaging gives the first claim, and kO-DLP⁡/DLP⁡=∑iDLP⁡i(kAi)/∑iDLP⁡ik\operatorname{O\text{-}DLP}/\operatorname{DLP}=\sum_i\operatorname{DLP}_i(kA_i)/\sum_i\operatorname{DLP}_i is the stated weighted mean; Ai∈[0,1]A_i\in[0,1] gives the range.

(ii) Write k′k' for the size at which the metric is read, let q=∣Ri∣q=|R_i|, and let Xj=1[pos⁡i(j)=j]X_j=\mathbf{1}[\operatorname{pos}_i(j)=j] for j∈Rij\in R_i. Under a uniform arrangement of the k′k' returned points, each returned point is equally likely to occupy any of the k′k' positions, so EXj=1/k′\mathbb{E}X_j=1/k' and E[XjXl]=1/(k′(k′−1))\mathbb{E}[X_jX_l] = 1/(k'(k' - 1)) for j≠lj \ne l. Hence E[#hits]=q/k′\mathbb{E}[\#\text{hits}] = q/k', which is E[Ai]=1/k′\mathbb{E}[A_i] = 1/k', and

Var⁡(#hits)=qk′(1−1k′)+q(q−1)[1k′(k′−1)−1k′2]=qk′(1−1k′)+q(q−1)k′2(k′−1)≤qk′,(36)\operatorname{Var}(\#\text{hits}) = \frac{q}{k'}\left(1-\frac{1}{k'}\right) + q(q-1)\left[\frac{1}{k'(k'-1)}-\frac{1}{k'^{2}}\right] = \frac{q}{k'}\left(1-\frac{1}{k'}\right) + \frac{q(q-1)}{k'^{2}(k'-1)} \le\frac{q}{k'}, \tag*{(36)}

the last step because q−1≤k′−1q-1 \le k'-1. Since k′ O-DLPi=#hitsk'\text{ O-DLP}_i = \#\text{hits} and q/k′=DLPiq/k' = \text{DLP}_i, this is Var⁡(k′ O-DLPi)≤DLPi\operatorname{Var}(k'\text{ O-DLP}_i) \le\text{DLP}_i. With independent arrangements across points, Var⁡(O-DLP)=n−2∑iVar⁡(O-DLPi)≤n−2k′−2∑iDLPi=DLP/(nk′2)\operatorname{Var}(\text{O-DLP}) = n^{-2}\sum_i \operatorname{Var}(\text{O-DLP}_i) \le n^{-2}k'^{-2}\sum_i \text{DLP}_i = \text{DLP}/(nk'^{2}), so the standard deviation of O-DLP is at most k′−1DLP/nk'^{-1}\sqrt{\text{DLP}/n}. Dividing by DLP/k′\text{DLP}/k' gives Eq. (21).

(iii) Since k′≤min⁡(k,m)k' \le\min(k,m), every one of the k′k' points the embedding returns for ii is a candidate in CiC_i: a returned neighbor lies among the kk neighbors, and a returned non-neighbor is one of the k′≤mk' \le m nearest non-neighbors. So the first k′k' entries of πi\pi_i are exactly the returned points, in embedded order. A retained neighbor j∈Rij \in R_i is a miss when pos⁡i(j)≠j\operatorname{pos}_i(j) \ne j. If pos⁡i(j)<j\operatorname{pos}_i(j) < j, the pos⁡i(j)−1≤j−2\operatorname{pos}_i(j)-1 \le j-2 entries ahead of jj cannot hold all j−1j-1 neighbors of higher relevance, so one of them follows jj in πi\pi_i: a strict inversion involving jj. If pos⁡i(j)>j\operatorname{pos}_i(j) > j, at least jj entries precede jj, and at most j−1j-1 of them have relevance above rel⁡j\operatorname{rel}_j; the remaining one is a neighbor of lower rank or an intruder of relevance zero, and (j,⋅)(j,\cdot) is a strict inversion involving jj. Every miss therefore belongs to at least one strict inversion of πi\pi_i, and a strict inversion contains at most two retained neighbors, so #misses≤2inv⁡(πi)\#\text{misses} \le2\operatorname{inv}(\pi_i) and DLPi(k′)−O-DLPi(k′)=#misses/k′≤2inv⁡(πi)/k′\text{DLP}_i(k')-\text{O-DLP}_i(k') = \#\text{misses}/k' \le2\operatorname{inv}(\pi_i)/k'. Lemma S1 gives inv⁡(πi)≤G∗ℓi/ΔK\operatorname{inv}(\pi_i) \le G^*\ell_i/\Delta_K; averaging over ii gives Eq. (22). □

What the plane can hold. The following bound makes the crowding remark of § 4.1 quantitative. Let deg⁡H(j)\deg^H(j) be the number of points that count jj among their kk high-dimensional neighbors, the in-degree of jj in the high-dimensional kk-NN digraph.

This proposition says that in the plane a point can be among the kk nearest neighbors of at most 7k7k others, so neighbor edges into high-dimensional hubs beyond that count cannot be kept by any embedding.

Proposition S1 (Crowding bound for order). For any embedding of nn points in R2\mathbb{R}^{2} with distinct pairwise distances,

O-DLP(k)≤DLP(k)≤1nk∑j=1nmin⁡(deg⁡H(j), 7k).(37)O\text{-}DLP(k) \le DLP(k) \le\frac{1}{nk}\sum_{j=1}^{n}\min(\deg^{H}(j),\,7k). \tag*{(37)}

Proof. The first inequality is Eq. (8). For the second, write DLP(k)=1nk∑i∣Ni∩N^i∣DLP(k)=\frac{1}{nk}\sum_i |N_i \cap\hat{N}_i| with N^i\hat{N}_i the kk nearest embedded points, and count pairs by their second element: ∑i∣Ni∩N^i∣=∑j#{i:j∈Ni, j∈N^i}≤∑jmin⁡(deg⁡H(j),deg⁡low(j))\sum_i |N_i \cap\hat{N}_i|=\sum_j \#\{i:j\in N_i,\ j\in\hat{N}_i\}\le\sum_j\min(\deg^{H}(j),\deg^{\mathrm{low}}(j)), where deg⁡low(j)\deg^{\mathrm{low}}(j) is the in-degree of jj in the embedded kk-NN digraph. Stone’s lemma [44] bounds that in-degree: partition the plane around yjy_j into seven cones of angle 2π/7<π/32\pi/7<\pi/3; if two points ya,yby_a,y_b lie in the same cone with ∥ya−yj∥<∥yb−yj∥\lVert y_a-y_j\rVert<\lVert y_b-y_j\rVert, the angle at yjy_j is below π/3\pi/3 and so ∥yb−ya∥<∥yb−yj∥\lVert y_b-y_a\rVert<\lVert y_b-y_j\rVert, hence yay_a is closer to yby_b than yjy_j is; only the kk points of a cone nearest to yjy_j can therefore have yjy_j among their kk nearest, and deg⁡low(j)≤7k\deg^{\mathrm{low}}(j)\le7k. □

The bound bites only through the hubs of the high-dimensional graph [45]: every neighbor edge that points into a node with deg⁡H>7k\deg^{H}>7k beyond the first 7k7k is lost in any planar embedding, whatever the loss. Table A.22 evaluates the bound on the eight benchmarks at k=10k=10. The hub tail is thin everywhere: the bound is 1.000 on MNIST and Covertype, where no point exceeds in-degree 70, and at least 0.968 on the rest, with CIFAR-10 the most hub-heavy at 3.2% of edges. The measured DLP of the best method lies between 0.18 and 0.65 (Table 5), so the in-degree ceiling of the plane is far from what limits membership, and the order that survives is set by the loss and the layout it reaches.

DatasetnnMax in-degreePoints above 70Unkeepable edgesBound
MNIST70 0005300.00%1.000
CIFAR-1060 0002776603.24%0.968
Fashion-MNIST70 0001882150.69%0.993
Speech Commands59 387110510.10%0.999
Renal-Cortex50 027126880.26%0.997
Brain-Cortex50 212142550.16%0.998
Covertype59 9993000.00%1.000
AG News60 0001792360.79%0.992

Table A.22. The crowding bound of Proposition S1 at k=10k=10: the largest high-dimensional in-degree, the number of points with in-degree above 7k=707k=70, the share of neighbor edges that no planar embedding can keep, and the right-hand side of Eq. (A.15).

The trading scorer. The scorer of Appendix A.§ 4 computes, for each bar ii that DBSCAN leaves outside every cluster, the Spearman correlation ρi\rho_i between the ranks 1,…,k′1,\ldots,k' of ii’s k′k' nearest input-space neighbors and the ranks of those same bars by distance to yiy_i in the embedding, with k′=10k'=10. The order half of the marker score is 1−ρi1-\rho_i.

This proposition says that the order half of the marker score is bounded by the same per-bar rank loss that SR-tSNE minimizes.

Proposition S2 (The scorer reads a controlled quantity). Let k′≤kk' \le k and let ℓi\ell_i be the hard per-point rank loss of § 3.6. Then, for every bar ii,

1−ρi≤12k′(k′+1)⋅G⋆ΔKℓi.(38)1-\rho_i \le\frac{12}{k'(k'+1)} \cdot\frac{G^\star}{\Delta_K}\ell_i. \tag*{(38)}

Proof. Write dj=∣j−rj∣d_j=|j-r_j| for the displacement of the jj-th input-space neighbor, rjr_j being its rank among the k′k' neighbors by embedded distance, and let inv⁡i(k′)\operatorname{inv}_i(k') be the number of pairs among the k′k' neighbors whose order the embedding inverts. Spearman’s coefficient on two permutations of {1,…,k′}\{1,\ldots,k'\} is ρi=1−6∑jdj2/(k′(k′2−1))\rho_i=1-6\sum_j d_j^2/(k'(k'^2-1)). The Diaconis–Graham inequality [46] gives ∑jdj≤2inv⁡i(k′)\sum_j d_j\le2\operatorname{inv}_i(k'), and each dj≤k′−1d_j\le k'-1, so ∑jdj2≤(k′−1)∑jdj≤2(k′−1)inv⁡i(k′)\sum_jd_j^2\le(k'-1)\sum_jd_j\le2(k'-1)\operatorname{inv}_i(k'). Hence 1−ρi≤12inv⁡i(k′)/(k′(k′+1))1-\rho_i\le12\operatorname{inv}_i(k')/(k'(k'+1)). Every pair of the first k′k' neighbors that the embedding inverts is a strict inversion of πi\pi_i, because the two carry distinct relevances k−j+1k-j+1 and the order among candidates is the order by embedded distance; so inv⁡i(k′)≤inv⁡(πi)\operatorname{inv}_i(k')\le\operatorname{inv}(\pi_i). Lemma S1 gives inv⁡(πi)≤(G⋆−Gihard)/ΔK=G⋆ℓi/ΔK\operatorname{inv}(\pi_i)\le(G^\star-G_i^{\mathrm{hard}})/\Delta_K=G^\star\ell_i/\Delta_K. □

The constant is uniform: it is fixed by kk, mm and k′k', and it does not grow with nn. At the setting used throughout this paper, k=m=k′=10k=m=k'=10, it is 8.8×1028.8\times10^2. The proposition identifies which objective drives the quantity down: under SR-tSNE what remains of 1−ρi1-\rho_i after training is the order the plane could not restore, whereas under t-SNE it also carries the order the objective never asked for. The min–max normalization of Appendix A.§ 4 then ranks bars by that residual.

This corollary says that a finite weight on the hard rank term makes the whole loss prefer the correct order of every pair.

Corollary S1 (A finite weight suffices). Let the embedding have squared diameter at most R2R^2. The total loss KL+λLrankhard\mathrm{KL}+\lambda\mathcal{L}^{\mathrm{hard}}_{\mathrm{rank}}, read on the distance arrays of Theorem 3, strictly prefers the correct order for every inverted pair of every point whenever

λ>λ⋆=nG⋆ln⁡(1+R2)ΔKmax⁡i,j<l(pil−pij)+.(39)\lambda>\lambda^\star=\frac{nG^\star\ln(1+R^2)}{\Delta_K}\max_{i,j<l}(p_{il}-p_{ij})^+. \tag*{(39)}

Since pab≤1/np_{ab}\le1/n for every pair, λ⋆≤G⋆ln⁡(1+R2)/ΔK\lambda^\star\le G^\star\ln(1+R^2)/\Delta_K, a bound that does not grow with nn.

Proof of Corollary S1. For an inverted pair, Theorem 3(a) bounds the KL preference from below by −(pil−pij)+ln⁡(1+R2)-(p_{il}-p_{ij})^{+}\ln(1+R^{2}), since 0<ln⁡(1+di,j2)−ln⁡(1+di,l2)≤ln⁡(1+R2)0<\ln(1+d_{i,j}^{2})-\ln(1+d_{i,l}^{2})\le\ln(1+R^{2}). Theorem 3(c) bounds the rank preference from below by λΔK/(nG⋆)\lambda\Delta_{K}/(nG^{\star}). The sum is positive when λ\lambda exceeds λ⋆\lambda^{\star}. Finally pab=(pb∣a+pa∣b)/2n≤1/np_{ab}=(p_{b|a}+p_{a|b})/2n\le1/n because each conditional distribution sums to one. The statement is about the hard rank loss; the soft loss of Eqs. (2)–(4) converges to it as τ→0\tau\to0 (Theorem 2), so for any pair with a fixed positive distance gap the soft preference converges to the hard one. □

References

  1. [1]L. Van der Maaten, G. Hinton, Visualizing data using t-sne., Journal of machine learning research 9 (11) (2008).
  2. [2]L. McInnes, J. Healy, J. Melville, Umap: Uniform manifold approximation and projection for dimension reduction, arXiv preprint arXiv:1802.03426 (2018).arxiv.org/abs/1802.03426
  3. [3]K. R. Moon, D. Van Dijk, Z. Wang, G. Sigante, D. B. Burkhardt, W. S. Chen, K. Yim, A. v. d. Elzen, M. J. Hirn, R. R. Coifman, N. B. Ivanova, G. Wolf, S. Krishnaswamy, Visualizing structure and transitions in high-dimensional biological data, Nature biotechnology 37 (12) (2019) 1482–1492.DOI
  4. [4]E. Amid, M. K. Warmuth, TriMap: Large-scale Dimensionality Reduction Using Triplets, arXiv preprint arXiv:190.00204 (2019).arxiv.org/abs/1910.00204
  5. [5]H. Han, W. Li, J. Wang, G. Qin, X. Qin, Enhance explainability of manifold learning, Neurocomputing 500 (2022) 877–895. doi:https://doi.org/10.1016/j.neucom.2022.05.119.
  6. [6]M. Belkin, P. Niyogi, Laplacian eigenmaps for dimensionality reduction and data representation, Neural Computation 15 (6) (2003) 1373–1396. doi:10.1162/089976603321780317.
  7. [7]Y. Wang, Y. Sun, H. Huang, C. Rudin, Dimension reduction with locally adjusted graphs, Proceedings of the AAAI Conference on Artificial Intelligence 39 (20) (2025) 2157–2165.arxiv.org/abs/2412.15426
  8. [8]N. Kury, D. Kobak, S. Damrich, DREAMS: Preserving both local and global structure in dimensionality reduction, Transactions on Machine Learning Research (2026).arxiv.org/abs/2508.13747
  9. [9]D. Olszewski, Data vulnerability index for the “crowding problem” in nonlinear dimensionality reduction, Neurocomputing 648 (2025) 130619. doi:https://doi.org/10.1016/j.neucom.2025.130619.
  10. [10]H. Jeon, J. Park, S. Shin, J. Seo, Stop misusing t-SNE and UMAP for visual analytics, IEEE Transactions on Visualization and Computer Graphics (2026).
  11. [11]J. Venna, S. Kaski, Local multidimensional scaling, Neural Networks 19 (6) (2006) 889–899. doi:https://doi.org/10.1016/j.neunet.2006.05.014.
  12. [12]J. Venna, J. Peltonen, K. Nybo, H. Aidos, S. Kaski, Information retrieval perspective to nonlinear dimensionality reduction for data visualization, Journal of Machine Learning Research 11 (2010) 451–490.
  13. [13]S. T. Roweis, L. K. Saul, Nonlinear dimensionality reduction by locally linear embedding, Science 290 (5500) (2000) 2323–2326. doi:10.1126/science.290.5500.2323. URL https://science.org/doi/abs/10.1126/science.290.5500.2323DOI
  14. [14]Y. Wang, H. Huang, C. Rudin, Y. Shaposhnik, Understanding how dimension reduction tools work: an empirical approach to deciphering t-sne, umap, trimap, and pacmap for data visualization, Journal of Machine Learning Research 22 (201) (2021) 1–73.
  15. [15]P. Lambert, E. Couplet, M. Verleysen, J. A. Lee, FUnc-SNE: A flexible, fast, and unconstrained algorithm for neighbour embeddings, arXiv preprint arXiv:2509.07681 (2025).arxiv.org/abs/2509.07681
  16. [16]F. Contreras, R. Peletier, K. Bunte, Adaptive locally aligned ant technique and manifold blurring mean shift for manifold detection and denoising, Neurocomputing 679 (2026) 133199. doi:https://doi.org/10.1016/j.neucom.2026.133199.
  17. [17]L. Van der Maaten, K. Weinberger, Stochastic triplet embedding, in: 2012 IEEE International Workshop on Machine Learning for Signal Processing, IEEE, 2012, pp. 1–6.DOI
  18. [18]S. Lespinats, B. Fertil, P. Villemain, J. Hérault, RankVis: Mapping from the neighborhood network, Neurocomputing 72 (13-15) (2009) 2964–2978.DOI
  19. [19]Y. Terada, U. von Luxburg, Local ordinal embedding, in: International Conference on Machine Learning, 2014, pp. 847–855.
  20. [20]J. A. Lee, M. Verleysen, Quality assessment of dimensionality reduction: Rank-based criteria, Neurocomputing 72 (7-9) (2009) 1431–1443.DOI
  21. [21]J. A. Lee, D. H. Peluffo-Ordóñez, M. Verleysen, Multi-scale similarities in stochastic neighbour embedding: Reducing dimensionality while preserving both local and global structure, Neurocomputing 169 (2015) 246–261.DOI
  22. [22]C. de Bodt, A. Diaz-Papkovich, M. Bleher, K. Bunte, C. Coupette, S. Damrich, E. Fita Sanmartin, F. A. Hamprecht, E.-Á. Horvát, D. Kohli, S. Krishnaswamy, J. A. Lee, B. P. F. Lelieveldt, L. McInnes, I. T. Nabney, M. Noichl, P. G. Poličar, B. Rieck, G. Wolf, G. Mishne, D. Kobak, Low-dimensional embeddings of high-dimensional data, arXiv preprint arXiv:2508.15929 (2025).arxiv.org/abs/2508.15929
  23. [23]K. Q. Weinberger, K. Saul, Distance metric learning for large margin nearest neighbor classification, Journal of Machine Learning Research 10 (2009) 207–244.
  24. [24]T. Qin, T.-Y. Liu, H. Li, A general approximation framework for direct optimization of information retrieval measures, Information Retrieval 13 (4) (2010) 375–397.DOI
  25. [25]L. Van Der Maaten, Accelerating t-SNE using tree-based algorithms, J. Mach. Learn. Res. 15 (1) (2014) 3221–3245.
  26. [26]J. B. Tenenbaum, V. de Silva, J. C. Langford, A global geometric framework for nonlinear dimensionality reduction, Science 290 (5500) (2000) 2319–2323.pubmed.ncbi.nlm.nih.gov/11125149
  27. [27]H. Huang, Y. Wang, C. Rudin, Navigating the effect of parametrization for dimensionality reduction, in: Advances in Neural Information Processing Systems, Vol. 37, 2024, pp. 12977–13019.arxiv.org/abs/2411.15894
  28. [28]D. Peng, Z. Gui, W. Wei, et al., Sampling-enabled scalable manifold learning unveils the discriminative cluster structure of high-dimensional data, Nature Machine Intelligence 7 (2025) 1669–1684.arxiv.org/abs/2401.01100
  29. [29]L. Sikkema, C. Ramírez-Suástegui, D. C. Strobl, et al., An integrated cell atlas of the lung in health and disease, Nature Medicine 29 (6) (2023) 1563–1577.
  30. [30]H. Han, J. Y.-L. Forrest, J. Wang, S. Yuan, F. Han, D. Li, Explainable machine learning for high frequency trading dynamics discovery, Information Sciences 684 (2024) 121286. doi:10.1016/j.ins.2024.121286.DOI
  31. [31]Y. Le, X. Yang, Tiny imagenet visual recognition challenge, CS 231N, Stanford University 7 (7) (2015) 3.
  32. [32]G. Cohen, S. Afshar, J. Tapson, A. van Schaik, EMNIST: extending mnist to handwritten letters, in: International Joint Conference on Neural Networks, 2017, pp. 2921–2926.DOI
  33. [33]A. Krizhevsky, Learning multiple layers of features from tiny images, Tech. rep., University of Toronto (2009).
  34. [34]J. J. Hull, A database for handwritten text recognition research, IEEE Transactions on Pattern Analysis and Machine Intelligence 16 (5) (1994) 550–554.DOI
  35. [35]Y. LeCun, L. Bottou, Y. Bengio, P. Haffner, Gradient-based learning applied to document recognition, Proceedings of the IEEE 86 (11) (1998) 2278–2324.
  36. [36]H. Xiao, K. Rasul, R. Vollgraf, Fashion-mnist: a novel image dataset for benchmarking machine learning algorithms, arXiv preprint arXiv:1708.07747 (2017).arxiv.org/abs/1708.07747
  37. [37]P. Warden, Speech commands: a dataset for limited-vocabulary speech recognition, arXiv preprint arXiv:1804.03209 (2018).arxiv.org/abs/1804.03209
  38. [38]A. Baevski, Y. Zhou, A. Mohamed, M. Auli, wav2vec 2.0: a framework for self-supervised learning of speech representations, in: Advances in Neural Information Processing Systems, Vol. 33, 2020, pp. 12449–12460.arxiv.org/abs/2006.11477
  39. [39]J. A. Blackard, D. J. Dean, Comparative accuracies of artificial neural networks and discriminant analysis in predicting forest cover types from cartographic variables, Computers and Electronics in Agriculture 24 (3) (1999) 131–151.DOI
  40. [40]X. Zhang, J. Zhao, Y. LeCun, Character-level convolutional networks for text classification, in: Advances in Neural Information Processing Systems, Vol. 28, 2015, pp. 649–657.arxiv.org/abs/1509.01626
  41. [41]J. Devlin, M.-W. Chang, K. Lee, K. Toutanova, BERT: pre-training of deep bidirectional transformers for language understanding, in: Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics, 2019, pp. 4171–4186.arxiv.org/abs/1810.04805
  42. [42]M. Ester, H.-P. Kriegel, J. Sander, X. Xu, A density-based algorithm for discovering clusters in large spatial databases with noise, in: Proceedings of the Second International Conference on Knowledge Discovery and Data Mining (KDD), 1996, pp. 226–231.
  43. [43]C. Spearman, The proof and measurement of association between two things, The American Journal of Psychology 15 (1) (1904) 72–101. doi:10.2307/1412159.DOI
  44. [44]C. J. Stone, Consistent nonparametric regression, The Annals of Statistics 5 (4) (1977) 595–620.
  45. [45]M. Radovanović, A. Nanopoulos, M. Ivanović, Hubs in space: Popular nearest neighbors in high-dimensional data, Journal of Machine Learning Research 11 (2010) 2487–2531.
  46. [46]P. Diaconis, R. L. Graham, Spearman’s footrule as a measure of disarray, Journal of the Royal Statistical Society: Series B 39 (2) (1977) 262–268.

Paper details

vlnc:W_bp49h64h
Version
v1 · latest
Published
Fresh — added to the Hub within the last 30 days
Added
License
CC BY 4.0
Usage

views · 1

downloads · 0

A view counts once per browser per day, a download each time the PDF is requested.

  • Computer Science

AI

View statement
Looking for related papers…
Contents