WorksComputer Science
Vendi Anomaly Scores for Efficient and Accurate Anomaly Detection
Abstract
Anomaly detection (AD) is a critical for trustworthy machine learning and scientific discovery. Existing methods typically define anomalies through local density, isolation heuristics, or distance from a fitted distribution, making them sensitive to neighborhood scale, partitioning choices, or restrictive distributional assumptions. In this work, we introduce a new paradigm by formulating anomaly detection from a diversity perspective. We propose the (VAS), which detects anomalies by quantifying how much a sample changes the diversity of the dataset, as measured by the Vendi score, when removed. VAS captures both local redundancy and global data structure without relying on density estimation or partitioning heuristics. Moreover, VAS requires no hyperparameter tuning, instead adapting automatically to the spectral structure of the data. VAS is non-parametric and scales linearly with dataset size. Across 148 benchmark embedding datasets derived from 10 source datasets and 4 embedding architectures, VAS achieves state-of-the-art AD performance. We further validate VAS on large-scale ImageNet experiments, where it remains robust as both contamination rate and dataset scale increase.
Introduction
Anomaly detection (AD) is essential for analyzing and curating machine learning datasets and for discovery [10, 8]. In particular, by identifying samples that are corrupted, mislabeled, out-of-distribution, or otherwise irrelevant, AD helps maintain data quality and improve model reliability [1, 40]. Since modern datasets are increasingly inspected through learned embeddings that encode semantic relationships, effective data curation requires AD methods that operate directly in these embedding spaces [45, 50, 9].
AD algorithms are commonly categorized by the supervision available during training. Supervised methods fit detectors using labeled mixtures of normal and anomalous samples [37, 57], while semi-supervised methods assume access to clean data and identify deviations from this reference set [11]. Unsupervised AD fits directly to the dataset being analyzed, making it especially attractive for modern data curation. As datasets increasingly span diverse modalities, it is often impractical to collect data on anomalies or to construct clean reference sets for every domain, making unsupervised detectors appealing.
Unsupervised AD methods typically define anomalies through isolation [32], local density [7], or distance from a fitted distribution [51]. Isolation-based methods score points by how easily they can be separated from the rest of the data, with classical examples including isolation forests (iForest) [32], extended isolation forests (EIF) [24], and robust random cut forests (RRCF) [22], as well as geometric variants such as the hypersphere ensembles of INNE [3] and ADERH [17]. Other approaches use local neighborhood structure, including -nearest neighbor scoring [5] and local outlier factor (LOF) [7], or fitted feature-space distributions [51]. These methods are often effective, but their performance depends on certain implicit assumptions: isolation methods seek points that are easy to separate based on some heuristic, local methods depend on a chosen neighborhood scale, and distributional methods rely on the adequacy of the fitted density model. On the other hand, deep unsupervised detectors can learn more flexible notions of abnormality [Xu et al., 2023a,b, Goodge et al., 2022], but they require training a model for each dataset, which adds computational cost and other deployment-related challenges. How can we develop AD methods that retain the efficiency of classical detectors while making fewer assumptions about the geometry of embedding data?
Contribution. In this paper, we answer this question and make several contributions and findings that we summarize below.
We introduce a new paradigm for AD based on diversity. We propose the Vendi Anomaly Score (VAS), which builds on the Vendi Scores (VS), a family of similarity-based diversity metrics [20, 46], and defines anomaly through changes in the eigenvalue spectrum of the embedding similarity matrix (Figure 1).
We make VAS practical and easy to deploy. We derive an efficient approximation to the exact leave-one-out score, allowing VAS to scale to large embedding datasets. With a simple heuristic, our method also automatically adjusts to different data geometries, eliminating the need for neighborhood selection, fitting clusters, or estimating parametric densities.
We demonstrate the effectiveness of VAS across diverse embedding datasets and large-scale contamination experiments. On ADBENCH image and text benchmarks [23], spanning 148 total datasets, VAS outperforms all other baselines. We also evaluate VAS on controlled ImageNet experiments with up to 100k samples [13], showing that VAS is especially effective for detecting contaminating subpopulations at scale.
Figure 1. Overview of VAS. Left: A multimodal dataset contaminated with an anomaly. Right: Removing a rare anomalous point induces a larger change in the eigenvalue spectrum than removing a normal sample. VAS detects these shifts by estimating the change in Vendi Score, derived from the Rényi entropy of order of the spectrum (Eq. (1)).
Related Work
Our work relates to three lines of research, which we describe below.
Unsupervised anomaly detection. Existing unsupervised AD methods typically identify anomalies based on criteria such as separability, local density, or reconstruction error.
geometric variants such as the hypersphere ensembles of INNE [3] and ADERH [17]. Other approaches use local neighborhood structure, including -nearest neighbor scoring [5] and local outlier factor (LOF) [7], or fitted feature-space distributions [51]. These methods are often effective, but their performance depends on certain implicit assumptions: isolation methods seek points that are easy to separate based on some heuristic, local methods depend on a chosen neighborhood scale, and distributional methods rely on the adequacy of the fitted density model. On the other hand, deep unsupervised detectors can learn more flexible notions of abnormality [Xu et al., 2023a,b, Goodge et al., 2022], but they require training a model for each dataset, which adds computational cost and other deployment-related challenges. How can we develop AD methods that retain the efficiency of classical detectors while making fewer assumptions about the geometry of embedding data?
Partitioning-based methods isolate anomalies by recursively subdividing the feature space. Isolation Forest (iForest) [32] and its variants, such as Extended Isolation Forest [24], identify anomalies as points that can be separated with fewer splits. While effective in low-dimensional or tabular settings, these methods rely on partitions of the feature space, which can struggle to capture complex geometry in high-dimensional embedding spaces.
A second class of methods estimates local data density. Techniques such as Local Outlier Factor (LOF) [7] and deep nearest neighbors (DN2) [5] identify anomalies as points that lie in low-density regions relative to their neighbors. However, these approaches require constructing neighborhoods of fixed size, making them sensitive to scale and limiting their ability to adapt to heterogeneous or multimodal data distributions.
Ensemble-based methods combine multiple weak signals to improve robustness. For example, Lightweight on-line Detector of Anomalies (LODA) [49] aggregates anomaly scores from random low-dimensional projections, while ADERH leverages ensembles of random hyperspheres to estimate density and support [17]. ADERH provides the strongest results to date for AD on tabular data, although it requires the strong assumption that normal data can be bounded within hyperspheres, which may not be true for embedding data.
Recent work has also turned to deep learning to improve AD in complex data. Many of these methods aim to construct representation spaces where anomalies are more easily separated. For example, Deep Isolation Forest [60] learns a non-linear transformation of the data that IForest can leverage. LUNAR [21] unifies local outlier methods such as LOF through a graph neural network message-passing framework, introducing learnability into neighborhood-based AD. Similarly, DeepSVDD [52] learns to project normal data within a hypersphere and anomalous data outside of it. RCA [31] and RDP [59] use collaboration between multiple models to generate a new space where anomalies are more separable. SLAD [61] and DTE [36] both measure how far a sample deviates from normal data. SLAD learns to predict self-supervised “scale” relationships within each sample and identifies anomalies via inconsistencies in these predictions, while DTE leverages diffusion models to estimate deviation from the normal data manifold.
Out-of-distribution detection. Out-of-distribution (OOD) detection is closely related to semi-supervised AD: both settings assume access to in-distribution data and aim to identify samples that deviate from it. In OOD detection, however, the goal is typically to determine whether a test input falls outside the training distribution of a particular predictive model. As a result, many OOD methods are designed around signals produced by that model, including prediction confidence [27], energy-based scores [33, 34], feature-space statistics such as Mahalanobis distance [30, 39], or other properties of the learned representation [19, 5, 58]. Standardized benchmarks such as OpenOOD [62] evaluate these methods in model-centric settings where the detector is built around a classifier trained on a specified in-distribution dataset.
In contrast, we study unsupervised AD. Here, the goal is to identify anomalous samples within the dataset being analyzed, using embeddings produced by a model trained independently on that dataset. We do not assume labels, a clean reference set, prediction logits, or model-specific confidence scores. Our method instead operates directly on the similarity structure of the embedding dataset.
Measuring rarity with the Vendi Scores. Friedman and Dieng [2022] introduced the Vendi Score (VS) as a similarity-based diversity measure. It has since been used to evaluate diversity in a wide variety of domains [54, 41, 12, 44, 4 Tom et al.] (Zammzi et al. 2025). Recent methods have explored how the Vendi Scores can be used to identify rare or novel samples in various settings. In active search, Nguyen and Dieng [2024] measure incremental changes to quality-weighted extensions of the VS, showing that changes in the VS can be used to guide the selection of diverse and informative samples in experimental design settings. Similarly, Askari Hemmat et al. [2024] leverage incremental changes to the VS to guide image generation toward underrepresented regions of the output space. The VS has also been used for OOD detection: the Vendi Novelty Scores (VNS) detect OOD samples by measuring how much a test sample increases the diversity of a reference dataset by class-conditional novelty estimates derived from a trained classifier [47]. VNS relies on incremental changes to diversity, but it is designed for inference-time OOD detection relative to a fixed training distribution and leverages model-dependent signals.
Method
We review the Vendi Scores (VS), and introduce our method for leveraging them in unsupervised AD.
The Vendi Scores. Let denote a dataset of samples. Given a positive semidefinite kernel , we construct the similarity matrix with entries . The VS [20] measures the diversity of via the entropy of the eigenvalues of , normalized such that .
This definition can be generalized using the Rényi entropy [46]. Let denote the th eigenvalue of and be the order of the VS. The order- is defined as
This simplifies to the exponential Shannon Entropy when . The parameter controls sensitivity to different parts of the spectrum: smaller values emphasize the tail, while larger values emphasize the leading eigenmodes [46].
Computing naively requires the eigendecomposition of the matrix , which requires complexity. However, as Friedman and Dieng [ [20] note, when working with -normalized features and using a cosine kernel, the VS can be computed more efficiently. In this setting, the similarity matrix is and has the same non-zero eigenvalues of the smaller matrix . We therefore can compute the spectrum using , reducing the computational cost to , which is considerably more efficient in the common ML setting where the number of samples greatly exceeds the feature dimension ().
Fast estimation of diversity influence for anomaly detection. We consider a dataset of samples represented by -dimensional embeddings , obtained from a pretrained model. The goal of unsupervised AD is to identify which samples are anomalous relative to the rest of the dataset. We aim to learn a scoring function that assigns a real-valued anomaly score to each sample based on its embedding. Higher scores correspond to more anomalous points. Only the induced ordering matters: any transformation of that preserves the ordering yields an equivalent solution.
We define anomaly scores based on the change in dataset diversity when a sample is removed. Assume normalized features and let denote the feature covariance matrix corresponding to the cosine kernel. For a sample , let denote the dataset with removed, and define . We then define the anomaly score as,
Intuitively, samples that contribute uniquely to the structure of the dataset will induce a larger decrease in diversity when removed, yielding larger values of , as shown in Figure 1.
However, computing naively for all samples requires time, which is prohibitive for high-dimensional datasets. Moreover, this formulation requires recomputing the spectrum of after removing each sample, preventing efficient vectorization and limiting the ability to leverage batched GPU computation. We can accelerate this computation by noting that each removal corresponds to a rank-one perturbation of of size . Proposition 3.1 shows that the resulting change in Vendi Score can be approximated by a single quadratic form involving .
Proposition 3.1 (First-order approximation in log-space). Assume that is twice continuously differentiable in a neighborhood of . Let
Then, for each ,
A proof is provided in Appendix Section A. The differentiability assumption excludes spectral boundary cases in which zero eigenvalues make terms such as or singular. In implementation, all spectral quantities are computed on the positive eigenspace of , equivalently restricting the calculation to the span of the data.
Because in Proposition 3.1 is constant across samples in a dataset, and we only care about ranking samples from our scoring function, we only need to compute the quantity for each sample . This yields a simple batched computation of the VAS:
We empirically demonstrate the accuracy of the approximation used in VAS in § 4.1.
VAS can also be interpreted as a spectral quadratic anomaly score, connecting it to classical methods such as Mahalanobis distance and PCA reconstruction error. While these methods use predefined rules for weighting different directions of variation, VAS derives its weighting from each sample’s influence on dataset diversity, with the relative importance of different directions controlled by the Vendi Score order . In Appendix Section B, we provide additional description of how VAS relates to other spectral scores.
Automatically adapting . VAS introduces a single hyperparameter: the order of the Vendi Score. Previous methods leveraging the Vendi Scores often fix a single value of across experiments [63, 47] or track the VS across a range of values to draw diversity profiles [15, 28]. However, in unsupervised AD, the optimal choice of can vary significantly depending on the geometry of the embedding space, and we are not afforded a validation set to tune hyperparameters. Therefore, we design a simple, data-dependent heuristic that adapts automatically across datasets.
The parameter controls how strongly the Vendi Score emphasizes the tail of the eigenvalue spectrum. The importance of this tail varies across datasets. In embedding spaces where the data is highly concentrated, such as BERT embeddings of text, most variance is captured by the leading eigenmode, resulting in a large principal eigenvalue of the trace-normalized similarity matrix. In this regime, AD requires amplifying the contribution of the spectral tail, which is achieved by using smaller values of . In contrast, in more isotropic embedding spaces, such as image representations spanning diverse visual features, the variance is more evenly distributed, leading to a smaller . In these settings, less emphasis on the tail is needed, and larger values of are appropriate.
To capture this behavior without manual tuning, we define as a function of the principal eigenvalue:
The lower bound of is introduced for numerical stability when evaluating for small . This heuristic assigns smaller values of to concentrated datasets and larger values to more dispersed ones, adapting automatically to the spectral structure of the data.
Interpreting VAS. Proposition 3.1 shows that VAS ranks samples by the quadratic form . Therefore, to understand what kinds of subpopulations VAS identifies as anomalous, it suffices to study the expected score for samples drawn from a component . Proposition 3.2 analyzes this quantity under a stylized orthogonal mixture model.
Proposition 3.2 (Expected VAS under orthogonal mixtures). Let be a mixture distribution on , and define . Assume each component is supported on a subspace , and that the subspaces are mutually orthogonal. Let . Using as defined in Proposition 3.1, for any and ,
Consequently, the expected VAS score of a sample from component is
The proof is provided in Appendix C. Proposition 3.2 shows that the expected VAS score for samples from component depends on the ratio . Here, is the component mass, while
| Dataset | Encoder | VAS | ADERH | DIF | IForest | LUNAR | DN2 | LODA | LOF | KPCA | SLAD | GMD | DTE |
| CIFAR10 | ResNet18 | 0.745 | 0.633 | 0.684 | 0.627 | 0.657 | 0.740 | 0.635 | 0.696 | 0.751 | 0.735 | 0.732 | 0.673 |
| CIFAR10 | ViT | 0.941 | 0.924 | 0.860 | 0.876 | 0.837 | 0.855 | 0.856 | 0.519 | 0.905 | 0.942 | 0.898 | 0.590 |
| FashionMNIST | ResNet18 | 0.911 | 0.865 | 0.868 | 0.831 | 0.852 | 0.858 | 0.829 | 0.605 | 0.915 | 0.904 | 0.912 | 0.848 |
| FashionMNIST | ViT | 0.930 | 0.910 | 0.856 | 0.890 | 0.815 | 0.860 | 0.862 | 0.519 | 0.917 | 0.919 | 0.925 | 0.657 |
| MNIST-C | ResNet18 | 0.793 | 0.763 | 0.744 | 0.734 | 0.780 | 0.763 | 0.724 | 0.628 | 0.765 | 0.782 | 0.793 | 0.759 |
| MNIST-C | ViT | 0.739 | 0.723 | 0.704 | 0.720 | 0.695 | 0.704 | 0.689 | 0.555 | 0.719 | 0.732 | 0.723 | 0.568 |
| MVTec-AD | ResNet18 | 0.799 | 0.755 | 0.750 | 0.745 | 0.768 | 0.797 | 0.704 | 0.763 | 0.741 | 0.788 | 0.790 | 0.705 |
| MVTec-AD | ViT | 0.896 | 0.809 | 0.795 | 0.808 | 0.858 | 0.875 | 0.781 | 0.842 | 0.847 | 0.851 | 0.891 | 0.697 |
| SVHN | ResNet18 | 0.657 | 0.579 | 0.615 | 0.576 | 0.609 | 0.662 | 0.589 | 0.638 | 0.657 | 0.655 | 0.659 | 0.592 |
| SVHN | ViT | 0.579 | 0.519 | 0.524 | 0.530 | 0.564 | 0.584 | 0.524 | 0.577 | 0.565 | 0.553 | 0.580 | 0.529 |
| 20News | BERT | 0.626 | 0.543 | 0.557 | 0.555 | 0.571 | 0.565 | 0.554 | 0.607 | 0.554 | 0.606 | 0.603 | 0.560 |
| 20News | RoBERTa | 0.749 | 0.636 | 0.624 | 0.643 | 0.698 | 0.732 | 0.607 | 0.723 | 0.720 | 0.703 | 0.716 | 0.503 |
| Agnews | BERT | 0.676 | 0.583 | 0.582 | 0.583 | 0.650 | 0.610 | 0.558 | 0.725 | 0.604 | 0.656 | 0.650 | 0.587 |
| Agnews | RoBERTa | 0.881 | 0.718 | 0.661 | 0.677 | 0.790 | 0.850 | 0.638 | 0.719 | 0.846 | 0.797 | 0.813 | 0.501 |
| Amazon | BERT | 0.608 | 0.577 | 0.565 | 0.558 | 0.608 | 0.595 | 0.531 | 0.578 | 0.585 | 0.606 | 0.604 | 0.522 |
| Amazon | RoBERTa | 0.544 | 0.485 | 0.518 | 0.509 | 0.536 | 0.551 | 0.520 | 0.548 | 0.551 | 0.532 | 0.530 | 0.548 |
| Imdb | BERT | 0.502 | 0.486 | 0.490 | 0.488 | 0.493 | 0.517 | 0.447 | 0.505 | 0.517 | 0.513 | 0.515 | 0.483 |
| Imdb | RoBERTa | 0.530 | 0.439 | 0.508 | 0.485 | 0.518 | 0.529 | 0.506 | 0.532 | 0.530 | 0.510 | 0.504 | 0.500 |
| Yelp | BERT | 0.679 | 0.623 | 0.617 | 0.603 | 0.664 | 0.672 | 0.573 | 0.667 | 0.647 | 0.659 | 0.676 | 0.467 |
| Yelp | RoBERTa | 0.566 | 0.511 | 0.537 | 0.531 | 0.578 | 0.594 | 0.548 | 0.581 | 0.577 | 0.568 | 0.575 | 0.532 |
| Average Rank | 2.35 | 8.70 | 8.55 | 9.20 | 6.10 | 4.05 | 10.15 | 6.25 | 4.70 | 4.35 | 3.80 | 9.80 | |
| p-Value | - | 1.34e-20 | 1.15e-24 | 6.27e-24 | 3.78e-22 | 3.14e-10 | 2.08e-25 | 5.23e-15 | 4.11e-15 | 4.49e-16 | 1.34e-08 | 2.07e-24 |
caption: AUROC across dataset families and encoders. For multi-split datasets, we report average AUROC. Orange marks best, blue marks 2nd. Bottom-row p-values are paired Wilcoxon tests: VAS vs. each baseline over all 148 dataset-split/encoder instances.
:::
is the component’s effective spectral dimension. Thus, VAS is sensitive to how much probability mass a component has relative to its internal diversity. Components that are small but spectrally diffuse can receive high anomaly scores, even if their samples are not isolated from one another. This component-level sensitivity emerges without explicitly identifying components or forming local neighborhoods; it arises implicitly from the spectrum of the full dataset.
Time and space complexity. The computational cost of VAS is dominated by forming , eigen-decomposing the resulting matrix, and computing the VAS scores from (2) for all samples. Forming costs time, the eigendecomposition costs time, and the batched score computation costs time. Thus, the total runtime is , with memory complexity . In contrast, naively recomputing the Vendi Score after each sample removal would require separate eigendecompositions, yielding time complexity. VAS therefore avoids per-sample spectral computations and reduces scoring to a single global eigendecomposition followed by batched matrix operations.
Experiments
In this section, we compare VAS against a diverse set of AD algorithms. We benchmark all algorithms on various image and text embedding datasets across multiple architectures.
Datasets and models. We evaluate VAS on the embedding-based image and text datasets from AD-BENCH [23]. These datasets span numerous image and text datasets and architectures. For all image datasets, we test methods against embeddings from ResNet18 [25] and ViT [16] architectures that had been pre-trained on ImageNet [13]. For text data, we test performance on embeddings from BERT [14] and RoBERTa [35]. The benchmark covers a wide range of AD scenarios. These include rare class detection (e.g., CIFAR-10 [29], SVHN [42]), where one class is treated as normal and samples from the remaining classes are downsampled to form anomalies; distributional corruption (e.g., MNIST vs. MNIST-C [38]), where anomalies arise from shifts in data generation; and semantic or structural anomalies (e.g., MVTec-AD [6]). In text datasets (e.g., Amazon [26], Yelp [65]), anomalies are constructed via shifts in semantic content, such as contrasting sentiment. Additional specifications for each dataset are provided in Appendix Section D.
We additionally perform AD experiments on ImageNet-1K [13], using embeddings extracted from the DinoV3 ViT-B/16 model [53].
Baselines. We compare against 11 unsupervised AD algorithms: ADERH [17], DIF [60], IForest [32], DN2 [5], LODA [49], LOF [7], KPCA [19], SLAD [61], DTE [36], and LUNAR [21]. We also include a custom Gaussian-mixture Mahalanobis distance baseline (GMD) to represent fitted-distribution methods. Inspired by Gaussian feature modeling for AD [51], GMD fits a Bayesian Information Criterion-selected Gaussian mixture model to the unlabeled embeddings and scores each sample by its Mahalanobis distance to its most likely component. Together, these baselines span feature-space partitions (IForest), local neighborhoods (LOF, DN2), randomized ensemble methods based on projections or hyperspheres (LODA, ADERH), deep anomaly detectors (SLAD, DIF, DTE, LUNAR), kernel reconstruction methods (KPCA), and fitted feature-space distributions (GMD). Additional descriptions of each algorithm and their relationship to VAS are provided in Appendix Section E.
Hyperparameters and preprocessing. All main-text experiments use the default hyperparameter recommendations from each method’s reference implementation. VAS uses the adaptive choice of described in § 3, with no validation-label tuning. To assess sensitivity of baselines to the default hyperparameter protocol, we additionally report a grid-search comparison in Appendix Section F.
For preprocessing, we use a method-specific protocol designed to match the assumptions of each baseline rather than imposing a single uniform pipeline. This avoids disadvantaging methods whose scoring rules depend on a particular feature scale. Specifically, ADERH uses MinMaxScaler and LUNAR uses StandardScaler [48]. SLAD, DIF, IForest, DTE, LOF, and LODA are applied to raw features, consistent with their standard usage. DN2 and GMD operate on -normalized features, following recent evidence that normalized embeddings improve distance- and Mahalanobis-based scoring in deep feature spaces [55, 39]. Finally, KPCA and VAS use a cosine kernel, so they operate on -normalized features.
Evaluation protocol. All methods are fit directly on the contaminated unlabeled dataset and then used to score the same samples, matching the standard unsupervised AD setting. We evaluate performance using Area Under the ROC Curve (AUROC) and Average Precision (AP). Higher values indicate better performance for both metrics.
Empirical Accuracy of VAS
Prior to benchmarking VAS against other baselines, we verify the accuracy of its approximation to the exact leave-one-out VS change. In Figure 2, we test the method on a 25,000-point, 100-component mixture of gaussians dataset in 512 dimensions. For all tested VS orders , we achieve a Spearman rank correlation of , showing that the approximation preserves the sample ranking. We also show the practical efficiency of VAS: Varying the dataset size from 1000 to 25000 points shows that VAS is orders of magnitude faster than the exact approach.
Figure 2. VAS provides accurate and efficient diversity influence measurements. Left: VAS diversity influence compared against the exact leave-one-out Vendi Score change for multiple orders of . Spearman rank correlation is shown for each choice of . Right: Varying dataset sizes shows that VAS provides orders of magnitude speedup over the exact approach.
We additionally evaluate the approximation on all 148 ADّBENCH benchmark instances by comparing VAS against the exact leave-one-out Vendi Score change. The approximation closely preserves the anomaly rankings, with an average Spearman over 0.98 on all tested benchmarks. The change in AUROC is at most 1.16%. Detailed results are provided in Table 2.
| Dataset | Encoder | Approx. AUROC | Exact AUROC | AUROC (%) | Spearman |
| CIFAR-10 | ResNet18 | 0.745 | 0.745 | -0.01 | 1.000 |
| CIFAR-10 | ViT | 0.941 | 0.941 | -0.02 | 1.000 |
| FashionMNIST | ResNet18 | 0.911 | 0.911 | -0.02 | 1.000 |
| FashionMNIST | ViT | 0.930 | 0.929 | 0.00 | 1.000 |
| MNIST-C | ResNet18 | 0.793 | 0.793 | 0.00 | 1.000 |
| MNIST-C | ViT | 0.739 | 0.739 | 0.01 | 1.000 |
| MVTec-AD | ResNet18 | 0.799 | 0.796 | -0.41 | 0.990 |
| MVTec-AD | ViT | 0.896 | 0.906 | 1.16 | 0.985 |
| SVHN | ResNet18 | 0.657 | 0.657 | 0.00 | 1.000 |
| SVHN | ViT | 0.579 | 0.579 | 0.06 | 1.000 |
| 20News | BERT | 0.626 | 0.630 | 0.48 | 0.997 |
| 20News | RoBERTa | 0.749 | 0.748 | -0.08 | 0.995 |
| AG News | BERT | 0.676 | 0.676 | 0.03 | 1.000 |
| AG News | RoBERTa | 0.881 | 0.881 | 0.01 | 1.000 |
| Amazon | BERT | 0.608 | 0.608 | 0.00 | 1.000 |
| Amazon | RoBERTa | 0.544 | 0.544 | 0.00 | 1.000 |
| IMDB | BERT | 0.502 | 0.502 | -0.01 | 1.000 |
| IMDB | RoBERTa | 0.530 | 0.530 | 0.00 | 1.000 |
| Yelp | BERT | 0.679 | 0.679 | 0.00 | 1.000 |
| Yelp | RoBERTa | 0.566 | 0.566 | 0.00 | 1.000 |
Table 2. Comparison of exact and approximate VAS across 148 ADBENCH benchmark instances. Results are averaged across dataset splits for each dataset–encoder combination. AUROC (%) is the percentage change from approximate to exact VAS. Spearman’s measures the rank correlation between exact and approximate anomaly scores. The approximation preserves both anomaly rankings and downstream detection performance across datasets.
ADBENCH Anomaly Detection Performance Results
Tables 1 and Table 3 show the performance of VAS relative to other baselines on the ADBENCH embedding benchmarks based on AUROC and AP, respectively. We report family-level averages across datasets and across the two embedding models available for each benchmark family. Statistical tests are also run across all 148 benchmark instances, where each instance corresponds to a dataset split paired with an embedding model. VAS achieves significantly stronger performance than the competing methods under a Wilcoxon signed-rank test for both AUROC and AP.
| Dataset | Encoder | VAS | ADERH | DIF | IForest | LUNAR | DN2 | LODA | LOF | KPCA | SLAD | GMD | DTE |
| CIFAR10 | ResNet18 | 0.153 | 0.093 | 0.112 | 0.088 | 0.104 | 0.163 | 0.091 | 0.122 | 0.166 | 0.145 | 0.141 | 0.103 |
| CIFAR10 | ViT | 0.659 | 0.541 | 0.326 | 0.360 | 0.389 | 0.386 | 0.365 | 0.072 | 0.486 | 0.670 | 0.491 | 0.109 |
| FashionMNIST | ResNet18 | 0.449 | 0.370 | 0.395 | 0.325 | 0.319 | 0.337 | 0.321 | 0.128 | 0.495 | 0.450 | 0.436 | 0.335 |
| FashionMNIST | ViT | 0.546 | 0.508 | 0.325 | 0.423 | 0.333 | 0.346 | 0.381 | 0.085 | 0.451 | 0.508 | 0.511 | 0.115 |
| MNIST-C | ResNet18 | 0.237 | 0.232 | 0.223 | 0.173 | 0.192 | 0.198 | 0.208 | 0.123 | 0.214 | 0.256 | 0.235 | 0.227 |
| MNIST-C | ViT | 0.226 | 0.269 | 0.220 | 0.242 | 0.192 | 0.216 | 0.200 | 0.090 | 0.200 | 0.231 | 0.201 | 0.091 |
| MVTec-AD | ResNet18 | 0.642 | 0.577 | 0.570 | 0.565 | 0.582 | 0.648 | 0.520 | 0.589 | 0.578 | 0.628 | 0.637 | 0.543 |
| MVTec-AD | ViT | 0.748 | 0.648 | 0.612 | 0.626 | 0.712 | 0.723 | 0.591 | 0.690 | 0.684 | 0.673 | 0.750 | 0.489 |
| SVHN | ResNet18 | 0.092 | 0.076 | 0.080 | 0.071 | 0.081 | 0.095 | 0.076 | 0.085 | 0.091 | 0.091 | 0.093 | 0.076 |
| SVHN | ViT | 0.066 | 0.056 | 0.055 | 0.056 | 0.062 | 0.067 | 0.056 | 0.068 | 0.063 | 0.060 | 0.066 | 0.057 |
| 20News | BERT | 0.088 | 0.066 | 0.069 | 0.065 | 0.075 | 0.076 | 0.066 | 0.083 | 0.069 | 0.080 | 0.084 | 0.069 |
| 20News | RoBERTa | 0.148 | 0.080 | 0.074 | 0.076 | 0.112 | 0.125 | 0.069 | 0.111 | 0.127 | 0.112 | 0.115 | 0.055 |
| Agnews | BERT | 0.094 | 0.066 | 0.065 | 0.064 | 0.088 | 0.077 | 0.062 | 0.130 | 0.072 | 0.080 | 0.087 | 0.069 |
| Agnews | RoBERTa | 0.334 | 0.125 | 0.089 | 0.089 | 0.199 | 0.249 | 0.081 | 0.115 | 0.272 | 0.188 | 0.213 | 0.054 |
| Amazon | BERT | 0.063 | 0.060 | 0.059 | 0.058 | 0.063 | 0.061 | 0.055 | 0.058 | 0.060 | 0.064 | 0.062 | 0.054 |
| Amazon | RoBERTa | 0.055 | 0.048 | 0.052 | 0.050 | 0.053 | 0.055 | 0.052 | 0.056 | 0.055 | 0.053 | 0.052 | 0.059 |
| Imdb | BERT | 0.047 | 0.047 | 0.047 | 0.047 | 0.047 | 0.049 | 0.044 | 0.050 | 0.050 | 0.050 | 0.049 | 0.047 |
| Imdb | RoBERTa | 0.056 | 0.046 | 0.053 | 0.050 | 0.053 | 0.054 | 0.053 | 0.056 | 0.055 | 0.052 | 0.052 | 0.049 |
| Yelp | BERT | 0.086 | 0.074 | 0.074 | 0.070 | 0.086 | 0.083 | 0.064 | 0.084 | 0.075 | 0.080 | 0.083 | 0.049 |
| Yelp | RoBERTa | 0.057 | 0.049 | 0.052 | 0.051 | 0.061 | 0.063 | 0.053 | 0.063 | 0.059 | 0.058 | 0.058 | 0.056 |
| Average Rank | 2.90 | 7.90 | 8.45 | 9.65 | 6.00 | 4.25 | 9.95 | 6.10 | 4.90 | 4.40 | 4.20 | 9.30 | |
| p-Value | - | 1.12e-14 | 2.14e-20 | 2.79e-20 | 4.42e-20 | 3.84e-09 | 2.42e-21 | 1.89e-12 | 1.07e-10 | 3.59e-08 | 6.62e-11 | 4.99e-22 |
Table 3. AP across dataset families and encoders. For multi-split datasets, we report average AP. Orange marks best, blue marks 2nd. Bottom-row p-values are paired Wilcoxon tests: VAS vs. each baseline over all 148 dataset-split/encoder instances.
Figure 5 shows the distribution of the adaptively selected Vendi orders . The order controls which parts of the spectrum drive the diversity-influence score: larger values emphasize dominant eigen-directions, while smaller values place relatively more weight on lower-variance directions. We observe that NLP embeddings tend to have larger principal eigenvalues than CV embeddings, suggesting stronger concentration around shared semantic context. Consequently, the adaptive rule selects smaller values of for NLP datasets, which makes VAS more sensitive to variation in the spectral tail.
Figure 5. Distribution of principal eigenvalue (left) and the selected orders (right) on all tested ADBENCH benchmarks. The geometry of CV datasets is significantly different than those from NLP datasets, so we select different orders to account for this.
To ensure that our comparisons are robust against hyperparameters, we perform a grid search over the hyperparameters of all baseline methods. For each method, we pick the hyperparameters that give the strongest performance relative to VAS. Even in this setting of letting methods overfit to the evaluation sets, VAS still provides the strongest results (Table 4 and Table 5). Additional details of this grid search experiment are in Appendix Section F.
| Dataset | Encoder | VAS | ADERH | DIF | IForest | LUNAR | DN2 | LODA | LOF | KPCA | SLAD | GMD | DTE |
| CIFAR10 | ResNet18 | 0.745 | 0.632 | 0.631 | 0.633 | 0.663 | 0.739 | 0.651 | 0.689 | 0.734 | 0.736 | 0.732 | 0.655 |
| CIFAR10 | ViT | 0.941 | 0.924 | 0.788 | 0.892 | 0.876 | 0.831 | 0.900 | 0.444 | 0.610 | 0.946 | 0.898 | 0.578 |
| FashionMNIST | ResNet18 | 0.911 | 0.869 | 0.832 | 0.839 | 0.867 | 0.838 | 0.853 | 0.576 | 0.895 | 0.904 | 0.912 | 0.823 |
| FashionMNIST | ViT | 0.930 | 0.911 | 0.776 | 0.896 | 0.820 | 0.837 | 0.893 | 0.493 | 0.900 | 0.923 | 0.925 | 0.656 |
| MNIST-C | ResNet18 | 0.793 | 0.772 | 0.708 | 0.742 | 0.791 | 0.752 | 0.750 | 0.599 | 0.791 | 0.780 | 0.793 | 0.727 |
| MNIST-C | ViT | 0.739 | 0.721 | 0.682 | 0.723 | 0.705 | 0.697 | 0.718 | 0.543 | 0.732 | 0.733 | 0.723 | 0.566 |
| MVTec-AD | ResNet18 | 0.799 | 0.760 | 0.730 | 0.746 | 0.776 | 0.801 | 0.747 | 0.757 | 0.783 | 0.791 | 0.790 | 0.733 |
| MVTec-AD | ViT | 0.896 | 0.813 | 0.753 | 0.815 | 0.866 | 0.880 | 0.799 | 0.841 | 0.718 | 0.864 | 0.891 | 0.672 |
| SVHN | ResNet18 | 0.657 | 0.583 | 0.585 | 0.583 | 0.613 | 0.665 | 0.592 | 0.637 | 0.673 | 0.654 | 0.659 | 0.597 |
| SVHN | ViT | 0.579 | 0.514 | 0.515 | 0.531 | 0.563 | 0.594 | 0.529 | 0.584 | 0.584 | 0.550 | 0.580 | 0.526 |
| 20News | BERT | 0.626 | 0.546 | 0.555 | 0.560 | 0.592 | 0.572 | 0.561 | 0.617 | 0.606 | 0.597 | 0.603 | 0.567 |
| 20News | RoBERTa | 0.749 | 0.627 | 0.597 | 0.651 | 0.711 | 0.733 | 0.639 | 0.708 | 0.731 | 0.709 | 0.716 | 0.539 |
| Agnews | BERT | 0.676 | 0.592 | 0.569 | 0.590 | 0.666 | 0.613 | 0.568 | 0.711 | 0.653 | 0.649 | 0.650 | 0.604 |
| Agnews | RoBERTa | 0.881 | 0.701 | 0.616 | 0.690 | 0.832 | 0.833 | 0.674 | 0.659 | 0.865 | 0.804 | 0.813 | 0.523 |
| Amazon | BERT | 0.608 | 0.578 | 0.548 | 0.564 | 0.608 | 0.594 | 0.571 | 0.573 | 0.608 | 0.606 | 0.604 | 0.525 |
| Amazon | RoBERTa | 0.544 | 0.474 | 0.502 | 0.511 | 0.537 | 0.555 | 0.507 | 0.552 | 0.544 | 0.530 | 0.530 | 0.547 |
| Imdb | BERT | 0.502 | 0.486 | 0.492 | 0.488 | 0.495 | 0.518 | 0.497 | 0.504 | 0.511 | 0.510 | 0.515 | 0.470 |
| Imdb | RoBERTa | 0.530 | 0.432 | 0.493 | 0.487 | 0.519 | 0.540 | 0.494 | 0.543 | 0.537 | 0.511 | 0.504 | 0.506 |
| Yelp | BERT | 0.679 | 0.628 | 0.579 | 0.605 | 0.684 | 0.672 | 0.603 | 0.656 | 0.673 | 0.660 | 0.676 | 0.529 |
| Yelp | RoBERTa | 0.566 | 0.507 | 0.501 | 0.543 | 0.579 | 0.598 | 0.546 | 0.577 | 0.560 | 0.573 | 0.575 | 0.544 |
| Average Rank | 2.60 | 8.60 | 10.65 | 8.80 | 5.25 | 4.25 | 8.50 | 6.70 | 4.20 | 4.60 | 3.95 | 9.90 | |
| p-Value vs VAS | - | 5.46e-20 | 3.10e-25 | 2.28e-23 | 8.92e-18 | 8.91e-10 | 6.01e-23 | 1.60e-14 | 6.22e-11 | 3.14e-14 | 1.34e-08 | 9.41e-24 |
Table 4. AUROC across dataset families and encoders using each method’s best hyperparameter setting. For multi-split datasets, we report average AUROC. Orange marks best, blue marks 2nd. Bottom-row p-values are paired Wilcoxon tests: VAS vs. each baseline over all 148 dataset-split/encoder instances.
| Dataset | Encoder | VAS | ADERH | DIF | IForest | LUNAR | DN2 | LODA | LOF | KPCA | SLAD | GMD | DTE |
| CIFAR10 | ResNet18 | 0.153 | 0.094 | 0.088 | 0.091 | 0.105 | 0.160 | 0.094 | 0.116 | 0.117 | 0.143 | 0.141 | 0.093 |
| CIFAR10 | ViT | 0.659 | 0.544 | 0.221 | 0.405 | 0.441 | 0.341 | 0.427 | 0.054 | 0.075 | 0.689 | 0.491 | 0.108 |
| FashionMNIST | ResNet18 | 0.449 | 0.372 | 0.315 | 0.329 | 0.327 | 0.310 | 0.359 | 0.102 | 0.383 | 0.459 | 0.436 | 0.264 |
| FashionMNIST | ViT | 0.546 | 0.510 | 0.205 | 0.433 | 0.329 | 0.317 | 0.435 | 0.073 | 0.393 | 0.537 | 0.511 | 0.129 |
| MNIST-C | ResNet18 | 0.237 | 0.237 | 0.169 | 0.183 | 0.196 | 0.190 | 0.226 | 0.109 | 0.220 | 0.255 | 0.235 | 0.169 |
| MNIST-C | ViT | 0.226 | 0.273 | 0.167 | 0.243 | 0.195 | 0.209 | 0.235 | 0.082 | 0.219 | 0.238 | 0.201 | 0.100 |
| MVTec-AD | ResNet18 | 0.642 | 0.584 | 0.539 | 0.567 | 0.588 | 0.653 | 0.572 | 0.576 | 0.625 | 0.635 | 0.636 | 0.569 |
| MVTec-AD | ViT | 0.748 | 0.652 | 0.554 | 0.635 | 0.722 | 0.734 | 0.613 | 0.687 | 0.554 | 0.692 | 0.750 | 0.468 |
| SVHN | ResNet18 | 0.092 | 0.077 | 0.071 | 0.074 | 0.081 | 0.096 | 0.077 | 0.084 | 0.098 | 0.092 | 0.093 | 0.077 |
| SVHN | ViT | 0.066 | 0.055 | 0.054 | 0.056 | 0.062 | 0.068 | 0.056 | 0.069 | 0.067 | 0.060 | 0.066 | 0.056 |
| 20News | BERT | 0.088 | 0.067 | 0.067 | 0.066 | 0.080 | 0.074 | 0.065 | 0.094 | 0.082 | 0.077 | 0.084 | 0.072 |
| 20News | RoBERTa | 0.148 | 0.079 | 0.073 | 0.079 | 0.117 | 0.128 | 0.075 | 0.105 | 0.135 | 0.108 | 0.115 | 0.064 |
| Agnews | BERT | 0.094 | 0.067 | 0.064 | 0.065 | 0.093 | 0.078 | 0.062 | 0.120 | 0.083 | 0.078 | 0.087 | 0.077 |
| Agnews | RoBERTa | 0.334 | 0.122 | 0.076 | 0.092 | 0.232 | 0.228 | 0.087 | 0.096 | 0.292 | 0.183 | 0.213 | 0.057 |
| Amazon | BERT | 0.063 | 0.060 | 0.057 | 0.058 | 0.063 | 0.061 | 0.059 | 0.058 | 0.063 | 0.063 | 0.062 | 0.054 |
| Amazon | RoBERTa | 0.055 | 0.047 | 0.050 | 0.050 | 0.053 | 0.056 | 0.050 | 0.057 | 0.055 | 0.052 | 0.052 | 0.058 |
| Imdb | BERT | 0.047 | 0.047 | 0.047 | 0.047 | 0.047 | 0.049 | 0.047 | 0.049 | 0.048 | 0.049 | 0.049 | 0.046 |
| Imdb | RoBERTa | 0.056 | 0.045 | 0.050 | 0.051 | 0.053 | 0.054 | 0.051 | 0.057 | 0.057 | 0.053 | 0.052 | 0.051 |
| Yelp | BERT | 0.086 | 0.075 | 0.067 | 0.070 | 0.090 | 0.083 | 0.068 | 0.082 | 0.084 | 0.079 | 0.083 | 0.057 |
| Yelp | RoBERTa | 0.057 | 0.049 | 0.049 | 0.052 | 0.061 | 0.064 | 0.052 | 0.062 | 0.057 | 0.058 | 0.058 | 0.058 |
| Average Rank | 3.05 | 7.60 | 10.60 | 8.75 | 5.40 | 4.65 | 8.40 | 6.30 | 4.70 | 4.35 | 4.30 | 9.90 | |
| p-Value vs VAS | - | 2.80e-14 | 8.96e-26 | 8.92e-20 | 1.35e-17 | 1.71e-08 | 3.89e-18 | 9.30e-13 | 1.95e-13 | 1.68e-05 | 6.62e-11 | 3.96e-23 |
Table 5. AP across dataset families and encoders using each method’s best hyperparameter setting. For multi-split datasets, we report average AP. Orange marks best, blue marks 2nd. Bottom-row p-values are paired Wilcoxon tests: VAS vs. each baseline over all 148 dataset-split/encoder instances.
We additionally evaluate the sensitivity of VAS to alternative rules for selecting , including different scaling factors, multiple leading eigenvalues, and the effective spectral rank. Our heuristic performs comparably to several alternatives and significantly outperforms others across 148 ADBENCH tasks, suggesting that its effectiveness is not overly sensitive to the choice of heuristic (Appendix § 13).
ImageNet Anomaly Detection Results
We next evaluate VAS in a controlled large-scale image setting using ImageNet embeddings from DINOv3 ViT-B/16 [53]. We construct contaminated datasets by sampling normal images from one set of ImageNet classes and anomalous images from a disjoint set of classes. We study two settings. First, we fix the normal set to 25,000 images from 25 classes and vary the number of anomalous images sampled from 25 disjoint classes, thereby sweeping the anomaly rate. Second, we fix the anomaly rate at 20% and scale the dataset by increasing the number of normal classes in increments of 10, while sampling anomalies from an equal number of disjoint classes. This increases both dataset size and semantic diversity as we scale to 100,000 normal images. Across both settings, VAS uses the same adaptive choice of as in the ADBENCH experiments and achieves the strongest performance among the evaluated methods (Figure 3). SLAD provides the most competitive performance, although as shown in the following section, it is orders of magnitude slower than VAS.
Figure 3. VAS achieves accurate ImageNet AD. We construct contaminated datasets by sampling normal and anomalous images from disjoint ImageNet classes. Left: AUROC as the anomaly rate varies with 25,000 normal images from 25 classes. Right: AUROC as dataset size increases at a fixed 20% anomaly rate, adding normal samples in increments of 10, 000. VAS performs best across both sweeps, showing robustness to increasing contamination and scale. Results are averaged over 5 replications, with standard deviation bars included.
These results match the behavior predicted by Proposition 3.2. The anomalous samples in this experiment are not isolated outliers: they are sampled from a select number of ImageNet classes and can therefore form coherent local neighborhoods in feature space. This structure can weaken local density methods, which may treat anomalous samples as typical once they have nearby anomalous neighbors. VAS instead scores each sample by its diversity influence. Under the mixture structure described in Proposition 3.2, this score depends on both component mass and internal diversity, enabling VAS to detect rare but coherent anomalous subpopulations.
To further investigate VAS across different anomaly types, we consider three additional constructions using ADBENCH embeddings: locally class-confusable observations, global directional perturbations, and disrupted feature dependencies. VAS performs well across constructions compared to the other best-performing baselines. We provide details of these experiments in Appendix § 14.
Latency Evaluation of VAS
We compare the latency of VAS and all tested baselines on simulated Gaussian datasets while varying either the number of samples or the embedding dimension (Figure 4). We report the time required to fit each method and score all samples. All experiments were performed on a single NVIDIA A6000 GPU. For ADERH, LODA, and LOF, we use custom GPU-friendly implementations to provide fairer latency comparisons, and verify that their outputs match the original implementations.
Figure 4. VAS supports efficient AD. Left: Latency of methods when increasing dataset size at a fixed dimensionality of 128. Right: Latency of methods with increasing dimensionality at a fixed dataset size of 5000. Results are averaged over repetitions.
Figure 4 shows that neural baselines are substantially slower across the tested regimes. As the number of samples increases, VAS exhibits linear scaling and becomes one of the fastest methods on larger datasets. This follows from its batched quadratic-form computation after the matrix has been formed. VAS has a stronger dependence on embedding dimension than alternatives such as IForest or LODA. However, in embedding-based AD, the dimension is fixed by the pretrained model. Thus, the more relevant large-scale regime is increasing at fixed , where VAS scales linearly.
Conclusion
This work introduces VAS, an unsupervised AD method for embedding data that scores each sample by its influence on dataset diversity. By deriving a first-order approximation to leave-one-out Vendi Scores, VAS avoids repeated eigendecompositions, requires no validation-based hyperparameter tuning, and adapts automatically to the geometry of the dataset. Across image and text AD benchmarks, VAS consistently improves over classical, kernel-based, fitted-distribution, and deep AD baselines. Our large-scale ImageNet experiments further show that VAS is especially effective for detecting coherent contamination, where anomalous samples form rare but structured subpopulations. Together, these results establish VAS as a scalable and effective approach to unsupervised AD.
Limitations. VAS depends on the quality of the embedding space and the chosen similarity function. If anomalies are indistinguishable from normal samples under the embedding or cosine similarity, VAS cannot reliably detect them. While VAS provides a principled anomaly ranking, the scalar score does not by itself explain the source of anomalousness. In addition, our adaptive rule for selecting is heuristic and may not be optimal for all data geometries.
Future work. Future work includes extending VAS to learned or task-specific kernels and developing principled rules for adapting the VS order to dataset geometry. More broadly, VAS may be useful for other data curation tasks, including coreset selection and duplicate detection.
Broader impacts. VAS can support dataset auditing by helping identify corrupted or mislabeled samples in large embedding datasets. However, anomaly scores depend on the embedding model and similarity function, and may reflect their biases. In sensitive applications, VAS should therefore be used as a tool for prioritizing human review rather than as an automatic rule for removing data.
Acknowledgments and Disclosure of Funding
Adji Bousso Dieng acknowledges support from the National Science Foundation CAREER grant CAIF #2541965. Amey Pasarkar is supported by an NSF-GRFP fellowship. The authors declare no competing interests.
AI use statement. In this work, we used generative AI tools for implementing methods in code. We have not used generative AI tools for generating data sets, developing theoretical models or conceptual frameworks, formulating mathematical claims, assisting in the writing of proofs, proposing hypotheses, cleaning and reformatting datasets, supporting qualitative and thematic data analysis, interpreting results or designing or providing feedback on research methodology. We have reviewed all AI-assisted work: All code went through testing to ensure we provided faithful reproductions of established baselines and that our code matched the experimental descriptions in the paper. AI tools were used as a small portion of the paper editing process and all changes were reviewed by the Authors. We take responsibility for the final content of this work, including text, claims, or artifacts produced with the aid of generative AI.
Proof of Proposition 3.1
Removing induces the perturbation
Moreover,
because and .
We now compute the first variation of .
Case .
Here denotes the directional derivative of at in the perturbation direction . Then,
Case .
Using spectral calculus,
Thus, in both cases,
By Taylor’s theorem,
Substituting gives
Using , we obtain
which completes the proof.
Comparison of VAS to Alternative Spectral Scores
VAS belongs to a broader family of spectral quadratic anomaly scores. For -normalized embeddings , consider the eigendecomposition
where . Writing , any spectral function defines a quadratic score
Different spectral detectors differ in how the choice of , and therefore how they weight the eigen-directions of . From Proposition 3.1, VAS assigns the weights
As in the main text, spectral functions are evaluated on the positive eigenspace of . For , VAS assigns larger positive weights to smaller eigenvalues. For , the weights are negative, with larger magnitudes on leading eigen-directions. Thus, the order controls how spectral directions contribute to diversity influence.
Mahalanobis distance. For mean-centered features, the covariance equals , and the squared Mahalanobis score is
where is the pseudoinverse. As , the VAS spectral weights become proportional to . More precisely,
Thus, VAS approaches the Mahalanobis ranking in this limit. Unlike standard Mahalanobis distance, however, VAS operates on the uncentered second-moment matrix and adaptively selects rather than fixing inverse eigenvalue weights.
PCA reconstruction error. Let contain the leading eigenvectors of . The corresponding PCA residual score is
PCA therefore assigns zero weight to the leading directions and equal weight to the remaining directions. Kernel PCA applies an analogous reconstruction principle in a kernel feature space. In contrast, VAS uses a smooth spectral weighting without imposing a hard rank cutoff.
Global similarity. For , the VAS score simplifies to
Thus, at , VAS is ranking-equivalent to negative average squared cosine similarity, favoring samples that are globally dissimilar to the dataset.
VAS is related to classical spectral detectors, but its weighting arises from the leave-one-out influence of a sample on dataset diversity. Rather than prescribing a fixed spectral filter, VAS adapts its weighting to the dataset through the Vendi Score order .
Proof of Proposition 3.2
Since , we have
Because each component is supported on subspace , the operator is supported on . Since the subspaces are mutually orthogonal, is block diagonal with block on . Therefore, for ,
Substituting into the definition of gives
Taking expectation over ,
Using ,
Finally, since
we have
Thus
which proves the claim.
ADBENCH Embedding Datasets
Here we summarize the ADBENCH embedding datasets. For all image datasets, we use embeddings from both ResNet18 [25] and ViT models [16]. For text datasets, we use embeddings from BERT [14] and RoBERTa [35].
CIFAR-10 (10 splits): Each split treats one class as normal and the remaining nine classes as anomalies, which are downsampled to 5% of the total dataset. Each split contains 5,263 samples. Results are averaged across all 10 choices of the normal class. Evaluated using both 512- and 1000-dimensional image embeddings.
FashionMNIST (10 splits): Same protocol as CIFAR-10, with one class designated as normal and others downsampled to 5% anomalies. Each split contains 6,315 samples. Results are averaged across splits. Evaluated with 512- and 1000-dimensional embeddings.
SVHN (10 splits): Same one-class protocol as CIFAR-10. Each split corresponds to a different digit as the normal class, with remaining digits downsampled to 5% anomalies. Sample sizes range from 4,904 to 10,000. Evaluated with 512- and 1000-dimensional embeddings.
MNIST-C (16 splits): Corrupted variants of MNIST, where original MNIST images are treated as normal and corrupted versions (e.g., noise, blur, etc.) are treated as anomalies. Each split corresponds to a different corruption type and contains 10,000 samples. Evaluated with 512- and 1000-dimensional embeddings.
MVTec-AD (15 splits): Industrial anomaly detection dataset consisting of images of manufactured objects with defects. Each object category (15 total) defines a separate anomaly detection task, where defect-free images are normal and defective images are anomalous. Sample sizes range from 102 to 501 per category. Evaluated with 512- and 1000-dimensional embeddings.
20 Newsgroups (6 splits): For each split, one topic group is treated as normal and the remaining groups are treated as anomalies. The six normal classes are computer, recreation, science, miscellaneous, politics, and religion. Sample sizes are 615, 1,532, 1,657, 2,497, 2,514, and 3,090. Evaluated using 768-dimensional text embeddings.
AG News (4 splits): Multi-class news dataset where each split treats one class as normal and the remaining classes are downsampled to 5% anomalies. Each split contains 10,000 samples. Evaluated with 768-dimensional embeddings.
Amazon Reviews (1 dataset): Binary sentiment dataset with 10,000 samples. Negative reviews are treated as anomalies and positive reviews as normal. Evaluated using 768-dimensional embeddings.
IMDB (1 dataset): Binary sentiment dataset with 10,000 samples. Negative reviews are treated as anomalies and positive reviews as normal. Evaluated using 768-dimensional embeddings.
Yelp (1 dataset): Review dataset with 10,000 samples. Reviews with 0–1 stars are treated as anomalies, while 3–4 star reviews are treated as normal. Evaluated using 768-dimensional embeddings.
Baselines
Here we review the baseline methods we compare against. In Figure 4, all baselines were run on a single NVIDIA A6000 GPU with 4 Intel Xeon Gold 5320, 2.20GHz CPUs, with results generated over 25 GPU hours.
IForest. Liu et al. [ [32] introduced Isolation Forest to take advantage of the idea that anomalies are easier to separate than a normal point from the rest of the dataset. IForest builds an ensemble of isolation trees based on random partitions of the feature space, and detects anomalies based on average how many partitions are needed to isolate a sample. The hyperparameters are the number of samples used to build each tree, and the total number of trees. IForest relies on the feature being easy to partition, which may not be true when features are highly correlated.
DIF. Xu et al. [ [60] improved on IForest by noting that the linear partitions are not expressive enough for most datasets. The resulting method, Deep Isolation Forest (DIF), augments isolation forests with random deep representations: instead of directly applying axis-parallel splits in the input space, DIF first maps data through an ensemble of randomly initialized neural networks, and then applies standard isolation trees in this transformed space.
DIF is primarily designed for tabular data where nonlinear feature transformations must be learned. On embedding datasets, the representation are already highly expressive and semantically structured. Applying additional random neural projections can distort this geometry, potentially destroying meaningful similarity structure and degrading distance-based anomaly signals. As a result, DIF may offer limited benefit on high-quality embedding datasets, while incurring additional computational overhead.
LOF. Breunig et al. [ [7] designed Local Outlier Factor (LOF) to detect anomalies based on local neighborhoods. In particular, LOF measures how dense a sample’s neighborhood is relative to its nearest neighbors. The main assumption is that anomalies will have significantly less dense neighborhoods than their neighbors. This identifies anomalies at a local level, but will ignore global structure, and requires the explicit construction of neighborhoods based on a fixed nearest-neighbor cutoff . LOF requires complexity to construct the nearest-neighbor distances.
DN2. K-Nearest Neighbors on neural embeddings was explored by Bergman et al. [ [5] as a simple heuristic for anomaly detection. Anomaly scores are computed as the average distance between a test embedding and its nearest neighbors in the training set. Identifying the nearest neighbors requires time and similar to LOF, requires the construction of explicit local neighborhoods that ignore global structure.
GMD. Gaussian feature modeling has been used for anomaly detection in pretrained deep feature spaces [51]. To this idea to our fully unsupervised setting, where the unlabeled data may be multi-modal and contaminated, we use a Gaussian-mixture Mahalanobis distance baseline. We fit a GMM with a tied covariance matrix , select the number of components by BIC, assign each sample to its most likely component with mean , and score it by
with larger values indicating greater deviation from the learned feature distribution. The main computational cost is EM fitting, which scales as for iterations, components, samples, and dimension . Following recommendations for Mahalanobis scoring in deep feature spaces [39], we apply GMD to -normalized features.
SLAD. Scale Learning for Anomaly Detection (SLAD) is a deep learning approach for identifying anomalies [61]. It learns representations by introducing auxiliary supervisory signals derived from scale consistency, encouraging the model to produce embeddings that are invariant across randomly sampled feature subspaces. SLAD constructs multiple random subspaces of the input features and trains a neural network to predict consistent representations across these subspaces, thereby capturing stable patterns of normal data while amplifying deviations for anomalies. To improve robustness, SLAD uses an ensemble of subspace learners, where each model operates on different sampled feature subsets and their outputs are aggregated into a final anomaly score.
ADERH. Adaptive Density Estimation via Random Hyperspheres (ADERH) is an unsupervised anomaly detection method that estimates local data density by constructing an ensemble of random hyperspheres in feature space [17]. Each hypersphere is defined by a randomly sampled center and radius, and its contribution is weighted based on how many training points fall inside it. Regions that consistently contain many points across sampled hyperspheres are considered high-density (normal), while points that fall into few or low-density hyperspheres are assigned higher anomaly scores. This ensemble-based strategy allows ADERH to approximate complex density structure without explicitly fitting a parametric model, and aligns with the general principle that anomalies correspond to low-density regions in the data. ADERH’s complexity is , where is the number of samples, is the number of hyperspheres, and is the feature dimension.
ADERH implicitly assumes that normal data can be well-approximated by a union of compact, spherical regions that are separable from anomalies. While this assumption is often reasonable for tabular data, it breaks down in modern embedding spaces. Embeddings tend to lie on continuous, dense manifolds rather than forming isolated clusters [18]. As a result, while ADERH can perform strongly on structured tabular datasets, it may be less appropriate for anomaly detection on modern embedding representations.
LODA. Lightweight On-line Detector of Anomalies (LODA) is an ensemble, training-free approach for anomaly detection based on sparse random projections [49]. Specifically, it projects high-dimensional data into many one-dimensional subspaces, fits simple histogram-based density estimators in each projection, and aggregates these to estimate the overall likelihood of a sample. LODA is quite fast: the runtime scales with for building projections and histograms in dimensions.
LODA assumes that meaningful structure can be captured through the random projections and corresponding densities. While this may be reasonable for tabular data, it breaks down in embedding spaces where information is distributed across many correlated dimensions and lies on a structured manifold. Random projections to 1D can destroy this geometry, making density estimates unreliable and reducing anomaly separability.
KPCA. Fang et al. [2025] demonstrated that Kernel PCA reconstruction errors can be a useful signal for OOD detection. Here, we apply their approach to anomaly detection. The KPCA approach uses a fused cosine-gaussian kernel to fit the data and compute reconstruction errors. Fang et al. [2025] showed that these errors can be computed efficiently for OOD detection using Random Fourier Features (RFFs). In all, this approach has time complexity for an sample dataset in dimensions approximated with a -dimensional RFF.
VAS and KPCA both rely on kernels, but use different motivations for anomaly detection. KPCA learns a low-rank approximation of the data and scores anomalies based on reconstruction error, implicitly assuming that normal data lies near a low-dimensional manifold. In contrast, VAS measures how each sample perturbs the eigenvalue spectrum of the similarity matrix. As a result, KPCA is sensitive primarily to deviations from the leading eigenvalues, whereas VAS captures changes to the full spectral distribution, including effects in the tail. This allows VAS to detect anomalies that may lie within the dominant subspace but still alter the overall diversity structure of the dataset.
LUNAR. LUNAR is a graph-neural-network-based local anomaly detector that learns to score samples from their nearest-neighbor graph [21]. It generalizes classical local outlier methods by replacing fixed neighborhood statistics with trainable message passing over local neighborhoods. Unlike fixed neighborhood scores, LUNAR incurs additional cost from constructing the nearest-neighbor graph and training a GNN, but can learn a more adaptive local anomaly score once the graph is built.
DTE. Diffusion Time Estimation (DTE) identifies anomalies based on how diffuse they are from the normal data distribution [36]. DTE estimates the posterior distribution over diffusion time for a given sample, which serves as a proxy for its distance from the data manifold. Intuitively, normal samples lie close to the data manifold and correspond to small diffusion times, while anomalous samples require larger diffusion times (i.e., more noise) to be explained. A neural network is trained to predict this diffusion time, and the anomaly score is given by the predicted mean or mode.
We focus on the categorical parametrization, which models diffusion time as a discrete distribution over bins. This variant is the empirically strongest.
Grid Search Experiment
We perform a discrete grid search over key hyperparameters for each method, while keeping all other settings fixed to their benchmark or paper-specified defaults. Below, we list for each method: (i) the default configuration, (ii) the hyperparameters swept over, and (iii) a brief description of each hyperparameter.
ADERH: Default: , subset size . Sweep: , . Here, is the number of hyperspheres in the ensemble, and is the number of samples used to define each hypersphere.
DIF: Default: , . Sweep: , . is the number of isolation trees, and is the number of random neural representations used.
DTE (Categorical): Default: num_bins (all other training hyperparameters fixed as in Livernoche et al. [2024]). Sweep: num_bins . num_bins is the number of discrete diffusion time bins in the categorical parametrization.
IForest: Default: , max_samples . Sweep: , max_samples . is the number of trees, and max_samples is the subsample size used to build each tree.
KNN: Default: (with internal row-wise normalization). Sweep: . is the number of nearest neighbors used to compute the anomaly score.
LODA: Default: , . Sweep: , . is the number of random projections (histograms), and is the number of bins per histogram.
LOF: Default: . Sweep: . is the number of neighbors used to estimate local density.
kPCA: Default: , rff_dim , exp_var_ratio . Sweep: , exp_var_ratio . controls the kernel bandwidth, rff_dim is the number of random Fourier features, and exp_var_ratio is the fraction of variance retained in PCA.
SLAD: Default: , subspace_pool_size . Sweep: , subspace_pool_size . is the number of subspace learners, and subspace_pool_size is the number of candidate feature subspaces.
LUNAR: Default: , sampling strategy = mixed. Sweep: , sampling strategy . is the number of neighbors used when constructing the KNN graph, and sampling strategy describes how the contrastive samples are constructed.
In tables 4 and 5, we report results using the best-performing configuration within the specified grid.
Alternative Heuristics for VAS
We evaluate the sensitivity of VAS to alternative data-dependent heuristics for selecting the Vendi Score order . Our default rule is , where is the largest eigenvalue of the trace-normalized feature covariance matrix .
We consider three families of alternatives:
The slope variants modify the scaling factor in our default rule, while the top- variants use the cumulative mass of the leading eigenvalues. The normalized Vendi Score heuristic instead uses the effective spectral dimension, where . We retain the lower bound of 0.1 for numerical stability.
Table 6 shows that our default heuristic is robust to several reasonable changes in the adaptation rule. Modifying the scaling factor from 2 to 1, or using the cumulative mass of the top 5 or 10 eigenvalues, produces similar AUROC values, with no statistically significant differences under the paired Wilcoxon tests. In contrast, increasing the scaling factor to 5 or using the normalized heuristic yields significantly worse performance. These results support the effectiveness of our simple default rule while indicating that VAS does not depend strongly on its precise formulation.
| Dataset | Encoder | Adaptive | Slope-1 | Slope-5 | Top-5 | Top-10 | |
| CIFAR-10 | ResNet18 | 0.745 | 0.732 | 0.725 | 0.711 | 0.737 | 0.729 |
| CIFAR-10 | ViT | 0.941 | 0.914 | 0.943 | 0.894 | 0.919 | 0.911 |
| FashionMNIST | ResNet18 | 0.911 | 0.908 | 0.876 | 0.897 | 0.907 | 0.903 |
| FashionMNIST | ViT | 0.930 | 0.928 | 0.910 | 0.926 | 0.929 | 0.928 |
| MNIST-C | ResNet18 | 0.793 | 0.794 | 0.755 | 0.791 | 0.794 | 0.793 |
| MNIST-C | ViT | 0.739 | 0.741 | 0.737 | 0.743 | 0.740 | 0.741 |
| MVTec-AD | ResNet18 | 0.799 | 0.800 | 0.794 | 0.790 | 0.800 | 0.800 |
| MVTec-AD | ViT | 0.896 | 0.900 | 0.873 | 0.904 | 0.900 | 0.900 |
| SVHN | ResNet18 | 0.657 | 0.656 | 0.633 | 0.646 | 0.655 | 0.652 |
| SVHN | ViT | 0.579 | 0.585 | 0.546 | 0.589 | 0.585 | 0.586 |
| 20News | BERT | 0.626 | 0.629 | 0.613 | 0.630 | 0.629 | 0.629 |
| 20News | RoBERTa | 0.749 | 0.749 | 0.749 | 0.750 | 0.749 | 0.749 |
| AG News | BERT | 0.676 | 0.678 | 0.663 | 0.680 | 0.678 | 0.678 |
| AG News | RoBERTa | 0.881 | 0.881 | 0.881 | 0.881 | 0.881 | 0.881 |
| Amazon | BERT | 0.608 | 0.608 | 0.607 | 0.608 | 0.608 | 0.608 |
| Amazon | RoBERTa | 0.544 | 0.544 | 0.544 | 0.544 | 0.544 | 0.544 |
| IMDB | BERT | 0.502 | 0.501 | 0.507 | 0.499 | 0.501 | 0.501 |
| IMDB | RoBERTa | 0.530 | 0.530 | 0.530 | 0.529 | 0.530 | 0.530 |
| Yelp | BERT | 0.679 | 0.679 | 0.678 | 0.679 | 0.679 | 0.679 |
| Yelp | RoBERTa | 0.566 | 0.566 | 0.566 | 0.566 | 0.566 | 0.566 |
| Wilcoxon p-value | – | 0.399 | 0.0058 | 0.371 | 0.0728 |
Table 6. Sensitivity of VAS to alternative adaptive rules for selecting across 148 ADBENCH benchmark instances. AUROC values are averaged across dataset splits for each dataset–encoder combination. Mean AUROC is computed across all benchmark instances. The bottom row reports unadjusted, one-sided paired Wilcoxon tests comparing our default adaptive rule against each alternative, with the alternative hypothesis that our default achieves higher AUROC.
Anomaly Detection Across Different Anomaly Types
Han et al. [ [23] define four types of anomalies: local, global, dependency, and clustered anomalies. Our ImageNet experiments investigate clustered anomalies: coherent anomalous subpopulations. Here we evaluate VAS on three additional constructions inspired by the remaining anomaly types. We create realistic anomalies using data from ADBENCH embedding datasets to construct anomalies with different geometric properties.
We evaluate four datasets (CIFAR-10, SVHN, 20News, and AG News), each with two embedding architectures, and compare VAS against five representative baselines: GMD, DN2, KPCA, LOF, and SLAD. Across all tables, we report mean AUROC over five independent trials and the average rank of each method across the eight dataset–encoder combinations.
Local Anomalies. We first investigate local anomalies - those that deviate from their local neighborhood. To construct these, we identify samples that are semantically different from a normal class but appear plausible under its local distribution. For each dataset, we split embeddings into fit and hold-out sets, fit one Gaussian per class, and assign each held-out sample to its highest-likelihood class.
For each predicted class , we form a dataset from samples assigned to . True- samples are normal; misclassified samples are anomalous. These are local anomalies because they lie in ’s region rather than being globally isolated.
Global Directional Anomalies. We next investigate anomalies that deviate from the global geometry of the normal data. We compute class-centroid directions from the genuine embeddings and define the subspace spanned by these centroids. For each anomaly, we sample a random Gaussian direction and project it onto the orthogonal complement of this subspace. Given a normalized genuine embedding , we construct an anomalous direction as
and restore the original embedding norm.
The parameter controls the magnitude of the directional perturbation. We report results for , representing a relatively small departure from the original embedding direction. Each trial contains 2,500 genuine normal observations and 278 synthesized anomalies, corresponding to approximately 10% contamination. Results are averaged across five independent trials.
Feature-Dependency Anomalies. Finally, we investigate anomalies that preserve feature marginals while breaking feature dependence.. For each semantic class, we sample up to 2,500 genuine embeddings, normalize them to unit length, and apply a random orthogonal rotation. We divide the rotated features into four approximately equal-sized blocks.
To construct an anomaly, we replace one randomly selected block with the corresponding block from a different observation. We then apply the inverse rotation and normalize the resulting embedding. This construction combines feature blocks from normal observations while disrupting their joint structure. We generate approximately 10% anomalies and average results across all available semantic classes and five random trials. For classes with fewer observations, sample counts are reduced accordingly.
Overall, VAS achieves the best average rank across all three anomaly constructions. It performs particularly well on global directional anomalies and on feature-dependency anomalies in text embeddings. For image-based dependency anomalies, LOF is also highly competitive. Local class-confusable anomalies are substantially more challenging across methods, although VAS retains the strongest average rank. These experiments provide complementary evidence of VAS’s ability to identify different forms of anomalous structure in pretrained embedding spaces.
| Dataset | Encoder | VAS | GMD | DN2 | KPCA | LOF | SLAD |
| CIFAR-10 | ResNet18 | 0.557 | 0.561 | 0.571 | 0.574 | 0.544 | 0.551 |
| CIFAR-10 | ViT | 0.733 | 0.723 | 0.719 | 0.721 | 0.659 | 0.727 |
| SVHN | ResNet18 | 0.610 | 0.610 | 0.617 | 0.617 | 0.579 | 0.600 |
| SVHN | ViT | 0.552 | 0.543 | 0.535 | 0.543 | 0.539 | 0.543 |
| 20News | BERT | 0.555 | 0.560 | 0.526 | 0.525 | 0.525 | 0.548 |
| 20News | RoBERTa | 0.604 | 0.560 | 0.560 | 0.561 | 0.536 | 0.538 |
| AG News | BERT | 0.661 | 0.608 | 0.609 | 0.609 | 0.620 | 0.632 |
| AG News | RoBERTa | 0.660 | 0.589 | 0.642 | 0.630 | 0.624 | 0.605 |
| Average Rank | 1.81 | 3.63 | 3.56 | 3.06 | 5.19 | 3.75 |
Table 7. AUROC for local class-confusable anomalies constructed from genuine ADBENCH embeddings. Results are averaged across assigned-class subsets and five independent trials. Anomaly rates vary with class overlap. Average ranks are computed across dataset–encoder combinations (lower is better).
| Dataset | Encoder | VAS | GMD | DN2 | KPCA | LOF | SLAD |
| CIFAR-10 | ResNet18 | 0.578 | 0.561 | 0.188 | 0.433 | 0.517 | 0.530 |
| CIFAR-10 | ViT | 0.526 | 0.531 | 0.194 | 0.215 | 0.521 | 0.488 |
| SVHN | ResNet18 | 0.669 | 0.558 | 0.289 | 0.554 | 0.530 | 0.560 |
| SVHN | ViT | 0.956 | 0.585 | 0.272 | 0.607 | 0.572 | 0.586 |
| 20News | BERT | 0.996 | 0.719 | 0.727 | 0.746 | 0.919 | 0.924 |
| 20News | RoBERTa | 1.000 | 0.995 | 1.000 | 1.000 | 0.956 | 1.000 |
| AG News | BERT | 0.991 | 0.712 | 0.657 | 0.752 | 0.932 | 0.896 |
| AG News | RoBERTa | 1.000 | 1.000 | 1.000 | 1.000 | 1.000 | 1.000 |
| Average Rank | 1.63 | 3.69 | 5.13 | 3.75 | 3.94 | 2.88 |
Table 8. AUROC for global directional anomalies constructed by perturbing embeddings outside the span of normal-class centroids. Results use and approximately 10% contamination, averaged across five trials. Average ranks are computed across dataset–encoder combinations (lower is better).
| Dataset | Encoder | VAS | GMD | DN2 | KPCA | LOF | SLAD |
| CIFAR-10 | ResNet18 | 0.825 | 0.789 | 0.844 | 0.864 | 0.894 | 0.747 |
| CIFAR-10 | ViT | 0.828 | 0.748 | 0.793 | 0.825 | 0.847 | 0.669 |
| SVHN | ResNet18 | 0.858 | 0.778 | 0.851 | 0.841 | 0.879 | 0.791 |
| SVHN | ViT | 0.959 | 0.694 | 0.799 | 0.818 | 0.835 | 0.716 |
| 20News | BERT | 0.922 | 0.685 | 0.814 | 0.744 | 0.895 | 0.775 |
| 20News | RoBERTa | 0.985 | 0.549 | 0.728 | 0.779 | 0.762 | 0.628 |
| AG News | BERT | 0.917 | 0.664 | 0.754 | 0.736 | 0.910 | 0.770 |
| AG News | RoBERTa | 0.999 | 0.531 | 0.810 | 0.814 | 0.827 | 0.611 |
| Average Rank | 1.63 | 5.75 | 3.63 | 3.38 | 1.75 | 4.88 |
Table 9. AUROC for feature-dependency anomalies constructed by swapping blocks of orthogonally rotated embeddings. Results use approximately 10% contamination and are averaged across semantic classes and five trials. Average ranks are computed across dataset–encoder combinations (lower is better).
References
- [1]Mohiuddin Ahmed, Nazim Choudhury, and Shahadat Uddin. Anomaly detection on big data in financial markets. In Proceedings of the 2017 IEEE/ACM International Conference on Advances in Social Networks Analysis and Mining 2017, pages 998–1001, 2017.DOI
- [3]Tharindu R Bandaragoda, Kai Ming Ting, David Albrecht, Fei Tony Liu, and Jonathan R Wells. Efficient anomaly detection by isolation using nearest neighbour ensemble. In 2014 IEEE International conference on data mining workshop, pages 698–705. IEEE, 2014.DOI
- [4]Roberto Barceló, Cristóbal Alcázar, and Felipe Tobar. Avoiding mode collapse in diffusion models fine-tuned with reinforcement learning. arXiv preprint arXiv:2410.08315, 2024.arxiv.org/abs/2410.08315
- [5]Liron Bergman, Niv Cohen, and Yedid Hoshen. Deep nearest neighbor anomaly detection. arXiv preprint arXiv:2002.10445, 2020.arxiv.org/abs/2002.10445
- [6]Paul Bergmann, Michael Fauser, David Sattlegger, and Carsten Steger. Mvtec ad–a comprehensive real-world dataset for unsupervised anomaly detection. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 9592–9600, 2019.DOI
- [7]Markus M Breunig, Hans-Peter Kriegel, Raymond T Ng, and Jörg Sander. LoF: identifying density-based local outliers. In Proceedings of the 2000 ACM SIGMOD international conference on Management of data, pages 93–104, 2000.
- [8]Elizabeth G Campolongo, Yuan-Tang Chou, Ekaterina Govorkova, Wahid Bhimji, Wei-Lun Chao, Chris Harris, Shih-Chieh Hsu, Hilmar Lapp, Mark S Neubauer, Josephine Namayanja, et al. Building machine learning challenges for anomaly detection in science. arXiv preprint arXiv:2503.02112, 2025.arxiv.org/abs/2503.02112
- [9]Yang Cao, Sikun Yang, Chen Li, Haolong Xiang, Lianyong Qi, Bo Liu, Rongsheng Li, and Ming Liu. Tad-bench: A comprehensive benchmark for embedding-based text anomaly detection. arXiv preprint arXiv:2501.11960, 2025.arxiv.org/abs/2501.11960
- [10]Varun Chandola, Arindam Banerjee, and Vipin Kumar. Anomaly detection: A survey. ACM computing surveys (CSUR), 41(3):1–58, 2009.
- [11]Wei Dai, Kai Hwang, and Jicong Fan. Unsupervised anomaly detection for tabular data using noise evaluation. arXiv preprint arXiv:2412.11461, 2024.arxiv.org/abs/2412.11461
- [12]Alan DenAdel, Madeline Hughes, Akshaya Thoutam, Anay Gupta, Andrew W Navia, Nicolo Fusi, Srivatsan Raghavan, Peter S Winter, Ava P Amini, and Lorin Crawford. Evaluating the role of pre-training dataset size and diversity on single-cell foundation model performance. bioRxiv, pages 2024–12, 2025.DOI
- [13]Jia Deng, Wei Dong, Richard Socher, Li-Jia Li, Kai Li, and Li Fei-Fei. Imagenet: A large-scale hierarchical image database. In 2009 IEEE conference on computer vision and pattern recognition, pages 248–255. Ieee, 2009.
- [14]Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. Bert: Pre-training of deep bidirectional transformers for language understanding. In Proceedings of the 2019 conference of the North American chapter of the association for computational linguistics: human language technologies, volume 1 (long and short papers), pages 4171–4186, 2019.arxiv.org/abs/1810.04805
- [15]Adji Bousso Dieng and Amey Pasarkar. A unified and predictive measure of functional diversity. arXiv preprint arXiv:2509.16133, 2025.arxiv.org/abs/2509.16133
- [16]Alexey Dosovitskiy, Lucas Beyer, Alexander Kolesnikov, Dirk Weissenborn, Xiaohua Zhai, Thomas Unterthiner, Mostafa Dehghani, Matthias Minderer, Georg Heigold, Sylvain Gelly, et al. An image is worth 16x16 words: Transformers for image recognition at scale. arXiv preprint arXiv:2010.11929, 2020.arxiv.org/abs/2010.11929
- [17]Walid Durani, Collin Leiber, Khalid Durani, Claudia Plant, and Christian Böhm. Anomaly detection by an ensemble of random pairs of hyperspheres. In The Thirty-ninth Annual Conference on Neural Information Processing Systems, 2025. URL https://openreview.net/forum?id=vM4PIjsJDG.DOI
- [18]Kawin Ethayarajh. How contextual are contextualized word representations? comparing the geometry of bert, elmo, and gpt-2 embeddings. In Proceedings of the 2019 conference on empirical methods in natural language processing and the 9th international joint conference on natural language processing (EMNLP-IJCNLP), pages 55–65, 2019.arxiv.org/abs/1909.00512
- [19]Kun Fang, Qinghua Tao, Mingzhen He, Kexin Lv, Runze Yang, Haibo Hu, Xiaolin Huang, Jie Yang, and Longbin Cao. Kernel pca for out-of-distribution detection: Non-linear kernel selections and approximations. arXiv preprint arXiv:2505.15284, 2025.DOI
- [20]Dan Friedman and Adji Bousso Dieng. The vendi score: A diversity evaluation metric for machine learning. arXiv preprint arXiv:2210.02410, 2022.
- [21]Adam Goodge, Bryan Hooi, See-Kiong Ng, and Wee Siong Ng. Lunar: Unifying local outlier detection methods via graph neural networks. In Proceedings of the AAAI conference on artificial intelligence, volume 36, pages 6737–6745, 2022.
- [22]Sudipto Guha, Nina Mishra, Gourav Roy, and Okke Schrijvers. Robust random cut forest based anomaly detection on streams. In International conference on machine learning, pages 2712–2721. PMLR, 2016.
- [23]Songqiao Han, Xiyang Hu, Hailiang Huang, Minqi Jiang, and Yue Zhao. Adbench: Anomaly detection benchmark. Advances in neural information processing systems, 35:32142–32159, 2022.
- [24]Sahand Hariri, Matias Carrasco Kind, and Robert J Brunner. Extended isolation forest. IEEE transactions on knowledge and data engineering, 33(4):1479–1489, 2019.DOI
- [25]Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recognition. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 770–778, 2016.arxiv.org/abs/1512.03385
- [26]Ruining He and Julian McAuley. Ups and downs: Modeling the visual evolution of fashion trends with one-class collaborative filtering. In proceedings of the 25th international conference on world wide web, pages 507–517, 2016.
- [27]Dan Hendrycks and Kevin Gimpel. A baseline for detecting misclassified and out-of-distribution examples in neural networks. arXiv preprint arXiv:1610.02136, 2016.
- [28]Xiuyuan Hu, Xuege Hou, Guoqing Liu, Yang Zhao, Jieran Li, Dongbiao Sun, José Miguel Hernández-Lobato, Hao Zhang, and Xue Liu. Evaluating the diversity of ai-generated content with diversity profiles. arXiv preprint arXiv:2608.17731, 2026.arxiv.org/abs/2608.17731
- [29]Alex Krizhevsky, Geoffrey Hinton, et al. Learning multiple layers of features from tiny images. 2009.
- [30]Kimin Lee, Kibok Lee, Honglak Lee, and Jinwoo Shin. A simple unified framework for detecting out-of-distribution samples and adversarial attacks. Advances in neural information processing systems, 31, 2018.
- [31]Boyang Liu, Ding Wang, Kaixiang Lin, Pang-Ning Tan, and Jiayu Zhou. Rca: A deep collaborative autoencoder approach for anomaly detection. In IJCAI: proceedings of the conference, volume 2021, page 1505, 2021.
- [32]Fei Tony Liu, Kai Ming Ting, and Zhi-Hua Zhou. Isolation forest. In 2008 eighth ieee international conference on data mining, pages 413–422. IEEE, 2008.
- [33]Weitang Liu, Xiaoyun Wang, John Owens, and Yixuan Li. Energy-based out-of-distribution detection. Advances in neural information processing systems, 33:21464–21475, 2020.arxiv.org/abs/2010.03759
- [34]Xixi Liu, Yaroslava Lochman, and Christopher Zach. Gen: Pushing the limits of softmax-based out-of-distribution detection. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 23946–23955, 2023.
- [35]Yinhan Liu, Myle Ott, Naman Goyal, Jingfei Du, Mandar Joshi, Danqi Chen, Omer Levy, Mike Lewis, Luke Zettlemoyer, and Veselin Stoyanov. Roberta: A robustly optimized bert pretraining approach. arXiv preprint arXiv:1907.11692, 2019.
- [36]Victor Livernoche, Vineet Jain, Yashar Hezaveh, and Siamak Ravanbakhsh. On diffusion modeling for anomaly detection. In The Twelfth International Conference on Learning Representations, page 7999, 2024.
- [37]Jiangang Ma, Le Sun, Hua Wang, Yanchun Zhang, and Uwe Aickelin. Supervised anomaly detection in uncertain pseudoperiodic data streams. ACM Transactions on Internet Technology (TOIT), 16 (1):1–20, 2016.
- [38]Norman Mu and Justin Gilmer. Mnist-c: A robustness benchmark for computer vision. arXiv preprint arXiv:1906.02337, 2019.arxiv.org/abs/1906.02337
- [39]Maximilian Mueller and Matthias Hein. Mahalanobis++: Improving ood detection via feature normalization. arXiv preprint arXiv:2505.18032, 2025.
- [40]Ali Bou Nassif, Manar Abu Talib, Qassim Nasir, and Fatima Mohamad Dakalbab. Machine learning for anomaly detection: A systematic review. Ieee Access, 9:78658–78700, 2021.
- [41]Mehdi Neshat, Menasha Thilakaratne, Mohammed El-Abd, Seyedali Mirjalili, Amir H Gandomi, and John Boland. Smart buildings energy consumption forecasting using adaptive evolutionary bagging extra tree learning models. Energy, page 137130, 2025.DOI
- [42]Yuval Netzer, Tao Wang, Adam Coates, Alessandro Bissacco, Baolin Wu, Andrew Y Ng, et al. Reading digits in natural images with unsupervised feature learning. In NIPS workshop on deep learning and unsupervised feature learning, volume 2011, page 4. Granada, 2011.
- [44]Bjarke Frost Nielsen, Amey P Pasarkar, Qiqi Yang, Bryan T Grenfell, and Adji Bousso Dieng. Applications of the vendi score in genomic epidemiology. arXiv preprint arXiv:2509.22520, 2025.arxiv.org/abs/2509.22520
- [45]Guansong Pang, Chunhua Shen, Longbing Cao, and Anton Van Den Hengel. Deep learning for anomaly detection: A review. ACM computing surveys (CSUR), 54(2):1–38, 2021.DOI
- [46]Amey P Pasarkar and Adji Bousso Dieng. Cousins of the vendi score: A family of similarity-based diversity metrics for science and machine learning. arXiv preprint arXiv:2310.12952, 2023.arxiv.org/abs/2310.12952
- [47]Amey P Pasarkar and Adji Bousso Dieng. Vendi novelty scores for out-of-distribution detection. arXiv preprint arXiv:2602.10062, 2026.arxiv.org/abs/2602.10062
- [48]Fabian Pedregosa, Gaël Varoquaux, Alexandre Gramfort, Vincent Michel, Bertrand Thirion, Olivier Grisel, Mathieu Blondel, Peter Prettenhofer, Ron Weiss, Vincent Dubourg, et al. Scikit-learn: Machine learning in python. the Journal of machine Learning research, 12:2825–2830, 2011.arxiv.org/abs/1201.0490
- [49]Tomáš Pevný. Loda: Lightweight on-line detector of anomalies. Machine Learning, 102(2):275–304, 2016.DOI
- [50]Tal Reiss, Niv Cohen, Liron Bergman, and Yedid Hoshen. Panda: Adapting pretrained features for anomaly detection and segmentation. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 2806–2814, 2021.arxiv.org/abs/2010.05903
- [51]Oliver Rippel, Patrick Mertens, and Dorit Merhof. Modeling the distribution of normal data in pre-trained deep features for anomaly detection. In 2020 25th International Conference on Pattern Recognition (ICPR), pages 6726–6733. IEEE, 2021.arxiv.org/abs/2005.14140
- [52]Lukas Ruff, Robert Vandemeulen, Nico Goernitz, Lucas Deecke, Shoaib Ahmed Siddiqui, Alexander Binder, Emmanuel Müller, and Marius Kloft. Deep one-class classification. In International conference on machine learning, pages 4393–4402. PMLR, 2018.
- [53]Oriane Siméoni, Huy V. Vo, Maximilian Seitzer, Federico Baldassarre, Maxime Oquab, Cijo Jose, Vasil Khalidov, Marc Szafraniec, Seungeun Yi, Michaël Ramamonjisoa, Francisco Massa, Daniel Haziza, Luca Wehrstedt, Jianyuan Wang, Timothée Darcet, Théo Moutakanni, Leonel Sentana, Claire Roberts, Andrea Vedaldi, Jamie Tolan, John Brandt, Camille Couprie, Julien Mairal, Hervé Jégou, Patrick Labatut, and Piotr Bojanowski. DINOv3, 2025. URL https://arxiv.org/abs/2508.10104.
- [54]George Stein, Jesse Cresswell, Rasa Hosseinzadeh, Yi Sui, Brendan Ross, Valentin Villecroze, Zhaoyan Liu, Anthony L Caterini, Eric Taylor, and Gabriel Loaiza-Ganem. Exposing flaws of generative model evaluation metrics and their unfair treatment of diffusion models. Advances in Neural Information Processing Systems, 36:3732–3784, 2023.arxiv.org/abs/2306.04675
- [55]Yiyou Sun, Yifei Ming, Xiaojin Zhu, and Yixuan Li. Out-of-distribution detection with deep nearest neighbors. In International conference on machine learning, pages 20827–20840. PMLR, 2022.arxiv.org/abs/2204.06507
- [57]Shay Vargaftik, Isaac Keslassy, Ariel Orda, and Yaniv Ben-Itzhak. Rade: resource-efficient supervised anomaly detection using decision tree-based ensemble methods. Machine Learning, 110(10):2835–2866, 2021.arxiv.org/abs/1909.11877
- [58]Haoqi Wang, Zhizhong Li, Litong Feng, and Wayne Zhang. Vim: Out-of-distribution with virtual-logit matching. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 4921–4930, 2022.arxiv.org/abs/2203.10807
- [59]Hu Wang, Guansong Pang, Chunhua Shen, and Congbo Ma. Unsupervised representation learning by predicting random distances. arXiv preprint arXiv:1912.12186, 2019.arxiv.org/abs/1912.12186
- [60]Hongzuo Xu, Guansong Pang, Yijie Wang, and Yongjun Wang. Deep isolation forest for anomaly detection. IEEE transactions on knowledge and data engineering, 35(12):12591–12604, 2023a.DOI
- [61]Hongzuo Xu, Yijie Wang, Juhui Wei, Songlei Jian, Yizhou Li, and Ning Liu. Fascinating supervisory signals and where to find them: Deep anomaly detection with scale learning. In International Conference on Machine Learning, pages 38655–38673. PMLR, 2023b.arxiv.org/abs/2305.16114
- [62]Jingkang Yang, Pengyun Wang, Dejian Zou, Zitang Zhou, Kunyuan Ding, Wenxuan Peng, Haoqi Wang, Guangyao Chen, Bo Li, Yiyou Sun, et al. Openood: Benchmarking generalized out-of-distribution detection. Advances in Neural Information Processing Systems, 35:329598–32611, 2022.arxiv.org/abs/2210.07242
- [63]Yuming Yang, Yang Nan, Junjie Ye, Shihan Dou, Xiao Wang, Shuo Li, Huijie Lv, Tao Gui, Qi Zhang, and Xuan-Jing Huang. Measuring data diversity for instruction tuning: A systematic analysis and a reliable metric. In Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 18530–18549, 2025.
- [65]Xiang Zhang, Junbo Zhao, and Yann LeCun. Character-level convolutional networks for text classification. Advances in neural information processing systems, 28, 2015.arxiv.org/abs/1509.01626




