betaFeedback

WorksComputer Science

TripletAD: Weakly Supervised Anomaly Detection in Embedding Space with Rare Anomalies and Scarce Labels

Abstract

Many practical anomaly detection problems come with only a handful of confirmed anomalies, and the unreviewed remainder of the data cannot be trusted to be normal. We study this setting under a curated-label protocol, in which unconfirmed anomalies are removed from training rather than relabelled as normal, and propose TripletAD, a weakly supervised detector that learns a triplet-margin embedding and calibrates a linear scoring head with two soft penalties: a partial cross-entropy on labelled anomalies and a relative-contrast hinge that separates the mean scores of anomalous and normal samples. Because every mined triplet contains a labelled anomaly, the few labels are reused across the entire training set instead of being diluted by it. On 24 ADBench datasets whose inputs are pre-trained embeddings (text, image and speech vectors), TripletAD outperforms established state-of-the-art weakly supervised detectors (DevNet, PReNet and DeepSAD) and gradient-boosted trees when anomalies are rare. At a 1% anomaly rate it has the highest mean AUCPR of six methods, 20% above DevNet; on the naturally rare Speech dataset it doubles the best baseline (0.394 vs. 0.196 AUCPR); and on MVTec-AD it is best when only a quarter or half of the anomalies are labelled. Pooled over 70 rare- or sparse-label settings, it has the best average rank (2.21 of 6) and outperforms each baseline under Wilcoxon signed-rank tests with Holm correction. When anomalies are more frequent it performs on par with these methods. The method needs no per-dataset tuning and is insensitive to its loss weights and training length; on raw tabular features, gradient-boosted trees remain the stronger choice.

Introduction

Anomaly detection in industry is rarely unsupervised and rarely fully supervised. In fraud review, defect inspection, content moderation and security monitoring, analysts inspect a small fraction of flagged cases, confirm a few dozen anomalies, and leave the rest unreviewed. Two properties recur across these settings: anomalies are rare, and labelled anomalies are scarce.

The data behind these decisions is also changing. Across industry, text, images, audio and logs are increasingly encoded as embeddings by pre-trained models, and language-model and agentic systems now represent documents, transactions and interactions as contextual embeddings. Anomaly detection is therefore moving from hand-built tabular features into embedding space, and the need for detectors that work there, with rare anomalies and only a handful of confirmed cases, is growing and widespread.

The state of the art depends on the input. On raw tabular features, gradient-boosted trees such as CatBoost ([26]) remain the method to beat, ahead of deep detectors ([11, 12]). On embedding inputs, the established state of the art for this weakly supervised setting is the family of deep detectors DevNet ([23]), PReNet ([25]) and DeepSAD ([29]), designed for a few labelled anomalies and a large unlabelled pool. These methods usually treat the unlabelled pool as (mostly) normal, and when labelled anomalies become very rare, two things work against them: each labelled anomaly appears in only a small fraction of training batches unless it is explicitly oversampled, and the objective has few positive examples from which to shape the decision boundary. Rare anomalies on embedding inputs, the case that matters most in practice, is therefore where improvement is most needed.

We take a metric-learning view. TripletAD trains an encoder with a triplet-margin loss [30, 15] over triplets mined so that every triplet contains a labelled anomaly: an anomalous anchor is paired with an anomalous positive and a normal negative, and a normal anchor with a normal positive and an anomalous negative. A linear head on the learned embedding is then calibrated with two soft terms: a partial cross-entropy that only rewards high scores on labelled anomalies, and a relative-contrast hinge that pushes the mean anomaly score of anomalous samples above that of normal samples. We pair the method with a curated-label protocol that drops unconfirmed anomalies from training instead of passing them off as normal, which removes one source of label noise and makes the comparison between methods cleaner.

Our main finding is that, in the rare-anomaly regime on embedding inputs, TripletAD outperforms established state-of-the-art weakly supervised detectors (DevNet, PReNet and DeepSAD) as well as gradient-boosted trees, and that it matches them elsewhere on embeddings. With anomalies at 1% of the data, it has the highest mean AUCPR of the six methods we compare, 20% above DevNet and 53% above CatBoost. On the ADBench Speech dataset, whose anomalies are naturally rare (1.65%), it doubles the best baseline. At 2% it again leads. At the ADBench default of about 5% anomalies it performs on par with the other methods, and on MVTec-AD [2], where anomalies make up 13–33% of the data, it is on par with full labels and the best method once labels become scarce. On raw tabular data, our results confirm that gradient-boosted trees remain stronger, so we position TripletAD for learned representations.

Contributions.

  • TripletAD, a simple weakly supervised detector that combines triplet-margin representation learning with two soft calibration penalties on a linear head (§ 3).

  • A curated-label evaluation protocol for sparse labels, in which unlabelled anomalies are excluded from training, together with an anomaly-rarity sweep that varies the anomaly rate on fixed embedding datasets (§ 4).

  • An empirical study over 34 datasets and six methods, with paired significance tests, showing that TripletAD leads in the rare-anomaly regime on embedding inputs (§ 5).

  • Ablation and sensitivity analyses showing that the relative-contrast term is the key calibration component and that results are stable across loss weights and training lengths (§ 6).

Related work

Weakly supervised anomaly detection. Surveys by Chandola et al. [5] and Pang et al. [24] cover the broader field. DevNet [23] learns a scalar score with a deviation loss that pulls unlabelled samples towards a Gaussian reference and pushes labelled anomalies several standard deviations away, using batches balanced between labelled anomalies and unlabelled data. PReNet [25] instead learns a pairwise relation score over pairs of anomalies and unlabelled samples. DeepSAD [29] extends Deep SVDD [28] by mapping unlabelled data near a centre and labelled anomalies away from it. ADBench [12] benchmarks these and many other detectors and supplies the datasets and reference implementations we use; NLP-ADBench [22] extends the comparison to text. These methods treat the unlabelled pool as approximately normal. The curated protocol we use is related in spirit to positive-unlabelled learning [9, 1], where treating unlabelled data as negative is known to bias the learner; we sidestep the issue by training only on confirmed anomalies and confirmed normals.

Metric and contrastive learning. Siamese networks ([4, 20]) and triplet networks ([15, 30, 14]) learn embeddings in which distance reflects class membership, and supervised contrastive learning ([18]) generalises the idea to many positives per anchor. Siamese networks have also been applied to weakly supervised anomaly detection in industry, for example to flag grey-market transactions ([32]). There, pairs of records are labelled similar or dissimilar, with unlabelled records treated as normal, and a new record is classified by its pairwise similarity to a reference set of labelled anomalies. TripletAD differs in three ways aimed at detection accuracy. First, each training triplet compares an anchor against both an anomaly and a normal sample, so the embedding learns a relative ordering rather than isolated pairwise similarities. Second, the curated protocol excludes unconfirmed anomalies instead of treating unlabelled data as normal, which removes a source of label noise. Third, a scoring head is calibrated directly for ranking anomalies above normal samples, so the model outputs an anomaly score rather than a similarity to be thresholded. As a side benefit, scoring requires a single forward pass and no reference set. In anomaly detection, triplet or contrastive objectives have otherwise mostly been used in self-supervised form. TriAD ([31]) uses tri-domain contrastive learning for time-series anomalies; TripleAD ([33]) distils three channels for graph anomalies; and MulSen-TripleAD ([21]) fuses three sensor modalities for object inspection. Despite the similar names, none of these use labelled anomalies inside the triplet or calibrate a score head as we do. TripletAD uses the triplet loss as a supervised representation learner in which the scarce anomalies appear in every triplet.

Trees versus deep models on tabular data. Grinsztajn et al. [11] and Gorishniy et al. [10] show that tree ensembles remain strong on tabular data with heterogeneous, individually informative features. Our results on raw tabular anomaly datasets agree. Our results on embedding inputs, where information is spread across many weakly informative coordinates, point the other way.

Method

Setting

We observe a training set D={(xi,yi)}i=1N\mathcal{D} = \{(x_i, y_i)\}_{i=1}^{N} with xi∈RDx_i \in\mathbb{R}^{D} and yi∈{0,1}y_i \in\{0,1\}, where yi=1y_i = 1 marks a confirmed anomaly and yi=0y_i = 0 a normal sample. Confirmed anomalies are few (∣A∣≪N|\mathcal{A}| \ll N, with A={i:yi=1}\mathcal{A} = \{i : y_i = 1\}) and, in the sparse-label experiments, form a random subset of all anomalies in the data. Unconfirmed anomalies are not in D\mathcal{D} (§ 4). The goal is a score s(x)∈[0,1]s(x) \in[0,1] that ranks anomalies above normal samples at test time.

Architecture

TripletAD has an encoder fθ:RD→Rdf_\theta: \mathbb{R}^{D} \to\mathbb{R}^{d} and a linear scoring head:

z=fθ(x),s(x)=σ(w⊤z+b),(1)z = f_\theta(x), \qquad s(x) = \sigma(w^\top z + b), \tag*{(1)}

where σ\sigma is the logistic function. In all experiments fθf_\theta is a multilayer perceptron with hidden widths 128 and 64, batch normalisation ([17]) and ReLU activations, and an embedding size of d=32d = 32. When the input is itself a pre-trained embedding (BERT, ResNet-18 or i-vector features), fθf_\theta learns a task-specific 32-dimensional embedding on top of it. The pre-trained feature extractor is not fine-tuned.

Anomaly-anchored triplet mining

Each epoch uses every training sample once as an anchor xax_a. For each anchor we draw a positive xpx_p from the same class (excluding the anchor itself) and a negative xnx_n from the opposite class, both uniformly at random:

  • anomalous anchor (ya=1y_a = 1): xp∈Ax_p \in\mathcal{A}, xn∉Ax_n \notin\mathcal{A};

  • normal anchor (ya=0y_a = 0): xp∉Ax_p \notin\mathcal{A}, xn∈Ax_n \in\mathcal{A}.

By construction, every triplet contains at least one labelled anomaly. With NN samples and ∣A∣|\mathcal{A}| anomalies, each anomaly appears in roughly N/∣A∣N/|\mathcal{A}| triplets per epoch, so a dataset with 1% anomalies reuses each label about a hundred times per epoch without explicit oversampling or batch balancing. Mining is O(N)O(N) per epoch.

Objective

For a batch of triplets, TripletAD minimises

J=Ltrip+λ1LPCE+λ2LRC.(2)J = \mathcal{L}_{\mathrm{trip}} + \lambda_1\mathcal{L}_{\mathrm{PCE}} + \lambda_2\mathcal{L}_{\mathrm{RC}}. \tag*{(2)}

Triplet loss. With z⋆=fθ(x⋆)z_\star= f_\theta(x_\star) and Euclidean distance,

Ltrip=1B∑t=1Bmax⁡(0,∥za(t)−zp(t)∥2−∥za(t)−zn(t)∥2+m),(3)\mathcal{L}_{\mathrm{trip}} = \frac{1}{B}\sum_{t=1}^{B}\max\left(0,\left\lVert z_a^{(t)}-z_p^{(t)}\right\rVert_2-\left\lVert z_a^{(t)}-z_n^{(t)}\right\rVert_2+m\right), \tag*{(3)}

with margin m=1m = 1. The term clusters anomalies with anomalies and normals with normals in embedding space.

Partial cross-entropy (PCE). The head is rewarded for assigning high scores to labelled anomalous anchors:

LPCE=−1B∑t=1B1[ya(t)=1]log⁡s(xa(t)).(4)\mathcal{L}_{\mathrm{PCE}}=-\frac{1}{B}\sum_{t=1}^{B}\mathbb{1}\left[y_a^{(t)}=1\right]\log s\left(x_a^{(t)}\right). \tag*{(4)}

The loss is “partial” because normal anchors contribute nothing. In deployment the normal pool may still hide anomalies, so the head is never pushed to drive any particular normal sample’s score to zero. The average runs over all BB triplets, so the term’s weight scales with the anomaly rate. We found that averaging over anomalous anchors only was never better and was less stable (Appendix § 10).

Relative contrast (RC). Let S+S_+ and S−S_- be the anomalous and normal members of the positives and negatives in the batch, {xp(t),xn(t)}t=1B\left\{x_p^{(t)},x_n^{(t)}\right\}_{t=1}^{B}. Under our mining rule each triplet contributes exactly one sample to each set. RC is a hinge on the gap between their mean scores:

LRC=max⁡(0,1−(1∣S+∣∑x∈S+s(x)−1∣S−∣∑x∈S−s(x))).(5)\mathcal{L}_{\mathrm{RC}}=\max\left(0,1-\left(\frac{1}{|S_+|}\sum_{x\in S_+}s(x)-\frac{1}{|S_-|}\sum_{x\in S_-}s(x)\right)\right). \tag*{(5)}

RC is a soft, batch-level ranking constraint. It asks the head to separate the two groups on average rather than to classify each sample. It also supplies the negative-side signal that PCE deliberately omits.

Training

Training has two stages (Algorithm 1). Stage 1 pre-trains the encoder with Ltrip\mathcal{L}_{\mathrm{trip}} alone for 50 epochs, during which the head receives no gradient. Stage 2 optimises the full objective (2) for 100 epochs. Each batch stacks its anchors, positives and negatives into a single forward pass, so batch normalisation sees all three roles. We use Adam (Kingma & Ba, [19]) with learning rate 10−310^{-3}, batches of 256 triplets, no weight decay, and the defaults λ1=1\lambda_1=1 and λ2=0.25\lambda_2=0.25 on every dataset. No hyperparameter is tuned per dataset. The anomaly score at test time is s(x)s(x) from Eq. (1).

Experimental setup

Datasets. All datasets come from ADBench (Han et al., [12]); Table 1 lists them. Our focus is the 24 datasets whose inputs are pre-trained embeddings: five text corpora encoded with BERT (Devlin et al., [8]) (20News, AGNews, Amazon, IMDB, Yelp), three image datasets encoded with ResNet-18 (He et al., [13]) (CIFAR-10, FashionMNIST, SVHN), the 15 MVTec-AD categories (Bergmann et al., [2]) encoded with ResNet-18, and Speech, whose inputs are 400-dimensional i-vectors (Rayana, [27]). For the text and image

DatasetNNDDAnomaliesRate (%)Best 1-feature AUROC
Embedding (text: BERT; image: ResNet-18)
20News3,0907681544.980.708
AGNews10,0007685005.000.730
Amazon10,0007685005.000.713
IMDB10,0007685005.000.698
Yelp10,0007685005.000.753
CIFAR-105,2635122635.000.782
FashionMNIST6,3155123154.990.779
SVHN5,2085122604.990.723
Embedding (speech: i-vectors)
Speech3,686400611.650.674
Raw tabular
Backdoor95,3291962,3292.440.887
Cover286,048102,7470.960.999
Fraud284,807294920.170.933
HTTP567,49832,2110.391.000
Waveform3,443211002.900.873
Yeast1,484850734.160.592
ALOI49,534271,5083.040.569
Campaign41,188624,64011.270.818
CelebA202,599394,5472.240.793
Census299,28550018,5686.200.787
Embedding (MVTec-AD, ResNet-18), 15 categories
range102–50151230–14112.8–32.50.601–0.996

Table 1. Datasets. “Best 1-feature AUROC” is the highest AUROC obtained by ranking on a single input coordinate (either sign).

Algorithm 1 TripletAD training
Require: curated training set $\mathcal{D}$, epochs $E_1 = 50$, $E_2 = 100$, weights $\lambda_1,\lambda_2$
1: for stage $\in \{1,2\}$ do
2:   for epoch $= 1,\ldots,E_{\mathrm{stage}}$ do
3:     Mine one triplet per anchor $x_a \in \mathcal{D}$ (anomaly-anchored rule); shuffle into batches of 256
4:     for each batch do
5:       $J \leftarrow \mathcal{L}_{\mathrm{trip}}$ if stage $= 1$ else $\mathcal{L}_{\mathrm{trip}} + \lambda_1\mathcal{L}_{\mathrm{PCE}} + \lambda_2\mathcal{L}_{\mathrm{RC}}$
6:       Update $\theta$ (and $w,b$ in stage 2) with Adam
7:     return $s(x) = \sigma(w^{\mathsf{T}}f_\theta(x) + b)$

datasets we use ADBench’s first class split (class 0 is the anomaly class). To delimit the method’s scope, we also evaluate ten raw tabular datasets. The last column of Table 1 reports the best AUROC achievable by thresholding any single input feature. Embedding coordinates are individually weak (0.67–0.78), while most tabular datasets contain at least one strong feature.

Splits and preprocessing. For each of five seeds we make a stratified 70/30 train/test split and fit min–max scaling on the training part only. Every method sees exactly the same split, scaling and labels for a given seed. We report AUCPR (average precision), which is more informative than AUROC under heavy class imbalance [6], averaged over the five seeds.

Curated sparse labels. To emulate a review queue, a label ratio ρ\rho keeps a random ⌊ρ⋅∣Atrain∣⌋\lfloor\rho\cdot|A_{\mathrm{train}}| \rfloor of the training anomalies as labelled. The remaining training anomalies are removed from the training set rather than relabelled as normal. The test set is never modified. We skip a (dataset, ρ\rho) cell when fewer than ten anomalies would remain labelled, so cells differ slightly in which datasets they contain (noted in each table). All baselines are trained on the same curated sets.

Anomaly-rarity sweep. To separate the effect of anomaly rarity from that of dataset identity, we subsample the anomalies of each of the eight text and image datasets, keeping all normal samples, so that anomalies make up 2% or 1% of the data. The subsample is redrawn for each of the five seeds, so the reported variance includes which anomalies were kept. Appendix A repeats the sweep with a single fixed subsample and reaches the same conclusion. At 1% the training sets contain 21 (20News) to 67 (AGNews, Amazon, IMDB, Yelp) labelled anomalies, comparable to Speech (43).

Baselines. We compare against two tree ensembles and three deep weakly supervised detectors, all trained on identical inputs:

  • random forest [3]: scikit-learn, 100 trees;

  • CatBoost [26]: library defaults;

  • DevNet [23]: our re-implementation following the original paper, with one hidden layer of 20 units, deviation loss with margin 5, and balanced batches of 512;

  • PReNet [25] and DeepSAD [29]: ADBench’s reference implementations at their default hyperparameters.

No method, including ours, is tuned per dataset. ADBench’s DeepSAD fails on a few very small MVTec training sets (its data loader yields zero batches). Those cells are omitted for DeepSAD and marked in the tables.

Statistics. Following Demšar [7], the unit of analysis is a (dataset, setting) cell, with AUCPR averaged over seeds. We compare TripletAD with each baseline using a two-sided Wilcoxon signed-rank test over paired cells and correct the five resulting pp-values with the Holm procedure [16]. We also report average ranks and a Friedman test over cells where all six methods have a result.

Compute. All runs use a CPU with one thread per process; no GPU is needed.

Results on embedding inputs, from rare to frequent anomalies

We organise the embedding results by anomaly rate: rare (≤ 2%), moderate (about 5%) and frequent (> 10%). All six methods, including the tree ensembles, are compared in every setting.

Rare anomalies (2%): TripletAD outperforms the established state of the art

1% anomalies. With anomalies at 1% of the data (Table 2, Figure 1), TripletAD has the highest mean AUCPR of the six methods (0.244), ahead of DeepSAD (0.210), PReNet (0.207), DevNet (0.204), CatBoost (0.159) and random forest (0.118). That is 20% above DevNet and 53% above CatBoost. It is the best method, or tied for best, on five of the eight datasets, and its average rank is 1.88. The text datasets are where it is strongest: it matches or beats DevNet and PReNet on all five, and on AGNews it nearly doubles them (0.325 against 0.175 and 0.171).

DatasetTripletADDeepSADDevNetPReNetCatBoostRF
Anomaly rate 1%
20News0.3410.2120.3410.3290.1520.106
AGNews0.3250.2620.1750.1710.1590.082
Amazon0.1110.0950.0990.1070.0570.023
IMDB0.1260.1180.0950.0810.0690.022
Yelp0.1810.1960.1510.1500.1170.047
CIFAR-100.1350.1300.1790.1860.1420.116
FashionMNIST0.5750.5730.4110.4310.4900.491
SVHN0.1540.0920.1830.2030.0890.061
Average0.2440.2100.2040.2070.1590.118
Anomaly rate 2%
20News0.4410.3960.4630.4470.3670.250
AGNews0.3320.3920.3040.3000.2930.169
Amazon0.2400.1690.1580.1520.1420.048
IMDB0.1970.1820.1680.1610.1360.061
Yelp0.3190.3640.2940.2880.2610.099
CIFAR-100.2810.2170.2950.2870.2500.191
FashionMNIST0.6600.6210.6030.6000.6120.575
SVHN0.2950.2660.3760.3530.2470.173
Average0.3460.3260.3330.3240.2890.196
Anomaly rate 5%
20News0.5790.6230.6230.6180.5860.449
AGNews0.6080.6200.4620.4530.5410.339
Amazon0.3240.3460.3350.3200.3330.161
IMDB0.3400.4360.3040.2920.3350.183
Yelp0.5150.5640.4880.4620.4990.276
CIFAR-100.4680.3970.4690.4870.4230.359
FashionMNIST0.7720.7750.7520.7690.7710.754
SVHN0.5540.5150.5810.5930.5120.401
Average0.5200.5350.5020.4990.5000.365

Table 2. Anomaly-rarity sweep: AUCPR (mean of five seeds, with the anomaly subsample redrawn for each seed) when anomalies make up 1%, 2% or 5% (original) of each dataset. Best per row in bold.

Figure 1. Anomaly-rarity sweep on eight text and image embedding datasets. Left: mean AUCPR over datasets and five seeds as the anomaly rate falls from 5% to 1%. Right: the same, divided by DevNet’s score. TripletAD’s lead grows to 20% over DevNet at 1%, while the tree ensembles fall further behind as anomalies become rarer.

Speech, 1.65% anomalies. Speech is the ADBench dataset closest to the target setting: anomalies are naturally rare, each training split holds only 43 of them, and the i-vector coordinates are individually weak. Here TripletAD’s advantage is largest (Table 3). With all labels it reaches 0.394 AUCPR, twice the best baseline (CatBoost, 0.196) and more than twice DevNet (0.178) and PReNet (0.165). The lead holds as labels are removed: 0.317 at 75% and 0.252 at 50%, against at most 0.181 and 0.163 for any baseline. Only at 25% of labels, ten labelled anomalies in all, are PReNet and DevNet slightly ahead.

Labelled anomaliesTripletADDeepSADDevNetPReNetCatBoostRF
100%0.3940.0730.1780.1650.1960.070
75%0.3170.0600.1800.1810.1520.068
50%0.2520.0500.1570.1630.1130.071
25%0.0930.0320.1060.1090.0530.036

Table 3. Speech (1.65% anomalies): AUCPR as the fraction of labelled training anomalies varies under the curated protocol.

2% anomalies. At 2%, TripletAD again has the highest mean AUCPR (0.346, against 0.333 for DevNet, 0.326 for DeepSAD and 0.289 for CatBoost) and the best average rank (2.00).

Moderate anomaly rate (about 5%): on par

At the ADBench default of about 5% anomalies (bottom block of Table 2), the six methods are close: TripletAD (0.520) performs on par with DevNet (0.502), PReNet (0.499) and CatBoost (0.500), with DeepSAD slightly ahead (0.535). Removing labels at this rate (Appendix § 9, Table 9) keeps TripletAD on par with DevNet and PReNet and well ahead of DeepSAD and the trees (20 or more wins out of 21 cells).

Labelled anomaliesTripletADDeepSADDevNetPReNetCatBoostRF
25% (8 sets)0.4450.4020.4480.4390.3690.258
10% (8 sets)0.3570.2770.3580.3530.2490.194
5% (5 sets)0.3190.2400.3340.3260.2320.183
TripletAD wins–20/219/2110/2120/2121/21

Table 9. Eight text and image embedding datasets at their original 5% anomaly rate, with curated sparse labels: mean AUCPR. Cells with fewer than ten labelled anomalies are skipped, so the 5% row covers five datasets.

Read from 5% down to 1%, every method loses AUCPR as anomalies thin out, but the baselines lose it faster, so TripletAD’s advantage opens up exactly in the rare regime that motivates this work. Over the 24 cells of the sweep it beats CatBoost on 21 and random forest on all 24 (Holm $p < 10^{-4}$), and it beats DeepSAD,

DevNet and PReNet on 14–16 of the 24. Those three comparisons become significant in the pooled analysis of § 5.4.

Frequent anomalies (>10%> 10\%): best when labels are scarce

The 15 MVTec-AD categories have 13–33% anomalies (Table 4). With most or all anomalies labelled, the methods converge: CatBoost, DevNet and TripletAD are within 0.013 of each other. Once labels become scarce, TripletAD is the best method. With a quarter of the anomalies labelled, its mean AUCPR is 0.704, ahead of CatBoost (0.698), PReNet (0.683), DevNet (0.671) and DeepSAD (0.625). With half labelled it again leads (0.734). Over the 27 category cells at 25% and 50% labels where all six methods ran, it has the best average rank (2.48).

Labels (categories)TripletADDeepSADDevNetPReNetCatBoostRFTripletAD best
25% (13)0.7040.6250.6710.6830.6980.6916/13
50% (15)0.7340.644†0.7140.7100.7320.7196/15
75% (15)0.7420.665†0.7350.7260.7480.7337/15
100% (15)0.7450.676†0.7490.7370.7580.7424/15

Table 4. MVTec-AD (ResNet-18 embeddings, 13–33% anomalies): mean AUCPR over categories under curated sparse labels, and the number of categories on which TripletAD is the best of the six methods. †DeepSAD mean over the categories where it ran.

Statistical summary

Table 5 pools every rare- or sparse-label embedding cell: the 2% and 1% rarity cells, the high-dimensional sparse-label cells, MVTec-AD at 25% and 50% labels, and Speech, for 70 cells in total. TripletAD has the best average rank (2.21, against 2.84 for DevNet and 3.04 for PReNet; Friedman p=2×10−18p = 2 \times10^{-18}). It also outperforms every baseline individually after Holm correction, by 0.021–0.023 AUCPR on average over DevNet and PReNet and by 0.05–0.10 over DeepSAD and the trees.

TripletAD vs.CellsW / LMean AUCPR gainHolm pp
DeepSAD6862 / 6+0.0712×10−102 \times10^{-10}
Random forest6957 / 12+0.1004×10−94 \times10^{-9}
CatBoost6954 / 15+0.0536×10−76 \times10^{-7}
DevNet6944 / 24+0.0210.020
PReNet6944 / 24+0.0230.015

Table 5. Pooled comparison over 70 rare- or sparse-label embedding cells (AUCPR averaged over five seeds per cell). W/L: cells where TripletAD is better/worse. pp: two-sided Wilcoxon signed-rank, Holm-corrected over the five baselines. A few cells are missing for DeepSAD; one DevNet and one PReNet cell are tied. Average rank over 68 complete cells (1 = best): TripletAD 2.21, DevNet 2.84, PReNet 3.04, CatBoost 3.84, DeepSAD 4.22, RF 4.85.

Cost. TripletAD is also cheap to train. On the eight text and image datasets with full labels, one fit plus scoring takes 28 s on a single CPU thread on average, against 33 s for PReNet, 39 s for DeepSAD, 41 s for DevNet and 130 s for CatBoost (random forest: 23 s). Scoring a new sample is a single forward pass.

Ablation and sensitivity

Components. Table 6 removes each component on Speech and on two tabular datasets where the effect is easy to measure. Removing RC is the most damaging: Fraud drops from 0.773 to 0.514 and Cover becomes unstable (0.847 ± 0.237). On Speech, removing RC costs only 0.014, so its importance varies by dataset. Removing PCE or the triplet pre-training stage costs at most 0.014 anywhere. With neither calibration term, the head is never trained and the scores collapse, which confirms that the embedding alone is not a detector. RC therefore carries most of the calibration, and PCE and pre-training are small, consistent refinements.

VariantSpeechFraudCover
Full objective0.394 ± 0.0830.773 ± 0.0400.995 ± 0.001
w/o PCE (λ1=0\lambda_{1}=0)0.391 ± 0.0860.761 ± 0.0480.994 ± 0.004
w/o RC (λ2=0\lambda_{2}=0)0.380 ± 0.0730.514 ± 0.0740.847 ± 0.237
Triplet only0.129 ± 0.1150.157 ± 0.3420.072 ± 0.150
w/o triplet pretraining0.380 ± 0.0750.762 ± 0.0440.990 ± 0.005

Table 6. Ablation: AUCPR (mean ± s.d. over five seeds), full labels.

Loss weights and training length. Figure 2 (left) shows mean test AUCPR on nine embedding datasets (the eight text and image datasets plus Speech) over a grid of λ1∈{0.1,0.5,1}\lambda_{1} \in\{0.1, 0.5, 1\} and λ2∈{0.05,0.1,0.25,0.5}\lambda_{2} \in\{0.05, 0.1, 0.25, 0.5\}, with three seeds each. All twelve settings fall between 0.477 and 0.495. The default (1, 0.25) scores 0.483, within 0.012 of the best setting (0.1, 0.25). We keep the default to avoid tuning on the evaluation datasets. Figure 2 (right) shows test AUCPR as stage-2 training runs from 10 to 200 epochs on four datasets with different label ratios. The curves are flat beyond about 50 epochs, with a slight decline at 200. TripletAD needs no early stopping or validation-based tuning in our experiments.

Figure 2. Sensitivity. Left: mean test AUCPR over nine embedding datasets for each (λ1\lambda_1, λ2\lambda_2); the colour scale spans ±0.05 around the mean so the small spread is shown at true scale; the red box is the default. Right: test AUCPR against the number of joint-training epochs (dashed line: default of 100). The percentage after each dataset name is its label ratio.

Scope and limitations

  • Learned representations. TripletAD is designed for embedding inputs. On raw tabular features, gradient-boosted trees remain the stronger choice (Appendix § 9, Table 10), consistent with prior work [11]; among deep detectors, TripletAD still outperforms DevNet on seven of ten tabular datasets.

DatasetTripletADDeepSADDevNetPReNetCatBoostRF
Backdoor0.974–0.882–0.9980.996
Cover0.995–0.956–1.0000.999
Fraud0.773–0.743–0.8660.849
HTTP0.998–0.997–0.9990.999
Waveform0.5050.5380.2090.2340.6050.550
Yeast0.5880.5360.4870.4900.6620.655
ALOI0.2050.1450.0460.0480.4200.538
Campaign0.4500.5860.5030.5160.6620.636
CelebA0.2500.4350.3370.3230.4660.390
Census0.4150.6030.5390.5290.6970.619

Table 10. Raw tabular datasets with full labels: AUCPR (mean of five seeds). PReNet and DeepSAD were not run on Backdoor, Cover, Fraud and HTTP.

  • Image embeddings at 1%. On CIFAR-10 and SVHN at 1% anomalies, PReNet and DevNet remain ahead. Understanding which embedding geometries favour each approach is future work.

  • Synthetic rarity. The 2% and 1% datasets are made by subsampling anomalies from existing benchmarks. Naturally rare embedding datasets, such as Speech, show the same pattern, but there are few of them in public benchmarks.

  • Frozen feature extractor. TripletAD learns its embedding on top of fixed pre-trained features. Training the triplet objective end to end through the feature extractor may help further and is left for future work.

  • Default hyperparameters for all methods. No method is tuned per dataset. The λ\lambda study suggests TripletAD would gain little from tuning.

  • Small cells. At 1% anomalies and on small MVTec categories, test sets contain few anomalies (9–29 at 1%; 9–43 on MVTec-AD), so per-cell AUCPR is noisy. Our conclusions rest on pooled, paired tests rather than individual cells.

Conclusion

TripletAD combines triplet-margin representation learning, anomaly-anchored mining and two soft calibration penalties into a weakly supervised detector with no per-dataset tuning. On pre-trained embedding inputs it outperforms established state-of-the-art weakly supervised detectors and tree ensembles when anomalies are rare: at a 1% anomaly rate it is the strongest of the six methods we compare, and pooled over 70 rare-or sparse-label settings it ranks first, with significant gains over each baseline. At higher anomaly rates it performs on par with these methods, and on raw tabular data gradient-boosted trees remain the stronger choice. A natural next step is to apply TripletAD to embeddings of language-model outputs and agent interaction traces, for example to flag anomalous agent behaviour. We hope the curated-label protocol and the rarity sweep will be useful for evaluating weakly supervised detectors in the regime practitioners actually face: few labels, rare anomalies, and an unreviewed pool that cannot be assumed clean.

Reproducibility. Code, configurations and per-run results for every table and figure are available from the authors upon request. All datasets are public through ADBench.

Per-dataset results

Additional studies

Label contamination. We also tested robustness to mislabelled training anomalies, where a fraction of the training anomalies is flipped to normal, on Fraud with full labels. AUPCR (mean ±\pm s.d., five seeds) degrades gracefully:

0%1%5%10%
0.773±0.0400.773 \pm0.0400.766±0.0320.766 \pm0.0320.766±0.0370.766 \pm0.0370.729±0.0540.729 \pm0.054

Table 13.

DatasetTripletADDeepSADDevNetPReNetCatBoostRF
20News0.5790.6230.6230.6180.5860.449
AGNews0.6080.6200.4620.4530.5410.339
Amazon0.3240.3460.3350.3200.3330.161
IMDB0.3400.4360.3040.2920.3350.183
Yelp0.5150.5640.4880.4620.4990.276
CIFAR-100.4680.3970.4690.4870.4230.359
FashionMNIST0.7720.7750.7520.7690.7710.754
SVHN0.5540.5150.5810.5930.5120.401

Table 7. Text and image embedding datasets at the original 5% anomaly rate with full labels: AUCPR (mean of five seeds).

DatasetTripletADDeepSADDevNetPReNetCatBoostRF
Anomaly rate 1%
20News0.2870.2180.2300.2490.1520.103
AGNews0.2010.1950.1280.1260.1030.070
Amazon0.1240.1140.0810.0740.0690.021
IMDB0.1750.1910.1040.0800.0670.031
Yelp0.2900.2540.1900.1680.1220.043
CIFAR-100.1130.1380.1430.1480.1900.145
FashionMNIST0.5340.5240.4080.4240.5020.530
SVHN0.1190.0690.1630.1590.0810.051
Average0.2300.2130.1810.1780.1610.124
Anomaly rate 2%
20News0.4670.5450.5430.5530.4460.346
AGNews0.4120.4350.2660.2760.3250.168
Amazon0.2100.2020.1460.1390.1390.055
IMDB0.2190.2220.1420.1360.1580.053
Yelp0.2740.2780.2770.2560.2150.097
CIFAR-100.2290.1840.2660.2870.2130.169
FashionMNIST0.6820.6800.6310.6250.6570.633
SVHN0.3100.1910.3050.3120.2080.145
Average0.3500.3420.3220.3230.2950.208
Anomaly rate 5%
20News0.5790.6230.6230.6180.5860.449
AGNews0.6080.6200.4620.4530.5410.339
Amazon0.3240.3460.3350.3200.3330.161
IMDB0.3400.4360.3040.2920.3350.183
Yelp0.5150.5640.4880.4620.4990.276
CIFAR-100.4680.3970.4690.4870.4230.359
FashionMNIST0.7720.7750.7520.7690.7710.754
SVHN0.5540.5150.5810.5930.5120.401
Average0.5200.5350.5020.4990.5000.365

Table 8. Anomaly-rarity sweep with a single fixed anomaly subsample (drawn once with seed 0 and reused for all five split seeds). The conclusions match Table 2: TripletAD has the best mean at 2% (0.350) and 1% (0.230), and its lead over DevNet grows from 4% to 27%.

PCE averaging. Averaging LPCE\mathcal{L}_{\mathrm{PCE}} over anomalous anchors only, instead of over all triplets, was never better than the default on the datasets we tried and was unstable on Backdoor. We keep averaging over all triplets.

DatasetLabelsTripletADDeepSADDevNetPReNetCatBoostRF
20News25%0.4680.4160.5440.5110.3000.264
AGNews25%0.5230.5270.4630.4450.4470.223
Amazon25%0.3100.2440.3140.3030.2450.109
IMDB25%0.3090.3050.2800.2590.2450.113
Yelp25%0.4860.4540.4680.4480.3760.185
CIFAR-1025%0.3180.2540.3430.3620.3190.247
FashionMNIST25%0.7190.7040.6870.6980.6760.656
SVHN25%0.4300.3140.4860.4830.3450.264
20News10%0.3790.1970.3580.3540.1790.166
AGNews10%0.4460.3590.3480.3250.2560.145
Amazon10%0.2420.1480.2580.2460.1450.078
IMDB10%0.2210.1920.2140.2030.1640.075
Yelp10%0.3960.3180.4300.4160.2340.148
CIFAR-1010%0.1880.1840.2230.2250.1650.158
FashionMNIST10%0.6370.6110.6270.6320.6000.583
SVHN10%0.3480.2040.4090.4240.2490.200
AGNews5%0.3020.1930.3200.3110.2100.126
Amazon5%0.1840.0970.2070.1960.1230.068
IMDB5%0.2020.0890.1970.1870.1240.063
Yelp5%0.3000.2260.3250.3100.1650.098
FashionMNIST5%0.6080.5970.6240.6280.5400.562

Table 11. Text and image embedding datasets with curated sparse labels: AUCPR (mean of five seeds).

CategoryLabelsTripletADDeepSADDevNetPReNetCatBoostRF
bottle100%0.9470.9480.9510.9300.9580.947
bottle50%0.9630.9530.9270.9420.9540.949
bottle25%0.9470.9480.9200.9200.9480.948
cable100%0.5340.5450.5400.5550.5430.571
cable50%0.5820.4920.4970.4880.5260.539
cable25%0.5680.4950.4520.4550.5480.562
capsule100%0.7430.7040.6610.6590.6750.669
capsule50%0.7240.6760.6280.6290.6980.671
capsule25%0.7080.6420.5620.6080.7220.705
carpet100%0.6620.6390.7350.7080.7370.698
carpet50%0.6510.6020.7110.6750.7140.687
carpet25%0.6440.5750.7100.6870.7050.667
grid100%0.7180.5500.7080.6730.6870.678
grid50%0.6590.5220.6870.6550.6480.624
grid25%0.5810.4960.5520.5700.5650.546
hazelnut100%0.6370.6410.6890.6860.7810.752
hazelnut50%0.6400.6150.6380.6400.7460.729
hazelnut25%0.6200.5530.5890.6400.7150.683
leather100%0.9700.9380.9950.9510.9920.992
leather50%0.9200.9430.9930.9730.9920.990
leather25%0.9450.9360.9920.9910.9900.985
metal nut100%0.7660.5780.7520.7470.7260.681
metal nut50%0.7160.5600.7160.7270.6680.631
metal nut25%0.7120.4980.6880.7050.6070.592
pill100%0.5970.5840.6370.6280.6010.578
pill50%0.6240.5470.6110.6110.6100.604
pill25%0.5560.5270.5930.5860.5840.570
screw100%0.5980.4930.6020.5940.5590.512
screw50%0.5870.4650.5540.5520.4680.472
screw25%0.4720.4240.4570.4640.3870.388
tile100%0.7860.7330.7960.7980.7900.783
tile50%0.7820.7370.7660.7740.7730.790
tile25%0.7310.6920.6530.6790.7510.769
toothbrush100%0.892–0.8900.8910.8800.879
toothbrush50%0.823–0.8170.8050.8360.821
transistor100%0.5860.5380.5970.5870.7160.698
transistor50%0.6140.4690.5420.5710.6670.640
wood100%0.7840.6920.7160.6940.7810.796
wood50%0.7560.6660.6640.6580.7720.761
wood25%0.7260.6220.6200.6360.6640.707
zipper100%0.9520.8830.9660.9520.9400.898
zipper50%0.9640.7670.9560.9450.9090.873
zipper25%0.9370.7210.9310.9330.8800.854

Table 12. MVTec-AD per category: AUCPR (mean of five seeds). “–”: DeepSAD’s data loader failed.

References

  1. [1]Jessa Bekker and Jesse Davis. Learning from positive and unlabeled data: A survey. Machine Learning, 109 (4):719–760, 2020.DOI
  2. [2]Paul Bergmann, Michael Fauser, David Sattleger, and Carsten Steger. MVTec AD – a comprehensive real-world dataset for unsupervised anomaly detection. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 9592–9600, 2019.DOI
  3. [3]Leo Breiman. Random forests. Machine Learning, 45(1):5–32, 2001.
  4. [4]Jane Bromley, Isabelle Guyon, Yann LeCun, Eduard Säckinger, and Roopak Shah. Signature verification using a “Siamese” time delay neural network. In Advances in Neural Information Processing Systems, volume 6, 1993.
  5. [5]Varun Chandola, Arindam Banerjee, and Vipin Kumar. Anomaly detection: A survey. ACM Computing Surveys, 41(3):1–58, 2009.
  6. [6]Jesse Davis and Mark Goadrich. The relationship between precision-recall and ROC curves. In Proceedings of the 23rd International Conference on Machine Learning, pp. 233–240, 2006.DOI
  7. [7]Janez Demšar. Statistical comparisons of classifiers over multiple data sets. Journal of Machine Learning Research, 7:1–30, 2006.DOI
  8. [8]Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. BERT: Pre-training of deep bidirectional transformers for language understanding. In Proceedings of NAACL-HLT, pp. 4171–4186, 2019.arxiv.org/abs/1810.04805
  9. [9]Charles Elkan and Keith Noto. Learning classifiers from only positive and unlabeled data. In Proceedings of the 14th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, pp. 213–220, 2008.DOI
  10. [10]Yury Gorishniy, Ivan Rubachev, Valentin Khrulkov, and Artem Babenko. Revisiting deep learning models for tabular data. In Advances in Neural Information Processing Systems, volume 34, 2021.arxiv.org/abs/2106.11959
  11. [11]Léo Grinsztajn, Edouard Oyallon, and Gaël Varoquaux. Why do tree-based models still outperform deep learning on typical tabular data? In Advances in Neural Information Processing Systems (Datasets and Benchmarks Track), volume 35, 2022.
  12. [12]Songqiao Han, Xiyang Hu, Hailiang Huang, Minqi Jiang, and Yue Zhao. ADBench: Anomaly detection benchmark. In Advances in Neural Information Processing Systems (Datasets and Benchmarks Track), volume 35, 2022.
  13. [13]Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recognition. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 770–778, 2016.arxiv.org/abs/1512.03385
  14. [14]Alexander Hermans, Lucas Beyer, and Bastian Leibe. In defense of the triplet loss for person re-identification. arXiv preprint arXiv:1703.07737, 2017.arxiv.org/abs/1703.07737
  15. [15]Elad Hoffer and Nir Ailon. Deep metric learning using triplet network. In Similarity-Based Pattern Recognition (SIMBAD), pp. 84–92, 2015.arxiv.org/abs/1412.6622
  16. [16]Sture Holm. A simple sequentially rejective multiple test procedure. Scandinavian Journal of Statistics, 6 (2):65–70, 1979.DOI
  17. [17]Sergey Ioffe and Christian Szegedy. Batch normalization: Accelerating deep network training by reducing internal covariate shift. In Proceedings of the 32nd International Conference on Machine Learning, pp. 448–456, 2015.
  18. [18]Prannay Khosla, Piotr Teterwak, Chen Wang, Aaron Sarna, Yonglong Tian, Phillip Isola, Aaron Maschinot, Ce Liu, and Dilip Krishnan. Supervised contrastive learning. In Advances in Neural Information Processing Systems, volume 33, 2020.arxiv.org/abs/2004.11362
  19. [19]Diederik P. Kingma and Jimmy Ba. Adam: A method for stochastic optimization. In International Conference on Learning Representations, 2015.arxiv.org/abs/1412.6980
  20. [20]Gregory Koch, Richard Zemel, and Ruslan Salakhutdinov. Siamese neural networks for one-shot image recognition. In ICML Deep Learning Workshop, 2015.
  21. [21]Wengiao Li, Bozhong Zheng, Xiaohao Xu, Jinye Gan, Fading Lu, Xiang Li, Na Ni, Zheng Tian, Xiaonan Huang, Shenghua Gao, and Yingna Wu. Multi-sensor object anomaly detection: Unifying appearance, geometry, and internal properties. arXiv preprint arXiv:2412.14592, 2024a.arxiv.org/abs/2412.14592
  22. [22]Yuangang Li, Jiaqi Li, Zhuo Xiao, Tiankai Yang, Yi Nian, Xiyang Hu, and Yue Zhao. NLP-ADBench: NLP anomaly detection benchmark. arXiv preprint arXiv:2412.04784, 2024b.arxiv.org/abs/2412.04784
  23. [23]Guansong Pang, Chunhua Shen, and Anton van den Hengel. Deep anomaly detection with deviation networks. In Proceedings of the 25th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining, pp. 353–362, 2019.arxiv.org/abs/1911.08623
  24. [24]Guansong Pang, Chunhua Shen, Longbing Cao, and Anton van den Hengel. Deep learning for anomaly detection: A review. ACM Computing Surveys, 54(2):1–38, 2021.DOI
  25. [25]Guansong Pang, Chunhua Shen, Huidong Jin, and Anton van den Hengel. Deep weakly-supervised anomaly detection. In Proceedings of the 29th ACM SIGKDD Conference on Knowledge Discovery and Data Mining, 2023.arxiv.org/abs/1910.13601
  26. [26]Liudmila Prokhorenkova, Gleb Gusev, Aleksandr Vorobev, Anna Veronika Dorogush, and Andrey Gulin. CatBoost: Unbiased boosting with categorical features. In Advances in Neural Information Processing Systems, volume 31, 2018.arxiv.org/abs/1706.09516
  27. [27]Shebuti Rayana. ODDS library. https://odds.cs.stonybrook.edu, 2016.
  28. [28]Lukas Ruff, Robert Vandermeulen, Nico Görtz, Lucas Deecke, Shoaib Ahmed Siddiqui, Alexander Binder, Emmanuel Müller, and Markus Kloft. Deep one-class classification. In Proceedings of the 35th International Conference on Machine Learning, pp. 4393–4402, 2018.
  29. [29]Lukas Ruff, Robert A. Vandermeulen, Nico Görtz, Alexander Binder, Emmanuel Müller, Klaus-Robert Müller, and Markus Kloft. Deep semi-supervised anomaly detection. In International Conference on Learning Representations, 2020.arxiv.org/abs/1906.02694
  30. [30]Florian Schroff, Dmitry Kalenichenko, and James Philbin. FaceNet: A unified embedding for face recognition and clustering. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 815–823, 2015.DOI
  31. [31]Yuting Sun, Guansong Pang, Guanhua Ye, Tong Chen, Xia Hu, and Hongzhi Yin. Unraveling the “anomaly” in time series anomaly detection: A self-supervised tri-domain solution. arXiv preprint arXiv:2311.11235, 2023.arxiv.org/abs/2311.11235
  32. [32]Sumit Wadhwa and Souvik Nath. Machine learning techniques for anomaly detection. US Patent 12,517,940 B1, 2026. Filed September 18, 2024; issued January 6, 2026.
  33. [33]ChunJing Xiao, Jiahui Lu, Xovee Xu, Fan Zhou, Tianshu Xie, Wei Lu, and Lifeng Xu. Reconciling attribute and structural anomalies for improved graph anomaly detection. arXiv preprint arXiv:2506.23469, 2025.DOI

Paper details

vlnc:W_qpddpxdd
Version
v1 · latest
Published
Fresh — added to the Hub within the last 30 days
Added
License
CC BY 4.0
Usage

views · 8

downloads · 0

A view counts once per browser per day, a download each time the PDF is requested.

  • Computer Science

Not recorded for this paper

View statement
Looking for related papers…
Contents