betaFeedback

WorksComputer Science

Universal Semantic Enhancement for Weakly Supervised Anomaly Detection

Abstract

Weakly supervised anomaly detection on tabular data is an essential task in many label-scarce applications, since tabular data is widely encountered and it is commonly observed in practice that only a limited number of labeled anomalies are available alongside abundant unlabeled samples. Most existing methods try to learn features that capture the inherent distribution patterns in the target domain. However, such features often struggle to provide a complete description of possible abnormal characteristics and are highly susceptible to overfitting the extremely scarce labels. To address this challenge, we propose a weakly supervised anomaly detection method based on universal semantic enhancement for tabular data, which fuses the universal knowledge from a pre-trained tabular foundation model with the domain-specific knowledge learned on the target dataset. The two types of knowledge are naturally complementary: universal semantics describe the common patterns of tabular data across domains, while the domain-specific features capture the inherent distributions of normal and anomalous behaviors in the target dataset. A prototype-conditioned modulation mechanism is then designed to effectively integrate the two types of features, allowing them to reinforce each other for improved anomaly detection. Extensive experiments on public datasets show that the proposed method outperforms competitive methods when only eight labeled anomalies are available in the training procedure.

Introduction

Tabular data is one of the most prevalent data forms in real-world applications, and is widely used in areas such as financial risk management, medical diagnosis, and equipment monitoring [7, 18]. In these areas, detecting anomalies, i.e., 1 the records that deviate from the majority of the data, makes it possible to discover potential risk events such as fraudulent transactions and intrusion behaviors in time, and to reduce the corresponding losses [18]. Building an effective deep anomaly detection model usually requires a large number of labeled samples to guide the training procedure. However, anomalous events occur rarely in practice, and accurately annotating them requires extensive expert knowledge of the domain, so that collecting a sufficient number of labeled anomalies is extremely expensive [18, 12]. As a result, practical scenarios usually provide only a small number of labeled anomalies together with abundant unlabeled samples. Weakly supervised anomaly detection has therefore attracted considerable attention, since it learns detection models from exactly such data and does not rely on large-scale annotation [12, 13, 11].

To make the best use of the scarce labels, most existing weakly supervised anomaly detection methods for tabular data train a feature encoder together with an anomaly scoring function on the target dataset [12, 21, 11, 26]. The features learned in this way reflect the inherent distribution patterns of a single domain, namely the domain priors carried by the few labeled anomalies and the statistical patterns of normal behaviors in historical data. However, anomalies may appear in highly diverse forms, and the features learned from a single domain can hardly give a complete description of these possible forms; in addition, with extremely few labeled anomalies, the learned model tends to overfit these limited samples. Recently, pre-trained tabular foundation models [8] have shown that universal semantics of tabular data can be learned from large-scale and diverse data, and the resulting sample representations describe the common patterns shared across domains. Such universal knowledge naturally complements the domain-specific knowledge learned on the target dataset, while how to effectively combine the two types of knowledge for weakly supervised anomaly detection remains largely unexplored. Motivated by this observation, this paper studies how to enhance weakly supervised anomaly detection with universal semantics on tabular data.

In this paper, we propose a weakly supervised anomaly detection method for tabular data based on universal semantic enhancement. The core idea is to introduce the cross-domain universal semantics contained in a pre-trained tabular foundation model into the domain-specific detection model, so that the universal knowledge and the domain-specific knowledge complement each other. The two views are complementary: the universal semantic features provide a cross-domain prior that helps find anomalies hard to discover from single-domain data, while the domain-discriminative features capture the normal and anomalous distributions of the target dataset. Specifically, the proposed method first uses a pre-trained tabular foundation model to extract the universal semantic features for each input sample. Then, an autoencoder-based feature extractor is trained on the target dataset to obtain the domain-discriminative features, which describe how a sample deviates from the normal patterns of this domain. Since the two types of features come from different feature spaces and differ in both distribution and semantic level, directly concatenating them tends to cause mutual interference, while fusion modules with many extra parameters are hard train reliably with the scarce labels. A prototype-conditioned modulation mechanism is therefore designed, which builds the fusion signal on class prototypes and adaptively integrates the two types of features for each sample; on top of the integrated features, an anomaly scoring network is trained with a weakly supervised loss. Experiments on various public datasets show that the proposed method outperforms competitive methods, especially when only a few labeled anomalies are available in the training procedure.

The main contributions of this paper are summarized as follows.

  • We introduce universal semantic enhancement into weakly supervised anomaly detection on tabular data. By bringing in the cross-domain knowledge from a pre-trained tabular foundation model, the proposed method addresses the limitation that existing methods rely only on the priors of a single domain, and allows the universal and domain-specific knowledge to complement each other.

  • We design a prototype-conditioned modulation mechanism, which seamlessly integrates the universal semantic features and the domain-discriminative features for each sample, and can be trained stably even when the labeled anomalies are extremely scarce.

  • We conduct extensive experiments on public datasets, and the results show that the proposed method outperforms competitive methods when only a few labeled anomalies are available. Ablation studies further verify the effectiveness of each component.

The rest of this paper is organized as follows. Section 2 reviews the related works. Section 3 presents the proposed method in details. Section 4 reports the experimental results and analyses, and Section 5 concludes this paper.

Related Work

Weakly Supervised Anomaly Detection

Weakly supervised anomaly detection methods learn detection models from a small set of labeled anomalies alongside abundant unlabeled data. DevNet [12] enforces a statistical deviation loss between labeled anomalies and Gaussian reference priors; Deep SAD [21] extends Deep SVDD [20] to a semi-supervised hypersphere objective; FEAWAD [11] enriches the scoring network with three complementary signals derived from a deep autoencoder. More recently, PReNet [13] amplifies the effective supervision by learning pairwise relation features over labeled pairs, while RoSAS [26] addresses label scarcity by generating continuous-target pseudo-anomalies via interpolation between labeled anomalies and unlabeled samples. Despite these advances, all these methods learn representations only from the target dataset, performing poorly on distributions outside the target data. In the field of visual anomaly detection, researchers have attempted to address this issue using pre-trained models. PatchCore [19] and WinCLIP [9], pre-trained on ImageNet [3] and CLIP [17] backends respectively, have outperformed methods trained only on the target data, showing that cross-domain semantic knowledge can be transferred to anomaly detection. However, how to effectively utilize this cross-domain universal semantic knowledge in anomaly detection on tabular data remains an unexplored direction. Our approach aims to fill this gap by leveraging the universal semantics of a pre-trained model on tabular data to enhance the anomaly detection capabilities of domain-specific models.

Pre-trained Knowledge Integration and Feature Fusion

Combining complementary knowledge from heterogeneous sources is a broad strategy across diverse tasks. In vision-language modelling, CLIP [17] demonstrates that visual and linguistic features encode orthogonal aspects of the same input, enabling zero-shot generalization that neither modality achieves alone. In natural language processing, augmenting pre-trained representations such as BERT [4] with domain-specific features has become standard for low-resource tasks, where the pre-trained encoder supplies cross-domain priors that compensate for scarce labels [16]. For feature fusion, feature-wise linear modulation (FiLM) [14] achieves cross-modal conditioning via feature-wise affine transformation; more expressive designs, including gated multimodal unit (GMU) [1], cross-attention [24], and DCN v2 [25] model higher-order interactions at the cost of substantially more trainable parameters. Prototypical networks [22] offer a complementary perspective: anchoring representation learning to non-trainable class prototype vectors enables effective few-shot learning without extensive parameter optimization. At inference time, zero-parameter ensemble methods such as predictive dynamic fusion (PDF) [5] combine complementary prediction scores by weighting each view by its confidence and cross-modal consistency. These existing parameterized fusion modules, however, often fail under extreme label scarcity because a handful of labeled anomalies provides insufficient signal to reliably optimize additional parameters. Our method bridges this gap by grounding the fusion signal in fixed class prototypes, achieving per-sample adaptive modulation without trainable conditioning vectors.

Methods

Problem Formulation

Let X={xi}i=1n⊂Rd\mathcal{X} = \{x_i\}_{i=1}^{n} \subset\mathbb{R}^{d} be a tabular dataset, where xi∈Rdx_i \in\mathbb{R}^{d} is the feature vector of the ii-th instance and yi∈{0,1}y_i \in\{0,1\} denotes its ground-truth label (yi=1y_i = 1 for anomaly, yi=0y_i = 0 for normal). In the weakly supervised setting, only a small subset of anomalous samples M={(xj,yj)∣yj=1}\mathcal{M} = \{(x_j, y_j) \mid y_j = 1\} with ∣M∣≪n|\mathcal{M}| \ll n is labeled. The remaining n−∣M∣n - |\mathcal{M}| samples form the unlabeled set U\mathcal{U}, which is predominantly normal but may contain a small fraction of undetected anomalies. The goal is to learn a function s:Rd→Rs : \mathbb{R}^{d} \to\mathbb{R} that maps each instance to an anomaly score, assigning labeled anomalies xp∈Mx_p \in\mathcal{M} distinctly higher scores than unlabeled samples xq∈Ux_q \in\mathcal{U}, so that the two are clearly separated in the score space.

Framework Overview

As shown in Figure 1, our framework consists of four components. (1) Universal Semantic Feature Extraction: a frozen pre-trained model on tabular data generates cross-domain embeddings ti∈R512t_i \in\mathbb{R}^{512} without any gradient updates. (2) Domain Discriminative Feature Extraction: a reconstruction-based feature extractor captures the domain specific knowledge of the target dataset and produces a hidden embedding hih_i, a normalized residual rir_i, and a reconstruction error eie_i. (3) ProtoFiLM-Dynamic: our prototype-conditioned modulation module adaptively encodes hih_i via per-sample soft attention over class prototypes, producing the fused representation fif_i; during training, a label interpolation strategy generates pseudo-samples xmixx_{\mathrm{mix}} with continuous supervision targets between labeled anomalies and unlabeled data, which densifies the scarce supervision signal. (4) Anomaly Scoring Network: a three-layer MLP takes (ri,fi,ei)(r_i,f_i,e_i) as input and outputs the anomaly score sis_i; at inference, a zero-parameter score ensemble combines this score with the pre-trained model’s class probability ptp_t according to their confidence and consistency.

Figure 1. Overview of the proposed framework. Universal semantic features tit_i are extracted by the frozen pre-trained model, and domain-specific features (hi,ri,ei)(h_i,r_i,e_i) by the AutoEncoder. Our ProtoFiLM-Dynamic fuses hih_i and tit_i via per-sample prototype attention to produce the fused representation fif_i. The scoring network takes (ri,fi,ei)(r_i,f_i,e_i) as input and outputs the anomaly score sis_i. During training, label interpolation between a labeled anomaly and an unlabeled sample with coefficient α\alpha generates a pseudo-sample xmixx_{\mathrm{mix}} whose score is supervised toward the interpolated soft target αa\alpha a (aa is the score margin); at inference, a zero-parameter score ensemble combines the anomaly score with the pre-trained model’s class probability ptp_t into the final score sfinals_{\mathrm{final}}.

Module Design

Universal Semantic Features. We choose TabPFN [8], a prior-data-fitting network that is meta-trained on a large number of synthetic tabular classification tasks sampled from Bayesian priors. During testing, it generates calibrated class probabilities by learning context by treating the entire training set as a cue, without any gradient updates. We use its internal representation as a frozen feature extractor: for each sample xix_i, we extract

ti=TabPFN⁡(xi∣X)∈R512,(1)t_i = \operatorname{TabPFN}(x_i \mid\mathcal{X}) \in\mathbb{R}^{512}, \tag*{(1)}

where X\mathcal{X} denotes the full training set used as in-context examples for TabPFN. Because TabPFN has accessed distributions far exceeding the scope of any single target dataset, tit_i encodes cross-domain statistical regularities, such as which feature combinations are statistically anomalous in multiple domains. More importantly, TabPFN is completely frozen throughout all training phases: its only overhead is a forward propagation of the training set before joint training begins to cache tit_i, which reduces our training overhead.

Domain Discriminative Features. TabPFN provides universal semantic context; complementarily, we need a feature extractor that exposes how each sample deviates from the structural regularities of the target dataset. We adopt the reconstruction-based feature extraction module from FEAWAD [11], which is built upon a symmetric encoder–decoder pair: the encoder fe:Rd→R64f_e : \mathbb{R}^{d} \to\mathbb{R}^{64} compresses each sample into a compact structural embedding through two fully connected layers (d→128→64d \to128 \to64), and the decoder fd:R64→Rdf_d : \mathbb{R}^{64} \to\mathbb{R}^{d} mirrors it to reconstruct the input through two fully connected layers (64→128→d64 \to128 \to d). Pre-trained on the predominantly normal unlabeled data, the extractor reconstructs normal patterns accurately while producing large reconstruction errors on samples that deviate from them, giving it an inherent capability to expose domain-specific anomalies. From the input xix_i and the reconstructed x^i=fd(fe(xi))\hat{x}_i = f_d(f_e(x_i)), we extract three features:

hi=fe(xi)∈R64,(2)h_i = f_e(x_i) \in\mathbb{R}^{64}, \tag*{(2)}
ri=x^i−xi∥x^i−xi∥2∈Rd,(3)r_i = \frac{\hat{x}_i - x_i}{\lVert\hat{x}_i - x_i \rVert_2} \in\mathbb{R}^{d}, \tag*{(3)}
ei=∥x^i−xi∥2∈R.(4)e_i = \lVert\hat{x}_i - x_i \rVert_2 \in\mathbb{R}. \tag*{(4)}

Each feature plays a unique role: the hidden embedding hih_i carries a compressed structural representation of the sample; the normalized residual rir_i encodes the direction of the reconstruction bias, providing information about the anomaly type; and the scalar error eie_i quantifies the magnitude of the bias and is injected as a persistent signal into each layer of the scoring network. Together, these three features provide complementary domain-specific information that may not be fully captured by the semantic representations derived from TabPFN.

ProtoFiLM-Dynamic Fusion While universal semantic and domain-specific features are complementary, effectively fusing them under extremely scarce supervision remains a critical challenge. The standard FiLM [14] applies a global affine transformation to one feature space and conditions it with another feature space, using a single conditional vector shared by all samples. This approach works well in data-rich scenarios but its performance degrades for anomaly detection under severely limited anomaly supervision.

First, before joint training, we compute two class prototype vectors [22] in TabPFN embedding space. The anomaly prototype cac_a is the mean embedding of labeled anomalies in M\mathcal{M}; the normal prototype cn\mathbf{c}_n is the mean of a random subsample U0⊂U\mathcal{U}_0 \subset\mathcal{U}, serving as a proxy for the normal class:

ca=1∣M∣∑j∈Mtj∈R512,(5)\mathbf{c}_a = \frac{1}{|\mathcal{M}|} \sum_{j \in\mathcal{M}} \mathbf{t}_j \in\mathbb{R}^{512}, \tag*{(5)}
cn=1∣U0∣∑j∈U0tj∈R512.(6)\mathbf{c}_n = \frac{1}{|\mathcal{U}_0|} \sum_{j \in\mathcal{U}_0} \mathbf{t}_j \in\mathbb{R}^{512}. \tag*{(6)}

Both prototypes are stored as non-trainable buffers throughout joint training. Second, for each sample we compute squared Euclidean distances to both prototypes in TabPFN space and convert them to soft mixing coefficients via softmax:

[αin,αia]=softmax⁡([−∥ti−cn∥22,−∥ti−ca∥22]),(7)[\alpha_i^n, \alpha_i^a] = \operatorname{softmax}([-{\lVert\mathbf{t}_i - \mathbf{c}_n \rVert}_2^2, -{\lVert\mathbf{t}_i - \mathbf{c}_a \rVert}_2^2]), \tag*{(7)}

where αin\alpha_i^n and αia\alpha_i^a are the soft attention weights toward the normal and anomaly prototypes, respectively. We then form the per-sample conditioning vector as their convex combination δi=αincn+αiaca∈R512\boldsymbol{\delta}_i = \alpha_i^n \mathbf{c}_n + \alpha_i^a \mathbf{c}_a \in\mathbb{R}^{512}. Samples close to the anomaly prototype receive αia≈1\alpha_i^a \approx1; clearly normal samples receive αin≈1\alpha_i^n \approx1. Third, two zero-initialized linear layers (Wγ,Wβ∈R64×512)(W_\gamma, W_\beta\in\mathbb{R}^{64 \times512}) map δi\boldsymbol{\delta}_i to per-channel scale and shift parameters γi=Wγδi\boldsymbol{\gamma}_i = W_\gamma\boldsymbol{\delta}_i and βi=Wβδi\boldsymbol{\beta}_i = W_\beta\boldsymbol{\delta}_i, which modulate the autoencoder hidden embedding:

hi′=(1+γi)⊙hi+βi,(8)\mathbf{h}'_i = (1 + \boldsymbol{\gamma}_i) \odot\mathbf{h}_i + \boldsymbol{\beta}_i, \tag*{(8)}

where ⊙\odot denotes element-wise multiplication. Finally, we get the fused representation fi=[hi′;ti]∈R576\mathbf{f}_i = [\mathbf{h}'_i; \mathbf{t}_i] \in\mathbb{R}^{576}. Since δi\boldsymbol{\delta}_i is a convex combination of fixed non-trainable vectors, gradient flow into WγW_\gamma and WβW_\beta remains bounded and well-conditioned regardless of label count, which directly avoids gradient starvation.

Anomaly Scoring Network. The scoring network receives the normalized residual ri\mathbf{r}_i, the fused representation fi\mathbf{f}_i, and the reconstruction error scalar eie_i, and generates anomaly scores. The reconstruction error is concatenated in every hidden layer, ensuring that this strong geometric signal remains visible throughout the computation regardless of how the preceding layers transform ri\mathbf{r}_i and fi\mathbf{f}_i.

z1=ReLU⁡(W1[ri;fi;ei])∈R256,(9)\mathbf{z}_1 = \operatorname{ReLU}(W_1[\mathbf{r}_i; \mathbf{f}_i; e_i]) \in\mathbb{R}^{256}, \tag*{(9)}
z2=ReLU⁡(W2[z1;ei])∈R32,(10)\mathbf{z}_2 = \operatorname{ReLU}(W_2[\mathbf{z}_1; e_i]) \in\mathbb{R}^{32}, \tag*{(10)}
si=W3[z2;ei]∈R,(11)s_i = W_3[\mathbf{z}_2; e_i] \in\mathbb{R}, \tag*{(11)}

where all weight matrices are Xavier-initialized [6]. The scalar output sis_i is used directly in the deviation loss during training and, at inference, mapped to the open interval (0,1)(0, 1) through the sigmoid function for score fusion.

Training and Inference

Training proceeds in three consecutive phases, as summarized in Algorithm 1. In the joint-training phase, each mini-batch B\mathcal{B} is class-balanced, containing an equal number of normal samples drawn from the unlabeled set U\mathcal{U} and anomalous samples resampled from the labeled anomalies M\mathcal{M} (with replacement, since only ∣M∣|\mathcal{M}| are available). The model is trained by minimizing the loss averaged over each batch, with the loss per-sample being:

L=Ld+Le+λ⋅Lrosas.(12)\mathcal{L}=\mathcal{L}_{d}+\mathcal{L}_{e}+\lambda\cdot\mathcal{L}_{\mathrm{rosas}}. \tag*{(12)}

Part of our objective is inspired by the loss proposed in FEAWAD [11]. For a sample xix_i with label yiy_i, the deviation loss Ld\mathcal{L}_{d} drives the anomaly score sis_i toward 0 for a normal sample and above the margin a=5a=5 for a labeled anomaly, and the reconstruction constraint Le\mathcal{L}_{e} mirrors this objective on the reconstruction error eie_i:

Ld=(1−yi)∣si∣+yimax⁡(0,a−si),(13)\mathcal{L}_{d}=(1-y_i)|s_i|+y_i\max(0,a-s_i), \tag*{(13)}
Le=(1−yi)ei+yimax⁡(0,a−ei).(14)\mathcal{L}_{e}=(1-y_i)e_i+y_i\max(0,a-e_i). \tag*{(14)}

The RoSAS interpolation loss [26] Lrosas\mathcal{L}_{\mathrm{rosas}} densifies the supervision signal with continuous-target pseudo-samples, and is defined on the pairs of anomaly and unlabeled samples formed within the batch B\mathcal{B}, and each pair is interpolated into exactly one pseudo-sample xmix\mathbf{x}_{\mathrm{mix}}. Specifically, for an anomalous sample xp\mathbf{x}_p we pair it one-to-one with a distinct normal sample xq\mathbf{x}_q from the unlabeled half of B\mathcal{B}, draw an independent coefficient α∼Uniform⁡(0,1)\alpha\sim\operatorname{Uniform}(0,1), and blend the two in feature space into xmix\mathbf{x}_{\mathrm{mix}}:

xmix=(1−α)xq+αxp,(15)\mathbf{x}_{\mathrm{mix}}=(1-\alpha)\mathbf{x}_q+\alpha\mathbf{x}_p, \tag*{(15)}

whose score is regressed toward the soft target αa\alpha a:

Lrosas=∣s(xmix)−αa∣,(16)\mathcal{L}_{\mathrm{rosas}}=\left|s(\mathbf{x}_{\mathrm{mix}})-\alpha a\right|, \tag*{(16)}

where s(xmix)s(\mathbf{x}_{\mathrm{mix}}) is obtained by passing xmix\mathbf{x}_{\mathrm{mix}} through the same encoder–fusion–scoring pipeline. Since αa\alpha a is a continuous soft label rather than a constant, these pseudo-samples populate the whole score interval between the two endpoints {0,a}\{0,a\} that (Ld+Le)(\mathcal{L}_{d}+\mathcal{L}_{e}) supervises, which encourages a smooth scoring function and mitigates overfitting under extreme label scarcity. Each pseudo-sample thus corresponds one-to-one to a sample of B\mathcal{B} and contributes an independent regression term, sharing the network parameters with the batch samples it is built from but without interacting with either of them in the loss.

Inference-Time Score Fusion. At inference, two complementary anomaly signals are available: the deviation-based score sd=σ(s(x)/a)∈[0,1]s_d=\sigma(s(\mathbf{x})/a)\in[0,1], where σ(⋅)\sigma(\cdot) is the sigmoid function and aa is the score margin, and the TabPFN class probability pt∈[0,1]p_t\in[0,1]. We apply predictive dynamic fusion (PDF) [5] as an auxiliary zero-parameter ensemble, which weights each signal by its own prediction confidence and cross-modal consistency:

wd∝∣sd−0.5∣+(1−∣sd−pt∣),(17)w_d\propto|s_d-0.5|+(1-|s_d-p_t|), \tag*{(17)}
wt∝∣pt−0.5∣+(1−∣sd−pt∣),(18)w_t\propto|p_t-0.5|+(1-|s_d-p_t|), \tag*{(18)}
sfinal=wdsd+wtptwd+wt,(19)s_{\mathrm{final}}=\frac{w_d s_d+w_t p_t}{w_d+w_t}, \tag*{(19)}

Algorithm 1 Three-Phase Training Procedure

Input: Dataset X\mathcal{X}, labeled anomalies M\mathcal{M}, hyperparameter λ\lambda

Output: Trained parameters of the whole model

  1. Phase 1: Autoencoder Pretraining (100 epochs):

  2. for each batch B⊂U\mathcal{B} \subset\mathcal{U} do

  3. Update autoencoder by minimizing Lae=1∣B∣∑i∈B∥x^i−xi∥22\mathcal{L}_{\mathrm{ae}} = \frac{1}{|\mathcal{B}|}\sum_{i \in\mathcal{B}} \lVert\hat{x}_i - x_i \rVert_2^2

  4. end for

  5. Phase 2: Feature Caching:

  6. Compute and cache {ti}←TabPFN⁡(X)\{t_i\} \leftarrow\operatorname{TabPFN}(\mathcal{X})

  7. Compute prototypes cac_a from M\mathcal{M}, cnc_n from U0\mathcal{U}_0

  8. Phase 3: Joint Training (30 epochs):

  9. for each class-balanced batch B\mathcal{B} do

  10. Extract (hi,ri,ei)(h_i, r_i, e_i) from autoencoder; retrieve tit_i from cache

  11. Compute δi\delta_i via prototype soft attention (Eqs. (5)–(7))

  12. Modulate hi→hi′h_i \to h_i' via FiLM (Eq. (8)); form fif_i

  13. Compute score sis_i via scoring network

  14. For each anomaly in B\mathcal{B}, pair it with a distinct normal sample in B\mathcal{B}; draw α∼Uniform⁡(0,1)\alpha\sim\operatorname{Uniform}(0,1), build xmixx_{\mathrm{mix}} (Eq. (15)) and score it

  15. Compute L=Ld+Le+λ⋅Lrosas\mathcal{L} = \mathcal{L}_d + \mathcal{L}_e + \lambda\cdot\mathcal{L}_{\mathrm{rosas}}, averaged over B\mathcal{B}

  16. Update autoencoder, WγW_\gamma, WβW_\beta, scoring network via Adam

  17. end for

where ∝\propto denotes proportionality: the two unnormalized weights are defined up to a common factor that cancels when they are normalized by wd+wtw_d + w_t in sfinals_{\mathrm{final}}, so only their relative magnitude matters. Here ∣s−0.5∣\lvert s - 0.5 \rvert measures prediction confidence as the distance from the decision boundary, and (1−∣sd−pt∣)(1 - \lvert s_d - p_t \rvert) rewards cross-modal consistency between the two views. This fusion introduces no additional training cost and serves as a lightweight complement to the learned framework. Together, the four components and three-phase training procedure form a cohesive framework that systematically addresses cross-domain feature fusion and gradient starvation under extreme label scarcity in weakly supervised anomaly detection on tabular data.

Experiments

Experimental Setup

We evaluated our model using the ADBench benchmark [7]. ADBench is a standardized dataset containing 57 real-world tabular anomaly detection datasets from various domains, including cyber intrusion, medical diagnosis, and financial fraud. Consistent with previous research, we excluded datasets with fewer than 30 anomalies, resulting in 42 datasets for evaluation. The 42 datasets span five application domains: medical diagnosis (annthyroid, breastw, Cardio, Cardiotocography, Hepatitis, Lymphography, mammography, Pima, thyroid, vertebral, WBC, WDBC, WPBC), network intrusion (HTTP, InternetAds, SMTP), image and signal recognition (ALOI, landsat, letter, mnist, optdigits, Page-Blocks, pendigits, satellite, satimage-2, speech, Waveform), financial fraud and spam (campaign, SpamBase, Stamps), and other scientific and industrial data (cover, fault, glass, Ionosphere, magic.gamma, musk, shuttle, skin, Wilt, vowels, wine, yeast). Dataset sizes range from a few hundred to tens of thousands of samples, with anomaly ratios between 1% and 35%. Each dataset was stratified and split into training and test sets at an 80/20 ratio. The unlabeled training set contains 2% undetected anomalies, reflecting the practical reality that the unlabeled data is not purely normal but is inevitably contaminated with a small fraction of anomalies. In the training set, we fed the model only ∣M∣=8|\mathcal{M}| = 8 labeled anomalies, which is a stricter setting than the commonly used ∣M∣=30|\mathcal{M}| = 30 [12, 11, 13]. All experiments were repeated 10 times with different random seeds, and the mean AUROC and mean AUPRC values are reported. All models ran on NVIDIA RTX 3090 GPUs and were trained using the Adam optimizer [10] (β1=0.9\beta_1 = 0.9, β2=0.999\beta_2 = 0.999, ε=10−8\varepsilon= 10^{-8}, lr =10−3= 10^{-3}). The autoencoder was pre-trained for 100 epochs, and the whole framework is jointly optimized for 30 epochs.

We compare against three categories of baselines. (1) Traditional ML classifiers: Random Forest (RF) [2] and CatBoost [15] are trained as supervised binary classifiers, treating the 8 labeled anomalies and all unlabeled training samples as positive and negative classes respectively. (2) Deep weakly supervised methods: DevNet [12] learns an anomaly scoring function via a deviation loss with a Gaussian prior; DeepSAD [21] extends a hypersphere objective to the semi-supervised setting; FEAWAD [11] enriches the scoring network with three autoencoder-derived feature signals; PReNet [13] amplifies supervision by learning pairwise relation features over labeled pairs. (3) Foundation model baseline: standalone TabPFN [8] is applied directly as an in-context classifier.

Performance Comparison with Competing Methods

We evaluate our framework against 7 baselines on all 42 datasets with ∣M∣=8|\mathcal{M}| = 8 labeled anomalies, covering three categories as stated in § 4.1. Table 1 and Table 2 report per-dataset AUROC and AUPRC on 20 representative datasets.

DatasetRFCatBoostDevNetFEAWADPReNetDeepSADTabPFNOurs
ALOI0.51300.52700.51060.51870.49530.53880.60980.6466
Campaign0.53230.87410.58680.58930.68090.63290.85700.8668
Cardio0.92560.97220.85780.94690.93330.95450.98700.9964
Cardiotocography0.82120.92390.80600.81710.86590.87090.93400.9424
Fault0.65190.73660.58630.62860.60860.75740.76240.7826
Hepatitis0.83850.81540.77180.70000.78460.68200.85640.8827
InternetAds0.79890.82780.72790.70490.69820.82060.85870.8883
Landsat0.68640.83850.72930.73760.74120.81420.87240.8843
Letter0.66920.74870.57580.77450.74260.91950.94550.9588
Lymphography0.97760.97240.92410.94140.98970.91380.98960.9961
Mammography0.63540.92510.80580.86300.89060.85910.92620.9386
Pima0.68130.77150.60210.62490.61260.69020.77310.8077
CoverOOMOOM0.83880.72680.68060.94120.99840.9995
SpamBase0.83950.91850.80300.83430.89490.77490.93770.9453
Speech0.49510.60830.57910.61740.55040.47300.62500.6104
Stamps0.94760.96020.96940.92980.97230.94810.98140.9914
Vertebral0.76130.82580.79960.81470.85040.73690.88330.9091
Waveform0.67590.90870.69150.82490.82650.81870.93160.9475
WPBC0.50380.59680.67990.63190.70470.49210.72420.7385
Yeast0.54440.60730.56660.57620.62610.61130.62020.6494
Avg. Rank (20)6.43.76.55.64.95.32.11.1
Avg. Rank (42)6.43.66.05.55.15.12.31.1

Table 1. Per-dataset AUROC on 20 representative ADBench datasets (∣M∣=8|\mathcal{M}| = 8, 10 runs). Bold = best, underline = second best. OOM = GPU out of memory. Avg. Rank (20) is computed over the 20 datasets shown, and Avg. Rank (42) over all 42 evaluated datasets.

DatasetRFCatBoostDevNetFEAWADPReNetDeepSADTabPFNOurs
ALOI0.03500.03600.03860.03540.03470.04290.08140.0697
Campaign0.14310.45920.18600.18860.22610.19490.41730.4419
Cardio0.81090.85230.77160.85350.85410.80690.91510.9204
Cardiotocography0.69100.79780.69780.65670.73330.73850.82710.8375
Fault0.51060.61750.49540.53050.51750.60360.63140.6050
Hepatitis0.62210.59610.62390.45980.55970.43960.70780.7143
InternetAds0.60470.69320.56810.58180.57810.64760.73170.7053
Landsat0.44400.61670.52360.55170.48970.58880.68050.6171
Letter0.18970.22290.23770.31410.25970.59250.67210.6253
Lymphography0.81230.74990.73020.81520.85940.64200.85940.9605
Mammography0.18480.53550.41720.44850.46780.36050.56620.4700
Pima0.52830.63870.48730.50360.50750.55410.64660.6624
CoverOOMOOM0.27440.04380.06060.64490.85010.9643
SpamBase0.80050.91210.76480.79730.88490.71120.93280.9335
Speech0.02160.05810.07450.10610.09260.05460.10990.1033
Stamps0.81140.84110.78940.72400.80330.62860.89070.9363
Vertebral0.45310.53570.54750.51790.62530.50130.72480.6962
Waveform0.10210.33080.29330.34700.34860.34250.46880.4721
WPBC0.28230.38410.48150.44470.49060.28140.54080.5188
Yeast0.39880.45030.45050.44260.46640.44750.46980.4702
Avg. Rank (20)6.74.35.95.64.75.71.71.6
Avg. Rank (42)6.63.95.25.44.95.52.31.5

Table 2. Per-dataset AUPRC on 20 representative ADBench datasets (∣M∣=8|\mathcal{M}| = 8, 10 runs). Bold = best, underline = second best. OOM = GPU out of memory. Avg. Rank (20) is computed over the 20 datasets shown, and Avg. Rank (42) over all 42 evaluated datasets.

Table 1 and Table 2 show that our method achieves the best average rank on both AUROC (1.1) and AUPRC (1.6), and this ranking advantage also holds on the full 42-dataset average (1.1 / 1.5). The two most informative reference points are the individual components of our framework: FEAWAD, relying solely on domain-specific features, ranks 5.6 on both metrics, showing that single-domain representations are insufficient under extreme label scarcity; while TabPFN, relying solely on universal semantics, already achieves rank 2.1 / 1.7, substantially better than all domain-only methods. Our method surpasses both, confirming that the two knowledge sources are genuinely complementary. The benefit is most pronounced on datasets where anomalies occupy semantically coherent regions in TabPFN space: InternetAds, Landsat, and Fault yield the largest gains over TabPFN (+0.0296, +0.0119, +0.0202 AUROC respectively), while Cover, Stamps, and Lymphography see our method lead on both metrics simultaneously. The one exception is Speech, where TabPFN outperforms our method

on both metrics: Speech anomalies lack stable semantic coherence, so prototype conditioning could introduce a misleading modulation signal under extreme label scarcity, which is a known limitation of prototype-based designs under high intra-class variability [22].

Label Efficiency

To investigate how performance varies with the number of available labeled anomalies, we evaluate all methods with ∣M∣∈{8,15,30,60,120}|\mathcal{M}| \in\{8, 15, 30, 60, 120\} on five representative datasets. All other settings follow § 4.1. Figure 2 reports per-dataset AUROC (top row) and AUPRC (bottom row).

Figure 2. AUROC (top) and AUPRC (bottom) vs. number of labeled anomalies ∣M∣|\mathcal{M}| on five representative datasets. Ours and TabPFN are shown with solid thick lines; all other baselines are shown with semi-transparent dashed lines.

Across the tested values of ∣M∣|\mathcal{M}|, our method maintains a clear advantage on most datasets. On Cover and HTTP, our method reaches near-ceiling AUROC and AUPRC even at ∣M∣=8|\mathcal{M}| = 8, indicating that universal semantic features supply sufficient cross-domain context to identify anomalies before any domain-specific pattern can be reliably learned from so few labels. On Stamps and ALOI, the gap over FEAWAD is largest at ∣M∣=8|\mathcal{M}| = 8 and gradually narrows as more labels are added, which directly confirms that universal semantic enhancement is most valuable precisely when domain-specific supervision is most scarce. The advantage over standalone TabPFN holds across most datasets and supervision levels, indicating that domain-specific autoencoder features provide complementary discriminative signals that remain beneficial even as the number of labeled anomalies increases.

Ablation Study

The goal of this experiment is to quantify the individual contribution of each proposed component and to examine how the components interact. Unlike § 4.2, which compares our framework against external methods, this section holds all external comparisons fixed and instead ablates each internal design choice one at a time. The ablation baseline is TabPFN-FEAWAD (concat + dev loss, AUROC 0.9174); all other settings are identical to § 4.2: 42 datasets, ∣M∣=8|\mathcal{M}| = 8 labeled anomalies, 10 independent runs. Results are reported in Table 3.

ConfigurationAUROCAUPRCΔ\Delta
Baseline (concat + dev loss)0.91740.7750—
+ FiLM (static prototype)0.91780.7703+0.0004
+ ProtoFiLM-Dynamic0.92380.7775+0.0064
+ RoSAS (loss only)0.91830.7745+0.0009
+ PDF (score fusion only)0.92420.7782+0.0068
+ ProtoFiLM-Dynamic + PDF0.92470.7776+0.0073
+ ProtoFiLM-Dynamic + RoSAS + PDF0.93080.7799+0.0134

Table 3. Component ablation. Each row adds one design choice to the previous configuration. Δ\Delta is relative to the ablation baseline AUROC (0.9174).

Fusion Module (ProtoFiLM-Dynamic). Replacing ProtoFiLM-Dynamic with a plain concat baseline (row 1 vs. row 3) reduces AUROC by 0.0064, confirming that per-sample prototype conditioning is the key driver of the improvement. Figure 3 visualizes the penultimate-layer embeddings and confirms that ProtoFiLM-Dynamic produces a tighter, more separable anomaly cluster than the concat baseline.

Figure 3. t-SNE [23] of the 32-dim penultimate scoring layer on the Waveform dataset (∣M∣=8|\mathcal{M}| = 8, one run). Blue: normal samples (n=669n = 669); orange: anomalies (n=20n = 20). Anomaly points in the raw feature space (left) are indistinguishable from normals. The Concat baseline (center) produces partial clustering but with significant normal–anomaly mixing. ProtoFiLM-Dynamic (right) yields a tight, spatially isolated anomaly cluster with a clean decision boundary.

Label Augmentation. RoSAS applied to the concat baseline alone yields only +0.0009 AUROC, indicating that pseudo-sample interpolation cannot by itself bridge the representation gap. Its contribution becomes substantial in the full system (+0.0134 total), demonstrating that RoSAS is effective as a complement to ProtoFiLM-Dynamic rather than a standalone strategy.

Inference-Time Score Fusion. PDF provides an additional gain on top of the full training-time system. This indicates that our model’s deviation-based score sds_d and TabPFN’s class probability ptp_t are not redundant: even with no extra parameters or training, simply combining the two signals yields a further improvement. The full system (ProtoFiLM-Dynamic + RoSAS + PDF) achieves the best overall performance (+0.0134 AUROC over the ablation baseline).

Hyperparameter Sensitivity

The full training objective L=Ld+Le+λ⋅Lrosas\mathcal{L} = \mathcal{L}_d + \mathcal{L}_e + \lambda\cdot\mathcal{L}_{\mathrm{rosas}} introduces a single additional hyperparameter λ\lambda that controls the relative weight of the RoSAS augmentation loss. Figure 4 reports AUROC and AUPR on 42 datasets over 10 runs as λ\lambda varies across a 100× range from 0.05 to 5.0.

Figure 4. Sensitivity of AUROC (left axis, blue) and AUPR (right axis, orange) to the RoSAS loss weight λ\lambda on 42 datasets (∣M∣=8|\mathcal{M}| = 8, 10 runs). Dashed lines mark the ablation baseline (concat + dev loss) without RoSAS.

According to the results, most tested values of λ\lambda outperform the ablation baseline (concat + dev loss). The curve is non-monotone rather than showing a clear degradation at either extreme: λ=0.5\lambda=0.5 achieves the peak AUROC (0.9308) and λ=2.0\lambda=2.0 is a close second (0.9303), while even the single dip at λ=0.1\lambda=0.1 (0.9156) recovers quickly at λ=0.2\lambda=0.2 (0.9286). This broad optimal region around 0.5–2.0 indicates that the RoSAS signal is beneficial across a wide range of strengths, and the method does not require careful tuning of this hyperparameter. We select λ=0.5\lambda=0.5 as the default.

Conclusion

In this paper, we propose a universal semantic enhancement method for weakly supervised anomaly detection on tabular data, which combines the universal knowledge from a frozen tabular foundation model with the domain-specific features learned by a target-dataset autoencoder. To integrate them under extreme label scarcity, we design ProtoFiLM-Dynamic, a prototype-conditioned modulation module that adaptively enhances each sample by grounding the modulation signal in fixed class prototypes, and adopt a RoSAS-style label-interpolation loss to densify the scarce supervision. On 42 public datasets with as few as eight labeled anomalies, the proposed method attains the best average rank among competitive baselines, with the universal semantics being most beneficial when domain labels are scarcest. Ablation studies further show that the per-sample conditioning of ProtoFiLM-Dynamic is complementary to a parameter-free inference-time score fusion, which brings an additional gain. One limitation of the current design is that class prototypes are computed once before joint training and remain fixed throughout. Future work could focus on online prototype refinement that updates class representations as the scoring network improves, which may further stabilize semantic enhancement under extremely sparse supervision. More broadly, we hope this work motivates further exploration of how universal semantic knowledge from large-scale pretrained models can be systematically integrated with domain-specific representations in other data-scarce detection settings.

References

  1. [1]Arevalo, J., Solorio, T., Montes-y-Gómez, M., González, F.A.: Gated multimodal units for information fusion. In: Workshop at ICLR (2017)arxiv.org/abs/1702.01992
  2. [2]Breiman, L.: Random forests. Mach. Learn. 45(1), 5–32 (2001)
  3. [3]Deng, J., Dong, W., Socher, R., Li, L.J., Li, K., Fei-Fei, L.: ImageNet: A large-scale hierarchical image database. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009)
  4. [4]Devlin, J., Chang, M.W., Lee, K., Toutanova, K.: BERT: pre-training of deep bidirectional transformers for language understanding. In: Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pp. 4171–4186. ACL (2019)arxiv.org/abs/1810.04805
  5. [5]Fu, Z., Chen, J., Yang, Y., Wen, J., Jing, X., He, H., Re: Predictive dynamic fusion. In: Proceedings of the 41st International Conference on Machine Learning. PMLR (2024)
  6. [6]Glorot, X., Bengio, Y.: Understanding the difficulty of training deep feedforward neural networks. In: Proceedings of the 13th International Conference on Artificial Intelligence and Statistics, pp. 249–256. PMLR (2010)
  7. [7]Han, S., Hu, X., Huang, H., Jiang, M., Zhao, Y.: ADBench: anomaly detection benchmark. In: Advances in Neural Information Processing Systems, vol. 35, pp. 32142–32159 (2022)
  8. [8]Hollmann, N., Müller, S., Purucker, L., Krishnakumar, A., Körfer, M., Hoo, S.B., Schirrmeister, R.T., Hutter, F.: Accurate predictions on small data with a tabular foundation model. Nature 637(8045), 319–327 (2025)DOI
  9. [9]Jeong, J., Zou, Y., Kim, T., Zhang, D., Ravichandran, A., Dabeer, O.: WinCLIP: zero-/few-shot anomaly classification and segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 19606–19616 (2023)arxiv.org/abs/2303.14814
  10. [10]Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. In: Proceedings of the 3rd International Conference on Learning Representations. ICLR (2015)arxiv.org/abs/1412.6980
  11. [11]Ma, X., Li, C., Yang, B., Wang, S., Zhang, J.: Feature encoding with autoencoders for weakly supervised anomaly detection. IEEE Trans. Knowl. Data Eng. 34(12), 5521–5532 (2022)
  12. [12]Pang, G., Shen, C., Cao, L., Hengel, A.V.D.: Deep anomaly detection with deviation networks. In: Proceedings of the 25th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining, pp. 353–362. ACM (2019)arxiv.org/abs/1911.08623
  13. [13]Pang, G., Shen, C., Jin, H., Hengel, A.V.D.: Deep weakly-supervised anomaly detection. In: Proceedings of the 29th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining, pp. 1795–1807. ACM (2023)arxiv.org/abs/1910.13601
  14. [14]Perez, E., Strub, F., de Vries, H., Dumoulin, V., Courville, A.: FiLM: visual reasoning with a general conditioning layer. In: Proceedings of the 32nd AAAI Conference on Artificial Intelligence, pp. 3942–3951. AAAI Press (2018)arxiv.org/abs/1709.07871
  15. [15]Prokhorenkova, L., Gusev, G., Vorobev, A., Dorogush, A.V., Gulin, A.: CatBoost: unbiased boosting with categorical features. In: Advances in Neural Information Processing Systems, vol. 31 (2018)arxiv.org/abs/1706.09516
  16. [16]Qiao, H., Niu, C., Chen, L., Pang, G.: AnomalyGFM: graph foundation model for zero/few-shot anomaly detection. In: Proceedings of the 31st ACM SIGKDD Conference on Knowledge Discovery and Data Mining, pp. 2326–2337 (2025)
  17. [17]Radford, A., Kim, J.W., Hallacy, C., Ramesh, A., Goh, G., Agarwal, S., Sastry, G., Askell, A., Mishkin, P., Clark, J., Krueger, G., Sutskever, I.: Learning transferable visual models from natural language supervision. In: Proceedings of the 38th International Conference on Machine Learning, pp. 8748–8763. PMLR (2021)arxiv.org/abs/2103.00020
  18. [18]Ren, J., Tang, T., Jia, H., Xu, Z., Fayek, H., Li, X., Ma, S., Xu, X., Xia, F.: Foundation models for anomaly detection: vision and challenges. AI Mag. 46(4) (2025)arxiv.org/abs/2502.06911
  19. [19]Roth, K., Pemula, L., Zepeda, J., Schölkopf, B., Brox, T., Gehler, P.: Towards total recall in industrial anomaly detection. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 14318–14328 (2022)arxiv.org/abs/2106.08265
  20. [20]Ruff, L., Vandermeulen, R., Goernitz, N., Deecke, L., Siddiqui, S.A., Binder, A., Müller, E., Kloft, M.: Deep one-class classification. In: Proceedings of the 35th International Conference on Machine Learning, pp. 4393–4402. PMLR (2018)
  21. [21]Ruff, L., Vandermeulen, R.A., Görnitz, N., Binder, A., Müller, E., Müller, K.R., Kloft, M.: Deep semi-supervised anomaly detection. In: Proceedings of the 8th International Conference on Learning Representations. ICLR (2020)arxiv.org/abs/1906.02694
  22. [22]Snell, J., Swersky, K., Zemel, R.S.: Prototypical networks for few-shot learning. In: Advances in Neural Information Processing Systems, vol. 30 (2017)arxiv.org/abs/1703.05175
  23. [23]Van der Maaten, L., Hinton, G.: Visualizing data using t-SNE. J. Mach. Learn. Res. 9, 2579–2605 (2008)
  24. [24]Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A.N., Kaiser, Ł., Polosukhin, I.: Attention is all you need. In: Advances in Neural Information Processing Systems, vol. 30 (2017)arxiv.org/abs/1706.03762
  25. [25]Wang, R., Shivanna, R., Cheng, D., Jain, S., Lin, D., Hong, L., Chi, E.: DCN V2: improved deep & cross network and practical lessons for web-scale learning to rank systems. In: Proceedings of the Web Conference 2021, pp. 1785–1797 (2021)
  26. [26]Xu, H., Pang, G., Wang, Y., Wang, Y.: RoSAS: deep semi-supervised anomaly detection with contamination-resilient continuous supervision. Inf. Process. Manag. 60(5), 103459 (2023)arxiv.org/abs/2307.13239

Paper details

vlnc:W_hnmpjbam
Version
v1 · latest
Published
Fresh — added to the Hub within the last 30 days
Added
License
CC BY 4.0
Usage

views · 0

downloads · 0

A view counts once per browser per day, a download each time the PDF is requested.

  • Computer Science

Not recorded for this paper

View statement
Looking for related papers…
Contents