betaFeedback

WorksComputer Science

AggCLIP: Aggregating CLIP for Zero-Shot Multimodal Anomaly Detection

Abstract

Zero-Shot Multimodal Anomaly Detection (ZSMAD) aims to identify both appearance and geometric defects in novel categories without relying on target-specific training data, making it especially valuable in real-world scenarios constrained by data scarcity, privacy concerns, or high annotation costs. While recent approaches successfully adapt vision-language models like CLIP to achieve this open-vocabulary generalization, they still face significant challenges: (i) severe geometric information loss during 3D-to-2D point cloud rendering; (ii) spatially noisy 2D feature maps produced by raw CLIP patch embeddings, compounded by shallow late-stage modality fusion; and (iii) severe cross-modal semantic misalignment due to decoupled processing streams. To address these limitations, we propose AggCLIP, a novel framework that adapts frozen CLIP representations for ZSMAD by explicitly incorporating local geometric structure and visual spatial context. Specifically, to accurately localize fine-grained defects, we design a Geometry-Aware Spatial Aggregation (GSA) module that refines back-projected 3D point features via density-adaptive neighborhood weighting, alongside a Context-Aware Spatial Aggregation (CSA) module that smooths multi-scale 2D patch embeddings using a lightweight trainable adapter. Furthermore, to prevent semantic drift between modalities, we introduce a Dual-Stream Prompt Contrastive (DSPC) loss, which aligns the abstract concepts of normality and abnormality across the point cloud and RGB streams. Extensive experiments demonstrate that the proposed AggCLIP framework achieves state-of-the-art performance on the MVTec 3D-AD and Eyecandies benchmarks under both one-vs-rest and cross-dataset zero-shot settings, notably surpassing the strongest competitive methods by up to 2.9% in pixel-level AUPRO.

Introduction

Industrial anomaly detection (IAD) is a critical process in modern manufacturing, playing a key role in ensuring stable operations and minimizing economic losses [2, 12, 32]. Previous methods have primarily focused on RGB images for detection [10, 18, 32, 41]. While these methods are effective for discovering texturing and surface defects, they inherently fail to capture structural and shape anomalies that are indiscernible in 2D projections. Consequently, modern industrial inspection systems increasingly demand the integration of paired RGB images and 3D point clouds for Multimodal Anomaly Detection (MAD).

Traditional MAD methods [8, 9, 15, 23, 33, 34, 37, 39, [46] typically rely on unsupervised learning schemas, which require training on a massive amount of defect-free samples from the target category. However, in real-world scenarios, acquiring sufficient training data for novel or diverse target categories is often difficult due to issues like commercial confidentiality, data privacy concerns, and prohibitive annotation costs. To address these challenges, Zero-Shot Multimodal Anomaly Detection (ZSMAD) has emerged as an essential task, aiming to identify and localize geometric and appearance defects on unseen product categories without relying on any target-specific training data.

CLIP [31], pre-trained on hundreds of millions of image-text pairs, establishes a powerful joint vision-language representation space with strong generalizability. Prompted by this capability, recent studies [13, 27, 48] have applied CLIP to zero-shot multimodal anomaly detection. The predominant paradigm involves rendering 3D point clouds into multi-view 2D images, processing these images with CLIP to derive anomaly concepts, and subsequently back-projecting the outputs to 3D space. However, as illustrated in Figure 1, existing ZSMAD methods still face three major limitations:

Paradigm comparison for zero-shot multimodal anomaly detection

Figure 1. (a) Existing ZSMAD pipelines typically suffer from three fundamental limitations: Geometric Loss during 3D-to-2D rendering, Spatial Noise and Weak Integration due to raw CLIP embeddings and simplistic late-fusion, and Semantic Misalignment between prompts and multimodal features. (b) The proposed AggCLIP framework effectively addresses these gaps. It refines noisy back-projected point features via the Geometry-Aware Spatial Aggregation (GSA) module, suppresses 2D spatial noise using Context-Aware Spatial Aggregation (CSA) paired with a learnable Adapter, and directly enforces deep cross-modal semantic interaction via the Dual-Stream Prompt Contrastive (DSPC) loss.

1) Geometric Information Loss. Projecting point clouds into 2D views discards fine-grained 3D structure, forcing CLIP to represent structural anomalies through 2D visual proxies and limiting its sensitivity to subtle geometric defects.

2) Spatial Noise and Superficial Modality Integration. Existing methods often discard paired RGB images or fuse them only at the final stage. Moreover, raw CLIP patch embeddings lack local spatial inductive biases, producing fragmented and noisy maps for dense anomaly localization.

3) Cross-Modal Semantic Misalignment. Processing multi-modal visual inputs without an explicit semantic regularization mechanism inevitably causes domain drift. This leads to a severe misalignment in the conceptual understanding of “normal” and “abnormal” boundaries between the text prompt space and the decoupled RGB and point cloud representation streams.

To overcome these limitations, we propose AggCLIP, a simple yet highly effective framework for zero-shot multimodal anomaly detection. Our key insight is that effective zero-shot inference requires explicitly incorporating local geometric structure and visual spatial context into frozen CLIP representations while strictly preserving cross-modal semantic consistency. Towards this end, we design two modality-specific spatial aggregation modules tailored for anomaly perception. For the 3D branch, we introduce the Geometry-Aware Spatial Aggregation (GSA) module, which refines noisy back-projected point features using density-adaptive neighborhoods defined on the original 3D coordinates. For the 2D branch, we design the Context-Aware Spatial Aggregation (CSA) module, paired with a lightweight adapter, to smooth hierarchical 2D patch embeddings and suppress spatial noise. Furthermore, to mitigate semantic drift, we propose a Dual-Stream Prompt Contrastive (DSPC) loss, which dynamically regularizes the textual prompt learners to rigorously align the abstractions of normality and abnormality across both modalities.

Our main contributions are summarized as follows:

  • We propose AggCLIP, a universally applicable zero-shot multimodal anomaly detection frame-

work that effectively bridges frozen CLIP representations with unified 2D appearance and 3D geometric reasoning.

  • We design GSA and CSA to refine back-projected point features with local 3D neighborhood context and RGB patch features with multi-scale spatial context, producing spatially coherent anomaly predictions.

  • We introduce the Dual-Stream Prompt Contrastive (DSPC) loss, which explicitly enforces semantic alignment between the decoupled text prompt spaces and the multimodal visual feature streams.

  • Extensive experiments on the MVTec 3D-AD and Eyecandies demonstrate that AggCLIP establishes new state-of-the-art object- and pixel-level performance under both one-vs-rest and challenging cross-dataset zero-shot settings.

Related Work

Classical Anomaly Detection

2D Anomaly Detection. Image-based industrial anomaly detection has been studied extensively under unsupervised and one-class settings. Existing methods can be broadly categorized into three major paradigms based on their underlying mechanisms. Feature embedding-based methods [10, 16, 32, 38] construct compact representations of normal patterns extracted from pre-trained networks, subsequently detecting anomalies by measuring structural deviations against memory banks or learned generative density models. Reconstruction-based methods [1, 12, 17, 18, 19, 25, 35, 36, 40, 45] leverage autoencoders and knowledge-distillation architectures trained exclusively on normal data, identifying anomalous regions by exploiting high spatial reconstruction errors. Furthermore, synthesis-based methods [6, 41, 43, 44] artificially generate pseudo-anomalous training samples through procedural texture pasting or diffusion-based synthesis, thereby enabling discriminative classifiers to learn explicit decision boundaries. Despite their widespread success, these approaches operate entirely on 2D RGB images and are inherently incapable of capturing geometric structural defects that remain invisible in planar textures.

Multimodal Anomaly Detection. To mitigate the limitations of single-modality perception, the release of comprehensive large-scale datasets such as MVTec 3D-AD [3] and Eyecandies [4] has driven rapid progress in multimodal anomaly detection. By jointly integrating 2D texture features and 3D spatial information, this paradigm enables a highly enriched and effective representation of complex physical anomaly patterns. Representative approaches have explored a diverse array of cross-modal fusion strategies. These include augmenting classical handcrafted 3D descriptors with learned 2D features [20], conducting bidirectional cross-modal feature mapping [9] [37], applying reconstruction-style multimodal distillation [15], and leveraging prototype and shape-aware 3D geometric reasoning [8, 23, 33, 34, 39, 46]. While demonstrating strong in-domain detection capabilities, these conventional multimodal frameworks remain fundamentally dependent on a massive corpus of defect-free normal samples from the target category for unsupervised training. In real-world industrial environments, strict constraints involving data privacy boundaries and prohibitive collection costs often make it exceptionally difficult to acquire sufficient normal samples. This intrinsic bottleneck significantly restricts their scalability, directly motivating the urgent exploration of zero-shot paradigms.

Zero-Shot Anomaly Detection

Zero-Shot 2D Anomaly Detection. ZSAD on unseen categories has become feasible by leveraging the generalized image-text alignment capabilities of CLIP [7, 11, 22, 26, 31, 50]. WinCLIP [21] first applied CLIP to ZSAD tasks by aggregating sliding window features with compositional text prompts. Subsequently, various prompt learning frameworks [5, 29, 30, 47] replaced static templates with learnable, object-agnostic, or context-aware tokens to adaptively capture localized defect semantics. However, these methods operate exclusively on planar 2D inputs, inherently restricting their capability to detect stereoscopic structural anomalies.

Zero-Shot Multimodal Anomaly Detection. Extending ZSAD to multimodal inputs aims to jointly localize both geometric and appearance defects without any target-specific training data. PointCLIP [42, 51] established the standard paradigm of rendering 3D point clouds into multi-view images for CLIP processing and back-projecting the features to 3D space. Building on this, PointAD [48] first adapted this paradigm to zero-shot anomaly detection by evaluating 3D structures from both point and pixel perspectives. Subsequent models like GS-CLIP [13] and ZUMA [27] further improved this pipeline via geometry-aware prompt regularization and training-free multimodal scoring, respectively. A fundamental limitation shared by these frameworks is their superficial incorporation of the native RGB modality and the total absence of explicit cross-modal semantic alignment. Our AggCLIP addresses this by incorporating local geometric structure and spatial context via modality-specific spatial aggregation modules (GSA and CSA) and enforcing multimodal semantic consistency via the DSPC loss.

Problem Formulation

Given a multimodal input X=(P,I)\mathcal{X} = (\mathcal{P}, \mathcal{I}), we aim to learn a unified model fθ:X→(s,M^)f_{\theta}: \mathcal{X} \to(s,\hat{\mathcal{M}}) that jointly predicts a sample-level anomaly score s∈[0,1]s \in[0,1] and a pixel-level anomaly map M^∈[0,1]H×W\hat{\mathcal{M}} \in[0,1]^{H \times W}. Here, the input consists of a 3D point cloud P∈RM×3\mathcal{P} \in\mathbb{R}^{M \times3} and a geometrically registered RGB image I∈RH×W×3\mathcal{I} \in\mathbb{R}^{H \times W \times3}. Sample-level anomaly detection can be formally defined as a binary classification problem, where the model aims to classify samples X\mathcal{X} as either normal (y=0y = 0) or abnormal (y=1y = 1). Anomaly segmentation extends this concept to the pixel level, utilizing a binary ground truth mask M∈{0,1}H×W\mathcal{M} \in\{0,1\}^{H \times W}. We operate within the Zero-Shot Anomaly Detection (ZSAD) context. Crucially, “zero-shot” here refers to the target product categories, meaning that the model has access to zero labeled examples from the target categories during training, but it is not training-free. Specifically, we train on an auxiliary dataset Dtrain={(Xi,Mi,yi)}i=1N\mathcal{D}_{\mathrm{train}} = \{(\mathcal{X}_i,\mathcal{M}_i,y_i)\}_{i=1}^{N} whose categories are entirely disjoint from those in the target testing set Dtest\mathcal{D}_{\mathrm{test}}, i.e., Ctrain∩Ctest=∅\mathcal{C}_{\mathrm{train}} \cap\mathcal{C}_{\mathrm{test}} = \varnothing. Despite the distribution shift P(Xtest)≠P(Xtrain)P(\mathcal{X}_{\mathrm{test}}) \ne P(\mathcal{X}_{\mathrm{train}}), the model is optimized on Dtrain\mathcal{D}_{\mathrm{train}} to generalize to unseen categories in Dtest\mathcal{D}_{\mathrm{test}}.

AggCLIP

Overview

As illustrated in Figure 2, we propose AggCLIP, a unified multimodal framework specifically designed for zero-shot 2D RGB and 3D point cloud anomaly detection. The primary objective of AggCLIP is to adapt the open-vocabulary generalized representations of a frozen vision-language model (CLIP) to the fine-grained demands of multimodal anomaly segmentation.

Overview of AggCLIP

Figure 2. Overview of AggCLIP. GSA refines back-projected point-cloud features, CSA and a lightweight adapter refine RGB patch features, and DSPC aligns modality-specific text prompt embeddings before multimodal anomaly scoring.

The AggCLIP framework operates in three sequential phases. First, we independently extract base appearance and geometric representations by passing the native RGB image and multi-view rendered point clouds through a shared, frozen CLIP vision encoder (§ 4.2). Second, to compensate for the inherent lack of local spatial inductive biases in raw CLIP features, we refine these embeddings via two modality-specific modules: the Geometry-Aware Spatial Aggregation (GSA) module for density-adaptive refinement of back-projected point features, and the Context-Aware Spatial Aggregation (CSA) module paired with a trainable adapter for 2D spatial noise suppression (§ 4.3). Finally, to prevent semantic drift between the isolated visual representations during adaptation, we introduce decoupled prompt learners regulated by a novel Dual-Stream Prompt Contrastive (DSPC) loss. This explicitly enforces cross-modal consistency by aligning the modality-specific text prompt embeddings for normality and abnormality across the point cloud and RGB streams (§ 4.4). By orchestrating these components around a frozen backbone, AggCLIP produces fine-grained anomaly score maps via text-visual similarity computation, enabling accurate zero-shot anomaly localization.

Multimodal Feature Extraction

Point Cloud Modality Feature Extraction

To leverage the open-vocabulary priors of CLIP for 3D data without training a separate 3D encoder, we adopt the Render-Extract-Backproject strategy. Given an input point cloud P∈RM×3\mathcal{P} \in\mathbb{R}^{M \times3}, we place KK virtual cameras around the object and render the point cloud from each viewpoint kk into an image Ik=R(P,k)\mathcal{I}_k = \mathcal{R}(\mathcal{P},k), converting 3D geometric structures into 2D visual representations. Each rendered image is fed into the frozen CLIP encoder Φimg\Phi_{\mathrm{img}} to extract local features Lk∈Rh′×w′×D\mathbf{L}_k \in\mathbb{R}^{h' \times w' \times D} and global features gk∈RD\mathbf{g}_k \in\mathbb{R}^{D}. The per-point base features FPbase∈RM×D\mathbf{F}_{\mathcal{P}}^{\mathrm{base}} \in\mathbb{R}^{M \times D} are then obtained by back-projecting Lk\mathbf{L}_k onto each 3D point via the pinhole camera model, sampling with bilinear interpolation, filtering occluded views with a Z-buffer visibility test, and averaging the valid features across all viewpoints (full derivation in the Appendix).

RGB Image Modality Feature Extraction

The input RGB image Irgb∈RH×W×3\mathcal{I}_{\mathrm{rgb}} \in\mathbb{R}^{H \times W \times3} is processed by the same frozen encoder Φimg\Phi_{\mathrm{img}} to obtain {grgb,Lrgb}=Φimg(Irgb)\{\mathbf{g}_{\mathrm{rgb}},\mathbf{L}_{\mathrm{rgb}}\} = \Phi_{\mathrm{img}}(\mathcal{I}_{\mathrm{rgb}}), where grgb∈RD\mathbf{g}_{\mathrm{rgb}} \in\mathbb{R}^{D} and Lrgb∈Rh′×w′×D\mathbf{L}_{\mathrm{rgb}} \in\mathbb{R}^{h' \times w' \times D} denote the global and local features, respectively. Sharing Φimg\Phi_{\mathrm{img}} across modalities ensures that RGB and rendered point cloud features reside in the same semantic space.

Multimodal Feature Spatial Aggregation

A fundamental challenge in adapting CLIP for zero-shot anomaly detection is that its features, inherently trained for image-level alignment, are noisy and lack fine-grained spatial or geometric awareness. To bridge this gap, we design two core components of our AggCLIP framework: Geometry-Aware Spatial Aggregation (GSA) for the 3D modality and Context-Aware Spatial Aggregation (CSA) for the 2D modality. These modules explicitly incorporate local geometric structure and spatial context into the frozen CLIP features.

Geometry-Aware Spatial Aggregation (GSA)

The projected 3D point features FPbase\mathbf{F}_{P}^{\mathrm{base}} initially represent isolated semantic observations, lacking geometric context. Anomalies on 3D surfaces are structurally defined by their deviations from local neighborhoods. Furthermore, raw point clouds exhibit highly non-uniform density, being dense at sharp edges and sparse on flat surfaces. Traditional fixed-scale aggregation struggles with this variance. To address this, GSA introduces a Density-Adaptive Geometry Enhancement mechanism.

First, we construct a kk-Nearest Neighbor (kk-NN) graph on the 3D coordinates pi∈R3\mathbf{p}_i \in\mathbb{R}^{3} and compute a local sparsity factor ρi=1k∑j∈Nk(i)∥pi−pj∥2\rho_i = \frac{1}{k}\sum_{j\in\mathcal{N}_k(i)} \lVert\mathbf{p}_i-\mathbf{p}_j\rVert_2, i.e., the average Euclidean distance to its kk spatial neighbors. GSA then derives a density-adaptive aggregation bandwidth σi=γ⋅ρi\sigma_i = \gamma\cdot\rho_i, where γ\gamma is a scaling coefficient. This bandwidth shrinks the receptive field in dense regions to preserve fine defect details and expands it in sparse regions to gather sufficient structural context.

We then aggregate neighborhood features with a residual connection. The unnormalized aggregated feature f~i\tilde{\mathbf{f}}_i for point pi\mathbf{p}_i is:

f~i=fibase+∑j∈Nk(i)exp⁡(−∥pi−pj∥222σi2)Ωi⋅fjbase(1)\tilde{\mathbf{f}}_i = \mathbf{f}_i^{\mathrm{base}} + \sum_{j\in\mathcal{N}_k(i)} \frac{\exp\left(-\frac{\lVert\mathbf{p}_i-\mathbf{p}_j\rVert_2^2}{2\sigma_i^2}\right)}{\Omega_i}\cdot\mathbf{f}_j^{\mathrm{base}} \tag*{(1)}

where Ωi\Omega_i is the sum of the Gaussian kernel values over Nk(i)\mathcal{N}_k(i), normalizing the weights into a distribution so that the aggregation term forms a convex combination of neighbor features. To ensure the aggregated features remain aligned with the pre-trained CLIP normalized embedding space, we apply ℓ2\ell_2-normalization to the output: firefined=f~i/∥f~i∥2\mathbf{f}_i^{\mathrm{refined}} = \tilde{\mathbf{f}}_i/\lVert\tilde{\mathbf{f}}_i\rVert_2. This yields the ℓ2\ell_2-normalized refined features FPrefined\mathbf{F}_{P}^{\mathrm{refined}}, which promote local 3D neighborhood consistency while retaining the original back-projected features through the residual connection.

Context-Aware Spatial Aggregation (CSA)

While the 2D local features Lrgb\mathbf{L}_{\mathrm{rgb}} capture rich appearance semantics, the patch-based processing of the ViT backbone [14] lacks the translation invariance and local inductive biases inherent in CNNs. In pixel-level tasks such as anomaly segmentation, this directly causes “blocky” artifacts and disjointed anomaly scores. To enforce spatial continuity without retraining the backbone, we propose the CSA module.

CSA employs a multi-scale smoothing strategy directly on the patch-resolution feature tokens across a predefined set of kernel sizes S={κ1,…,κn}\mathcal{S}=\{\kappa_1,\ldots,\kappa_n\}. For a given scale κ\kappa, the contextually aggregated feature at position (u,v)(u,v) on the patch grid is computed as:

L~rgbκ(u,v)=∑(Δu,Δv)∈RκG(Δu,Δv;σ)⋅Lrgb(u+Δu,v+Δv)(2)\widetilde{L}^{\kappa}_{\mathrm{rgb}}(u,v)=\sum_{(\Delta u,\Delta v)\in\mathcal{R}_{\kappa}}G(\Delta u,\Delta v;\sigma)\cdot L_{\mathrm{rgb}}(u+\Delta u,v+\Delta v) \tag*{(2)}

where G(⋅;σ)G(\cdot;\sigma) is a fixed Gaussian kernel shared across all scales and Rκ\mathcal{R}_{\kappa} defines the local window of size κ×κ\kappa\times\kappa. This parameter-free operation effectively mitigates local noise and ensures cohesive defect boundaries.

Although the aforementioned smoothing enforces spatial continuity, both the CLIP backbone and the Gaussian kernels remain completely frozen. To introduce trainable capacity without eroding the open-vocabulary behavior of the frozen backbone, we introduce the Anomaly-Aware Patch Adapter, a lightweight self-attention module that re-projects smoothed features into an anomaly-discriminative space. The aggregated features from all scales share one Adapter instance. We flatten L~rgbκ\widetilde{L}^{\kappa}_{\mathrm{rgb}} into a token sequence Zκ∈RN×DZ^{\kappa} \in\mathbb{R}^{N \times D} (N=h′×w′N=h^{\prime}\times w^{\prime}) and project it into queries, keys, and values:

Qκ=ZκWQ,Kκ=ZκWK,Vκ=ZκWV(3)Q^{\kappa}=Z^{\kappa}W_{Q},\quad K^{\kappa}=Z^{\kappa}W_{K},\quad V^{\kappa}=Z^{\kappa}W_{V} \tag*{(3)}

where WQ,WK,WV∈RD×DW_{Q},W_{K},W_{V}\in\mathbb{R}^{D\times D} are learnable projections following Layer Normalization. The adapted features are then obtained via self-attention with a residual connection and a subsequent feed-forward network:

L^rgbκ=Reshape⁡(FFN⁡(Zκ+Attention⁡(Qκ,Kκ,Vκ)))(4)\widehat{L}^{\kappa}_{\mathrm{rgb}}=\operatorname{Reshape}\left(\operatorname{FFN}\left(Z^{\kappa}+\operatorname{Attention}\left(Q^{\kappa},K^{\kappa},V^{\kappa}\right)\right)\right) \tag*{(4)}

where FFN⁡(⋅)\operatorname{FFN}(\cdot) is a two-layer feed-forward network with GELU activation. Here, the self-attention layer models global inter-patch dependencies, and the subsequent FFN maps the enriched features into a lower-dimensional anomaly-discriminative subspace.

Finally, we fuse the multi-scale adapted features in the similarity space. For each scale κ\kappa, we compute the raw cosine similarity between the adapted patch features and the text embeddings PrsP_{r}^{s} (defined in § 4.4), and accumulate these similarities across all scales before applying a single softmax:

Srgb(u,v)=softmax⁡s∈{n,a}(1τvla∑κ∈S⟨L^rgbκ(u,v),Prs⟩)(5)S_{\mathrm{rgb}}(u,v)=\operatorname{softmax}_{s\in\{n,a\}}\left(\frac{1}{\tau_{\mathrm{vla}}}\sum_{\kappa\in\mathcal{S}}\left\langle\widehat{L}^{\kappa}_{\mathrm{rgb}}(u,v),P_{r}^{s}\right\rangle\right) \tag*{(5)}

where τvla\tau_{\mathrm{vla}} is a temperature parameter shared across all visual-language alignment computations. Applying the softmax across the semantic state s∈{n,a}s\in\{n,a\} models the binary anomaly probability. This late-fusion strategy allows each scale to contribute its raw discriminative signal before normalization, preventing premature probability saturation that would occur if softmax were applied per-scale. Together, the Adapter serves as a lightweight domain-transfer bridge, where the self-attention layer captures inter-patch spatial dependencies and the FFN re-projects the contextualized features into an anomaly-aligned space for downstream multimodal fusion and scoring.

Dual-Stream Prompt Contrastive Learning

In multimodal zero-shot anomaly detection, a critical challenge is maintaining consistent semantic boundaries for “normal” and “abnormal” across physically distinct modality representations, given that 3D geometric anomalies and 2D appearance anomalies manifest with vastly different visual patterns. Existing prompt-based methods typically adopt one of two strategies: either employing a single shared prompt template for all modalities, which cannot adequately capture the distinct characteristics of geometric versus appearance defects; or optimizing separate prompt learners for each modality in complete isolation without any cross-modal regularization. In the latter case, the independently learned semantic representations inevitably drift apart. Specifically, the 3D branch may associate “anomaly” predominantly with geometric deformations, while the 2D branch anchors it to textural irregularities. Both strategies ultimately lead to conflicting anomaly predictions during multimodal fusion, degrading detection and localization performance.

To address this, we introduce a Dual-Stream Prompt Contrastive (DSPC) learning strategy that decouples modality-specific prompts while encouraging cross-modal semantic consistency. We initialize independent sets of learnable context vectors for each modality. Let m∈{p,r}m\in\{p,r\} denote the modality (where pp represents Point Cloud and rr represents RGB image), and s∈{n,a}s\in\{n,a\} denote the semantic state (normal and abnormal, respectively). The prompt template for a specific modality and state is formulated as:

Tms=[Wm,1s][Wm,2s]⋯[Wm,Ls]⊕state⁡s⊕cm(6)T_{m}^{s}=[W_{m,1}^{s}][W_{m,2}^{s}]\cdots[W_{m,L}^{s}]\oplus\operatorname{state}^{s}\oplus c_{m} \tag*{(6)}

where Wm,ksW_{m,k}^{s} are the learnable context tokens, LL is the context length, state⁡s∈{“normal”,“abnormal”}\operatorname{state}^{s}\in\{\text{“normal”},\text{“abnormal”}\} denotes the semantic state keyword, and cm∈{“point cloud”,“RGB image”}c_{m}\in\{\text{“point cloud”},\text{“RGB image”}\} is the modality identifier. By appending both the state label and the modality tag, the final prompt reads naturally as, e.g., “[context] normal point cloud”, grounding the text embedding to its corresponding visual branch. This dual-stream design allows each branch to learn modality-specific context representations while sharing a common state vocabulary.

Dual-Stream Prompt Contrastive (DSPC) Loss. While decoupled learners provide domain flexibility, independent optimization may cause the cross-modal semantic representations to gradually drift apart. To regularize this, we propose the DSPC loss. Let PmsP_m^s denote the ℓ2\ell_2-normalized text embedding generated by the frozen CLIP text encoder for a given prompt TmsT_m^s. We construct positive pairs from cross-modal prompts sharing the same semantic state: (Ppn,Prn)(P_p^n, P_r^n) and (Ppa,Pra)(P_p^a, P_r^a). Negative pairs include both cross-modal conflicting states (Ppn,Pra)(P_p^n, P_r^a), (Ppa,Prn)(P_p^a, P_r^n) and intra-modality opposite states (Pms,Pmsˉ)(P_m^s, P_m^{\bar{s}}).

To ensure fully symmetric alignment, we let each modality take turns as the anchor. Letting ϕ(a,b)=⟨a,b⟩/τdspc\phi(a,b)=\langle a,b\rangle/\tau_{dspc} where ⟨⋅,⋅⟩\langle\cdot,\cdot\rangle denotes the cosine similarity and τdspc\tau_{dspc} is the temperature, we define the per-anchor contrastive term for modality mm with state ss as:

Lms=−log⁡eϕ(Pms,Pmˉs)eϕ(Pms,Pmˉs)+eϕ(Pms,Pmˉsˉ)+eϕ(Pms,Pmsˉ)(7)\mathcal{L}_m^s=-\log\frac{e^{\phi(P_m^s,P_{\bar{m}}^s)}}{e^{\phi(P_m^s,P_{\bar{m}}^s)}+e^{\phi(P_m^s,P_{\bar{m}}^{\bar{s}})}+e^{\phi(P_m^s,P_m^{\bar{s}})}} \tag*{(7)}

where mˉ\bar{m} denotes the complementary modality. For each anchor PmsP_m^s, the denominator contains three terms: (i) the positive PmˉsP_{\bar{m}}^s (same state, cross-modal), (ii) the cross-modal negative PmˉsˉP_{\bar{m}}^{\bar{s}} (opposite state, cross-modal), and (iii) the intra-modal negative PmsˉP_m^{\bar{s}} (opposite state, same modality). Together, terms (ii) and (iii) enforce cross-modal semantic consistency and sharpen the intra-modal normal–abnormal decision boundary. The full DSPC loss averages over all four anchor combinations:

Ldspc=14∑m∈{p,r}∑s∈{n,a}Lms(8)\mathcal{L}_{dspc}=\frac{1}{4}\sum_{m\in\{p,r\}}\sum_{s\in\{n,a\}}\mathcal{L}_m^s \tag*{(8)}

This bidirectional formulation guarantees that both the point cloud and RGB branches are equally regularized: each modality’s intra-modal normal–abnormal boundary is explicitly sharpened, and the cross-modal semantic alignment is enforced symmetrically.

Training Objectives and Inference

Anomaly Score Map. For a target instance (P,I)(\mathcal{P},\mathcal{I}), the global sample-level classification score ss is obtained by averaging the softmax-normalized probabilities from the global visual features of both modalities:

s=12(sr+sp),s=\frac{1}{2}(s_r+s_p),
sm=1∣Gm∣∑g∈Gmexp⁡(⟨g,Pma⟩/τvla)∑s∈{n,a}exp⁡(⟨g,Pms⟩/τvla)(9)s_m=\frac{1}{|\mathcal{G}_m|}\sum_{g\in\mathcal{G}_m}\frac{\exp(\langle g,P_m^a\rangle/\tau_{vla})}{\sum_{s\in\{n,a\}}\exp(\langle g,P_m^s\rangle/\tau_{vla})} \tag*{(9)}

where Gr={grgb}\mathcal{G}_r=\{g_{\mathrm{rgb}}\} for the RGB image and Gp={gk}k=1K\mathcal{G}_p=\{g_k\}_{k=1}^{K} for the KK rendered 3D views.

Simultaneously, the dense anomaly probability map pmap_m^a is derived analogously using the local aggregated visual features vm,iv_{m,i}, where vm,iv_{m,i} denotes the feature vector at the ii-th spatial position from the GSA-refined point features (m=pm=p) or the CSA-adapted RGB features (m=rm=r). Before fusion, ppap_p^a is bilinearly upsampled to the original image resolution H×WH\times W. We apply a Gaussian filter Gσ(⋅)G_\sigma(\cdot) over both branches to mitigate noise. The final segmentation map Mfinal\mathcal{M}_{\mathrm{final}} is defined as:

Mfinal=β⋅Gσ(pra)+(1−β)⋅Gσ(ppa)(10)\mathcal{M}_{\mathrm{final}}=\beta\cdot G_\sigma(p_r^a)+(1-\beta)\cdot G_\sigma(p_p^a) \tag*{(10)}

where β∈[0,1]\beta\in[0,1] is a balancing weight.

Loss Function. During training on Dtrain\mathcal{D}_{\mathrm{train}}, we freeze the vision-language encoders and solely optimize the prompt context vectors alongside the patch adapter. We employ a composite cross-entropy classification loss Lcls=Lclsview+Lclsglobal\mathcal{L}_{\mathrm{cls}}=\mathcal{L}_{\mathrm{cls}}^{\mathrm{view}}+\mathcal{L}_{\mathrm{cls}}^{\mathrm{global}} to supervise both individual patch-level and global features and their intra-modality ensemble average. The pixel-level Visual-Language Alignment (VLA) loss Lvlam\mathcal{L}_{\mathrm{vla}}^m relies on Dice [28] and Focal [24] terms to explicitly enforce spatial consistency with the binary ground-truth mask M\mathcal{M}:

Lvlam=Focal⁡([pmn,pma],M)+Dice⁡(pmn,M)+Dice⁡(pmn,1−M)(11)\mathcal{L}_{\mathrm{vla}}^m=\operatorname{Focal}([p_m^n,p_m^a],\mathcal{M}) +\operatorname{Dice}(p_m^n,\mathcal{M}) +\operatorname{Dice}(p_m^n,1-\mathcal{M}) \tag*{(11)}

The total objective function is formulated as:

Ltotal=∑m∈{p,r}Lvlam+Lcls+λ⋅Ldspc(12)\mathcal{L}_{\mathrm{total}}=\sum_{m\in\{p,r\}}\mathcal{L}_{\mathrm{vla}}^m+\mathcal{L}_{\mathrm{cls}}+\lambda\cdot\mathcal{L}_{\mathrm{dspc}} \tag*{(12)}

where λ\lambda governs the DSPC regularization scaling.

Experiments

Experimental Settings

Datasets. We evaluate our approach on two widely used industrial anomaly detection benchmarks: the real-world MVTec 3D-AD [3] and the synthetic Eyecandies [4]. Both datasets provide paired RGB images and point clouds, along with image-level labels and pixel-level masks for comprehensive evaluation.

Evaluation Protocols. Following established zero-shot formulations (§ 3), we evaluate the generalization capability of AggCLIP under two distinct settings. First, following the protocol established in [48], we adopt the one-vs-rest setting to assess within-domain generalization: AggCLIP is trained on a single category and evaluated on the remaining unseen categories. Second, we explore a more challenging cross-dataset setting to measure generalization against domain shifts. To evaluate model generalizability on unseen datasets, we train AggCLIP on MVTec 3D-AD and test it on Eyecandies.

Evaluation Metrics. We evaluate our method using four standard metrics. For object-level detection, we employ O-AUROC and O-AP, with the latter better reflecting performance on inherently imbalanced industrial datasets. For pixel-level localization, we utilize P-AUROC and P-AUPRO. We emphasize AUPRO as it weights distinct anomaly components equally, preventing large defects from dominating the evaluation. All metrics are reported as percentages.

Implementation Details. We adopt CLIP ViT-L/14@336px as the frozen multimodal backbone (D=1024D = 1024). RGB images and rendered views are resized to 336×336336 \times336. We use the Open3D library [49] to generate K=9K = 9 multi-view 2D rendered images by rotating the point cloud around the X-axis at angles of {−45π,−35π,…,35π,45π}\{-\frac{4}{5}\pi, -\frac{3}{5}\pi, \ldots, \frac{3}{5}\pi, \frac{4}{5}\pi\}. In GSA, we set the neighborhood size strictly to k=10k = 10 and scaling coefficient γ=0.5\gamma= 0.5. In CSA, we use multi-scale Gaussian kernel sizes S={1,3,5}S = \{1, 3, 5\}. The standard deviation for both the CSA Gaussian kernels and the post-processing filter Gσ(⋅)G_{\sigma}(\cdot) is set to σ=4\sigma= 4. The hidden dimension of the two-layer FFN in the Adapter is 512. The learnable prompt context length is set to L=12L = 12. The temperature parameters are τvla=0.07\tau_{vla} = 0.07 and τdspc=0.5\tau_{dspc} = 0.5, and the multimodal fusion weight is β=0.5\beta= 0.5. Training runs for 15 epochs with Adam (batch size 4, learning rate 0.001, β1=0.5\beta_{1} = 0.5, β2=0.999\beta_{2} = 0.999), and the DSPC loss weight is λ=1\lambda= 1. All experiments are implemented in PyTorch on a single RTX 3090 GPU. More details are available in the Appendix.

Main Experimental Results

Quantitative Analysis. Tables 1 and 2 demonstrate that AggCLIP achieves state-of-the-art zero-shot performance under both the one-vs-rest and cross-dataset settings. In the one-vs-rest evaluation, AggCLIP consistently ranks first across all reported metrics for both the MVTec 3D-AD and Eyecandies. Specifically, on MVTec 3D-AD, it outpaces PointAD [48] by 1.4% in O-AUROC and 3.7% in P-AUPRO, while

MethodsMVTec 3D-AD Object-level (O-R, O-A)MVTec 3D-AD Pixel-level (P-R, P-P)Eyecandies Object-level (O-R, O-A)Eyecandies Pixel-level (P-R, P-P)
CLIP [31](60.4, 86.4)(-, 56.0)(73.0, 73.9)(78.0, 31.8)
AnomalyCLIP [47](66.2, 87.6)(91.6, 70.9)(65.0, 67.5)(85.0, 56.2)
PointAD [48](86.9, 96.1)(97.2, 90.2)(77.7, 80.4)(95.3, 84.3)
ZUMA-FT [27](88.2, -)(97.8, 93.1)(78.2, -)(97.5, 89.5)
GS-CLIP [13](88.2, 97.5)(97.6, 91.3)(79.3, 82.2)(95.8, 86.4)
AggCLIP (Ours)(88.3, 97.5)(98.4, 93.9)(79.8, 82.4)(98.2, 89.8)

Table 1. Performance comparison of zero-shot multimodal anomaly detection under the one-vs-rest setting on the MVTec 3D-AD and Eyecandies datasets. Best results are highlighted in red, and runners-up in blue. Throughout all tables, O-R, O-A, P-R, and P-P denote O-AUROC, O-AP, P-AUROC, and P-AUPRO, respectively.

MethodsObject-level (O-R, O-A)Pixel-level (P-R, P-P)
AnomalyCLIP [47](65.7, 68.1)(86.2, 61.3)
PointAD [48](78.6, 80.8)(94.0, 80.7)
ZUMA-FT* [27](79.1, 81.2)(94.8, 81.9)
GS-CLIP* [13](80.2, 82.2)(94.6, 82.4)
AggCLIP (Ours)(81.4, 84.2)(95.8, 85.3)

Table 2. Performance comparison of zero-shot multimodal anomaly detection under the cross-dataset setting (trained on MVTec 3D-AD, tested on Eyecandies). * Indicates results obtained by running the official code.

also achieving more precise pixel-level localization than strong baselines like GS-CLIP [13] and ZUMA-FT [27]. On the Eyecandies dataset, AggCLIP establishes new peaks of 79.8%/82.4% in O-AUROC/O-AP at the object level and 98.2%/89.8% in P-AUROC/P-AUPRO at the pixel level. Furthermore, under the more challenging cross-dataset setting, AggCLIP maintains its superiority against all competitors. It surpasses the runner-up by absolute margins of 1.2% and 2.0% in O-AUROC and O-AP over GS-CLIP [13], alongside notable improvements of 1.0% in P-AUROC over ZUMA-FT [27] and 2.9% in P-AUPRO over GS-CLIP, successfully verifying its stronger generalization against domain shifts. More quantitative results are available in the Appendix.

Qualitative Analysis. Figure 3 shows that AggCLIP produces anomaly maps with clearer boundaries and fewer false activations than GS-CLIP [13]. It localizes subtle or irregular defective regions more completely, while suppressing responses on normal textured areas. We attribute this improvement to our spatial aggregation modules (GSA and CSA), which incorporate local geometric structure and appearance spatial context into the frozen CLIP features, enabling more fine-grained and spatially coherent anomaly localization. This indicates that aggregating appearance and geometric information helps the model better distinguish true defects from category-specific visual variations. More qualitative results are available in the Appendix.

Qualitative results on MVTec 3D-AD and Eyecandies

Figure 3. Qualitative results on MVTec 3D-AD and Eyecandies. From top to bottom: input RGB images, 3D point clouds, ground truth, anomaly maps from GS-CLIP, and anomaly maps from AggCLIP.

Computational Efficiency

Because inference speed and resource management are critical for real-world industrial deployment, we evaluated the computational overhead of various methods, as detailed in Table 3. AggCLIP consistently outperforms both ZUMA-FT [27] and GS-CLIP [13] across all efficiency metrics. Although it is marginally slower than PointAD [48], AggCLIP yields substantial gains in anomaly detection and localization accuracy. This demonstrates that AggCLIP achieves an optimal trade-off between inference speed and detection performance.

MethodsGFLOPsFPSTime (s)Memory (MB)
CLIP* [31]2330.253.640.272270
PointAD* [48]4252.453.200.312610
ZUMA-FT* [27]4469.100.531.893856
GS-CLIP* [13]9591.981.120.893677
AggCLIP (Ours)4259.973.070.332645

Table 3. Efficiency comparison of different methods. * Indicates results obtained by running the official code.

Ablation Studies

Ablation on Different Components. Table 4 evaluates the contribution of each proposed component. The base-

VersionGSACSAAdapterLdspc\mathcal{L}_{\mathrm{dspc}}Object-level (O-R, O-A)Pixel-level (P-R, P-P)
V1✗✗✗✗(81.2, 89.6)(90.3, 88.9)
V2✓✗✗✗(87.4, 96.7)(97.8, 93.0)
V3✗✓✗✗(85.5, 94.3)(95.2, 90.4)
V4✗✓✓✗(86.8, 95.7)(96.2, 91.3)
V5✓✓✓✗(87.7, 97.1)(98.0, 93.6)
V6✓✓✓✓(88.3, 97.5)(98.4, 93.9)

Table 4. Ablation on different components.

line (V1) retains only the dual-branch architecture with decoupled learnable prompts, omitting GSA, CSA, the Adapter, and Ldspc\mathcal{L}_{\mathrm{dspc}}. Notably, integrating GSA into the 3D branch (V2) yields the most substantial single-module improvement (+6.2% O-AUROC, +4.1% P-AUPRO), confirming that density-adaptive geometric aggregation is critical for stabilizing noisy back-projected point features. A finer-grained comparison of different aggregation strategies within GSA is provided in the Appendix. Conversely, in the RGB branch, applying CSA (V3) and further equipping the Adapter (V4) progressively boost performance. The Adapter contributes an additional +1.3% O-AUROC and +0.9% P-AUPRO over CSA alone, validating the necessity of learnable feature adaptation for 2D appearance features. When the spatial aggregation modules from both modalities are jointly combined (V5), strong 2D–3D complementarity emerges, pushing P-AUROC to 98.0%. Finally, the full framework (V6), augmented with the Ldspc\mathcal{L}_{\mathrm{dspc}} loss, achieves the highest performance across all metrics, verifying that explicit cross-modal prompt alignment effectively tightens semantic consistency.

Ablation on the Number of Rendering Views. The number of rendered views KK controls the richness of 3D structural information transferred to point features. Figure 4(a) demonstrates that increasing KK steadily improves performance by providing more comprehensive multi-view geometric information. Performance saturates at K=9K = 9, which we adopt to balance accuracy and efficiency.

Ablation on key hyperparameters

Figure 4. Ablation on key hyperparameters on MVTec 3D-AD under the one-vs-rest setting. (a) Varying the number of rendered views KK. (b) Varying the GSA neighborhood size kk.

Ablation on the Number of Neighbors in GSA. Figure 4(b) reports GSA performance as a function of the neighborhood size kk. Both object- and pixel-level metrics peak at k=10k = 10, which we adopt as the default. Performance varies only marginally across k∈{5,10,15,20,25}k \in\{5, 10, 15, 20, 25\}, confirming that GSA is stable under the choice of neighborhood scale.

Ablation on the Multi-Scale Gaussian Kernels in CSA. As shown in Table 5, we investigate the influence of the Gaussian kernel set SS in the CSA module. While single-scale kernels provide a baseline, they suffer from a limited receptive field. Specifically, {1} captures fine details but lacks spatial smoothing, whereas {3} and {5} aggregate broader context at the expense of subtle defect boundaries. Combining these scales establishes a strong hierarchical representation. Integrating {1, 3} effectively bridges local and intermediate features, yielding notable improvements. Ultimately, the full multi-scale setting {1, 3, 5} achieves optimal performance across all metrics, peaking at 88.3% O-AUROC and 98.4% P-AUROC. This validates that combining distinct kernel sizes allows the CSA module to simultaneously preserve fine-grained structural integrity and encode rich regional context for accurate anomaly localization.

Kernel SizesObject-level (O-R, O-A)Pixel-level (P-R, P-P)
{1}(88.0, 97.1)(97.3, 92.9)
{3}(87.9, 96.8)(97.7, 92.6)
{5}(87.7, 97.2)(97.8, 93.0)
{1, 3}(88.1, 97.4)(98.1, 93.5)
{1, 3, 5}(88.3, 97.5)(98.4, 93.9)

Table 5. Ablation study on the multi-scale Gaussian kernel sizes SS in the CSA module.

Ablation on Modality. To evaluate the necessity of multimodal inputs, Table 6 compares single-modality baselines against our full framework. Relying solely on either 2D RGB or 3D point cloud data yields suboptimal performance. In contrast, by jointly fusing 2D and 3D features, AggCLIP achieves substantial gains across both the MVTec 3D-AD and Eyecandies datasets. This confirms that integrating complementary multimodal information is crucial for effective anomaly detection. Ablations on hyperparameter sensitivity and rendering condition are available in the Appendix.

DatasetModalityObject-level (O-R, O-A)Pixel-level (P-R, P-P)
MVTec 3D-ADRGB(82.3, 88.2)(93.4, 86.7)
MVTec 3D-ADPoint Cloud(84.1, 96.6)(97.2, 88.3)
MVTec 3D-ADRGB + Point Cloud(88.3, 97.5)(98.4, 93.9)
EyecandiesRGB(74.5, 77.3)(93.6, 78.8)
EyecandiesPoint Cloud(73.7, 78.5)(92.4, 79.6)
EyecandiesRGB + Point Cloud(79.8, 82.4)(98.2, 89.8)

Table 6. Ablation on modality.

Conclusion

In this paper, we presented AggCLIP, a novel zero-shot multimodal anomaly detection framework that adapts a frozen CLIP backbone for dense industrial inspection. To bridge the gap between image-level pre-training and fine-grained defect localization, AggCLIP explicitly synergizes 2D appearance and 3D geometric information through modality-specific spatial aggregation and aligns them via DSPC learning. This design effectively overcomes the global bias inherent in pre-trained vision-language models. Extensive evaluations demonstrate that AggCLIP establishes new state-of-the-art performance on the MVTec 3D-AD and Eyecandies datasets across both one-vs-rest and cross-dataset settings, all while maintaining high inference efficiency.

Acknowledgments

This work was supported by the Beijing Natural Science Foundation under Grant No. L243018.

References

  1. [1]Kilian Batzner, Lars Heckler, and Rebecca König. 2024. EfficientAD: Accurate Visual Anomaly Detection at Millisecond-Level Latencies. In Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision (WACV). 128–138.
  2. [2]Paul Bergmann, Michael Fauser, David Sattleger, and Carsten Steger. 2019. MVTec AD – A Comprehensive Real-World Dataset for Unsupervised Anomaly Detection. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). 9592–9600.DOI
  3. [3]Paul Bergmann, Xin Jin, David Sattleger, and Carsten Steger. 2022. The MVTec 3D-AD Dataset for Unsupervised 3D Anomaly Detection and Localization. In Proceedings of the 17th International Joint Conference on Computer Vision, Imaging and Computer Graphics Theory and Applications (VISIGRAPP). 202–213. doi:10.5220/001086500003124DOI
  4. [4]Luca Bonfiglioli, Marco Toschi, Davide Silvestri, Nicola Fioraio, and Daniele De Gregorio. 2022. The Eyecandies Dataset for Unsupervised Multimodal Anomaly Detection and Localization. In Asian Conference on Computer Vision (ACCV). 459–475.arxiv.org/abs/2210.04570
  5. [5]Yunkang Cao, Jiangning Zhang, Luca Frittoli, Yuqi Cheng, Weiming Shen, and Giacomo Boracchi. 2024. AdaCLIP: Adapting CLIP with Hybrid Learnable Prompts for Zero-Shot Anomaly Detection. In European Conference on Computer Vision (ECCV), Vol. 15093. 55–72.DOI
  6. [6]Qiyu Chen, Huiyuan Luo, Chengkan Lv, and Zhengtao Zhang. 2024. A Unified Anomaly Synthesis Strategy with Gradient Ascent for Industrial Anomaly Detection and Localization. In European Conference on Computer Vision (ECCV), Vol. 15125. 37–54. doi:10.1007/978-3-031-72855-6_3DOI
  7. [7]Xuhai Chen, Jiangning Zhang, Guanzhong Tian, Haoyang He, Wuhao Zhang, Yabiao Wang, Chengjie Wang, and Yong Liu. 2024. CLIP-AD: A Language-Guided Staged Dual-Path Model for Zero-Shot Anomaly Detection. In Human Activity Recognition and Anomaly Detection (IJCAI 2024 Workshop). Springer, 17–33. doi:10.1007/978-981-97-9003-6_2DOI
  8. [8]Yu-Min Chu, Chieh Liu, Ting-I Hsieh, Hwann-Tzong Chen, and Tyng-Luh Liu. 2023. Shape-Guided Dual-Memory Learning for 3D Anomaly Detection. In International Conference on Machine Learning (ICML). 6185–6194.
  9. [9]Alex Costanzino, Pierluigi Zama Ramirez, Giuseppe Lisanti, and Luigi Di Stefano. 2024. Multimodal Industrial Anomaly Detection by Crossmodal Feature Mapping. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). 17234–17243.arxiv.org/abs/2312.04521
  10. [10]Thomas Defard, Aleksandr Setkov, Angelique Loesch, and Romaric Audigier. 2021. PaDiM: A Patch Distribution Modeling Framework for Anomaly Detection and Localization. In International Conference on Pattern Recognition (ICPR). 475–489.arxiv.org/abs/2011.08785
  11. [11]Chenghao Deng, Haote Xu, Xiaolu Chen, Haodi Xu, Xiaotong Tu, Xingchao Ding, and Yue Huang. 2024. SimCLIP: Refining Image-Text Alignment with Simple Prompts for Zero-/Few-shot Anomaly Detection. In Proceedings of the 32nd ACM International Conference on Multimedia (MM). 1761–1770. doi:10.1145/3664647.3681376
  12. [12]Hanqiu Deng and Xingyu Li. 2022. Anomaly Detection via Reverse Distillation from One-Class Embedding. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). 9737–9746.arxiv.org/abs/2201.10703
  13. [13]Zehao Deng, An Liu, and Yan Wang. 2026. GS-CLIP: Zero-shot 3D Anomaly Detection by Geometry-Aware Prompt and Synergistic View Representation Learning. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). 35587–35596.arxiv.org/abs/2602.19206
  14. [14]Alexey Dosovitskiy, Lucas Beyer, Alexander Kolesnikov, Dirk Weissenborn, Xiaohua Zhai, Thomas Unterthiner, Mostafa Dehghani, Matthias Minderer, Georg Heigold, Sylvain Gelly, Jakob Uszkoreit, and Neil Houlsby. 2021. An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale. In International Conference on Learning Representations (ICLR).
  15. [15]Zhihao Gu, Jiangning Zhang, Liang Liu, Xu Chen, Jinlong Peng, Zhenye Gan, Guannan Jiang, Annan Shu, Yabiao Wang, and Lizhuang Ma. 2024. Rethinking Reverse Distillation for Multi-Modal Anomaly Detection. In Proceedings of the AAAI Conference on Artificial Intelligence. 8445–8453.DOI
  16. [16]Denis Gudovskiy, Shun Ishizaka, and Kazuki Kozuka. 2022. CFLOW-AD: Real-Time Unsupervised Anomaly Detection with Localization via Conditional Normalizing Flows. In Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision (WACV). 98–107.arxiv.org/abs/2107.12571
  17. [17]Jia Guo, Shuai Lu, Lei Fan, Zelin Li, Donglin Di, Yang Song, Weihang Zhang, Wenbing Zhu, Hong Yan, Fang Chen, Huiqi Li, and Hong’en Liao. 2025. One Dinomaly2 Detect Them All: A Unified Framework for Full-Spectrum Unsupervised Anomaly Detection. CoRR abs/2510.17611 (2025). Extended version of CVPR 2025. doi:10.48550/arXiv.2510.17611
  18. [18]Jia Guo, Shuai Lu, Weihang Zhang, Fang Chen, Huiqi Li, and Hong’en Liao. 2025. Dinomaly: The Less Is More Philosophy in Multi-Class Unsupervised Anomaly Detection. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). 20405–20415.arxiv.org/abs/2405.14325
  19. [19]Haoyang He, Yuhu Bai, Jiangning Zhang, Qingdong He, Hongxu Chen, Zhenye Gan, Chengjie Wang, Xi-angtai Li, Guanzhong Tian, and Lei Xie. 2024. Mam-baAD: Exploring State Space Models for Multi-Class Unsupervised Anomaly Detection. In Advances in Neural Information Processing Systems (NeurIPS), Vol. 37. 71162–71187. doi:10.5220/079017–2274arxiv.org/abs/2404.06564
  20. [20]Eliahu Horwitz and Yedid Hoshen. 2023. Back to the Feature: Classical 3D Features Are (Almost) All You Need for 3D Anomaly Detection. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) Workshops. 2968–2977.arxiv.org/abs/2203.05550
  21. [21]Jongheon Jeong, Yang Zou, Taewan Kim, Dongqing Zhang, Avinash Ravichandran, and Onkar Dabeer. 2023. WinCLIP: Zero-/Few-Shot Anomaly Classification and Segmentation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). 19606–19616.arxiv.org/abs/2303.14814
  22. [22]Xurui Li, Ziming Huang, Feng Xue, and Yu Zhou. 2024. MuSc: Zero-Shot Industrial Anomaly Classification and Segmentation with Mutual Scoring of the Unlabeled Images. In International Conference on Learning Representations (ICLR).arxiv.org/abs/2401.16753
  23. [23]Yiting Li, Xulei Yang, Jing Zhang, Sichao Tian, Jingyi Liao, and Fayao Liu. 2026. PIRN: Prototypical-based Intra-modal Reconstruction with Normality Communication for Multi-modal Anomaly Detection. In International Conference on Learning Representations (ICLR). Poster.
  24. [24]Tsung-Yi Lin, Priya Goyal, Ross Girshick, Kaiming He, and Piotr Dollár. 2017. Focal Loss for Dense Object Detection. In Proceedings of the IEEE International Conference on Computer Vision (ICCV). 2980–2988.
  25. [25]Wei Luo, Yunkang Cao, Haiming Yao, Xiaotian Zhang, Jianan Lou, Yuqi Cheng, Weiming Shen, and Wenyong Yu. 2025. Exploring Intrinsic Normal Prototypes within a Single Image for Universal Anomaly Detection. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). 9974–9983.arxiv.org/abs/2503.02424
  26. [26]Wenxin Ma, Xu Zhang, Qingsong Yao, Fenghe Tang, Chenxu Wu, Yingtai Li, Rui Yan, Zihang Jiang, and S. Kevin Zhou. 2025. AA-CLIP: Enhancing Zero-Shot Anomaly Detection via Anomaly-Aware CLIP. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). 4744–4754.arxiv.org/abs/2503.06661
  27. [27]Yunfeng Ma, Min Liu, Shuai Jiang, Jingyu Zhou, Yuan Bian, Xueping Wang, and Yaonan Wang. 2026. ZUMA: Training-free Zero-shot Unified Multimodal Anomaly Detection. IEEE Transactions on Pattern Analysis and Machine Intelligence 48, 6 (2026), 6601–6614. doi:10.1109/TPAMI.2026.3658856DOI
  28. [28]Fausto Milletari, Nassir Navab, and Seyed-Ahmad Ahmadi. 2016. V-Net: Fully Convolutional Neural Networks for Volumetric Medical Image Segmentation. In International Conference on 3D Vision (3DV). 565–571.arxiv.org/abs/1606.04797
  29. [29]Zhen Qu, Xian Tao, Xinyi Gong, ShiChen Qu, Qiyu Chen, Zhengtao Zhang, Xingang Wang, and Guiguang Ding. 2025. Bayesian Prompt Flow Learning for Zero-Shot Anomaly Detection. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). 30398–30408.
  30. [30]Zhen Qu, Xian Tao, Mukesh Prasad, Fei Shen, Zhengtao Zhang, Xinyi Gong, and Guiguang Ding. 2024. VCP-CLIP: A Visual Context Prompting Model for Zero-Shot Anomaly Segmentation. In European Conference on Computer Vision (ECCV). 301–317.arxiv.org/abs/2407.12276
  31. [31]Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, et al. 2021. Learning Transferable Visual Models from Natural Language Supervision. In International Conference on Machine Learning (ICML). 8748–8763.arxiv.org/abs/2103.00020
  32. [32]Karsten Roth, Latha Pemula, Joaquin Zepeda, Bernhard Schölkopf, Thomas Brox, and Peter Gehler. 2022. Towards Total Recall in Industrial Anomaly Detection. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). 14318–14328.arxiv.org/abs/2106.08265
  33. [33]Marco Rudolph, Tom Wehrbein, Bodo Rosenhahn, and Bastian Wandt. 2023. Asymmetric Student-Teacher Networks for Industrial Anomaly Detection. In Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision (WACV). 2592–2602.arxiv.org/abs/2210.07829
  34. [34]Wenbo Sui, Daniel Lichau, Josselin Lefèvre, and Harold Philippeau. 2026. Incomplete Multimodal Industrial Anomaly Detection via Cross-Modal Distillation. Information Fusion 126 (2026), 103572. doi:10.1016/j.inffus.2025.103572DOI
  35. [35]Guodong Wang, Shumin Han, Errui Ding, and Di Huang. 2021. Student-Teacher Feature Pyramid Matching for Anomaly Detection. In British Machine Vision Conference (BMVC). Article 349, 14 pages. doi:10.5244/C.35.349
  36. [36]Xiaolei Wang, Xiaoyang Wang, Huihui Bai, Eng Gee Lim, and Jimin Xiao. 2025. DecAD: Decoupling Anomalies in Latent Space for Multi-Class Unsupervised Anomaly Detection. In Proceedings of the Yue Wang, Jinlong Peng, Jiangning Zhang, Ran Yi, Yabiao Wang, and Chengjie Wang. 2023. Multimodal Industrial Anomaly Detection via Hybrid Fusion. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). 8032–8041.
  37. [38]Sheng Wu, Yimi Wang, Xudong Liu, Yuguang Yang, Runqi Wang, Guodong Guo, David Doermann, and Baochang Zhang. 2025. DFM: Differentiable Feature Matching for Anomaly Detection. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). 15224–15233.DOI
  38. [39]An Xiang, Zixuan Huang, Xitong Gao, Kejiang Ye, and Cheng-Zhong Xu. 2025. BridgeNet: A Unified Multimodal Framework for Bridging 2D and 3D Industrial Anomaly Detection. CoRR abs/2507.19253 (2025). doi:10.48550/arXiv.2507.19253arxiv.org/abs/2507.19253
  39. [40]Zhiyuan You, Lei Cui, Yujun Shen, Kai Yang, Xin Lu, Yu Zheng, and Xinyi Le. 2022. A Unified Model for Multi-class Anomaly Detection. In Advances in Neural Information Processing Systems (NeurIPS), Vol. 35. 4571–4584. doi:10.52202/068431–0330arxiv.org/abs/2206.03687
  40. [41]Vitjan Zavrtanik, Matej Kristan, and Danijel Skočaj. 2021. DRAEM – A Discriminatively Trained Reconstruction Embedding for Surface Anomaly Detection. In Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV). 8330–8339.arxiv.org/abs/2108.07610
  41. [42]Renrui Zhang, Ziyu Guo, Wei Zhang, Kunchang Li, Xupeng Miao, Bin Cui, Yu Qiao, Peng Gao, and Hongsheng Li. 2022. PointCLIP: Point Cloud Understanding by CLIP. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). 8552–8562.arxiv.org/abs/2112.02413
  42. [43]Xuan Zhang, Shiyu Li, Xi Li, Ping Huang, Jiulong Shan, and Ting Chen. 2023. DeSTSeg: Segmentation Guided Denoising Student-Teacher for Anomaly Detection. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). 3914–3923.arxiv.org/abs/2211.11317
  43. [44]Ximiao Zhang, Min Xu, and Xiuzhuang Zhou. 2024. RealNet: A Feature Selection Network with Realistic Synthetic Anomaly for Anomaly Detection. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). 16699–16708.arxiv.org/abs/2403.05897
  44. [45]Ying Zhao. 2023. OmniAL: A Unified CNN Framework for Unsupervised Anomaly Localization. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). 3924–3933.DOI
  45. [46]Yuan Zhao, Youwei Pang, Lihe Zhang, Hanqi Liu, Jiaming Zuo, Huchuan Lu, and Xiaoqi Zhao. 2026. UniMMAD: Unified Multi-Modal and Multi-Class Anomaly Detection via MoE-Driven Feature Decompression. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). 28502–28511.arxiv.org/abs/2509.25934
  46. [47]Qihang Zhou, Guansong Pang, Yu Tian, Shibo He, and Jiming Chen. 2024. AnomalyCLIP: Object-agnostic Prompt Learning for Zero-shot Anomaly Detection. In The Twelfth International Conference on Learning Representations (ICLR).arxiv.org/abs/2310.18961
  47. [48]Qihang Zhou, Jiangtao Yan, Shibo He, Wenchao Meng, and Jiming Chen. 2024. PointAD: Comprehending 3D Anomalies from Points and Pixels for Zero-shot 3D Anomaly Detection. In Advances in Neural Information Processing Systems (NeurIPS), Vol. 37. 84866–84896. doi:10.52202/079017–2695arxiv.org/abs/2410.00320
  48. [49]Qian-Yi Zhou, Jaesik Park, and Vladlen Koltun. 2018. Open3D: A Modern Library for 3D Data Processing. CoRR abs/1801.09847 (2018). doi:10.48550/arXiv.1801.09847DOI
  49. [50]Jiawen Zhu and Guansong Pang. 2024. Toward Generalist Anomaly Detection via In-context Residual Learning with Few-shot Sample Prompts. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). 17826–17836.arxiv.org/abs/2403.06495
  50. [51]Xiangyang Zhu, Renrui Zhang, Bowei He, Ziyu Guo, Ziyao Zeng, Zipeng Qin, Shanghang Zhang, and Peng Gao. 2023. PointCLIP V2: Prompting CLIP and GPT for Powerful 3D Open-world Learning. In Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV). 2639–2650.arxiv.org/abs/2211.11682

Paper details

vlnc:W_9egx9uhv
Version
v1 · latest
Published
Fresh — added to the Hub within the last 30 days
Added
License
CC BY 4.0
Usage

views · 6

downloads · 0

A view counts once per browser per day, a download each time the PDF is requested.

  • Computer Science

Not recorded for this paper

View statement
Looking for related papers…
Contents