betaFeedback

WorksComputer Science

When AUC Meets Generative Reconstruction: A Unified Principle for Balanced Visual Representation Learning

Abstract

Multimodal large language models (MLLMs) require visual encoders that extract fine-grained visual features while constructing semantically disentangled and discriminative representations. Conventional methodologies encounter a fundamental trade-off: contrastive learning enhances inter-class discrimination by suppressing intra-class variability, whereas generative reconstruction emphasizes perceptual fidelity but lacks explicit comparative supervision. The naive aggregation of these objectives via composite loss functions often induces optimization conflicts and instability due to gradient interference within the shared latent space. To resolve this, we introduce a unified representation-learning paradigm that reformulates generative reconstruction as a discriminative ranking problem within the generative output manifold. Our approach maintains perceptual reconstruction as a supervisory signal and integrates a ranking-based discriminative constraint by systematically ordering matched and mismatched reconstruction outputs. This formulation embeds discriminative information seamlessly into the generative process, alleviating the risk of loss competition. Specifically, we propose AUC Generative Reconstruction (AUCGR), which optimizes the area under the ROC curve (AUC) by ensuring higher scores for matched pairs than for mismatched pairs, and Partial AUC Generative Reconstruction (pAUCGR), which addresses supervision bias by focusing on mismatched comparisons within reliable feature subspaces. The proposed objectives are designed to be plug-and-play, requiring no architectural modifications to existing generative reconstruction frameworks. Theoretical analysis establishes rigorous connections to AUC/pAUC optimization and provides generalization guarantees. Extensive experiments validate that our method advances both discriminative accuracy and fine-grained perceptual representation.

Introduction

In recent years, Multimodal Large Language Models (MLLMs) [1], [2], [3] have undergone rapid advancement, with their performance increasingly governed by the quality of multimodal encoders. As a crucial component, vision encoders map images into compact visual representations that facilitate visual understanding and reasoning in downstream tasks [4], [5], [6], [7], [8], [9], [10]. Most encoders leverage contrastive pretraining, benefiting from strong zero-shot generalization capabilities and effective cross-modal alignment [11]. Nonetheless, the self-supervised text-image alignment achieved by these models remains relatively coarse [12]. Consequently, the resulting representations are often suboptimal, posing a major bottleneck for broader deployment [13].

A key challenge in representation learning is that the goal of a good representation is inherently abstract. The seminal review by Bengio et al. suggests that effective representations are linked to their capacity to capture the underlying factors of variation within data [14]. In addition, LeCun, Bengio, and Hinton’s Nature paper demonstrates that deep neural networks learn hierarchical representations: lower layers encode local structures while higher layers capture object-level abstractions [15]. This hierarchy exposes a fundamental tension between semantic abstraction and perceptual preservation. Representations must be invariant enough to enable category-level discrimination, while still retaining local details necessary for fine-grained understanding. This phenomenon is also observed in Kornblith, Shlens, and Quoc V. Le’s work, which finds that ImageNet-trained features are less general than previously thought, especially for fine-grained tasks {citep}[16]`. Building on these insights, recent foundation models increasingly advocate that a robust visual representation should simultaneously exhibit two complementary properties. The first is Discriminative Ability (D-Ability), which assesses whether the representation can identify the primary category or detect the presence of a target concept. The second is Fine-grained Perceptual Ability (P-Ability), which evaluates whether the representation retains pose, orientation, local attributes, and subtle visual differences.

Existing approaches employ distinct optimization strategies for these two capabilities. (1) Various forms of contrastive learning [17, 18, 19, 20, 21] encourage models to suppress local variations and low-level noise, thereby promoting stable semantic representations and strengthening D-Ability. As depicted in Figure 1(b), this process tends to cluster learned representations into compact semantic groups. (2) With the emergence of generative models, recent research leverages generative feedback for further enhancement [22, 23, 24, 25]. In this setting, generative models reconstruct images conditioned on visual features, where accurate reconstruction implies that the visual representation preserves detailed information from the input. This approach defines a reconstruction loss via a mean squared error (MSE) objective, thereby enhancing P-Ability. As shown in Figure 1(a), this objective produces a sparser representation space that maintains fine-grained variations across samples.

Figure 1. Motivation of our method. (a) Reconstructive learning improves P-Ability by preserving a sparser representation space. (b) Contrastive learning improves D-Ability by organizing representations into compact semantic clusters. (c) The naive method searches for a balanced representation through heuristic weight tuning between contrastive learning and reconstructive learning, but the proper weights are difficult to determine. (d) The target balanced representation, which is directly optimized by our method without heuristic weight tuning.

However, these two approaches typically optimize only one objective at a time, resulting in a trade-off between D-Ability and P-Ability. As illustrated in Figure 1(d), an ideal balanced representation should be sparse while maintaining clear semantic separability. This leads to a natural question:

Is there an intuitive optimization strategy that embraces both discriminative and fine-grained perceptual ability?

In this paper, we answer this question affirmatively. We first motivate our approach by proposing a straightforward solution that linearly combines these two seemingly contradictory objectives within one framework, as depicted in Figure 1(c). However, both theoretical analysis (§ 5.1.1) and empirical results (§ 5.1.2) indicate that this naive solution cannot directly resolve the conflict of the two goals.

To circumvent the need for explicit balancing, we propose a unified supervision strategy within generative reconstruction. The key insight is that reconstruction intrinsically promotes P-Ability, while comparison-based supervision fundamentally underpins the strong D-Ability observed in contrastive learning. Rather than introducing an additional contrastive branch, we seamlessly embed comparison signals into the reconstruction process. Specifically, for each image, we sample a generative timestep alongside its corresponding noisy observation, and compare different one-step denoising results under varying conditioning. (1) When the condition is derived from the same image, such as an augmented view, the reconstruction should remain highly consistent. These results are treated as positive examples. (2) When the condition originates from a different image, the reconstruction should diverge. These results serve as negative examples. By integrating comparisons directly into the generative process, we avoid directly optimizing two potentially conflicting objectives.

We instantiate this comparison structure using the area under the ROC curve (AUC), as depicted in Figure 3(b). Based on the definition of positive/negative examples shown above, optimizing the AUC objective encourages the score of a randomly chosen positive to exceed that of a randomly chosen negative [26]. This formulation precisely enforces the desired ordering within the generated image space at an arbitrary scale (since the comparison timestep is randomly chosen each time). On top of this, we introduce AUC Generative Reconstruction (AUCGR) as a tractable training objective. Theoretical analysis further demonstrates that this AUC-driven comparison structure supports both D-Ability and P-Ability.

Overview of the proposed method

Figure 3. An overview of our method. (a) A generic comparison principle for generative reconstruction. (b) AUC-oriented instantiations, including AUCGR and pAUCGR. (c) Reconstruction learning pipeline. Our method serves as a plug-and-play optimization module that replaces the original optimization strategy in existing reconstruction learning methods, thereby yielding balanced enhanced visual representations.

AUCGR still encounters a practical challenge in mini-batch training: the pseudo-negative set generated above may include noisy negatives that are semantically similar to the anchor, especially in long-tailed data distributions. Blindly telling apart those noisy examples from positives could introduce additional bias into supervision. To address this, we propose Partial AUC Generative Reconstruction (pAUCGR), yielding a partial-AUC interpretation that preserves uniform generalization guarantees.

Finally, we conduct comprehensive experiments across diverse benchmark datasets and multimodal large language models to verify the effectiveness of our proposed method.

This work extends our CVPR 2026 paper [27], where we initially proposed a unified contrastive reconstruction objective to resolve the gradient conflict between contrastive learning and generative reconstruction. In this version, we incorporate systematic enhancements in experimental settings, methodology, and empirical evaluation, resulting in a more general and robust generative reconstruction framework for visual representation learning. In summary, the new contributions are as follows:

  • New Methods. This version extends the contrastive reconstruction objective into a generic comparison-based theoretical framework (see § 6). As a fundamental principle, it can be instantiated by different comparison strategies. Moreover, it serves as a plug-and-play optimization module that enhances existing generative reconstruction methods and produces more balanced visual representations. We develop an AUC-oriented instantiation of the comparison strategy in the unified paradigm, termed AUC Generative Reconstruction (AUCGR). To mitigate biased supervision arising from noisy negatives, we further introduce Partial AUC Generative Reconstruction (pAUCGR), which restricts negative comparisons to reliable regions.

  • New Theory. We further provide theoretical analyses for the effectiveness of AUCGR and pAUCGR, establish corresponding generalization bounds (see § 7), and show that the conference version is naturally covered as the logistic-surrogate case of AUCGR.

  • New Empirical Results. We conduct a broad range of new empirical studies to evaluate the effectiveness of our method. Specifically, we introduce more generative reconstruction backbones, extend the evaluation to over 30 benchmarks (including zero-shot clustering, zero-shot classification, zero-shot retrieval, fine-grained perception, and MLLM evaluation), and perform more fine-grained ablation studies and visualization analyses.

  • Miscellaneous Contents. We also improve some existing contents to make our work more complete, including the abstract, introduction, review of prior arts (§ 2 and § 3), methodology and theory (Sec. 4, § 6 and § 7), and experiments (§ 8).

Related Work

Vision Encoders

Vision encoders serve as the visual backbone of MLLMs [1, 28], transforming raw images into high-level feature representations. Among various architectures [11, 29, 30, 31, 32, 33], CLIP [11] is one of the most widely adopted models. It also provides transferable representations for a wide range of tasks, including detection [34, 35, 36, 37, 38], segmentation [39, 40], video understanding [41, 42], and generation [43, 44, 45, 46, 47, 48, 49]. Reliability-aware visual learning has also received increasing attention. SELongVLM [50] effectively suppresses task-irrelevant visual content through self-corrective clip selection, while [51] explicitly model knowledge uncertainty via Monte Carlo dropout to reduce unreliable supervision in object detection distillation. Overall, recent studies have focused on enhancing vision encoders’ understanding capability to improve downstream performance. Some primarily strengthen its discriminative ability [17, 18, 52], while others improve its fine-grained perceptual ability [22, 23, 24, 25, 53]. However, an ideal visual encoder should achieve a balance between both aspects. To bridge this gap, this paper aims to develop an enhanced vision encoder with balanced understanding abilities, providing a stronger foundation for MLLMs.

Generative Reconstruction Learning

Generative modeling has recently become a powerful paradigm for improving visual representations. Early studies [54, 55, 56] use generative models as data augmenters to expand training data and boost generalization. More recent works [22, 23, 24, 25] add self-supervised reconstruction objectives to diffusion frameworks, enabling models to learn richer spatial details and fine-grained semantics. Among them, DIVA [23] introduces diffusion-based visual feedback conditioned on CLIP embeddings, which refines internal representations through generative reconstruction. GenHancer [24] extends this reconstruction process to discrete latent spaces, while un2^2CLIP [25] resolves the mismatch between CLIP’s pretrained feature space and generative model latents. However, these methods often overemphasize fine-grained reconstruction while neglecting discriminative learning, leading to suboptimal representations. In contrast, our work simultaneously strengthens discriminative ability and fine-grained perceptual ability, producing a more effective vision representation.

Vision-understanding Benchmarks

The benchmarks can be broadly divided into two camps. The first focuses on assessing discriminative ability. This ability can be evaluated through zero-shot clustering, zero-shot classification, and zero-shot retrieval. Zero-shot clustering examines whether image representations form separable semantic structures on datasets such as ImageNet [57], Caltech-101 [58], and DTD [59]. Zero-shot classification [11] further tests whether the learned visual features support class-level recognition. Zero-shot retrieval [60, 61] evaluates whether visual representations can distinguish correct image-text pairs from many candidates. The second group emphasizes fine-grained perceptual ability. MMVP [53] introduces a challenging benchmark with nine visual patterns, CV-Bench [62] extends evaluation to spatial relations, counting, and 3D understanding, and NaturalBench [63] adds natural adversarial examples to assess perceptual robustness. Together, these benchmarks provide a comprehensive evaluation of visual representations from both discriminative and fine-grained perceptual perspectives.

AUC Optimization

AUC optimization develops from classical machine learning to modern deep learning. Early work clarifies the distinct motivation for this line of research by showing that maximizing AUC is generally inconsistent with minimizing classification error [64]. This observation motivates a series of optimization methods for linear models, including logistic regression [65] and support vector machines [66, 67]. Subsequent studies extend AUC maximization to online and stochastic settings [68, 69, 70, 71], while generalization and consistency analyses [72], [73], [74], [75], [76] provide the corresponding theoretical foundations. With the rise of deep models, AUC-oriented training further extends to deep neural networks [77], and a range of variants such as partial AUC, compositional AUC, and long-tailed or class-balanced extensions gradually enriches this line of research [78], [79], [80]. More recently, AUC optimization also extends beyond instance-level prediction to the pixel level [81], [82], [83]. In practice, AUC-based optimization demonstrates effectiveness in recommendation systems [84], [85], disease prediction [86], [87], dense-level segmentation [81], [82], and adversarial training [88], [89], [90]. In this paper, we introduce AUC optimization into generative reconstruction for visual representation enhancement.

Preliminaries

Notation

Let ϵθ\epsilon_\theta denote the generative model, fϕf_\phi denote the visual encoder, and hωh_\omega denote the projection module that maps visual features into the generative space. The training dataset is defined as D={xi∈RH×W×3}i=1n\mathcal{D} = \{x^i \in\mathbb{R}^{H \times W \times3}\}_{i=1}^{n}, where HH and WW denote the image height and width. A Variational autoencoder (VAE) encodes each image xx into a latent x~\tilde{x}. We denote the visual feature as z=fϕ(x)z = f_\phi(x), the condition as c=hω(z)c = h_\omega(z), and the model prediction as ϵ^θ=ϵθ(x~t,c,t)\hat{\epsilon}_\theta= \epsilon_\theta(\tilde{x}_t, c, t) at timestep tt, with the ground-truth ϵtgt\epsilon_t^{\mathrm{gt}}. For clarity, we include a table of symbol definitions in Sec. A.

Generative Models

We review two typical generative models: diffusion models and flow-based models.

Diffusion Models. Diffusion models [43], [91] learn a generative process by reversing a gradual noising procedure over a finite number of timesteps. Given a latent representation x~0\tilde{x}_0 encoded by a VAE, the model progressively adds Gaussian noise to obtain x~t\tilde{x}_t at timestep tt. A noise prediction network ϵθ(x~t,c,t)\epsilon_\theta(\tilde{x}_t, c, t) parameterizes the reverse process and predicts the ground-truth noise ϵtgt\epsilon_t^{\mathrm{gt}} added at timestep tt. Training diffusion models amounts to minimizing the following denoising objective:

LDM=Ex~0,ϵgt,t[∥ϵtgt−ϵ^θ∥22],(1)\mathcal{L}_{\mathrm{DM}} = \mathbb{E}_{\tilde{x}_0,\epsilon^{\mathrm{gt}},t}\left[\left\lVert\epsilon_t^{\mathrm{gt}}-\hat{\epsilon}_\theta\right\rVert_2^2\right], \tag*{(1)}

where tt is sampled from a predefined timestep distribution. This objective promotes accurate noise prediction at each diffusion step and implicitly defines the reverse generative dynamics. The model achieves conditional generation by injecting the condition cc into the noise prediction network.

Flow-based Models. Flow-based models [92], [93] learn a continuous transformation that maps samples from a base distribution to the target data distribution through deterministic dynamics. In the conditional setting, this transformation is often written as an ordinary differential equation:

dx~tdt=ϵθ(x~t,c,t),(2)\frac{d\tilde{x}_t}{dt} = \epsilon_\theta(\tilde{x}_t,c,t), \tag*{(2)}

where ϵθ(x~t,c,t)\epsilon_\theta(\tilde{x}_t,c,t) denotes a time-dependent conditional velocity function. Rectified Flow uses a simpler formulation by constructing linear interpolations between samples from the base distribution and the data distribution. Specifically,

let x~0∼N(0,I)\tilde{x}_0 \sim\mathcal{N}(0,\mathbf{I}) and x~1∼pdata\tilde{x}_1 \sim p_{\mathrm{data}}, and define x~t=tx~1+(1−t)x~0\tilde{x}_t = t\tilde{x}_1 + (1-t)\tilde{x}_0. The conditional velocity field is learned by minimizing the regression objective:

LFM=Et,x~0,x~1[∥ϵtgt−ϵ^θ∥22],(3)\mathcal{L}_{\mathrm{FM}} = \mathbb{E}_{t,\tilde{x}_0,\tilde{x}_1}\left[\left\lVert\epsilon_t^{\mathrm{gt}}-\hat{\epsilon}_\theta\right\rVert_2^2\right], \tag*{(3)}

where ϵtgt=x~1−x~0\epsilon_t^{\mathrm{gt}} = \tilde{x}_1-\tilde{x}_0. It directly supervises the instantaneous velocity along straight-line paths conditioned on cc, defining a deterministic conditional generative process without requiring likelihood estimation or numerical solvers.

Although diffusion and flow-based models define different generative processes, their training objectives share the same basic form. Both supervise the model prediction to match a target signal. They provide a reliable supervision signal for generative reconstruction.

AUC Optimization

Area under the Receiver Operating Characteristic Curve (AUC) is a ranking metric widely used in binary classification [26]. It measures the probability that a scoring function assigns a higher score to a positive sample than to a negative sample, namely

AUC⁡(hθ)=P(hθ(x+)>hθ(x−)∣y+=1, y−=0),(4)\operatorname{AUC}(h_\theta) = \mathbb{P}\left(h_\theta(x^+) > h_\theta(x^-) \mid y^+ = 1,\ y^- = 0\right), \tag*{(4)}

where (x+,y+)(x^+,y^+) and (x−,y−)(x^-,y^-) denote positive and negative samples, respectively. A larger AUC indicates a stronger ability to rank positives ahead of negatives.

For a finite dataset, AUC maximization is commonly implemented through an empirical pairwise surrogate objective [78], [94], [95]. Let {xi+}i=1n+\{x_i^+\}_{i=1}^{n^+} and {xj−}j=1n−\{x_j^-\}_{j=1}^{n^-} denote the positive and negative sets. The corresponding empirical objective can be written as

L^AUC(hθ)=1n+n−∑i=1n+∑j=1n−ℓ(hθ(xi+)−hθ(xj−)),(5)\widehat{\mathcal{L}}_{\mathrm{AUC}}(h_\theta) = \frac{1}{n^+n^-}\sum_{i=1}^{n^+}\sum_{j=1}^{n^-}\ell\left(h_\theta(x_i^+)-h_\theta(x_j^-)\right), \tag*{(5)}

where ℓ\ell denotes a differentiable surrogate loss [94]. Minimizing this objective reduces pairwise ordering violations and therefore improves the induced AUC.

This formulation shows that AUC optimization depends on the relative order between positive and negative scores rather than their absolute calibration. Such a property is particularly valuable under imbalanced or long-tailed data distributions [96], [97], [98], [99], where comparison-based objectives can alleviate the bias caused by skewed class frequencies. This perspective provides the foundation for our later AUC instantiation of comparison-based generative reconstruction.

The Task of Visual Representation Learning

Visual representation learning aims to learn an encoder fϕ:X→Zf_\phi:\mathcal{X}\to\mathcal{Z} that maps an image xx into a visual representation zz. The goal is to obtain a representation with rich understanding capacity, typically characterized by two key properties: strong Discriminative Ability (D-Ability) and Fine-grained Perceptual Ability (P-Ability) [100], [101].

Property 1 (D-Ability): Discriminative ability describes a model’s ability to accurately and robustly distinguish between different visual categories. A representation with strong discriminative power can effectively separate distinct

classes in the feature space, keeping samples from the same class tightly clustered while pushing samples from different classes farther apart [11, 102]. Mathematically, for a mini-batch with class set Y\mathcal{Y} and feature set Zy={zi∣yi=y}\mathcal{Z}_y = \{z_i \mid y_i = y\}, we define the class center as μy=1∣Zy∣∑z∈Zyz\mu_y = \frac{1}{|\mathcal{Z}_y|} \sum_{z \in\mathcal{Z}_y} z. It is formulated as

min⁡Sinner,max⁡Sinter,(6)\min S_{\mathrm{inner}},\quad\max S_{\mathrm{inter}}, \tag*{(6)}
Sinner=1∣Y∣∑y∈Y1∣Zy∣∑z∈Zy∥z−μy∥22,(7)S_{\mathrm{inner}} = \frac{1}{|\mathcal{Y}|} \sum_{y \in\mathcal{Y}} \frac{1}{|\mathcal{Z}_y|} \sum_{z \in\mathcal{Z}_y} \lVert z-\mu_y\rVert_2^2, \tag*{(7)}
Sinter=1∣Y∣(∣Y∣−1)∑y,y′∈Y,y≠y′∥μy−μy′∥22.(8)S_{\mathrm{inter}} = \frac{1}{|\mathcal{Y}|(|\mathcal{Y}|-1)} \sum_{y,y' \in\mathcal{Y},y\ne y'} \lVert\mu_y-\mu_{y'}\rVert_2^2. \tag*{(8)}

Property 2 (P-Ability): Fine-grained perceptual ability reflects the model’s capacity to capture fine-grained visual details, enabling a comprehensive understanding of object attributes, spatial relations, and subtle semantic cues within an image. Following previous studies [22, 23, 24, 25], this perception is usually evaluated through reconstruction consistency. When a generative model can accurately reconstruct the original image from its learned representation, the representation is considered sufficiently rich. It is formulated as

References

  1. [1]H. Liu, C. Li, Y. Li, and Y. J. Lee, “Improved baselines with visual instruction tuning,” in CVPR, 2024, pp. 26 296–26 306.
  2. [2]D. Zhu, J. Chen, X. Shen, X. Li, and M. Elhoseiny, “MiniGPT-4: Enhancing vision-language understanding with advanced large language models,” in ICLR, 2024.arxiv.org/abs/2304.10592
  3. [3]A. Marafioti, O. Zohar, M. Farré, M. Noyan, E. Bakouch, P. Cuenca, C. Zakka, L. B. Allal, A. Lozhkov, N. Tazi et al., “Smolvlm: Redefining small and efficient multimodal models,” arXiv preprint arXiv:2504.05299, 2025.arxiv.org/abs/2504.05299
  4. [4]H. Yao, R. Zhang, and C. Xu, “Tcp: Textual-based class-aware prompt tuning for visual-language model,” in CVPR, 2024, pp. 23 438–23 448.
  5. [5]K. Zhou, J. Yang, C. C. Loy, and Z. Liu, “Conditional prompt learning for vision-language models,” in CVPR, 2022, pp. 16 816–16 825.
  6. [6]F. Zhang, S. Qu, F. Shi, and C. Xu, “Overcoming the pitfalls of vision-language model for image-text retrieval,” in ACMMM, 2024, pp. 2350–2359.DOI
  7. [7]B. Han, Q. Xu, S. Bao, Z. Yang, R. Cui, and Q. Huang, “Understanding-enhanced model collaboration for long-tailed egocentric mistake detection,” arXiv preprint arXiv:2606.02120, 2026.arxiv.org/abs/2606.02120
  8. [8]C. Hua, Q. Xu, S. Bao, Z. Yang, and Q. Huang, “Reconboost: Boosting can achieve modality reconcilement,” in ICML, 2024, pp. 19 573–19 597.arxiv.org/abs/2405.09321
  9. [9]Y. Wei, Y. Miao, D. Zhou, and D. Hu, “Moka: Multimodal low-rank adaptation for mlms,” in NeurIPS, 2025, pp. 137 470–137 492.arxiv.org/abs/2506.05191
  10. [10]X. Wang, X. Yang, Z. Huang, and Y. Chen, “Clip-sd: Clip-enhanced self-distillation for visual recognition,” TMM, vol. 28, pp. 2201–2213, 2026.DOI
  11. [11]A. Radford, J. W. Kim, C. Hallacy, A. Ramesh, G. Goh, S. Agarwal, G. Sastrу, A. Askell, P. Mishkin, J. Clark et al., “Learning transferable visual models from natural language supervision,” in ICML, 2021, pp. 8748–8763.arxiv.org/abs/2103.00020
  12. [12]W. Zhao, Z. Huang, J. Feng, and X. Wang, “Superclip: Clip with simple classification supervision,” in NeurIPS, 2025.arxiv.org/abs/2512.14480
  13. [13]S. Kim, M. Kang, D. Kim, J. Park, and S. Kwak, “Extending clip’s image-text alignment to referring image segmentation,” in ACL, 2024, pp. 4611–4628.
  14. [14]Y. Bengio, A. Courville, and P. Vincent, “Representation learning: A review and new perspectives,” TPAMI, vol. 35, no. 8, pp. 1798–1828, 2013.
  15. [15]Y. LeCun, Y. Bengio, and G. Hinton, “Deep learning,” nature, vol. 521, no. 7553, pp. 436–444, 2015.
  16. [16]S. Kornblith, J. Shlens, and Q. V. Le, “Do better imagenet models transfer better?” in CVPR, 2019, pp. 2661–2671.
  17. [17]K. Zhou, J. Yang, C. C. Loy, and Z. Liu, “Learning to prompt for vision-language models,” IJCV, vol. 130, no. 9, pp. 2337–2348, 2022.DOI
  18. [18]Y. Wei, H. Hu, Z. Xie, Z. Liu, Z. Zhang, Y. Cao, J. Bao, D. Chen, and D. Guo, “Improving clip fine-tuning performance,” in ICCV, 2023, pp. 5439–5449.DOI
  19. [19]S. Dai, Q. Xu, P. Wen, Y. Liu, and Q. Huang, “Exploring structural degradation in dense representations for self-supervised learning,” in NeurIPS, 2025, pp. 16 715–16 764.
  20. [20]Y. Liu, Q. Xu, P. Wen, S. Dai, and Q. Huang, “When the future becomes the past: Taming temporal correspondence for self-supervised video representation learning,” in CVPR, 2025, pp. 24 033–24 044.arxiv.org/abs/2503.15096
  21. [21]S. Liu, J. Su, C.-D. Wang, S. Sun, C. Lin, and Z. Huang, “Adaptive temporal expert routing with hierarchical wavelet enhancement for multi-modal sequential recommendation,” TOIS, vol. 44, no. 4, pp. 1–27, 2026.DOI
  22. [22]H. Wang, A. Zheng, Y. Zhao, T. Wang, Z. Ge, X. Zhang, and Z. Zhang, “Reconstructive visual instruction tuning,” in ICLR, 2025.
  23. [23]W. Wang, Q. Sun, F. Zhang, Y. Tang, J. Liu, and X. Wang, “Diffusion feedback helps clip see better,” in ICLR, 2025.arxiv.org/abs/2407.20171
  24. [24]S. Ma, Y. Ge, T. Wang, Y. Guo, Y. Ge, and Y. Shan, “Genhancer: Imperfect generative models are secretly strong vision-centric enhancers,” in ICCV, 2025.arxiv.org/abs/2503.19480
  25. [25]Y. Li, J. Zhao, H. Chang, R. Hou, S. Shan, and X. Chen, “un2clip: Improving clip’s visual detail capturing ability via inverting unclip,” in NeurIPS, 2025.arxiv.org/abs/2505.24517
  26. [26]J. A. Hanley and B. J. McNeil, “The meaning and use of the area under a receiver operating characteristic (roc) curve.” Radiology, vol. 143, no. 1, pp. 29–36, 1982.
  27. [27]B. Han, Q. Xu, S. Bao, Z. Yang, R. Cui, X. Zhao, and Q. Huang, “Guiding diffusion-based reconstruction with contrastive signals for balanced visual representation,” in CVPR, 2026.arxiv.org/abs/2603.04803
  28. [28]H. Liu, C. Li, Q. Wu, and Y. J. Lee, “Visual instruction tuning,” in NeurIPS, 2023, pp. 34 892–34 916.arxiv.org/abs/2304.08485
  29. [29]M. Caron, H. Touvron, I. M㆟sra, J. Jégou, J. Mairal, P. Bojanowski, and A. Joulin, “Emerging properties in self-supervised vision transformers,” in ICCV, 2021, pp. 9650–9660.
  30. [30]M. Oquab, T. Darcet, T. Moutakanni, H. Vo, M. Szafraniec, V. Khalidov, P. Fernandez, D. Haziza, F. Massa, A. El-Nouby et al., “Dinov2: Learning robust visual features without supervision,” TMLR, pp. 1–31, 2024.
  31. [31]O. Siméoni, H. V. Vo, M. Seitzer, F. Baldassarre, M. Oquab, C. Jose, V. Khalidov, M. Szafraniec, S. Yi, M. Ramamonjisoa et al., “Dinov3,” arXiv preprint arXiv:2508.10104, 2025.arxiv.org/abs/2508.10104
  32. [32]A. Luthra, T. Yang, and T. Galanti, “Self-supervised contrastive learning is approximately supervised contrastive learning,” in NeurIPS, 2026, pp. 41 701–41 738.
  33. [33]A. Mehta, X. Wei, X. Chen, and T. Yang, “Breaking the limits of open-weight clip: An optimization framework for self-supervised fine-tuning of clip,” arXiv preprint arXiv:2601.09859, 2026.arxiv.org/abs/2601.09859
  34. [34]S. Subramanian, W. Merrill, T. Darrell, M. Gardner, S. Singh, and A. Rohrbach, “Reclip: A strong zero-shot baseline for referring expression comprehension,” in ACL, 2022, pp. 5198–5215.arxiv.org/abs/2204.05991
  35. [35]N. Mu, A. Kirillov, D. Wagner, and S. Xie, “Slip: Self-supervision meets language-image pre-training,” in ECCV, 2022, pp. 529–544.
  36. [36]Z. Yang, Q. Xu, Z. Wang, S. Li, B. Han, S. Bao, X. Cao, and Q. Huang, “Harnessing hierarchical label distribution variations in test agnostic long-tail recognition,” in ICML, 2024, pp. 56 624–56 664.DOI
  37. [37]G. Lin, S. Yang, W.-S. Zheng, Z. Li, and Z. Huang, “A semantically guided and focused network for occluded person re-identification,” TIFS, vol. 20, pp. 9716–9731, 2025.
  38. [38]R. Zhong, G. Lin, and Z. Huang, “Modality decoupling and coupling network for visible-infrared person re-identification,” TIFS, 2026.DOI
  39. [39]C. Zhou, C. C. Loy, and B. Dai, “Extract free dense labels from clip,” in ECCV, 2022, pp. 696–712.
  40. [40]M. Xu, Z. Zhang, F. Wei, Y. Lin, Y. Cao, H. Hu, and X. Bai, “A simple baseline for open-vocabulary semantic segmentation with pre-trained vision-language model,” in ECCV, 2022, pp. 736–753.
  41. [41]H. Xu, G. Ghosh, P.-Y. Huang, D. Okhonko, A. Aghajanyan, F. Metze, L. Zettlemoyer, and C. Feichtenhofer, “Videoclip: Contrastive pre-training for zero-shot video-text understanding,” in EMNLP, 2021, pp. 6787–6800.arxiv.org/abs/2109.14084
  42. [42]M. Tang, Z. Wang, Z. Liu, F. Rao, D. Li, and X. Li, “ClipCaption: Clip for video caption,” in ACMMM, 2021, pp. 4858–4862.
  43. [43]R. Rombach, A. Blattmann, D. Lorenz, P. Esser, and B. Ommer, “High-resolution image synthesis with latent diffusion models,” in CVPR, 2022, pp. 10 684–10 695.
  44. [44]A. Q. Nichol, P. Dhariwal, A. Ramesh, P. Shyam, P. Mishkin, B. McGrew, I. Sutskever, and M. Chen, “Glide: Towards photorealistic image generation and editing with text-guided diffusion models,” in ICML, 2022, pp. 16 784–16 804.
  45. [45]B. Han, Q. Xu, S. Bao, Z. Yang, K. Zi, and Q. Huang, “Lightfair: Towards an efficient alternative for fair 2i diffusion via debiasing pre-trained text encoders,” in NeurIPS, 2025, pp. 22 671–22 724.arxiv.org/abs/2509.23639
  46. [46]F. Li, Q. Xu, S. Bao, B. Han, Z. Yang, and Q. Huang, “Hybrid generative fusion for efficient and privacy-preserving face recognition dataset generation,” in ICCV workshop, 2025, pp. 495–501.arxiv.org/abs/2508.10672
  47. [47]F. Li, Q. Xu, S. Bao, Z. Yang, X. Cao, and Q. Huang, “One image is worth a thousand words: A usability preservable text-image collaborative erasing framework,” in ICML, 2025, pp. 36 486–36 529.arxiv.org/abs/2505.11131
  48. [48]B. Gao, X. Gao, X. Wu, Y. Zhou, Y. Qiao, L. Niu, X. Chen, and Y. Wang, “The devil is in the prompts: Retrieval-augmented prompt optimization for text-to-video generation,” in CVPR, 2025, pp. 3173–3183.arxiv.org/abs/2504.11739
  49. [49]B. Gao, Q. Ma, X. Wu, S. Yang, G. Lan, H. Zhao, J. Chen, Q. Liu, Y. Qiao, X. Chen et al., “Rapo++: Cross-stage prompt optimization for text-to-video generation via data alignment and test-time scaling,” arXiv preprint arXiv:2510.20206, 2025.arxiv.org/abs/2510.20206
  50. [50]K. Zhang, Z. Yang, M. Han, Y. Zhuge, H. Hao, C. Li, Z. Li, and X. Chang, “Selongvlm: Empowering long video language models with self-corrective clip selection,” TPAMI, vol. 48, no. 7, pp. 8694–8709, 2026.
  51. [51]J. Yi, H. Zhang, J. Mao, T. Liu, M. Li, S. Lin, H. Gu, Z. Li, X. Chang, and Y. Wang, “Distilling object detectors via monte carlo dropout,” TPAMI, vol. 48, no. 8, pp. 9050–9063, 2026.DOI
  52. [52]S. Gong, L. Haoyou, Q. Dou, and F. Farnia, “Boosting the visual interpretability of clip via adversarial fine-tuning,” in ICLR, 2025.arxiv.org/abs/2610.14832
  53. [53]S. Tong, Z. Liu, Y. Zhai, Y. Ma, Y. L. Chen, and S. Xie, “Eyes wide shut? exploring the visual shortcomings of multimodal llms,” in CVPR, 2024, pp. 9568–9578.arxiv.org/abs/2401.06209
  54. [54]J. Shipard, A. Wiliem, K. N. Thanh, W. Xiang, and C. Fookes, “Diversity is definitely needed: Improving model-agnostic zero-shot classification via stable diffusion,” in CVPR, 2023, pp. 769–778.arxiv.org/abs/2302.03298
  55. [55]B. Li, X. Xu, X. Wang, Y. Hou, Y. Feng, F. Wang, X. Zhang, Q. Zhu, and W. Che, “Semantic-guided generative image augmentation method with diffusion models for image classification,” in AAAI, 2024, pp. 3018–3027.
  56. [56]Y. Tian, L. Fan, P. Isola, H. Chang, and D. Krishnan, “Stablerep: Synthetic images from text-to-image models make strong visual representation learners,” in NeurIPS, 2023, pp. 48 382–48 402.arxiv.org/abs/2306.00984
  57. [57]J. Deng, W. Dong, R. Socher, L.-J. Li, K. Li, and L. Fei-Fei, “Imagenet: A large-scale hierarchical image database,” in CVPR, 2009, pp. 248–255.
  58. [58]L. Fei-Fei, R. Fergus, and P. Perona, “Learning generative visual models from few training examples: An incremental bayesian approach tested on 101 object categories,” in CVPR workshop, 2004, pp. 178–178.
  59. [59]M. Cimpоi, S. Maji, I. Kokkinos, S. Mohamed, and A. Vedaldi, “Describing textures in the wild,” in CVPR, 2014, pp. 3606–3613.
  60. [60]P. Young, A. Lai, M. Hodosh, and J. Hockenmaier, “From image descriptions to visual denotations: New similarity metrics for semantic inference over event descriptions,” TACL, vol. 2, pp. 67–78, 2014.DOI
  61. [61]T.-Y. Lin, M. Maire, S. Belongie, J. Hays, P. Perona, D. Ramanan, P. Dollár, and C. L. Zitnick, “Microsoft coco: Common objects in context,” in ECCV, 2014, pp. 740–755.arxiv.org/abs/1405.0312
  62. [62]P. Tong, E. Brown, P. Wu, S. Woo, A. J. V. IYER, S. C. Akula, S. Yang, J. Yang, M. Middepogо, Z. Wang et al., “Cambrian-1: A fully open, vision-centric exploration of multimodal llms,” in NeurIPS, 2024, pp. 87 310–87 356.arxiv.org/abs/2406.16860
  63. [63]B. Li, Z. Lin, W. Peng, J. d. D. Nyandwi, D. Jiang, Z. Ma, S. Khanuja, R. Krishna, G. Neubig, and D. Ramanan, “Natural-bench: Evaluating vision-language models on natural adversarial samples,” in NeurIPS, 2024, pp. 17044–17068.arxiv.org/abs/2410.14669
  64. [64]C. Cortes and M. Mohri, “Auc optimization vs. error rate minimization,” in NeurIPS, 2003.
  65. [65]A. Herschтал and B. Raskutti, “Optimising area under the roc curve using gradient descent,” in ICML, 2004, p. 49.
  66. [66]T. Joachims, “A support vector method for multivariate performance measures,” in ICML, 2005, pp. 377–384.DOI
  67. [67]———, “Training linear svms in linear time,” in SIGKDD, 2006, pp. 217–226.
  68. [68]P. ZHAO, S. C. HOI, R. JIN, and T. YANG, “Online auc maximization,” in ICML, 2011, pp. 233–240.
  69. [69]W. Gao, R. Jin, S. Zhu, and Z.-H. Zhou, “One-pass auc optimization,” in ICML, 2013, pp. 906–914.
  70. [70]Y. Ying, L. Wen, and S. Lyu, “Stochastic online auc maximization,” in NeurIPS, 2016.
  71. [71]M. Natole, Y. Ying, and S. Lyu, “Stochastic proximal algorithms for auc maximization,” in ICML, 2018, pp. 3710–3719.
  72. [72]S. Agarwal, T. Graepel, R. Herbrich, S. Har-Peled, D. Roth, and M. I. Jordan, “Generalization bounds for the area under the roc curve,” Journal of Machine Learning Research, vol. 6, no. 4, 2005.
  73. [73]N. Usunier, M.-R. Amini, and P. Gallinari, “A data-dependent generalisation error bound for the auc,” in ICML Workshop, 2005.
  74. [74]S. Clémençon, G. Lugosi, and N. Vayatis, “Ranking and empirical minimization of u-statistics,” The Annals of Statistics, 2008.
  75. [75]S. Agarwal, “Surrogate regret bounds for bipartite ranking via strongly proper losses,” The Journal of Machine Learning Research, vol. 15, no. 1, pp. 1653–1674, 2014.
  76. [76]W. Gao and Z.-H. Zhou, “On the consistency of auc pairwise optimization,” arXiv preprint arXiv:1208.0645, 2012.arxiv.org/abs/1208.0645
  77. [77]M. Liu, Z. Yuan, Y. Ying, and T. Yang, “Stochastic auc maximization with deep neural networks,” in ICLR, 2020.
  78. [78]Z. Yang, Q. Xu, S. Bao, Y. He, X. Cao, and Q. Huang, “When all we need is a piece of the pie: A generic framework for optimizing two-way partial auc,” in ICML, 2021, pp. 11 820–11 829.
  79. [79]T. Yang and Y. Ying, “Auc maximization in the era of big data and ai: A survey,” ACM computing surveys, vol. 55, no. 8, pp. 1–37, 2022.arxiv.org/abs/2203.15046
  80. [80]Z. Yuan, D. Zhu, Z.-H. Qiu, G. Li, X. Wang, and T. Yang, “Libauc: A deep learning library for x-risk optimization,” in KDD, 2023, pp. 5487–5499.
  81. [81]B. Han, Q. Xu, Z. Yang, S. Bao, P. Wen, Y. Jiang, and Q. Huang, “Aucseg: Auc-oriented pixel-level long-tail semantic segmentation,” in NeurIPS, 2024, pp. 126 863–126 907.arxiv.org/abs/2409.20398
  82. [82]S. Bao, Q. Xu, F. Li, B. Han, Z. Yang, X. Cao, and Q. Huang, “Towards size-invariant salient object detection: A generic evaluation and optimization approach,” TPAMI, vol. 48, no. 2, pp. 1115–1132, 2025.
  83. [83]B. Han, Q. Xu, S. Bao, Z. Yang, S. Li, and Q. Huang, “Dual-stage reweighted moe for long-tailed egocentric mistake detection,” arXiv preprint arXiv:2509.12990, 2025.arxiv.org/abs/2509.12990
  84. [84]S. Bao, Q. Xu, Z. Yang, X. Cao, and Q. Huang, “Rethinking collaborative metric learning: Toward an efficient alternative without negative sampling,” IEEE TPAMI, vol. 45, no. 1, pp. 1017–1035, 2022.DOI
  85. [85]S. Bao, Q. Xu, Z. Yang, Y. He, X. Cao, and Q. Huang, “The minority matters: A diversity-promoting collaborative metric learning algorithm,” in NeurIPS, 2022, pp. 2451–2464.arxiv.org/abs/2209.15292
  86. [86]A. Westcott, D. P. Capaldi, D. G. McCormack, A. D. Ward, A. Fenster, and G. Parraga, “Chronic obstructive pulmonary disease: thoracic ct texture analysis and machine learning to predict pulmonary ventilation,” Radiology, vol. 293, no. 3, pp. 676–684, 2019.DOI
  87. [87]D. Gola, J. Erdmann, B. Müller-Myhsok, H. Schunkert, and I. R. Kőnig, “Polygenic risk scores outperform machine learning methods in predicting coronary artery disease status,” Genetic epidemiology, vol. 44, no. 2, pp. 125–138, 2020.DOI
  88. [88]W. Hou, Q. Xu, Z. Yang, S. Bao, Y. He, and Q. Huang, “Adauc: End-to-end adversarial auc optimization against long-tail problems,” in ICML, 2022, pp. 8903–8925.arxiv.org/abs/2206.12169
  89. [89]X. Xia, T. Liu, B. Han, M. Gong, J. Yu, G. Niu, and M. Sugiyama, “Sample selection with uncertainty of losses for learning with noisy labels,” in ICLR, 2022.
  90. [90]B. Han, I. W. Tsang, and L. Chen, “On the convergence of a family of robust losses for stochastic gradient descent,” in ECML PKDD, 2016, pp. 665–680.arxiv.org/abs/1605.01623
  91. [91]J. Ho, A. Jain, and P. Abbeel, “Denoising diffusion probabilistic models,” in NeurIPS, 2020, pp. 6840–6851.
  92. [92]Y. Lipman, R. T. Chen, H. Ben-Hamu, M. Nickel, and M. Le, “Flow matching for generative modeling,” in ICLR, 2023.
  93. [93]X. Liu, C. Gong et al., “Flow straight and fast: Learning to generate and transfer data with rectified flow,” in ICLR, 2023.arxiv.org/abs/2209.03003
  94. [94]Z. Yang, Q. Xu, S. Bao, X. Cao, and Q. Huang, “Learning with multiclass auc: Theory and algorithms,” IEEE TPAMI, vol. 44, no. 11, pp. 7747–7763, 2021.
  95. [95]Z. Yang, Q. Xu, S. Bao, Y. He, X. Cao, and Q. Huang, “Optimizing two-way partial auc with an end-to-end framework,” IEEE TPAMI, 2022.DOI
  96. [96]T. Fawcett, “An introduction to roc analysis,” Pattern recognition letters, vol. 27, no. 8, pp. 861–874, 2006.
  97. [97]X. Wu, F. Huang, and H. Huang, “Fast stochastic recursive momentum methods for imbalanced data mining,” in ICDM, 2022, pp. 578–587.DOI
  98. [98]F. Huang, X. Wang, J. Li, and S. Chen, “Adaptive federated minimax optimization with lower complexities,” in AISTATS, 2024, pp. 4663–4671.
  99. [99]Z. Yuan, Z. Guo, N. Chawla, and T. Yang, “Compositional training for end-to-end deep auc maximization,” in ICLR, 2021.
  100. [100]S. Kim, S. Kim, and J. Lee, “Hybrid generative-contrastive representation learning,” arXiv preprint arXiv:2106.06162, 2021.arxiv.org/abs/2106.06162
  101. [101]K. Nozawa and I. Sato, “Evaluation methods for representation learning: A survey,” in IJCAI, 2022, pp. 5556–5563.
  102. [102]C. Li and B. Wang, “Fisher linear discriminant analysis,” CCIS Northeastern University, vol. 6, 2014.
  103. [103]T. Chen, S. Kornblith, M. Norouzi, and G. Hinton, “A simple framework for contrastive learning of visual representations,” in ICML, 2020, pp. 1597–1607.arxiv.org/abs/2002.05709
  104. [104]K. He, X. Chen, S. Xie, Y. Li, P. Dollár, and R. Girshick, “Masked autoencoders are scalable vision learners,” in CVPR, 2022, pp. 16 000–16 009.
  105. [105]X. Liu, F. Zhang, Z. Hou, L. Mian, Z. Wang, J. Zhang, and J. Tang, “Self-supervised learning: Generative or contrastive,” TKDE, vol. 35, no. 1, pp. 857–876, 2021.
  106. [106]X. Xia, J. Liu, J. Yu, X. Shen, B. Han, and T. Liu, “Moderate coreset: A universal method of data selection for real-world data-efficient deep learning,” in ICLR, 2022.
  107. [107]J. Luo, F. Hong, J. Yao, B. Han, Y. Zhang, and Y. Wang, “Revive re-weighting in imbalanced learning by density ratio estimation,” in NeurIPS, 2024, pp. 79 909–79 934.
  108. [108]H. Xu, S. Xie, X. Tan, P.-Y. Huang, R. Howes, V. Sharma, S.-W. Li, G. Ghosh, L. Zettlemoyer, and C. Feichtenhofer, “Demystifying clip data,” in ICLR, 2024.
  109. [109]X. Zhai, B. Mustafa, A. Kolesnikov, and L. Beyer, “Sigmoid loss for language image pre-training,” in ICCV, 2023, pp. 11 975–11 986.
  110. [110]A. v. d. Oord, Y. Li, and O. Vinyals, “Representation learning with contrastive predictive coding,” arXiv preprint arXiv:1807.03748, 2018.
  111. [111]O. Sener and V. Koltun, “Multi-task learning as multi-objective optimization,” in NeurIPS, 2018.
  112. [112]X. Lin, H.-L. Zhen, Z. Li, Q.-F. Zhang, and S. Kwong, “Pareto multi-task learning,” in NeurIPS, 2019.
  113. [113]T. Yu, S. Kumar, A. Gupta, S. Levine, K. Hausman, and C. Finn, “Gradient surgery for multi-task learning,” in NeurIPS, 2020, pp. 5824–5836.
  114. [114]A. Rzhevskey, “Learning multiple layers of features from tiny images,” Master's thesis, University of Toronto, 2009.
  115. [115]Y. LeCun, L. Bottou, Y. Bengio, and P. Haffner, “Gradient-based learning applied to document recognition,” Proceedings of the IEEE, vol. 86, no. 11, pp. 2278–2324, 2002.
  116. [116]J. Xiao, J. Hays, K. A. Ehinger, A. Oliva, and A. Torralba, “Sun database: Large-scale scene recognition from abbey to zoo,” in CVPR, 2010, pp. 3485–3492.
  117. [117]S. Maji, E. Rahtu, J. Kannala, M. Blaschko, and A. Vedaldi, “Fine-grained visual classification of aircraft,” arXiv preprint arXiv:1306.5151, 2013.arxiv.org/abs/1306.5151
  118. [118]J. Krause, M. Stark, J. Deng, and L. Fei-Fei, “3d object representations for fine-grained categorization,” in ICCV workshop, 2013, pp. 554–561.
  119. [119]P. Helber, B. Bischke, A. Dengel, and D. Borth, “Eurosat: A novel dataset and deep learning benchmark for land use and land cover classification,” JSTARS, vol. 12, no. 7, pp. 2217–2226, 2019.
  120. [120]I. J. Goodfellow, D. Erhan, P. L. Carrier, A. Courville, M. Mirza, B. Hamner, W. Cukierski, Y. Tang, D. Thaler, D.-H. Lee et al., “Challenges in representation learning: A report on three machine learning contests,” in ICONIP, 2013, pp. 117–124.
  121. [121]M.-E. Nilsback and A. Zisserman, “Automated flower classification over a large number of classes,” in ICVGIP, 2008, pp. 722–729.
  122. [122]L. Bossard, M. Guillaumin, and L. Van Gool, “Food-101–mining discriminative components with random forests,” in ECCV, 2014, pp. 446–461.DOI
  123. [123]J. Stallkamp, M. Schlipsing, J. Salmen, and C. Igel, “Man vs. computer: Benchmarking machine learning algorithms for traffic sign recognition,” Neural networks, vol. 32, pp. 323–332, 2012.DOI
  124. [124]B. S. Veeling, J. Linmans, J. Winkens, T. Cohen, and M. Welling, “Rotation equivariant cnns for digital pathology,” in MICCAI, 2018, pp. 210–218.
  125. [125]O. M. Parkhi, A. Vedaldi, A. Zisserman, and C. Jawahar, “Cats and dogs,” in CVPR, 2012, pp. 3498–3505.
  126. [126]G. Cheng, J. Han, and X. Lu, “Remote sensing image scene classification: Benchmark and state of the art,” Proceedings of the IEEE, vol. 105, no. 10, pp. 1865–1883, 2017.
  127. [127]A. Coates, A. Ng, and H. Lee, “An analysis of single-layer networks in unsupervised feature learning,” in AISTATS, 2011, pp. 215–223.
  128. [128]M. Everingham, L. Van Gool, C. K. Williams, J. Winn, and A. Zisserman, “The pascal visual object classes (voc) challenge,” IJCV, vol. 88, no. 2, pp. 303–338, 2010.
  129. [129]B. Recht, R. Roelofs, L. Schmidt, and V. Shankar, “Do imagenet classifiers generalize to imagenet?” in ICML, 2019, pp. 5389–5400.arxiv.org/abs/1902.10811
  130. [130]D. Hendrycks, K. Zhao, S. Basart, J. Steinhardt, and D. Song, “Natural adversarial examples,” in CVPR, 2021, pp. 15 262–15 271.
  131. [131]D. Hendrycks, S. Basart, N. Mu, S. Kadavath, F. Wang, E. Dorundo, R. Desai, T. Zhu, S. Parajuli, M. Guo et al., “The many faces of robustness: A critical analysis of out-of-distribution generalization,” in ICCV, 2021, pp. 8340–8349.
  132. [132]A. Barbu, D. Mayo, J. Alverio, W. Luo, C. Wang, D. Gutfreund, J. Tenenbaum, and B. Katz, “Objectnet: A large-scale bias-controlled dataset for pushing the limits of object recognition models,” in NeurIPS, 2019, pp. 9453–9463.
  133. [133]Y. Li, Y. Du, K. Zhou, J. Wang, W. X. Zhao, and J.-R. Wen, “Evaluating object hallucination in large vision-language models,” in EMNLP, 2023, pp. 292–305.
  134. [134]P. Lu, S. Mishra, T. Xia, L. Qiu, K.-W. Chang, S.-C. Zhu, O. Tajford, P. Clark, and A. Kalyan, “Learn to explain: Multimodal reasoning via thought chains for science question answering,” in NeurIPS, 2022, pp. 2507–2521.
  135. [135]T. Guan, F. Liu, X. Wu, R. Xian, Z. Li, X. Liu, X. Wang, L. Chen, F. Huang, Y. Yacoob et al., “Hallusionbench: An advanced diagnostic suite for entangled language hallucination and visual illusion in large vision-language models,” in CVPR, 2024, pp. 14 375–14 385.
  136. [136]B. Liu, X. Liu, X. Jin, P. Stone, and Q. Liu, “Conflict-averse gradient descent for multi-task learning,” in NeurIPS, 2021, pp. 18 878–18 890.
  137. [137]Z. Chen, V. Badrinarayanan, C.-Y. Lee, and A. Rabinovich, “Gradnorm: Gradient normalization for adaptive loss balancing in deep multitask networks,” in ICML, 2018, pp. 794–803.
  138. [138]G. C. David, “Bi-lipschitz pieces between manifolds.” Revista Mathematica Iberoamericana, vol. 32, no. 1, 2016.
  139. [139]Y. Zhang, H. Wu, and L. Cheng, “Some new deformation formulas about variance and covariance,” in ICMIC, 2012, pp. 987–992.
  140. [140]W. Hoeffding, “Probability inequalities for sums of bounded random variables,” Journal of the American statistical association, vol. 58, no. 301, pp. 13–30, 1963.
  141. [141]Y. Li, P. Hu, D. Peng, J. Lv, J. Fan, and X. Peng, “Image clustering with external guidance,” in ICML, 2024, pp. 27 890–27 902.
  142. [142]B. Zhou, H. Zhao, X. Puig, S. Fidler, A. Barriuso, and A. Torralba, “Scene parsing through ade20k dataset,” in CVPR, 2017, pp. 633–641.
  143. [143]H. Caesar, J. Uijlings, and V. Ferrari, “Coco-stuff: Thing and stuff classes in context,” in CVPR, 2018, pp. 1209–1218.

Paper details

vlnc:W_ra8p9tjf
Version
v1 · latest
Published
Fresh — added to the Hub within the last 30 days
Added
License
CC BY 4.0
Usage

views · 0

downloads · 0

A view counts once per browser per day, a download each time the PDF is requested.

  • Computer Science

Not recorded for this paper

View statement
Looking for related papers…
Contents