betaFeedback

WorksComputer Science

Anomaly Detection for Hybrid Butterfly Subspecies via Probability Filtering

Abstract

Detecting butterfly hybrids requires knowledge of the parent subspecies, and the process can be tedious when encountering a new subspecies. This study focuses on a specific scenario where a model trained to recognize hybrid species A can generalize to species B when B biologically mimics A. Since species A and B share similar patterns, we leverage BioCLIP as our feature extractor to capture features based on their taxonomy. Consequently, the algorithm designed for species A can be transferred to B, as their hybrid and non-hybrid patterns exhibit similar relationships. To determine whether a butterfly is a hybrid, we adopt proposed probability filtering and color jittering to augment and simulate the mimicry. With these approaches, we achieve second place in the official development phase. Our code is publicly available at https://github.com/Justin900429/NSF-HDR-Challenge.

Introduction

Butterfly species can diverge into different subspecies due to geographical or habitat separation, leading to distinct variations in their visual appearances, such as wing color patterns. Typically, individuals of the same subspecies mate and produce offspring with predictable appearances. However, when subspecies encounter one another in overlapping regions, interbreeding may occur, resulting in hybrid offspring. These hybrids inherit characteristics from both parent subspecies, displaying intermediate visual patterns.

This study focuses on developing an anomaly detection algorithm to differentiate between hybrids and non-hybrids within a species. Furthermore, we explore whether such an algorithm can generalize to another species that visually mimics the first, providing insights into the generalization of the provided approach. The mimics represent two distinct species that evolve similar visual patterns when they share geographic ranges. This mimicry can serve as a defense mechanism against predators, particularly when one or both species are toxic or unpalatable.

Methods

Problem Formulation. Given an input image of butterfly wings, assign an anomaly score in the range [0, 1]. A higher score indicates a higher probability of a hybrid butterfly.

Backbone

Since the challenge requires transferring results from species A to the unseen species B, our model must incorporate a feature extractor capable of distinguishing species while capturing patterns that remain consistent across them. To achieve this, we adopt BioCLIP (Stevens et al. [3]) as our backbone feature extractor, as it recognizes inputs and generates features based on biological taxonomy.

To further refine feature extraction for butterfly species, we fine-tune the backbone without freezing its weights. However, updating the backbone parameters may compromise its original generalizability. To preserve its original ability, we apply a smaller learning rate to the backbone, ensuring that updates remain subtle and do not deviate significantly from the original weights.

Probability Filtering

Since the subspecies within a particular species are fixed, each image can be directly classified into a specific subspecies. To achieve this, we incorporate a three-layer classifier into the BioCLIP visual model. For non-hybrid butterflies, we assign a ground-truth class probability of 1 to their respective subspecies. In contrast, since hybrid butterflies originate from two different parent subspecies, we distribute their probabilities equally between both classes, assigning each a value of 0.5. The model architecture is illustrated in Figure 1. We train the entire model using cross-entropy loss. After training, we expect the model to predict non-hybrid butterflies with high confidence in a single class while assigning approximately equal confidence scores to two classes for hybrid cases.

Illustration of the training pipeline

Figure 1. Illustration of the training pipeline.

To transform the predicted probabilities into an anomaly score, we first examine whether the maximum probability pmax1p_{max1} exceeds a predefined threshold tt. If pmax1>tp_{max1} > t, the butterfly is likely a non-hybrid from a single subspecies, and we define its anomaly score as 1−pmax11 - p_{max1}. Conversely, if pmax1≤tp_{max1} \le t, the specimen is considered a hybrid. In this case, we compute the anomaly score as the sum of the two highest probabilities, pmax1+pmax2p_{max1} + p_{max2}, where pmax2p_{max2} is the second-highest probability. Formally, the anomaly score AA is defined as:

A={1−pmax1,if pmax1>t,pmax1+pmax2,otherwise.(1)A = \begin{cases} 1 - p_{max1}, & \text{if } p_{max1} > t, \\ p_{max1} + p_{max2}, & \text{otherwise}. \end{cases} \tag*{(1)}

Experiment

Dataset

For training, we utilize the dataset provided by Imageomics for the NSF HDR Challenge1. The dataset consists of 91 hybrid and 2001 non-hybrid images for species A. An illustration of the dataset is provided in Figure 2.2 Model evaluation is conducted on the official platform using a private dataset including both species A and species B. Performance is assessed using recall and AUC as evaluation metrics.

Illustration of the hybrid and non-hybrid cases

Figure 2. Illustration of the hybrid and non-hybrid cases.

Setup

We train our model for 200 epochs with a batch size of 32 and set tt to 0.75. To prevent excessive drift in the backbone used for feature extraction, we set a low learning rate of 3×10−63 \times10^{-6}. Meanwhile, the classifier head is trained with a larger learning rate of 3×10−43 \times10^{-4} to ensure effective adaptation. Since the mimic species exhibit similar coloration, we apply color jittering to enhance generalization. Additionally, to address the class imbalance between hybrid and non-hybrid samples, we employ a weighted sampler during training. As the provided images contain a significant amount of unused information, we further improve model robustness by applying cropping and linear affine transformations to refine feature extraction and reduce redundancy.

Results

We present the comparison results in Table 1. The results indicate that for species A, our method achieves the highest AUC for hybrid and non-hybrid classification. Regarding the mimicry performance, our model ranks second in both recall and AUC for species B hybrid cases. In terms of the overall score, we secure second place.

Table 1: Results on the private dataset from the official evaluation platform of the top 6 team. AA denotes the AUC, and RR denotes the recall score. The subscript indicates whether the result corresponds to species A or B, while the superscript denotes hybrid and non-hybrid cases, respectively.

ParticipantAAHA_A^HAANA_A^NRBR_BABHA_B^HScore ↑\uparrow
anomalyfactory0.950.150.500.700.44
Tater TOT0.990.130.760.890.37
20iterations0.980.120.270.550.29
kylechen12110.860.090.340.530.14
motoki0.760.090.180.350.12
Ours0.990.210.680.810.44

Table 1.

Ablation Study

To gain deeper insights into our approach, we conduct ablation study by evaluating each component also with the official evaluation platform as shown in Table 2.

ApproachAAHA_A^HAANA_A^NRBR_BABHA_B^HScore ↑\uparrow
w. DINO+SVM0.990.080.710.810.15
w. BioCLIP+SVM1.000.150.540.770.26
w.o jittering0.990.150.660.740.27
Ours0.990.210.680.810.44

Table 2. Ablation study of each component. The notations are the same to Table 1.

Backbone. We examine the backbone by comparing BioCLIP with DINOv2 (Oquab et al. [2]). The results demonstrate that BioCLIP achieves superior performance due to its strong capability to extract biologically relevant features.

Probability Filtering. We compare our probability filtering with an SVM-based approach (Hearst et al. [1]). In this experiment, we freeze BioCLIP and train only the SVM for prediction. The results indicate that probability matching provides a more effective strategy for detecting hybrid cases.

Color Jittering. We analyze the effect of color jittering and find that it further enhances performance by simulating variations observed in different mimicry species.

Conclusion

In this study, we developed an anomaly detection framework to identify hybrid butterflies and evaluated its generalizability to mimicry species. Our results demonstrate the effectiveness of BioCLIP for feature extraction and highlight the benefits of probability filtering. Notably, our approach achieved first place on the private dataset. Additionally, we conducted an ablation study to analyze the impact of each component on model performance, providing a comprehensive understanding of the proposed approach.

References

  1. [1]Hearst, M. A.; Dumais, S. T.; Osuna, E.; Platt, J.; and Scholkopf, B. 1998. Support vector machines. IEEE Intelligent Systems and their Applications, 13(4): 18–28.
  2. [2]Oquab, M.; Darcet, T.; Moutakanni, T.; Vo, H. V.; Szafraniec, M.; Khalidov, V.; Fernandez, P.; Haziza, D.; Massa, F.; El-Nouby, A.; Assran, M.; Ballas, N.; et al. 2024. DINOv2: Learning Robust Visual Features without Supervision. Transactions on Machine Learning Research.arxiv.org/abs/2304.07193
  3. [3]Stevens, S.; Wu, J.; Thompson, M. J.; Campolongo, E. G.; Song, C. H.; Carlyn, D. E.; Dong, L.; Dahdul, W. M.; Stewart, C.; Berger-Wolf, T.; et al. 2024. Bioclip: A vision foundation model for the tree of life. In Computer Vision and Pattern Recognition, 19412–19424.arxiv.org/abs/2311.18803

Paper details

vlnc:W_v7xw26fq
Version
v1 · latest
Published
Fresh — added to the Hub within the last 30 days
Added
License
CC BY 4.0
Usage

views · 0

downloads · 0

A view counts once per browser per day, a download each time the PDF is requested.

  • Computer Science

Not recorded for this paper

View statement
Looking for related papers…
Contents