WorksPhysics and Astronomy
Contrastive Astromer: Two-Stage Contrastive Representation Learning for Astronomical Light Curve Classification
Abstract
Large astronomical surveys collect abundant light curves, but reliable class labels remain scarce. ASTROMER 2 learns from unlabeled light curves by reconstructing masked observations, yet this objective does not explicitly encourage class separation in a compact representation of each light curve. We introduce Contrastive Astromer, a two-stage approach that makes these representations more useful for classification without changing the ASTROMER 2 encoder architecture. First, unsupervised contrastive learning on single-band MACHO light curves brings together representations of augmented views of the same object. Supervised contrastive learning, supported by an auxiliary classification loss, then uses a separate labeled Alcock sample to strengthen class discrimination. We evaluate the resulting representations by freezing the encoder and training a separate classifier. On a held-out set of 600 objects, this classifier achieves a macro-F1 of 0.7943 with 500 labeled examples per class, exceeding our reproduction of the ASTROMER 2 finetune-and-classify pipeline by 5.10 percentage points. With downstream classifier training budgets of 20 and 100 examples per class, macro-F1 reaches 0.7078 and 0.7704, respectively, also surpassing this baseline. Embedding-space analyses further show that supervised adaptation improves local class consistency and global class separation. These results show that contrastive adaptation makes pretrained light-curve representations more effective for classification and more informative about the structure of variability classes.
Introduction
Light curves provide a record of stellar variability, but turning those records into reliable classifications remains challenging. Their periods, amplitudes, and shapes carry information about variability class and the underlying stellar processes [11, 7]. Wide-field surveys such as ZTF and ATLAS provide large collections of these observations [2, 23]. Yet light curves are irregularly sampled, noisy, and often incomplete, while reliable class labels remain scarce. These challenges motivate models that learn from large unlabeled archives and produce representations that can be reused for classification [9] (Bommasani et al. 2021).
Masked reconstruction enables light-curve encoders to learn from observations without requiring class labels. ASTROMER and ASTROMER 2 use this approach by hiding some observed magnitudes during training and predicting them from the remaining temporal context [10, 9]. The encoder produces a sequence of feature vectors, one for each input observation. A simple way to obtain a fixed-length summary is to average these vectors over the visible observations within an input window. We refer to this summary as the pooled representation. This vector provides a convenient input to a classifier, but the reconstruction objective does not explicitly require it to distinguish between variability classes.
Contrastive learning offers a way to encourage useful relationships between representations during training. It brings representations of selected positive pairs closer together while separating them from negative pairs [17, 5]. For example, two independently perturbed versions of the same input can form a positive pair, allowing the model to learn which features should remain consistent across those perturbations. When labels are available, different objects from the same class can also form positive pairs [12]. We therefore investigate whether contrastive adaptation can make the pooled ASTROMER 2 representation more useful for variable-star classification while preserving the encoder architecture and single-band input.
We introduce Contrastive Astromer, a two-stage approach that combines learning from unlabeled light curves with supervised adaptation. Starting from the pretrained ASTROMER 2 encoder, Stage 1 applies unsupervised contrastive learning to MACHO light curves, encouraging consistency between augmented views of each object. Stage 2 then combines supervised contrastive learning with an auxiliary classification loss on a separate labeled Alcock sample. This second stage uses class labels to strengthen discrimination between variability classes. The resulting encoder produces pooled representations that can be used by a separately trained classifier.
To assess what each stage contributes, we compare the representations before and after adaptation on the Alcock variable-star benchmark [9, 19]. We hold the original, Stage 1, and Stage 2 encoders fixed and train the same classifier architecture on their pooled representations, using 20, 100, and 500 labeled examples per class for downstream classifier training. These budgets apply to the downstream classifier; Stage 2 uses its own separate labeled sample. We also examine class organization directly in the representation space, allowing us to evaluate the learned representations beyond their performance with a particular classifier.
Our main contributions are as follows:
We introduce a two-stage contrastive adaptation of ASTROMER 2 that combines unlabeled MACHO light curves with labeled Alcock data while retaining the original encoder architecture.
We show that contrastive adaptation improves classification using frozen pooled representations across three downstream label budgets. On a separate 600-object holdout, the full model achieves a macro-F1 of 0.7943 with 500 labeled examples per class for downstream classifier training, exceeding our reproduction of the ASTROMER 2 finetune-and-classify pipeline by 5.10 percentage points.
We show through direct embedding-space diagnostics that supervised adaptation improves local class consistency and global class separation in the pooled representations.
The remainder of this paper is organized as follows. Section 2 reviews the relevant background and related work. Section 3 presents Contrastive Astromer, and Sections 4 and 5 describe the experiments and results. Section 6 discusses the implications and limitations, and Section 7 concludes.
Background and Related Work
ASTROMER and Masked Reconstruction
ASTROMER and ASTROMER 2 use Transformer encoders to learn representations of irregular light curves through masked-observation prediction: parts of a light curve are hidden, and the encoder predicts their magnitudes from the surrounding temporal context [10, 9]. ASTROMER 2 uses a deeper Transformer on fixed windows of 200 observations and is pretrained on approximately 1.5 million single-band MACHO light curves [9, 19].
Masked reconstruction encourages the encoder to capture temporal context but does not explicitly require a pooled representation to separate variable-star classes. Here, a pooled representation is a single fixed-length summary vector formed by taking an equally weighted mean of the encoder’s output vectors for valid observation tokens within a 200-position input window. Information useful for predicting missing magnitudes need not make class distinctions readily accessible in this summary.
Contrastive Representation Learning
Contrastive learning shapes representations through comparisons between examples. In instance-level contrastive learning, two augmented views of the same input form a positive pair, while views from other inputs in the minibatch serve as negatives. Each view acts in turn as an anchor, or reference view. InfoNCE encourages the anchor to be more similar to its positive than to the competing negative views [17]. SimCLR uses this pairing scheme with a normalized, temperature-scaled contrastive loss and a projection head that is discarded for downstream transfer [5].
The choice of augmented views determines which variations the representation is encouraged to ignore. For light curves, these variations may include small time shifts, changes in sampling, and photometric perturbations. The views should preserve the variability information relevant to the task; otherwise, the model may learn to ignore features needed for classification. Augmentation therefore defines what consistency between two views should mean.
Class labels allow positive pairs to extend beyond two views of the same object. Without labels, instance-level learning treats different objects as negatives even when they belong to the same variability class. Supervised contrastive learning instead includes other views from the anchor’s class in its positive set, while views from different classes remain negatives [12]. This change encourages class-level organization in the representation space. Figure 1 illustrates the two pairing rules using a simple three-object minibatch.

Figure 1. Positive and negative views for one anchor in a three-object minibatch. For each object, one sampled light-curve window supplies two independently augmented views, represented here by their projected embeddings (subscripts 1 and 2). Objects A and B share a class; C belongs to a different class. In Stage 1, class labels are not used: only the companion view is positive, while both views of B are negatives because B is a different object, despite sharing A’s class. In Stage 2, class labels define the positive set, so both views of B become additional positives. The anchor itself is excluded from both sets. Only comparisons involving anchor are drawn; every view serves as an anchor in turn. The objects illustrate the pairing rules, not shared training data between stages.
Representation Learning in Astronomy
Contrastive learning has already been applied directly to astronomical light curves. StarCLR constructs positive pairs from partially overlapping subsequences of the same light curve and pretrains on TESS data [8]. Its use of temporal subsequences illustrates how related views can be constructed from the observations themselves, connecting the general contrastive framework to the structure of astronomical time series.
Other astronomical representation models draw on multiple photometric bands or complementary types of observation. Astro-MoE uses a Transformer with a mixture-of-experts module to model multiband astronomical time series [4]. Among multimodal approaches, AstroCLIP aligns galaxy images and spectra in a shared embedding space, while AstroM³ integrates light curves, spectra, and astrophysical metadata [13, 18, 21]. Maven uses contrastive learning to align supernova photometry and spectroscopy (Zhang et al., 2024).
Contrastive Astromer focuses on adapting representations learned from single-band light curves. We retain the ASTROMER 2 encoder architecture and examine how unsupervised contrastive learning followed by supervised adaptation changes the usefulness and class organization of its pooled representations. This setting complements work that broadens the observational inputs by studying what can be gained from adapting an existing single-band encoder.
Methodology
Overview
Contrastive Astromer is a representation-learning extension of the official ASTROMER 2 light-curve Transformer. It learns a reusable light-curve representation by continuing to train a pretrained ASTROMER 2 encoder in two stages. Stage 1 uses unlabeled MACHO light curves to encourage consistency between augmented views of the same object. Stage 2 uses labeled Alcock light curves to strengthen discrimination between variability classes.
Both stages follow the same path from observations to a contrastive embedding, as illustrated in Figure 2. A light-curve window is transformed into two augmented views. For each view, the encoder produces a vector at every input position, and averaging the vectors for retained observations gives a fixed-length representation . A projection head then maps to a normalized embedding , on which the contrastive loss is computed. Stage 2 also trains a temporary classifier on , providing an additional class-prediction signal. Figure 1 illustrates how the positive and negative pairs differ between stages.
Figure 2. Contrastive Astromer methodology. (a) Each augmented view passes separately through the shared ASTROMER 2 encoder . Mean pooling over retained observations converts the token vectors into a 256-dimensional light-curve representation . The projection head maps to the normalized 128-dimensional embedding used for contrastive learning. The auxiliary linear classifier operates directly on and is used only in Stage 2. (b) Stage 1 learns from unlabeled MACHO light curves using InfoNCE. Its encoder and projection-head weights initialize Stage 2, which uses labeled Alcock light curves with supervised contrastive learning and auxiliary cross-entropy (CE). (c) After Stage 2, both training heads are discarded. A separate downstream classifier is trained on representations from unaugmented windows, with the encoder frozen. Teal denotes retained encoder and pooling components; orange denotes temporary training heads; gray denotes data and representations. Light-curve thumbnails show real observations; the network, token, and vector graphics are schematic.
The two stages change the training objective without introducing a reconstruction task. Although the initial ASTROMER 2 checkpoint was learned through masked reconstruction, neither contrastive stage predicts hidden magnitudes. Observation masking is used only to construct augmented views, while padding masks identify positions that do not contain observations. After training, we retain the encoder and pooling operation; the projection head and temporary classifier are discarded.
Input Representation and Base Encoder
Each encoder input represents a window containing at most 200 observations from one light curve. We use input positions and write the magnitude and time arrays as
where is the magnitude and is the observation time. A window containing observations is extended with padding positions, whose magnitude and time values are set to zero.
A separate binary indicator records which positions are used in the current view. We write , with for a retained observation and for padding or an observation excluded by augmentation. Thus, is an auxiliary mask, not an additional measured property of the light curve. It is supplied to the encoder’s attention mechanism and used in pooling. Our implementation converts to to match the mask convention expected by the official encoder. During inference, every real observation in the selected window has ; only padding positions have .
The base model is the six-layer ASTROMER 2 Transformer, initialized from its official MACHO-pretrained checkpoint and producing 256-dimensional latent representations [9, 19]. We retain its architecture, time encoding, and zero-mean normalization. The contrastive and pooled-representation pipelines read the stored magnitude values. During representation learning, the encoder weights are updated; during downstream classifier training and evaluation, they are held fixed.
Window selection determines which observations from a longer light curve are presented to the encoder. During Stage 1 and Stage 2 training, a light curve with observations supplies a randomly selected contiguous window of 200 observations. The start is sampled uniformly from the valid positions. If , all observations are retained. Two augmented views are constructed from this same source window, and each view is then padded to positions if necessary. Downstream classifier training and evaluation use one deterministic window per object. For , we select the central 200 observations, starting at
with zero-based indexing. The selected window contains observations through . For , we retain the full sequence and pad it. No stochastic augmentation is applied. This rule gives the original, Stage 1, and Stage 2 encoders the same observations when their pooled representations are compared. The central crop is a controlled evaluation choice rather than a claim that it is optimal.
Pooled Encoder Representation
Mean pooling converts the encoder’s sequence of outputs into one vector for the input window. Let denote the ASTROMER 2 encoder with parameters . Given an input window , the encoder returns a sequence of token-level hidden states, denoted by , with hidden dimension :
For a view containing at least one retained observation, its pooled representation is
The numerator sums the token vectors for retained observations, and the denominator counts those observations. Each therefore contributes equally to the resulting vector .
The mask selects which vectors enter the average; it does not define a reconstruction target. During contrastive training, the average excludes padding and observations removed by augmentation. At inference, it excludes only padding. If all 200 positions contain observations, (3) reduces to an ordinary mean over the 200 token vectors. The pooled representation supplies both the training heads and the separate downstream classifier.
Contrastive Projection Head
The projection head provides a space specifically for the contrastive objective. Following the design used in SimCLR, we compute the contrastive loss after projection and retain the representation before projection for downstream tasks [5]. We write
where consists of a fully connected layer with 256 units and a ReLU activation, a linear layer with 128 outputs, and normalization.
The projection head and encoder are learned together through the contrastive loss. The head is randomly initialized at the start of Stage 1, and its weights are carried forward into Stage 2. Normalization makes the inner product between projected embeddings equivalent to cosine similarity, so comparisons depend on their directions rather than their magnitudes. Gradients through the projection head update the encoder, but downstream classification uses directly and does not require .
Augmentation and Contrastive Views
Augmentation creates two related views of each source window for contrastive training. For a sampled window , we independently draw
where denotes the augmentation distribution. Each view contains its own magnitude array, time array, and observation mask. Only these two views enter the contrastive objective; the unaugmented source window is not included as a third view. Figure 1 shows how the resulting views form positive and negative pairs. Figure 3 illustrates this construction for a real light-curve window, showing the original observations, two independently sampled views, and the effects of the six individual augmentation operators.
Figure 3. Augmented views of a real MACHO light-curve window. Panel (a) shows 200 consecutive observations of the long-period variable MACHO 23.3302.11. Panels (b)–(c) show two independently sampled augmented views, using the Stage 1 probabilities and strengths in Table 1. Panels (d)–(i) illustrate each augmentation applied individually. Open gray circles show the original observations for reference, colored points show retained augmented observations, and crosses mark excluded observations. Temporal cropping retains a contiguous interval of observation indices; masking and cropping change visibility without introducing zero-valued measurements. All main panels share the same axes, original time origin, and original-window mean magnitude, which is subtracted for display only. Insets magnify the small changes from amplitude scaling and Gaussian noise without changing their strength.
The value-based augmentations perturb the measurements while retaining their array positions. A random time shift adds the same offset to all timestamps in a view, with the offset bounded by a fraction of the window’s time span. Amplitude scaling rescales magnitude deviations around the window mean, and a magnitude offset adds a constant to all magnitudes. Gaussian noise adds independent, zero-mean perturbations to the magnitude values. These transformations are intended to preserve the variability information relevant to classification.
Observation masking and temporal cropping instead change which positions are retained. Random masking independently excludes observations by setting their values to zero. Temporal cropping selects a contiguous interval of observation indices and sets outside that interval. The crop fraction refers to the number of positions retained within the interval, rather than a fraction of elapsed time. Neither operation deletes array entries: excluded positions are omitted as attention keys and from pooling.
Stage 2 uses the same augmentation family with lower application probabilities and smaller perturbations. Table 1 reports the settings for both stages.
| Augmentation operator | Stage 1: Probability | Stage 1: Strength / parameter | Stage 2: Probability | Stage 2: Strength / parameter |
| Time jitter | 0.5 | Maximum shift: 0.1 times time range | 0.2 | Maximum shift: 0.03 times time range |
| Amplitude scaling | 0.5 | Scale range: [0.95, 1.05] | 0.25 | Scale range: [0.98, 1.02] |
| Amplitude shift | 0.3 | Shift range: [−0.1, 0.1] | 0.1 | Shift range: [−0.03, 0.03] |
| Gaussian magnitude noise | 0.8 | Noise level: 0.01 | 0.35 | Noise level: 0.005 |
| Random observation masking | 0.3 | Drop probability: 0.1 | 0.1 | Drop probability: 0.05 |
| Temporal cropping | 0.2 | Crop fraction: 0.8 | 0.05 | Crop fraction: 0.9 |
Table 1. Augmentation probabilities and strengths for Stage 1 and Stage 2. Probability is the chance of applying the augmentation to a view. For observation masking, the separate drop probability applies to each currently visible observation when masking is applied.
Stage 1: Unsupervised Contrastive Learning on MACHO
Stage 1 trains the encoder to recognize two augmented views of the same source window. It starts from the MACHO-pretrained ASTROMER 2 encoder and a randomly initialized projection head. A minibatch of source windows produces normalized projected embeddings, which we index as . Each view acts once as an anchor, and its companion view from the same window is its positive. Both views of every other object serve as negatives.
We optimize the normalized temperature-scaled contrastive loss used in SimCLR, an InfoNCE-style objective [17, 5]. For anchor , let index its positive view. The loss is
Here, , since the projected embeddings are normalized. The denominator includes the positive and all negatives; only the anchor itself is excluded.
The temperature controls how strongly the loss emphasizes high-similarity comparisons. Smaller values place greater weight on the most similar negatives, which are hardest to distinguish from the positive. Larger values distribute that weighting more evenly across the competing views.
The Stage 1 objective averages the loss over all anchors:
This objective updates both the encoder and projection head. No class labels or magnitude-reconstruction targets are used. The training settings and checkpoint-selection procedure are described in Section 4.
Stage 2: Supervised Contrastive Learning on Alcock
Stage 2 uses class labels to extend the positive set beyond another view of the same object. It loads both the encoder and projection-head weights from a Stage 1 checkpoint and continues training on labeled Alcock light curves. The six classes are Cep_0, Cep_1, EC, LPV, RRab, and RRc.
Each labeled object again supplies two augmented views. Let denote the class label inherited by view . Its positive set contains every other view in the minibatch with the same label:
This set includes the companion view of the same object and, when present, views of other objects in the same class. Every anchor therefore has at least one positive, so the set of valid anchors is
The supervised contrastive objective averages the log-probability of the same-class positives for each anchor:
The denominator contains all non-anchor views, including every positive. Views with different class labels serve as negatives.
An auxiliary classifier provides a class-prediction signal directly to the representation retained for downstream use. We attach a randomly initialized linear classifier to , with six output logits. Its cross-entropy loss is averaged over both augmented views of every object:
Thus, supervised contrastive learning acts on , while the auxiliary classifier encourages class discrimination directly through . This classifier is used only during Stage 2 and is separate from the downstream classifier.
The two objectives are combined with fixed weights that also specify the training schedule:
The first epoch uses a cross-entropy warm-up,
which updates the encoder and the new auxiliary classifier. During epochs 2–15, we use
so both objectives update the encoder, with the projection head trained through the contrastive term and the auxiliary classifier trained through cross-entropy. Section 4 describes the data partitions, optimization settings, and selection of the final encoder.
Final Contrastive Astromer Encoder
The final model produces one fixed-length representation from an unaugmented light-curve window.
After Stage 2, we discard the projection head and auxiliary classifier, retaining the trained encoder and the mean-pooling operation in Equation (3).
For downstream use, we select the deterministic window described in § 3.2, retain every real observation, and mark only padding positions with . The encoder remains frozen while a separate classifier is trained on , as described in § 5.1.
Experiments
This section describes the data partitions, training settings, and validation-only selection procedure used before final holdout evaluation.
Datasets
We use the MACHO and Alcock datasets distributed through the ASTROMER 2 GitHub repository1 ([9]). To accommodate our two-stage setup and prevent data leakage between stages, we repartition the data into object-disjoint pools for Stage 1, Stage 2, downstream cross-validation, and final testing.
We initialize Stage 1 from the MACHO-pretrained ASTROMER 2 checkpoint for unsupervised contrastive learning. Because the MACHO data share objects with the labeled Alcock catalog, we first remove all overlapping objects from the Stage 1 training set. The resulting Stage 1 training set shares no objects with Stage 2, downstream cross-validation, or the final holdout.
The Alcock catalog contains 20,894 labeled objects spanning six classes: Cep_0, Cep_1, EC, LPV, RRab, and RRc. We use a subset of 17,294 objects for supervised contrastive adaptation in Stage 2. Class counts in this subset are unequal. The remaining 3,600 objects are split into a 3,000-object downstream pool (500 per class) and a separate 600-object The downstream pool contains 500 objects per class. Alcock20, Alcock100, and Alcock500 use 20, 100, and 500 objects per class from this pool (120, 600, and 3,000 objects in total), respectively. The final holdout is separate and contains 100 objects per class.
final holdout (100 per class). The Alcock20, Alcock100, and Alcock500 settings draw 20, 100, and 500 objects per class from the downstream pool, respectively, for totals of 120, 600, and 3,000 objects. Each setting is evaluated with five non-overlapping validation folds, giving train/validation counts per class of 16/4, 80/20, and 400/100, respectively. The same folds are used for the pooled- controls and the reproduced official ASTROMER 2 pipeline. The final holdout is excluded from training, validation, and encoder selection. Table 2 summarizes the mutually disjoint Alcock partitions.
| Partition | Objects | Per class | Role |
| Stage 2 pool | 17,294 | class-imbalanced | representation learning |
| Downstream CV pool | 3,000 | 500 | five-fold selection |
| Final holdout | 600 | 100 | one-time testing |
| Total | 20,894 | — | — |
Table 2. Object-disjoint Alcock partitions used in the experiments. Counts denote unique objects.
Training and Evaluation Setup
Table 3 separates encoder construction, validation-only encoder selection, and final evaluation. The encoder is updated only in Stage 1 and Stage 2. Downstream classification freezes the selected encoder and trains a new classifier on its pooled representation .
| Stage | Dataset and split | Labels used? | Encoder updated? | Projection/classifier head | Purpose |
| Stage 1 | MACHO; no objects overlapping with Alcock | No | Yes | Projection head trained | Unsupervised contrastive representation learning |
| Stage 2 | 17,294-object Alcock representation pool | Yes | Yes | Projection head and auxiliary classifier trained | Supervised contrastive learning |
| CV and selection | 3,000-object pool; five folds; Alcock20/100/500 | Yes | No | Pooled- MLP head trained per run | Select one Stage 1 checkpoint using validation only |
| Final evaluation | Setting-specific training pool; 600-object holdout | Yes | No | Final MLP head retrained | Report holdout performance |
Table 3. Data roles in the Contrastive Astromer workflow.
The encoder architecture remains unchanged throughout. Stage 1 begins from the official MACHO-pretrained ASTROMER 2 checkpoint. Stage 1 runs for 100 epochs, saving a checkpoint after every epoch. Each checkpoint initializes a separate 15-epoch Stage 2 run on the disjoint Alcock subset. Table 4 summarizes the training settings.
| Setting | Stage 1 (unsupervised contrastive) | Stage 2 (supervised contrastive) | Downstream classifier head |
| Data split | MACHO; no objects overlapping with Alcock | 17,294-object Alcock pool | 3,000-object CV pool; 600-object holdout |
| Input window | 200 observations | 200 observations | central 200-observation window |
| Encoder | updated | updated | frozen |
| Objective | InfoNCE / NT-Xent | supervised contrastive + auxiliary CE | sparse categorical cross-entropy |
| Optimizer | Adam | Adam | Adam |
| Learning rate | |||
| Batch size | 32 | 32 | 128 |
| Temperature | 0.07 | 0.07 | — |
| Augmentation | main (Table 1) | weak (Table 1) | none |
| Epochs / schedule | 100 | 15; epoch 1 CE warm-up () = (0, 1), epochs 2–15 (1, 0.1) | early stopping, patience 40 |
| Checkpoint use | saved every epoch | epoch-15 encoder used for each Stage 1 checkpoint | best validation-loss head restored within each fold |
| Trained head | projection head | projection head + auxiliary classifier on | MLP 1024–512–256 + LayerNorm, 6-class output |
Table 4. Training and evaluation settings for the two encoder-construction stages and the downstream classifier head.
Stage 2 follows the loss schedule defined in Section 3.7. The first epoch optimizes only the auxiliary classifier’s cross-entropy loss, , defined in Equation (10), with . Epochs 2–15 optimize the combined objective in Equation (11), with . Every Stage 2 run lasts exactly 15 epochs, and downstream evaluation uses the encoder saved at the end of epoch 15. This training duration is fixed across all runs. The validation procedure described below selects among the resulting encoders initialized from different Stage 1 checkpoints.
Downstream Cross-Validation and Final Evaluation
Downstream evaluation compares three fixed encoder representations: the original MACHO-pretrained ASTROMER 2 encoder, a Stage 1 encoder, and its corresponding Stage 2 encoder. For each light curve, we encode its central segment and apply the mean in Equation (3) to the resulting token representations to obtain the pooled representation . A separate classifier is trained on each encoder’s pooled representations using the same classifier architecture, training settings, and data splits; the encoder remains frozen, and the contrastive projection head is not used.
The downstream classifier is an MLP that takes the pooled representation as input. It has three ReLU hidden layers with 1024, 512, and 256 units, followed by layer normalization and a six-class output layer. These hidden-layer widths match those of the official ASTROMER 2 classifier. Our classifier receives only , the mean of the final Transformer layer’s output vectors over the real observations in the selected window. The official classifier instead combines pooled representations from the input embedding and all Transformer layers, as described in Section 4.3. We optimize sparse categorical cross-entropy from logits with Adam at learning rate , batch size 128, and random seed 42. Validation loss controls early stopping with patience 40, and the minimum-validation-loss checkpoint is restored.
Each candidate encoder is evaluated through a separately trained downstream classifier. For every Alcock setting and fold, we train an MLP with the encoder frozen and retain the classifier weights from the training epoch with the lowest validation cross-entropy loss. We record the validation macro-F1 at that same epoch. Thus, the reported validation macro-F1 corresponds to the loss-selected classifier checkpoint, rather than the highest macro-F1 reached during classifier training.
We then use these validation scores to select the final encoder. Each of the 100 Stage 1 checkpoints initializes a separate Stage 2 run, whose epoch-15 encoder is evaluated across the three Alcock settings and five folds. This gives downstream classifier runs. For each candidate encoder, we average the validation macro-F1 values from its 15 selected classifier checkpoints, giving every setting and fold equal weight. We select the encoder with the highest mean validation macro-F1; exact ties are resolved by lower mean validation loss and then by the earlier Stage 1 initialization checkpoint. Test metrics do not enter this selection. The selected Stage 2 encoder is used across all three label budgets.
We also evaluate Stage 1 without Stage 2. We compare all 100 Stage 1 checkpoints using the same validation criterion across the three Alcock settings and five folds, with the same tie-breaking rules. This comparison selects the checkpoint used for the standalone Stage 1 results. We additionally evaluate the checkpoint selected for the full pipeline before Stage 2, allowing a direct comparison of the same encoder before and after Stage 2 training.
After encoder selection, we choose the number of training epochs for each final classifier from the mean validation-loss curve across the five cross-validation folds, using only epochs observed in all five folds. We then initialize a new classifier and train it for that number of epochs on the combined training and validation objects for the corresponding Alcock setting, without further early stopping. The classifier is evaluated once on the separate 600-object holdout. For the official ASTROMER 2 baseline, the numbers of epochs for target finetuning and classifier training are likewise selected through cross-validation.
Baselines
We use two ASTROMER 2 baselines to compare the complete classification pipeline and the encoder representation itself. The first reproduces the official finetune-and-classify pipeline across all three Alcock settings using the same disjoint five-fold downstream data construction. Its classifier draws on representations from several depths of the encoder: the input embedding before the first Transformer layer and the output of every Transformer layer. At each depth, it averages the token vectors over non-padding observations to obtain one vector per light-curve window. It then combines these pooled vectors using learned layer weights, implemented by the GammaWeight layer, and passes the result to an MLP. This baseline therefore includes both the official finetuning procedure and its multilayer readout.
The second baseline evaluates the original MACHO-pretrained ASTROMER 2 encoder through the same frozen-encoder classifier used for our contrastive models. It uses only the final-layer pooled representation , with the encoder held fixed while a separate MLP is trained as described in § 5.1. Comparing this baseline with the Stage 1 and Stage 2 encoders measures the changes in classification performance under a common pooling rule, classifier architecture, and downstream training protocol.
Metrics
The primary metric is macro-F1, the unweighted mean of the class-specific F1 scores. It gives each class equal weight, regardless of how many objects the class contains. Metric choice does not itself rebalance the training data or loss.
We also record accuracy, macro precision, macro recall, per-class precision, recall, and F1, confusion matrices, validation loss, and the selected epoch. The final holdout contains 100 objects per class, so accuracy is not dominated by unequal class support; macro-F1 remains primary because it captures the balance between precision and recall within every class. For the checkpoint-selection trajectories in Figure 4, the curves show aggregate mean validation macro-F1 and the shaded bands show the sample standard deviation across the setting/fold runs at each Stage 1 epoch. Final-refit holdout metrics are reported as single values.
Figure 4. Downstream validation macro-F1 across all 100 Stage 1 checkpoints for standalone Stage 1 and the full two-stage pipeline. The horizontal axis identifies the Stage 1 checkpoint: the standalone curve evaluates that encoder directly, while the two-stage curve evaluates the encoder obtained after a further 15 epochs of Stage 2 training. Each point averages scores across three Alcock settings and five folds; shaded bands denote the sample standard deviation across these 15 runs. Validation selects Stage 1 epoch 47 for standalone classification and epoch 33 as the initialization for the full pipeline.
Implementation Details
The experiments are implemented in TensorFlow/Keras ([1, 6]) and retain the official ASTROMER 2 encoder architecture and checkpoint interface ([9, 19]). The appendix records the mask convention and the data-disjointness audit.
Results
Downstream validation favors different Stage 1 checkpoints depending on whether the encoder is used directly or receives further supervised adaptation. We evaluate each Stage 1 checkpoint both on its own and after a fixed 15 epochs of Stage 2 training, fitting a separate downstream MLP to each frozen encoder. Following the selection procedure in § 5.1, standalone Stage 1 achieves its highest mean validation macro-F1 at epoch 47 (0.6873), whereas the full two-stage pipeline achieves its highest score when initialized from Stage 1 epoch 33 (0.7639; Figure 4).
These two Stage 1 checkpoints serve distinct roles in the comparisons below. Epoch 47 represents Stage 1 selected for standalone classification. Epoch 33 provides the matched starting point for measuring the improvement introduced by Stage 2: we compare this encoder before and after the 15 epochs of supervised adaptation. Both the classification and embedding-space evaluations include this matched comparison, while the classification results also include the independently selected epoch-47 encoder.
Downstream Classification
Stage 2 achieves the highest test macro-F1 across all three downstream label budgets. Table 5 reports final-refit holdout scores for the selected Stage 2 encoder, both Stage 1 controls, the original ASTROMER 2 encoder under
| Model / Pipeline | Alcock20 | Alcock100 | Alcock500 |
| Official ASTROMER 2 finetune-and-classify | 0.6933 | 0.7313 | 0.7433 |
| Original ASTROMER 2 encoder pooled- | 0.1735 | 0.2700 | 0.3599 |
| Stage 1, epoch 47 (independently selected) | 0.6248 | 0.6801 | 0.7289 |
| Stage 1, epoch 33 (matched initialization) | 0.6056 | 0.6703 | 0.7429 |
| Stage 2 (from Stage 1, epoch 33) | 0.7078 | 0.7704 | 0.7943 |
Table 5. Final-refit test macro-F1 after five-fold cross-validation model and training-duration selection. Each final model is refit on the complete development pool and evaluated once on the fixed holdout.
the same pooled- interface, and the reproduced official ASTROMER 2 pipeline. For each contrastive model, the same encoder checkpoint is used across all three label budgets.
The advantage over the reproduced official pipeline grows with the downstream label budget. Stage 2 improves test macro-F1 by 0.0145, 0.0391, and 0.0510 on Alcock20, Alcock100, and Alcock500, respectively, with corresponding accuracies of 0.7033, 0.7700, and 0.7933. Performance increases with the downstream label budget for every model, and Stage 2 maintains its lead throughout.
Stage 2 also outperforms the Stage 1 encoder selected specifically for standalone classification. This epoch-47 encoder achieves final-refit test macro-F1 scores of 0.6248, 0.6801, and 0.7289 on Alcock20, Alcock100, and Alcock500, respectively. Its checkpoint was selected using downstream validation performance before any Stage 2 training.
The matched epoch-33 comparison reveals how each contrastive stage contributes to the final result. Under the common pooled- interface, Stage 1 improves over the original ASTROMER 2 encoder by 0.4321, 0.4003, and 0.3830 macro-F1 on Alcock20, Alcock100, and Alcock500, respectively. This Stage 1 encoder remains below the complete official pipeline by 0.0877, 0.0610, and 0.0004, as expected given the official pipeline’s additional target finetuning and different readout.
Stage 2 builds on these gains, with its largest incremental benefit in the lowest-label setting. Relative to the matched epoch-33 Stage 1 encoder, it adds 0.1022, 0.1001, and 0.0514 macro-F1 on Alcock20, Alcock100, and Alcock500, respectively. Supervised adaptation therefore improves classification at every downstream label budget. Because the comparison follows the same encoder before and after Stage 2, these gains measure the effect of supervised contrastive adaptation rather than per-setting checkpoint selection.
Allowing a separate Stage 1 initialization for each Alcock setting would select epoch 52 for Alcock20 and epoch 79 for Alcock100 and Alcock500. These per-setting selections are retrospective sensitivity analyses; the primary Stage 2 results use the globally selected epoch-33 initialization across all three budgets, with the same checkpoint providing the matched Stage 1 control.
Table 6 reports the Alcock500 class-level results for the final-refit classifier built on the selected Stage 2 encoder initialized from Stage 1 epoch 33. Values are computed on the fixed 600-object holdout, which contains 100 objects per class.
| Class | Precision | Recall | F1 | Support |
| Cep_0 | 0.8977 | 0.7900 | 0.8404 | 100 |
| Cep_1 | 0.6484 | 0.8300 | 0.7281 | 100 |
| EC | 0.7912 | 0.7200 | 0.7539 | 100 |
| LPV | 0.8353 | 0.7100 | 0.7676 | 100 |
| RRab | 0.8367 | 0.8200 | 0.8283 | 100 |
| RRc | 0.8091 | 0.8900 | 0.8476 | 100 |
Table 6. Per-class Alcock500 performance for the final-refit classifier built on the globally selected Stage 2 encoder.
Alcock500 F1 is lowest for Cep_1 (0.7281) and highest for RRc (0.8476), Cep_0 (0.8404), and RRab (0.8283). Figure 5 shows that the largest individual off-diagonal errors are Cep_0→Cep_1 (18), RRab→RRc (17), and LPV→Cep_1 (16). RRab/RRc remains the largest mutual confusion, with 17 and 10 cross-assignments.
Figure 5. Alcock500 confusion matrix for the final-refit classifier built on the globally selected Stage 2 encoder. Rows are true classes and columns are predictions on the fixed 600-object holdout; each row contains 100 objects.
The gains from Stage 2 extend to the Cepheid classes, which are strongly underrepresented in the adaptation data.
The Stage 2 representation-learning pool contains only 73 and 572 objects, compared with 6,214 EC and 6,787 RRab objects. Despite this imbalance, Stage 2 raises Alcock500 F1 from 0.5488 to 0.7281 for and from 0.7822 to 0.8404 for , relative to the matched epoch-33 Stage 1 encoder. The final classifier correctly identifies 83 of the 100 holdout objects, achieving a recall of 0.8300 rather than collapsing this minority class into a majority class. These results indicate that supervised contrastive adaptation
substantially strengthens the representation of difficult minority classes despite the imbalance in its adaptation data.
Embedding-Space Evaluation
Downstream classification establishes whether the embeddings support accurate prediction, but it does not reveal how the classes are arranged in the representation space. We therefore examine whether contrastive training makes the embeddings themselves more class-coherent and better separated. Projection and clustering-based diagnostics are established ways to inspect the structure of learned representation spaces [16] (Xie et al. 2016). We first inspect the 256-dimensional pooled representations visually. PCA, t-SNE, and UMAP reduce the embeddings to two dimensions without using the class labels [14, 15]. After each projection is computed, we color the points by their known classes solely to make class structure visible.
The visualization uses a class-balanced sample of 3,000 objects drawn from the complete 20,894-object Alcock pool, with 500 objects sampled from each class. The exact sampling, preprocessing, and projection settings are reported in Appendix § 11.§ 3.
Figure 6 compares the original ASTROMER 2, Stage 1, and Stage 2 representations. The Stage 2 projections show the clearest visual alignment between the embedding clusters and the class colors, with substantially better separation than in the original representation. This qualitative structure indicates that Stage 2 produces embeddings that organize the variable-star classes more effectively.
Figure 6. Two-dimensional views of the 256-dimensional pooled- representations from the original ASTROMER 2, Stage 1 epoch 33, and Stage 2 from epoch 33 encoders (columns), using PCA, t-SNE, and UMAP (rows). Every panel contains the same class-balanced sample of 3,000 objects, with 500 objects per class, drawn from the complete 20,894-object Alcock pool. This combined sample is used only for qualitative visualization and is not treated as held-out evaluation evidence. Projection coordinates have no direct physical interpretation and should not be compared numerically across panels.
We next quantify class-aligned organization directly in the original 256-dimensional pooled- space. Unlike the visualization sample in Figure 6, all quantitative diagnostics are evaluated on the fixed 600-object locked holdout, containing 100 objects per class. For each encoder, a StandardScaler is fitted on the corresponding 3,000-object Alcock500 development pool and applied unchanged to the holdout embeddings. No downstream MLP is trained or used in this analysis.
We report three complementary diagnostics. First, -nearest-neighbor (k-NN; ) macro-F1 measures local label consistency by fitting a Euclidean k-NN classifier on the 3,000 labeled development embeddings and evaluating it on the locked holdout. Second, the inter/intra ratio divides the mean Euclidean distance between objects from different classes by the mean distance between objects from the same class; values above one indicate that between-class distances are larger on average. Third, the silhouette coefficient compares each object’s average distance to its own class with its distance to the nearest other class [22]. The known classes define the reference groups for all three diagnostics, so these are label-referenced measures of embedding geometry rather than a fully unsupervised clustering experiment.
Table 7 shows that Stage 1 substantially improves local class consistency: k-NN macro-F1 increases from 0.4926 for the original ASTROMER 2 representation to 0.7133, while the silhouette coefficient changes from −0.1008 to 0.0101. The inter/intra ratio decreases from 1.5267 to 1.2038, however, indicating that Stage 1 does not improve every global separation statistic. This distinction suggests that unsupervised contrastive learning produces more label-consistent local neighborhoods without yet organizing all classes into uniformly compact and separated global groups.
| Encoder | k-NN macro-F1 | Inter/intra ratio | Silhouette coefficient |
| Original ASTROMER 2 | 0.4926 | 1.5267 | -0.1008 |
| Stage 1, epoch 33 | 0.7133 | 1.2038 | 0.0101 |
| Stage 2 from epoch 33 | 0.7462 | 1.7577 | 0.0926 |
Table 7. Direct embedding-space diagnostics evaluated on the fixed 600-object locked holdout. The k-NN classifier uses the 3,000-object development pool as its labeled reference set. Higher values are better for all three metrics.
Stage 2 produces the strongest overall class-aware geometry. Relative to the matched Stage 1 encoder, its k-NN macro-F1 increases from 0.7133 to 0.7462, its inter/intra ratio increases from 1.2038 to 1.7577, and its silhouette coefficient increases from 0.0101 to 0.0926. The absolute silhouette value remains modest, indicating that some class overlap persists, but the agreement among the neighborhood, distance-ratio, silhouette, and projection diagnostics supports the conclusion that supervised contrastive adaptation produces the most class-discriminative pooled representation.
Discussion
The classification and embedding-space results show that the two contrastive stages play complementary roles. Stage 1 makes the pooled ASTROMER 2 representation more useful for classification, while Stage 2 strengthens class organization and achieves the highest reported holdout macro-F1 across the three downstream label budgets. Together, these findings support adapting the representation-learning objective to improve the utility of a fixed encoder architecture.
Contrastive Learning and Pooled Representations
ASTROMER 2 learns token representations by reconstructing masked observations from the surrounding light curve. This objective does not explicitly encourage the average of those representations to distinguish between variable-star classes. Stage 1 trains the encoder to bring representations of augmented views of the same object closer together and separate those of different objects. The gains in Table 5 show that Stage 1 makes class information more accessible through this simple pooling operation.
Stage 2 then uses class labels to expand the positive set from another view of the same object to views of other objects in the same class. Together with the auxiliary cross-entropy objective on , this training encourages a representation that supports class discrimination. Stage 2 improves on its Stage 1 epoch-33 starting point at every downstream label budget, with the largest gain in Alcock20. This pattern is consistent with supervised contrastive adaptation making the frozen representation easier to use when the downstream classifier has limited labeled data.
Embedding Geometry and Minority Classes
The direct embedding diagnostics provide evidence of improved class organization beyond the downstream MLP results. Stage 1 improves nearest-neighbor classification even though its inter/intra distance ratio decreases (Table 7). This combination suggests that useful local class neighborhoods can emerge without a corresponding improvement in global separation. Stage 2 achieves the highest -NN macro-F1, inter/intra ratio, and silhouette coefficient (Table 7), linking its classification gains to stronger class organization in the pooled representation itself. The modest silhouette coefficient of 0.0926 indicates that the classes still overlap. The qualitative projections in Figure 6 complement these full-dimensional diagnostics.
The class-level results show that Stage 2 also benefits the Cepheid classes, which are underrepresented in the Stage 2 training data. In Alcock500, both Cepheid classes improve after Stage 2, with the larger gain for the less abundant Cep_1 class. These gains show that the aggregate improvement includes better discrimination of minority classes despite the imbalance in the Stage 2 training data. Because the downstream classifier uses a balanced training sample, this finding concerns the usefulness of representations learned from imbalanced Stage 2 training data. It is particularly relevant to astronomical applications in which less abundant variability classes remain scientifically valuable.
Comparison with the Official ASTROMER 2 Pipeline
The original ASTROMER 2 encoder’s weak performance under simple averaging does not, by itself, show that its representations are poor. The official pipeline achieves much better classification performance with additional fine-tuning and a readout that combines information from multiple Transformer layers. Our results show that contrastive training makes the encoder’s representations more useful for classification when token representations are simply averaged.
Future Work
This study focuses on making the pooled representation of a pretrained light-curve encoder more useful for classification. By retaining the ASTROMER 2 architecture and comparing frozen encoders through a common pooling rule and downstream classifier, we connect the observed gains to changes in the learned representation. The Stage 2 results establish the effectiveness of supervised contrastive learning combined with auxiliary cross-entropy. Separating these two training signals would address a complementary question: how much each contributes to class discrimination and embedding geometry.
Transfer to new variability classes would test how broadly the learned representation can be reused. In the present study, Stage 2 uses a separate labeled adaptation pool containing the same six classes as the downstream task; the reported label budgets specify the examples available to each downstream classifier. Evaluating classes excluded from Stage 2 would require a different partition of the class vocabulary, extending the current study from improving discrimination among known classes to transferring representations beyond the adaptation labels.
Richer readouts could reveal additional class information distributed across the encoder. Applying the same multi-layer readout to the original, Stage 1, and Stage 2 encoders would test whether combining information from different depths further improves classification. Partial encoder fine-tuning would examine how these representations respond to additional task-specific training. These experiments would explore the interaction between representation learning and downstream model design, extending our present comparison of fixed, final-layer pooled representations.
Reliable uncertainty estimates could turn improved classification into better decisions about follow-up observations. A Gaussian process classifier applied to the pooled representations offers one route toward this goal [20]. Its predictive uncertainties would need to be assessed for calibration and for their ability to identify ambiguous light curves and prioritize follow-up targets. This direction introduces a decision-making objective beyond the classification performance and embedding geometry evaluated here.
Cross-survey evaluation would test whether the learned class structure persists under different observing conditions. The present experiments use single-band MACHO/Alcock light curves and a fixed augmentation family. Applying the method to surveys such as ZTF and ATLAS would introduce changes in cadence, noise, passbands, and class composition [2, 23]. Evaluating transfer under these changes would also help determine whether the augmentations should be tailored to the target survey. Class-level performance and embedding-neighborhood analyses would reveal which benefits, including those for underrepresented classes, extend beyond Alcock.
Conclusion
Contrastive Astromer demonstrates that the pooled representation of a pretrained light-curve encoder can be made substantially more useful for variable-star classification while preserving its architecture. Starting from ASTROMER 2, unsupervised contrastive learning on MACHO encourages consistency across augmented views, and supervised contrastive adaptation with auxiliary cross-entropy on Alcock makes class distinctions more accessible in the resulting representation. The two stages connect learning from unlabeled light curves with the construction of a class-aware encoder that can be reused across downstream label budgets.
On Alcock, the adapted encoder achieves holdout macro-F1 scores of 0.7078, 0.7704, and 0.7943 with 20, 100, and 500 labeled examples per class for downstream classifier training after the separate labeled adaptation stage. These are the highest reported scores among the evaluated pipelines. Under the common frozen pooled- interface, Stage 1 substantially improves on the original encoder, and Stage 2 adds a further 0.0514–0.1022 macro-F1 over its matched Stage 1 initialization. The largest Stage 2 gain occurs at the smallest downstream label budget.
The benefits extend to the organization of the representation itself. Direct embedding diagnostics show stronger local class consistency and class separation after Stage 2, while the Cepheid results demonstrate improved discrimination of classes underrepresented in the adaptation data. Classification, embedding geometry, and class-level performance therefore provide complementary evidence that contrastive adaptation makes the pooled representation more informative about variability class. Contrastive Astromer provides a concrete path from reconstruction-pretrained light-curve encoders to compact representations that expose class structure for both prediction and direct analysis.
Funding
This research received no specific grant from public, commercial, or not-for-profit funding agencies.
References
Implementation and Data-Audit Details
Mask Convention
The contrastive wrapper uses a visible-observation convention: denotes an observed token and denotes padding or an augmentation-dropped token. The official ASTROMER 2 attention implementation uses the opposite blocking convention internally. Before calling the official encoder, the wrapper therefore passes as the attention mask. This conversion keeps the visible-observation convention used for contrastive pooling consistent with the blocking convention expected by the official attention layer.
Data Package Provenance and Split Audits
The object-ID audit verified that the three Alcock partitions in Table 2 are mutually exclusive. A separate MACHO-to-Alcock audit removed 453 overlapping objects from the Stage 1 contrastive dataset before training.
All expected downstream runs and pooled- and official-pipeline controls completed successfully.
Embedding-Projection Configuration
Figure 6 uses a class-balanced sample drawn from the complete Alcock collection under the revised disjoint protocol. The source pool combines the three partitions in Table 2. Using random seed 42, we sample 500 objects without replacement from each of the six classes, producing a 3,000-object visualization set. Per-class reservoir sampling is followed by a deterministic global shuffle. The sampled records and their ordering are identical for the original ASTROMER 2, Stage 1 epoch 33, and Stage 2 from epoch 33 encoders.
For each encoder independently, the 256 embedding coordinates are standardized to zero mean and unit variance across the same 3,000 sampled objects before dimensionality reduction. PCA, t-SNE, and UMAP are then fitted independently to each encoder’s standardized embeddings. Class labels are not supplied to any projection method and are used only after projection to color the points. All stochastic procedures use random seed 42. Table A.1 reports the projection configuration.
The projections were generated using Python 3.10.20, NumPy 1.26.4, scikit-learn 1.7.2, Matplotlib 3.10.9, and UMAP-learn 0.5.12. Because each projection is fitted independently for each encoder, the coordinate values, orientations, and scales should not be compared numerically across panels. The projections are used only to compare visible class organization within each representation space.
| Method | Configuration |
| Sampling | 500 objects per class sampled without replacement from 20,894 unique Alcock objects; 3,000 objects total; random seed 42. |
| Preprocessing | StandardScaler fitted independently to each encoder’s 3,000 sampled embeddings. |
| PCA | Two components; random seed 42. |
| t-SNE | Two components; perplexity 30; PCA initialization; automatic learning rate; random seed 42. |
| UMAP | Two components; 15 nearest neighbors; minimum distance 0.1; Euclidean distance; random seed 42. |
Table A.1. Sampling, preprocessing, and dimensionality-reduction settings used to generate Figure 6. Parameters not listed use the defaults of the recorded software versions.
References
- [1]Abadi, M., Agarwal, A., Barham, P., Brevdo, E., Chen, Z., Citro, C., Corrado, G.S., Davis, A., Dean, J., Devin, M., Ghemawat, S., Goodfellow, I., Harp, A., Irving, G., Isard, M., Jia, Y., Jozefowicz, R., Kaiser, L., Kudlur, M., Levenberg, J., Mané, D., Monga, R., Moore, S., Murray, D., Olah, C., Schuster, M., Shlens, J., Steiner, B., Sutskever, I., Talwar, K., Tucker, P., Vanhoucke, V., Vasudevan, V., Viégas, F., Vinyals, O., Warden, P., Wattenberg, M., Wicke, M., Yu, Y., Zheng, X., 2015. TensorFlow: Large-scale machine learning on heterogeneous systems. Software available from tensorflow.org. URL: https://www.tensorflow.org/.arxiv.org/abs/1603.04467
- [2]Bellm, E.C., et al., 2019. The Zwicky Transient Facility: System overview, performance, and first results. Publications of the Astronomical Society of the Pacific 131, 018002. doi:10.1088/1538-3873/aaecbe, arXiv:1902.01932.DOI
- [4]Cádiz-Leyton, M., Cabrera-Vives, G., Protopapas, P., Moreno-Cartagena, D., Becker, I., 2025. Astro-MoE: Mixture of experts for multiband astronomical time series. arXiv preprint arXiv:2507.12611. URL: https://arxiv.org/abs/2507.12611, doi:10.48550/arXiv.2507.12611, arXiv:2507.12611.
- [5]Chen, T., Kornblith, S., Norouzi, M., Hinton, G., 2020. A simple framework for contrastive learning of visual representations, in: Proceedings of the 37th International Conference on Machine Learning, PMLR. pp. 1597–1607. URL: https://proceedings.mlr.press/v119/chen20j.html.arxiv.org/abs/2002.05709
- [6]Chollet, F., et al., 2015. Keras. https://keras.io.
- [7]Debosscher, J., Sarro, L.M., Aerts, C., Cuypers, J., Vandenbussche, B., Garrido, R., Solano, E., 2007. Automated supervised classification of variable stars. I. Methodology. Astronomy & Astrophysics 475, 1159–1183. doi:10.1051/0004-6361:20077638.DOI
- [8]Ding, J., Chen, X., Gao, X., Tang, X., Wang, S., Huang, Y., Qi, X., Xue, G., Luo, A., Liu, J., 2026. StarCLL: Contrastive learning representation for astronomical light curves. The Astrophysical Journal 1003, 141. doi:10.3847/1538-4357/aea64ef, arXiv:2604.24516.arxiv.org/abs/2604.24516
- [9]Donoso-Oliva, C., Becker, I., Protopapas, P., Cabrera-Vives, G., Cádiz-Leyton, M., Moreno-Cartagena, D., 2026. Generalizing across astronomical surveys: Few-shot light curve classification with Astromer 2. Astronomy & Astrophysics 707, A170. doi:10.1051/0004-6361/202550426, arXiv:2502.02717.
- [10]Donoso-Oliva, C., Becker, I., Protopapas, P., Cabrera-Vives, G., Vishnu, M., Vardhan, H., 2023. ASTROMER: A transformer-based embedding for the representation of light curves. Astronomy & Astrophysics 670, A54. doi:10.1051/0004-6361/202243928, arXiv:2205.01677.
- [11]Eyer, L., Mowlawi, N., 2008. Variable stars across the observational HR diagram. Journal of Physics: Conference Series 118, 012010. doi:10.1088/1742-6596/118/1/012010, arXiv:0712.3797.DOI
- [12]Khosla, P., Teterwak, P., Wang, C., Sarna, A., Tian, Y., Isola, P., Maschinot, A., Liu, C., Krishnan, D., 2020. Supervised contrastive learning, in: Advances in Neural Information Processing Systems, pp. 18661–18673. URL: https://arxiv.org/abs/2004.11362.
- [13]Lanusse, F., Parker, L.H., Golkar, S., Bietti, A., Cranmer, M., Eickenberg, M., Krawczik, G., McCabe, M., Ohana, R., Pettee, M., et al., 2023. AstroCLIP: Cross-modal pre-training for astronomical foundation models, in: NeurIPS 2023 AI for Science Workshop. URL: https://openreview.net/forum?id=zt35qM†DbT, arXiv:2310.03024.
- [14]van der Maaten, L., Hinton, G., 2008. Visualizing data using t-SNE. Journal of Machine Learning Research 9, 2579–2605. URL: https://www.jmlr.org/papers/v9/vandermaaten08a.html.
- [15]McInnes, L., Healy, J., Saul, N., Groβberger, L., 2018. UMAP: Uniform manifold approximation and projection. Journal of Open Source Software 3, 861. doi:10.21105/joss.00861.DOI
- [16]Mitrovic, J., McWilliams, B., Rey, M., 2020. Less can be more in contrastive learning, in: Proceedings on “I Can’t Believe It’s Not Better!” at NeurIPS Workshops, PMLR. pp. 70–75. URL: https://proceedings.mlr.press/v137/mitrovic20a.html.
- [17]van den Oord, A., Li, Y., Vinyals, O., 2018. Representation learning with contrastive predictive coding. arXiv preprint arXiv:1807.03748 URL: https://arxiv.org/abs/1807.03748.
- [18]Parker, L., Lanusse, F., Golkar, S., Sarra, L., Cranmer, M., Bietti, A., Eickenberg, M., Krawczik, G., McCabe, M., Morel, R., Ohana, R., Pettee, M., Regaldo-Saint Blancard, B., Cho, K., Ho, S., Collaboration, P.A., 2024. AstroCLIP: A cross-modal foundation model for galaxies. Monthly Notices of the Royal Astronomical Society 531, 4990–5011. doi:10.1093/mnras/stae1450, arXiv:2310.03024.DOI
- [19]Protopapas, P., Cabrera-Vives, G., Becker, I., Moreno Cartagena, D., Cádiz-Leyton, M., 2026. Supplementary material for: Generalizing across astronomical surveys: Few-shot light curve classification with Astromer 2. Zenodo dataset. URL: https://zenodo.org/records/18207945, doi:10.5281/zenodo.18207945.DOI
- [20]Rasmussen, C.E., Williams, C.K.I., 2006. Gaussian Processes for Machine Learning. Adaptive Computation and Machine Learning, MIT Press, Cambridge, MA. URL: https://gaussianprocess.org/gpml/.DOI
- [21]Rizhko, M., Bloom, J.S., 2025. AstroM3: A self-supervised multimodal model for astronomy. The Astronomical Journal 170, 28. doi:10.3847/1538-3881/adcbad, arXiv:2411.08842.DOI
- [22]Rousseeuw, P.J., 1987. Silhouettes: A graphical aid to the interpretation and validation of cluster analysis. Journal of Computational and Applied Mathematics 20, 53–65. doi:10.1016/0377-0427(87)90125-7.DOI
- [23]Tonry, J.L., Denneau, L., Heinze, A.N., Stalder, B., Smith, K.W., Smartt, S.J., Stubbs, C.W., Weiland, H.J., Rest, A., 2018. ATLAS: A high-cadence all-sky survey system. Publications of the Astronomical Society of the Pacific 130, 064505. doi:10.1088/1538-3873/aabadf, arXiv:1802.00879.DOI




