ST-LoRA: Single Trajectory LoRA Ensemble for Uncertainty Aware Agricultural Segmentation

Highlights

• ST-LoRA builds diverse ensembles using less model parameters

• Feed-forward layers, not attention, are the most important LoRA target for dense prediction

• ST-LoRA emerges as a competitor to full-rank ensemble at significantly smaller checkpoint size

• ST-LoRA offers competitive calibration robustness under distribution shift.

• ST-LoRA delivers competitive performance for near/far OoD detection tasks.

Introduction

Digital agriculture powered by sensor data and Machine Learning (ML) offers promising tools to optimize current practices and explore innovative solutions for future food production. Deep Neural Networks (DNN), in particular, excels at processing large sensor-based datasets to uncover meaningful patterns Attri, Awasthi, Sharma and Rathee [6]. This capability has enabled a range of impactful applications, including crop classification Teixeira, Morais, Sousa and Cunha [96], plant growth modeling and forecasting Drees, Weber, Rußwurm and Roscher [29]; Drees, Demie, Paul, Leonhardt, Seidel, Döring and Roscher [28]; Quan, Feng, Lv, Wang, Zhang, Liu and Yuan [86], disease detection Mob (2025), yield estimation Menon, Aravinth, Sankaran and Kiran [72], and building foundation models for enhancing the performance on various downstream tasks by using multi-modal sensory inputs Att (2025). As ML models become more integrated into agricultural workflows, enhancing computational efficiency and evaluating the quality of their predictions become essential; not only to establish trust in the predictions, but to know if human oversight is needed and when abstaining is safer than a confidently wrong decision. Uncertainty quantification (UQ) plays a key role by indicating when predictions are trustworthy unlike point estimates, UQ enables probabilistic predictions that help diagnose issues arising from the data, the model, or both. In supervised learning, models generalize from labelled data to unseen examples, an inductive process that introduces epistemic uncertainty (i.e., model uncertainty) due to limited knowledge about the underlying data-generating process. In contrast, aleatoric uncertainty (i.e., data uncertainty) stems from inherent noise and variability in the data. Unlike epistemic uncertainty which can in principle be reduced with more data or improved models, aleatoric uncertainty is irreducible, regardless of model capacity or training set size. Gawlikowski, Tassi, Ali, Lee, Humt, Feng, Kruspe, Triebel, Jung, Roscher, Shahzad, Yang, Bamler and Zhu [36]; Hüllermeier and Waegeman [53].

Among existing UQ methods, Deep Ensembles Lakshminarayanan, Prizel and Blundell [60] are widely regarded as the most commonly used method due to their strong calibration, predictive performance, and ability to decompose total uncertainty into its epistemic and aleatoric components. However, they are computationally expensive, multiplying both training time and memory footprint by the ensemble size, an increasingly critical limitation as modern architectures scale to billions of parameters and Vision Transformers Vaswani, Shazeer, Parmar, Uszkoreit, Jones, Gomez, Kaiser and Polosukhin [97] become the dominant backbone for dense prediction tasks. Snapshot ensembling Huang, Li, Pleiss, Liu, Hopcroft and Weinberger [50] addresses the training cost by collecting $M$ checkpoints from a single cosine-annealing trajectory, reducing training time by a factor of $1/M$. However, at deployment time, the snapshot ensembling is affected by large inference latency and memory footprint due to the need of maintaining $M$ full model replicas.

The same scaling pressure that makes deep ensembles prohibitive also affects standard fine-tuning. As pretrained vision and language models grow larger, adapting them to downstream tasks becomes increasingly costly in both compute and storage. Parameter-efficient fine-tuning offers a principled solution to this bottleneck. Low-Rank Adaptation (LoRA) Hu, yelong shen, Wallis, Allen-Zhu, Li, Wang, Wang and Chen (2022), originally developed for large language models, injects trainable low-rank matrices into the frozen layers of a pretrained backbone, dramatically reducing the number of trainable parameters and training time.

This naturally has raised the question of whether LoRA adapters can serve as ensemble members in their own right. Recent work has begun to explore this direction for uncertainty quantification [42, 8], freezing the shared backbone and constructing diverse members through different low-rank initializations. However, existing efforts have been limited to classification tasks at image level, with narrow ablation studies that leave the effect of key hyperparameters largely unexplored. Dense prediction tasks such as semantic segmentation, which impose stricter demands on spatial representations and pixel-level calibration, remain entirely unaddressed. This leads to the question at the core of this paper:

Can uncertainty quantification methods for dense prediction tasks be efficient not only robust ?

To fill this gap, we introduce and explore Single Trajectory LoRA (ST-LoRA), a lightweight ensemble-based extension of LoRA ensemble method for pixel-wise uncertainty quantification that constructs the members from a shared pretrained vision backbone. Illustrated in Figure 1, our method emerges as a lightweight competitor to the Full Rank Ensemble (FRE) ¹ on calibration and segmentation quality but also with significantly lower computational overhead.

Our main contributions are:

Contributions

• We introduce ST-LoRA, which combines LoRA adapters with snapshot ensembling to reduce training time compared to deep ensemble, number of parameters, and inference latency while being competitive full-rank ensemble’s calibration and segmentation quality.

• We show that ST-LoRA is less computationally complex by testing it on a real edge hardware used in agricultural context.

• We show through extensive ablation that feed-forward layers are important LoRA target in vision transformers for dense prediction, contrary to LLM practice.

• We provide a practitioner’s guide for configuring ST-LoRA across datasets and architectures, grounded in ablation findings on two crop types and three backbones.

• ST-LoRA achieves competitive performance with the low cross-seed variance across in-distribution evaluation, distribution shift, and near/far out-of-distribution detection against Snapshot Ensemble, MC Dropout, and DDU.

The rest of this article is organized as follows. Section 2 reviews research gaps in uncertainty estimation for agricultural machine learning. Section 3 introduces the mathematical background and presents the proposed ST-LoRA approach. Then, we present in Section 4 the experimental setup while Section 5 shows the experiment 1 - benchmarking ST-LoRA against FRE in terms of segmentation quality calibration, and computational efficiency- , Section 6 shows an ablation study on the most influential hyperparameters. Followed by Section 7 that presents an assessment of calibration robustness under distribution shift and finally Section 8 shows a comparison with state-of-the art methods covering in-distribution performance, covariate shift, and out-of-distribution (OoD) detection. Section 9 presents the final remarks and Section 10 concludes the paper. Additional analysis and implementation details are provided in the Appendix A and Appendix B .

Through extensive evaluation, we demonstrate that ST-LoRA² achieves competitive uncertainty quantification and segmentation quality relative to both full-rank and efficient SOTA uncertainty quantification methods, while offering substantial advantages in computational efficiency, storage footprint, and deployment flexibility.

Related Work

In this section, We begin with a general overview of uncertainty in machine learning, then follow the uncertainty pipeline through its two building blocks — representation, and quantification — before reviewing the limitations of existing approaches that motivate this work.

Uncertainty in ML can be addressed from two main perspectives. One common approach assumes that predictive uncertainty can be decomposed into epistemic (model) and aleatoric (data) components, each estimated independently Shaker and Hüllermeier [90]. An alternative perspective challenges the validity of this decomposition Hüllermeier [52], focusing instead on estimating total predictive uncertainty to ensure model reliability Angelopoulos and Bates [5]. As noted by Gruber, Schenk, Schierholz, Kreuter and Kauermann [40], the origin and categorization of uncertainty in ML remain ambiguous, with additional factors such as missing data or deployment in dynamic environments contributing to a more complex uncertainty landscape.

Uncertainty Evaluation. Evaluating UQ methods is non-trivial, as no ground-truth uncertainty labels exist for model predictions. Instead, evaluation relies on indirect schemes that probe whether uncertainty estimates are meaningful and actionable in practice. Following Hofman, Sale and Hüllermeier [47], four paradigms have been established: calibration under distribution shift Moreno-Torres, Raeder, Alaz-Rodríguez, Chawla and Herrera [74] assesses whether confidence scores remain reliable when imaging conditions change — critical in agricultural settings where lighting and weather vary; OoD detection Salehi, Mirzaei, Hendrycks, Li, Rohban and Sabokrou [88] identifies semantically novel inputs, flagging them for human review rather than producing overconfident predictions; selective prediction Xin, Tang, Yu and Lin [108] extends this by allowing abstention on any low-confidence sample, preferring a partial prediction over a confident wrong one; and active learning Bachman, Sordoni and Trischler [7] guides annotation by prioritizing uncertain samples, though this is orthogonal to our deployment-oriented focus. This work evaluates the first three schemes, as they directly assess prediction reliability in real-world agricultural deployment.

Uncertainty Representation for UQ falls into two primary categories: stochastic and deterministic. Stochastic representation captures epistemic uncertainty by modeling a second-order distribution over model parameters or outputs Hofman, Sale and Hüllermeier [48]. In contrast, deterministic represents uncertainty directly through distributions over latent representations Postels, Segù, Sun, Sieber, Van Gool, Yu and Tombari [85].³

Uncertainty Quantification Methods. Stochastic methods approximate Bayesian Neural Networks (BNNs), tackling the intractability of posterior inference through techniques like Monte Carlo Dropout (MC-Dropout) Gal and Ghahramani [35], Ensemble Learning Lakshminarayanan et al. [60], and Flipout Wen, Vicol, Ba, Tran and Grosse [104].

Deterministic methods - sampling free approaches, estimate uncertainty in a single forward pass by treating model weights as fixed. Recent advances in Deterministic Uncertainty Modeling (DUMs) Mukhoti, Kirsch, van Amersfoort, Torr and Gal [76]; van Amersfoort, Smith, Teh and Gal [4]; Liu, Lin, Padhy, Tran, Bedrax-Weiss and Lakshminarayanan [64]; Benkert, Prabhushankar and Al-Regib [9] achieve reliable estimates by regularizing latent representations for smoothness and bi-Lipschitz continuity, while avoiding feature collapse. These approaches offer lower computational overhead however, DUMs can suffer under distribution shifts and may negatively impact predictive accuracy due to the smoothness constraints imposed on the learned representations Postels, Segu, Sun, Gool, Yu and Tombari [84]. Hence more efforts focused on enhancing and exploring more efficient ensemble strategies.

Efficient Ensembles. Post-hoc ensemble methods like Checkpoint Ensembles (CE) Chen, Lundberg and Lee [15] and Snapshot Ensembles (Snap-E) Huang et al. [50] reduce training cost by saving intermediate model states during a single run. In contrast, integrated ensemble methods, including Packed Ensembles (PE) Laurent, Lafage, Tartaglione, Daniel, Martinez, Bursuc and Franchi, Multi-Input Multi-Output (MIMO) Havasi, Jenatton, Fort, Liu, Snoek, Lakshminarayanan, Dai and Tran, and Batch Ensembles (BE) Wen, Tran and Ba, embed ensembling into the architecture itself. For instance, PE uses group convolutions for efficient computation, MIMO extracts sub-networks from a shared backbone, and BE represents weights using a shared matrix modulated by rank-one factors. Finally, MC-Dropout provides a lightweight stochastic alternative by activating dropout at inference time.

While these methods improve the efficiency of uncertainty estimation, several challenges remain. Post-hoc ensembles reduce training time but do not lower inference cost or memory usage, as each ensemble member must be evaluated independently. Integrated ensemble approaches often require architectural modifications, which can be complex and potentially degrade the base model’s performance; for instance, the group convolutions used in Packed Ensembles (PE) are incompatible with Vision Transformer (ViT) architectures Dosovitskiy, Beyer, Kolesnikov, Weissenborn, Zhai, Unterthiner, Dehghani, Minderer, Heigold, Gelly et al.. MC-Dropout tends to be miscalibrated on in-distribution data and overly confident on out-of-distribution inputs, limiting its applicability in safety-critical settings Lakshminarayanan et al. [60].

In the text Mao, Ge, Fan, Xu, Mi, Hu and Gao [69] and Zhu, Shen, Zhao, Wang, Wang, Zhao, Shen and Wang [116]; Espejo-Garcia, Güldenering, Nalpantidis and Fountas [31] domains, LoRA Hu et al. (2022) has been particularly impactful, enabling fine-tuning on downstream tasks at a fraction of the computational cost of full fine-tuning. This efficiency has motivated its extension to ensemble construction: rather than training $M$ independent full models, LoRA adapters can serve as lightweight ensemble members over a shared frozen backbone, preserving ensemble diversity.

Closely related to our work, Paul, Quercia, Fortuin, Nöh and Scharr (2024) explored adapter-based techniques combined with Bayesian inference for uncertainty estimation in monocular depth estimation, while Onal, Flöge, Caldwell, Sheverdin and Fortuin applied a Bayesian framework with LoRA to quantify uncertainty in large language models. Recent efforts to investigate the suitability of LoRA ensembles for uncertainty estimation have been presented in Halbheer et al. [42] and Balabanov and Linander [8]. However, these studies share several limitations that motivate our work. First, they are restricted to image classification and Natural Language Processing (NLP) benchmarks, leaving open whether LoRA ensemble uncertainty estimation transfers to dense prediction tasks where uncertainty is a spatial field of pixel-wise distributions requiring class-aware calibration metrics, region-level OoD detection — none of which have been evaluated. Second, while Halbheer et al. [42] showed that LoRA Ensemble performs well against other uncertainty the investigation of LoRA hyperparameters has been limited which heavily affects the performance. Third, existing work retains $M$-fold training cost through independent member initialization, treating snapshot ensembling as a weaker alternative rather than a complementary mechanism. While prior layer placement studies compare coarse blocks such as attention versus attention with MLP, fine-grained component-level ablation remains absent, and the optimal configuration in classification settings does not transfer to vision transformer encoders used for segmentation. Finally Farag, Emam, Leonhardt and Roscher [32] have shown that DEs and MC-Dropout are the most commonly used methods for uncertainty estimation in agricultural segmentation, while Espejo-Garcia et al. [31]; Wang, He, Ye, Men, Gao and Wu [102] have shown that LoRA can be used for efficient fine tuning in agriculture without exploring the uncertainty aspect. This reveals a significant gap in exploring alternative methods that offer a better trade-off between computational efficiency and predictive reliability. In conclusion we observe the following gaps:

Research Gaps

• LoRA ensembles are untested in dense prediction tasks. Existing work is confined to image classification and NLP benchmarks, with no evaluation on dense prediction tasks or agricultural datasets.

• Critical deployment properties are neglected. No prior LoRA-based UQ method evaluates calibration under distribution shift for dense prediction tasks which is essential for safe real-world deployment.

• Layer selection is an open question. All prior work applies LoRA exclusively to attention projections following LLM conventions, without investigating whether feed-forward layers are more effective in vision transformers.

• Agricultural UQ lacks efficient alternatives. Uncertainty estimation in agricultural segmentation relies solely on expensive methods Farag et al. [32], with no calibration-aware, resource-efficient solution available.

• LoRA literature in Agriculture overlooks uncertainty quantification. It has focused only on downstream task without integrating uncertainty.

Methodology

In this section we begin by establishing notation, then briefly review the uncertainty representations used in our evaluation, before introducing the ST-LoRA formulation and its integration into semantic segmentation architectures.

Problem Formulation: Pixel-Wise Classification

Let $\mathcal{F}_{\theta} : \mathbb{R}^{H \times W \times3} \to\mathbb{R}^{H \times W \times L}$ be a pre-trained segmentation model mapping an input image $X \in\mathbb{R}^{H \times W \times3}$ to pixel-wise logits over $L$ classes, where $H$ and $W$ denote image height and width. For each pixel $(i,j)$, the predicted class probability vector is obtained by applying a softmax:

where $\Delta^{L-1}$ is the $(L-1)$-dimensional probability simplex. The predicted class label is then:

The ground-truth segmentation mask $Y \in\{1,\ldots,L\}^{H \times W}$ assigns a single class label to each pixel. The model is trained by minimizing the cross-entropy loss over all pixels:

Under this formulation, uncertainty quantification amounts to estimating, for each pixel $(i,j)$, how reliable $\hat{y}_{ij}$ is — distinguishing pixels where the model is confidently correct from those where the prediction should be treated with caution or abstained from entirely. For clarity, we adopt the following notation throughout this work. Scalars are denoted by lowercase letters (e.g., $r$, $\tau$), vectors by bold lowercase letters (e.g., $\mathbf{y}$), and matrices by bold uppercase letters (e.g., $\mathbf{X}$). Model parameters are denoted by $\theta$, with $\theta_0$ indicating frozen pre-trained weights. Probability distribution is represented by $\mathcal{P}(\cdot)$ and Gaussian Mixture Model (GMM) Dempster, Laird and Rubin (1977) is represented as $Q(\cdot)$, and functions or models by calligraphic letters (e.g., $\mathcal{F}$, $\mathcal{G}$, $\mathcal{H}$). Sets and spaces are denoted by blackboard bold letters (e.g., $\mathbb{R}$, $\mathbb{M}$). A summary of the main symbols is provided in Table 1.

Symbol	Description
Data and Labels
$X$	Input image
$\mathbf{y}_l$	Label for class $l$
$D$, $L$, $H$, $W$	Dataset, classes, height, width
Model and Parameters
$\theta_0$	Frozen pre-trained weights
$\Delta\theta_m$	Low-rank update for member $m$
$A_m$, $B_m$	ST-LoRA decomposition matrices
$r$, $\alpha$, $M$	Rank, scaling factor, ensemble size
$\mathcal{F}_{\theta}$	Segmentation model
$\mathbb{M}_r$	Low-rank subspace
Uncertainty
TU, EU, AU	Total, epistemic, aleatoric uncertainty
$\mathcal{P}(Y \mid X)$	Predictive posterior
$\mathcal{P}(\theta_m \mid D)$	Posterior weight for member $m$
$\psi(X)$
DDU-specific
$Z$	Latent feature vector
$Q(Z \mid l)$	Class-conditional GMM density
$\mu_l$, $\Sigma_l$	GMM mean and covariance
$\Pi(l)$	Class prior probability

Uncertainty Representation and Quantification

In this work, we consider two uncertainty representation frameworks: 1) stochastic representation (e.g. second-Order Distributions) which explicitly model uncertainty over predictions and 2) latent space distributions which represent uncertainty directly through distributions over latent space (i.e., feature space). We briefly describe their mathematical formulation in the following subsections.

Second-Order Distributions

Stochastic methods represent epistemic uncertainty by modeling a distribution over model parameters. The total predictive uncertainty (TU) is decomposed as TU ≈ AU +

EU, where AU and EU denote aleatoric and epistemic uncertainty respectively. Using Shannon entropy Hofman et al. [48]:

where $\mathcal{P}(y_l \mid X_{\mathrm{test}})$ is the probability of class $l$ under the predictive posterior. The aleatoric component is the expected entropy across ensemble members:

and epistemic uncertainty follows as $\mathrm{EU} \approx\mathrm{TU} - \mathrm{AU}$.

Latent Space Distributions

Deterministic methods such as DDU Mukhoti et al. [76] estimate uncertainty from the density of the input’s latent representation, without requiring multiple forward passes.

A GMM is fitted per class on penultimate-layer features $Z = F_\theta(X)$:

Epistemic uncertainty is then the negative log-likelihood under the class-marginalized mixture:

where a high value indicates the sample lies far from the training manifold. Aleatoric uncertainty is estimated as the categorical entropy of the softmax output, consistent with Eq (4).

ST-LoRA: Efficient Ensemble Learning via Low-Rank Adaptation

We start by injecting LoRA trainable low-rank matrices into frozen layers. This constrains members’ diversity to a learned task-relevant subspace, analogous to how Principal

Component Analysis (PCA) Shlens [91] identifies low-dimensional structure — but unlike PCA, ST-LoRA learns this weight subspace end-to-end via gradient descent, aligning it with downstream objectives. For a linear layer with weight $\theta_{0} \in\mathbb{R}^{d \times k}$, the adapted output is:

where $A \in\mathbb{R}^{r \times k}$ and $B \in\mathbb{R}^{d \times r}$ with $r \ll\min(d,k)$, reducing trainable parameters from $dk$ to $r(d+k)$.

This design is theoretically motivated by two observations. First, Aghajanyan, Gupta and Zettlemoyer [3] show that pre-trained models have a low intrinsic dimension, suggesting low-rank reparameterizations can match full-rank fine-tuning. Second, Hu et al. (2022) demonstrate that $\Delta\theta$ is strongly correlated with $\theta_{0}$, implying the low-rank subspace retains sufficient task-relevant information for meaningful uncertainty estimation. An illustration is shown in Figure 2.

Deep Ensembles Lakshminarayanan et al. [60]; Loaiza-Ganem, Villecroze and Wang [65] approximate Bayesian inference by treating independently trained models as samples from the parameter posterior $\mathcal{P}(\theta\mid D)$. The ensemble predictive posterior is:

While effective, this requires training and storing $M$ full model replicas, making it prohibitive for large vision transformers.

Hence, we use snapshot approach to construct an ensemble from a single training trajectory, reducing the effective training cost by a factor of $1/M$ compared to training $M$ independent models. A cosine annealing learning rate scheduler introduces cyclic variation in the learning rate, allowing the model to converge to distinct local minima within a single run. At the end of each cycle, the current model weights are saved as a snapshot, and the final ensemble averages predictions across all $M$ collected checkpoints Huang et al. [50]. Figure 3 illustrates the learning rate schedule and the corresponding snapshot collection points.

Each ensemble member $m \in\{1,\ldots,M\}$ in ST-LoRA for Semantic Segmentation is defined by a distinct adapter initialized from a different seed, yielding the uniform mixture:

from which TU, AU, and EU are computed pixel-wise using Equations (4) and (5).

Experimental Setup

Datasets and Pre-Processing

BUP20⁴ sawtooth pepper phenotyping dataset Smitt, Halstead, Zaenker, Bennewitz and McCool [92], which contains high-resolution $(1280 \times720)$ video frames captured by a Unmanned Ground Vehicle (UGV) in a commercial greenhouse. A subset of 279 non-overlapping frames is manually annotated for both semantic and instance segmentation across six classes: background, red, yellow, green, mixed-red, and mixed-yellow peppers. The dataset poses real-world challenges, including occlusions, lighting variation, background clutter, and class overlap, making it a strong benchmark for uncertainty-aware models in agriculture. All images are normalized using the mean $[0.485, 0.456, 0.406]$ and the standard deviation $[0.229, 0.224, 0.225]$, consistent with the ADE20K dataset Zhou, Zhao, Puig, Xiao, Fidler, Barriuso and Torralba [115].

GrowliFlower⁵ Kierdorf, Junker-Frohn, Delaney, Olave, Burkart, Jaenicke, Muller, Rascher and Roscher [55] is an open-source dataset acquired in 2020 and 2021 by flying a UAV, It contains 14K samples of cauliflower plants with multiple modalities available, including RGB and multi-spectral images. The dataset also includes a comprehensive range of annotations, making it useful for computer vision tasks such as classification and segmentation. All images are resized to $256 \times256$⁶ and normalized to be in range [0, 1]. No augmentation has been used while it helps the ensembles to get more diverse, however we isolate its effect and investigate more its effect at § 10. Furthermore, we filter out samples that include only soil at the early growth stages. Samples from both datasets are shown in Figure 4.

LoRA Adaptation for Segmentation Models

In this work, we focus on Vision Transformer (ViT)–based segmentation models for two reasons. First, our aim is to study the effect of low-rank adaptation in a setting that reflects current practice, and recent advances in semantic segmentation have shifted heavily toward transformer-based architectures, such as SegFormer, DPT, MaskFormer, Mask2Former, EoMT, and SAM Xie, Wang, Yu, Anandkumar, Alvarez and Luo [107]; Cheng, Misra, Schwing, Kirillov and Girdhar [16]; Cheng, Schwing and Kirillov [17]; Ranftl, Bochkovskiy and Koltun [87]; Kirillov, Mintun, Ravi, Mao, Rolland, Gustafson, Xiao, Whitehead, Berg, Lo et al. [57]; Kerssies, Cavagnero, Hermans, Norouzi, Averta, Leibe, Dubbelman and de Geus [54]. Evaluating LoRA in this context therefore aligns our analysis with the predominant direction of modern segmentation research. Second, although LoRA can in principle be applied to convolutional networks, its formulation and most empirical validations have been developed around transformer blocks.

We adopt SegFormer, Mask2Former and EoMT as our primary models due to their distinct design philosophies. SegFormer follows a per-pixel classification paradigm, combining a ViT-based encoder with a lightweight MLP decoder. In contrast, Mask2Former/EoMT frame segmentation as a mask-classification problem, using a shared transformer-based⁷ encoder and without a decoder or with two decoders:

a convolutional decoder for pixel-wise upsampling and a transformer decoder for predicting mask embeddings. Lastly, SegFormer is a task-specific (non-universal) segmentor tailored primarily for semantic segmentation, whereas Mask2Former/EoMT are universal segmentors capable of handling semantic, instance, and panoptic segmentation within a unified framework.

To enable efficient and scalable uncertainty quantification, we integrate LoRA into selected modules of both architectures. We first choose the target components (e.g., MLP layers or Multi-Head Attention blocks) and specify a low rank (e.g., r=8). The remaining model parameters can either remain frozen or be partially fine-tuned alongside the LoRA adapters, depending on the configuration. During inference, only the LoRA adapter and partly fine tuned weights are loaded, enabling faster execution and reduced memory usage. This design supports the construction of lightweight ensembles with significantly lower computational cost, making the approach well suited for resource-constrained agricultural robotics.

Evaluation Schemes and Metrics

We organize our evaluation metrics into three categories aligned with the evaluation schemes described above: predictive performance, computational efficiency, and uncertainty quantification.

Predictive Performance and Computational Efficiency

We assess segmentation quality using mean Intersection over Union (mIoU), which measures the average overlap between predicted and ground-truth segments across all classes. We evaluate computational cost across multiple dimensions: the number of trainable parameters; deployment I/O overhead, measured as checkpoint loading latency for each ensemble strategy; training duration, tracked via wall-clock time and seconds per epoch; and energy consumption, measured as average GPU power, GPU energy, and total energy (GPU, CPU, and RAM) on a single NVIDIA A100-SXM4-80GB GPU (400 W maximum thermal power), from which we derive CO$_2$ emissions.⁸ Together, these metrics provide a comprehensive picture of deployment feasibility on resource-constrained agricultural platforms.

Uncertainty Method Quality Assessment

Robustness under Distribution Shift. Dataset shift occurs when the joint distribution of training and test data differ, formally expressed as $P_{\mathrm{train}}(X,\mathbf{y}) \ne P_{\mathrm{test}}(X,\mathbf{y})$ Moreno-Torres et al. [74]. Multiple shift types exist, including prior probability shift, concept shift, and covariate shift. This work focuses on covariate shift, defined as:

where the conditional relationship between inputs $X$ and labels $\mathbf{y}$ remains invariant, but the marginal input distribution shifts between training and test conditions. This scenario is particularly relevant for agricultural applications, where imaging conditions vary across seasons, weather, and lighting.

To evaluate calibration robustness, we adopt the concept of breakdown point from robust statistics Loh [66]. The breakdown point quantifies the maximum proportion of corrupted observations an estimator can tolerate before producing arbitrarily incorrect results. In our context, we assess how calibration quality (measured by ECE) Guo, Pleiss, Sun and Weinberger [41] computed with both $\ell_1$ and $\ell_\infty$ norms to capture average and worst-case miscalibration respectively and segmentation performance (measured by mIoU) degrade as the severity of distribution shift increases. A robust UQ method should maintain relatively stable calibration and performance across varying shift intensities.

Out of Distribution (OoD) detection An essential method for evaluating the quality of an uncertainty quantification technique is to observe its response to these OoD samples. This response reflects the model’s ignorance expressed as high uncertainty about these points.

The problem of OoD detection can be formulated as follows:

When $X_{n+1}$ is declared, the OoD detector assigns a score to the sample by using the $\psi$ which is the scoring function, if it is greater than or equals $\tau$ it is considered ID, otherwise it is rejected and classified as OoD. For scoring we can use confidence, entropy, or any function under the condition that it is able to differentiate between the classes or to rank them differently. Hence, OoD detection can be treated as a binary classification problem, the only difference is that we lack the reference values for the OoD samples. For out-of-distribution detection, we measure discrimination performance using the area under the receiver operating characteristic curve (AUROC), which quantifies the method’s ability to separate in-distribution from out-of-distribution samples across all possible confidence thresholds.

Design rationale

The experimental stages are deliberately ordered so that each is motivated by the findings of the previous one, building toward a complete picture of ST-LoRA’s deployment readiness.

We begin with in-distribution performance (Section 5), addressing the most fundamental question: can low-rank adapters substitute full model replicas without meaningful loss in accuracy or calibration? Establishing this is a prerequisite for everything that follows; if ST-LoRA fails here, its efficiency advantages are irrelevant.

Having confirmed in-distribution reliability, we turn to the ablation study (Section 6), which asks how ST-LoRA should be configured. Since prior work applies LoRA exclusively to attention projections Hu et al. (2022); Mühllematter, Halbheer, Becker, Narnhofer, Aasen, Schindler and Turkoglu (2024), we systematically investigate target components, rank, scaling factor, and dropout to explore the hyperparameter landscape for vision transformer segmentation. All subsequent experiments use the configuration identified here.

With a validated and well-configured model, we assess its robustness under distribution shift (Section 7). Agricultural imaging conditions vary across seasons, weather, and lighting - a model that is well calibrated in-distribution but degrades rapidly under shift offers limited practical value. This stage examines whether ST-LoRA’s calibration advantage is stable or fragile, and identifies which hyperparameter choices offers better robustness.

Finally, comparison against efficient baselines (Section 8) situates ST-LoRA within the broader landscape of uncertainty quantification and post-hoc calibration methods, asking whether it matches or outperforms established SOTA alternatives - Snapshot Ensemble, MC Dropout, and DDU⁹ - across In distribution performance, robustness under distribution shift, and out-of-distribution detection. Together, these four stages demonstrate that ST-LoRA’s uncertainty estimates are not only accurate in controlled conditions but practically reliable under the challenging and variable conditions of real agricultural deployments.

Experiment 1: In Distribution (ID) Performance and Computational Efficiency

We evaluate four model architectures (SegFormer-B2, SegFormer-B4, and Mask2Former Swin-Base $10$,¹⁰ and EoMT) with three uncertainty quantification approaches (Softmax as a baseline), Full-Rank Ensemble (referred to in literature as SnapShot ensemble, and ST-LoRA)¹¹. Predictive performance is assessed via mIoU, calibration via ECE, and computational efficiency via FLOPs and parameter count. Additionally, we quantify the deployment I/O overhead of each ensemble strategy by measuring checkpoint loading latency on both GPU available by Google Colab and edge hardware (Raspberry Pi 4B)¹², comparing per-switch disk reads against in-memory adapter switching., and finally we measure the energy dissipated and CO$_2$ emissions.

Segmentation Quality and Calibration

Table 2 reveals a consistent pattern across both datasets: ensembling reduces mIoU on GrowliFlower-L but improves it on BUP20 dataset while improving calibration on both reflected by lower ECE compared to single-model counterparts. This trade-off is expected, as ensemble averaging smooths overconfident predictions and can be seen for all models in our comparative analysis.

Model	Baseline		FRE				ST-LoRA (ours)
	mIoU(%) ↑	ECE ↓	mIoU(%) ↑	ΔmIoU(%)	ECE ↓	ΔECE(%)	mIoU(%) ↑	ΔmIoU(%)	ECE ↓	ΔECE(%)
GrowliFlower-L (Cauliflower)
SgF-B4	88.6	0.0052	83.9	−5.3	0.0039	−25.0	84.5	−4.6	0.0027	−48.1
SgF-B2	87.8	0.0061	83.9	−4.4	0.0034	−44.3	84.7	−3.5	0.0052	−14.8
BUP20 (Sweet Pepper)
M2F	67.8	0.0112	67.1	−1.0	0.0030	−73.2	68.1	+0.4	0.0043	−61.6
SgF-B2	66.1	0.0028	70.0	+6.0	0.0048	+71.4	69.3	+5	0.0021	+25
EoMT	67.2	0.0073	67.4	+0.30	0.0033	−54.8	67.5	+45	0.0032	−56.2

Comparing ST-LoRA directly to FRE, ST-LoRA achieves competitive performance with less trainable parameters

Category	Hyperparameter	Pixel wise binary classification — Baseline	Pixel wise binary classification — FRE / ST-LoRA	Pixel wise multiclass classification — Baseline / FRE, and ST-LoRA
Dataset	Name		GrowliFlower-L	BUP20
Dataset	Train		1381	124
Dataset	Validation		293	63
Dataset	Test		296	92
Pre-processing	Image size		$256 \times256$	$1280 \times720$
Pre-processing	Normalization		$[0,1]$	$\mu=[0.485,0.456,0.406],\ \sigma=[0.229,0.224,0.225]$
Pre-processing	Augmentation		None	None
Model	Architecture		SegFormer (B2 / B4)	Mask2Former (Swin-B)/EoMT-Large(DINOv2)
Model	Pre-trained weights		ImageNet	ADE20K
Model	LoRA rank	—	$r=8$	— / $r=8$
Training	Epochs	100	100	100
Training	Optimizer	Adam	Adam	Adam
Training	Learning rate	$1 \times10^{-4}$	$[2 \times10^{-4},2 \times10^{-2}]$	$2 \times10^{-4}/[2 \times10^{-5},2 \times10^{-3}]$
Training	LR schedule	Constant	Cosine annealing w/ warm restarts	Constant / Cosine annealing
Training	Cycle length	—	20 epochs	20 epochs
Training	Ensemble members	—	5	— / 5
Training	Batch size	32	32	4
Training	Early stopping	Patience = 100	—	—
Training	Loss		Cross-entropy	Cross-entropy
Training	Weight decay		0.0	0.0
Training	Hardware		NVIDIA A100 (80 GB)	NVIDIA A100 (80 GB)

across both datasets and all architectures. On BUP20, ST-LoRA matches or exceeds the FRE mIoU among all methods with similar ECE performance on average across models and datasets. On GrowliFlower-L with SegFormer-B4, ST-LoRA achieves the best ECE while also outperforming FRE in segmentation accuracy, demonstrating that low-rank adapters can effectively substitute full model replicas without significant performance degradation.

Furthermore, we use reliability diagrams Dimitriadis, Gneiting and Jordan [24] to assess model calibration by plotting prediction confidence on the x-axis against empirical accuracy on the y-axis. A perfectly calibrated model follows the diagonal line $y = x$, where accuracy equals confidence at all prediction levels. Deviations below the diagonal indicate overconfidence (predictions are more confident than accurate), while deviations above indicate underconfidence. Figure 5 reveals the substantial impact of ensembling on calibration. Both the baseline and single ST-LoRA models exhibit severe overconfidence, with their reliability curves falling significantly below the diagonal across most confidence bins. Ensembling significantly improves calibration for both approaches: FRE and ST-LoRA curves shift closer to the ideal diagonal, reducing the gap between predicted confidence and actual accuracy. FRE demonstrates marginally better calibration than ST-LoRA across most confidence bins, particularly in the 0.4–0.8 range. However, FRE exhibit underconfidence in the highest confidence bin (0.8–1.0), where the curves rise above the diagonal. This suggests that when the ensembles are most correct, they are actually too conservative in their predictions. This underconfidence at high confidence levels represents a different

Sizes: FRE 4×431.8 MB, ST-LoRA 4×5.5 MB (78× smaller).

calibration failure mode that may require targeted post-processing techniques as mentioned in Wang, Gong and Wang [100]..

Computational Efficiency, Energy Consumption, and CO2 Emissions

Beyond predictive quality, ST-LoRA is substantially more computationally efficient than FRE in terms of number of trainable parameters, memory footprint and I/O latency as shown in Table 4 and Table 3. However, we observe ST-LoRA increases FLOPs more compared to FRE which is a cost shared by both methods due to the ensemble averaging step¹⁴.

Device	Strategy	FRE	ST-LoRA (ours)
Colab	Sequential (ms)	1704.7	172.2
	Setup† (ms)	6402.0	2237.4
	Per switch (ms)	0.001	8.0
Rasp. Pi	Sequential (ms)	1017.0	8.0
	Setup† (ms)	4080.0	25.0
	Per switch (ms)	0.001	0.001

Approach	FLOPs (GMAC)	Train Params. (M)
Sgf-B4 (base)	12.54	61.37
FRE	62.70	306.84
ST-LoRA (ours)	64.78	70.84
Sgf-B2 (base)	5.34	24.72
FRE	26.64	123.61
ST-LoRA (ours)	27.64	29.85
M2F (base)	229.72	108.77
FRE	1148	534
ST-LoRA (ours)	1170	116

The latency reduction stems from the size asymmetry between LoRA adapters and full model replicas: since each adapter is significantly smaller than a full checkpoint, the I/O overhead that typically bottlenecks sequential model loading becomes negligible. To verify this, we benchmark two loading strategies for Mask2Former checkpoints under both FRE and ST-LoRA configurations, on Google colab (network-mounted filesystem) and an edge device (Raspberry Pi 4B, local ext4, 449 MB/s). The first strategy loads each checkpoint from disk before every forward pass, incurring full I/O cost per ensemble member. The second strategy preloads all checkpoints into RAM once at startup, then switches between them in each memory during inference, eliminating repeated disk reads entirely.

Table 3 reports ensemble switching latency under sequential and preloaded strategies on server and edge hardware. In the sequential setting, ST-LoRA reduces per-model loading time by 10× on the server and 127× on the edge device, a direct consequence of its 78× smaller checkpoint size. Under the preloaded strategy, ST-LoRA’s one-time setup cost is 2.9× lower on the server and 163× lower on the edge, reducing startup overhead from 4080 ms to just 25 ms on the Raspberry Pi 4B. Once preloaded, per-switch latency is negligible on both methods and both devices with an advantage for ST-LoRA as the memory is much smaller.

We have shown that the efficiency of our approach is reflected in reducing training time compared to deep ensemble, fewer trainable parameters, reduced storage requirements ,and lower I/O latency compared to the FRE method. However, all of these metrics have been measured after training. To assess the training-time energy implications of ST-LoRA, we train FRE and ST-LoRA for 40 epochs and monitor their energy footprint. To isolate the training loop from confounding factors, we exclude the first epoch

• corresponding to GPU warm-up - and omit the validation loop from monitoring.

As shown in Table 5, ST-LoRA behaves differently across the three architectures. For Mask2Former, ST-LoRA is greener than FRE, even though both draw a comparable average GPU power; the reduction comes instead from a shorter training duration, which lowers total energy and emissions. For SegFormer-B2, ST-LoRA is essentially with lower training time but with higher power compared to FRE: while it still reduces the storage footprint, it consumes a similar amount of energy and produces similar emissions. For EoMT, however, ST-LoRA performs worse than FRE, with higher values across all monitored metrics on average. This pattern holds consistently whether we consider the training loop alone or the full run including validation and warm-up.

Arch	Metric	FRE	ST-LoRA	%Δ	95% CI	$t$ $p$	Wilcoxon $p$	$d_z$
Mask2Former	CO$_2$ (kg)	0.08749 ± 0.00049	0.08193 ± 0.0026	−6.3%	[−8.0, −3.5]	0.011	0.062	+2.02
Mask2Former	Total energy (kWh)	0.2297 ± 0.0013	0.2151 ± 0.0069	−6.3%	[−8.0, −3.5]	0.011	0.062	+2.02
Mask2Former	GPU energy (kWh)	0.1795 ± 0.0015	0.1654 ± 0.0024	−7.8%	[−8.8, −6.8]	0.000	0.062	+6.01
Mask2Former	Wall-clock (s)	2154 ± 40	1980 ± 24	−8.1%	[−9.2, −7.2]	0.000	0.062	+5.55
Mask2Former	GPU power (W)	298.7 ± 7.4	299.3 ± 7.3	+0.2%	[−1.0, +2.3]	0.844	0.625	−0.09
SegFormer	CO$_2$ (kg)	0.01658 ± 0.00037	0.01677 ± 0.00056	+1.1%	[−1.2, +2.6]	0.387	0.625	−0.43
SegFormer	Total energy (kWh)	0.04353 ± 0.00097	0.04402 ± 0.0015	+1.1%	[−1.2, +2.6]	0.387	0.625	−0.43
SegFormer	GPU energy (kWh)	0.03325 ± 0.0011	0.03335 ± 0.0011	+0.3%	[−5.0, +5.1]	0.926	1.000	−0.04
SegFormer	Wall-clock (s)	444.3 ± 6.4	427.7 ± 11	−3.7%	[−6.5, −0.7]	0.100	0.188	+0.95
SegFormer	GPU power (W)	261.8 ± 12	273.5 ± 18	+4.5%	[−4.4, +13.3]	0.423	0.438	−0.40
EoMT	CO$_2$ (kg)	0.02035 ± 0.00094	0.02258 ± 0.0023	+10.9%	[+2.6, +21.5]	0.107	0.062	−0.92
EoMT	Total energy (kWh)	0.05342 ± 0.0025	0.05926 ± 0.0061	+10.9%	[+2.6, +21.5]	0.107	0.062	−0.92
EoMT	GPU energy (kWh)	0.04035 ± 0.0013	0.04299 ± 0.0022	+6.5%	[+2.7, +11.1]	0.049	0.062	−1.25
EoMT	Wall-clock (s)	566.5 ± 58	604.4 ± 71	+6.7%	[+0.1, +14.2]	0.166	0.312	−0.76
EoMT	GPU power (W)	253.1 ± 18	254.8 ± 16	+0.7%	[−3.3, +4.1]	0.767	0.625	−0.14
Mask2Former	GPU energy (train-only)	0.1384 ± 0.0012	0.1214 ± 0.0027	−12.3%	[−13.5, −10.6]	0.000	0.062	+6.61
Mask2Former	Wall-clock (train-only)	1683 ± 17	1498 ± 10	−11.0%	[−11.6, −10.4]	0.000	0.062	+12.04
SegFormer	GPU energy (train-only)	0.02018 ± 0.00043	0.01969 ± 0.0005	−2.4%	[−5.1, +0.2]	0.196	0.312	+0.69
SegFormer	Wall-clock (train-only)	331.5 ± 6.5	315.9 ± 7.9	−4.7%	[−7.8, −1.4]	0.072	0.188	+1.09
EoMT	GPU energy (train-only)	0.02055 ± 0.00047	0.02279 ± 0.00067	+10.9%	[+9.1, +13.4]	0.001	0.062	−3.87
EoMT	Wall-clock (train-only)	357.6 ± 0.92	406.5 ± 29	+13.7%	[+8.3, +20.8]	0.020	0.062	−1.68

We hypothesize that this architecture-dependent behaviour is driven by the fact that the frozen base-model weights still contribute substantially to total emissions, since forward and backward passes over the full-rank backbone remain energy-intensive regardless of how few parameters are actually updated. The proportion of fine-tuned parameters relative to the total model size also appears to play a role. As an example in Mask2Former, ST-LoRA fine-tunes roughly 1 million of 108 million parameters (≈ 0.9%); for SegFormer-B2, roughly 3.5 million of 30,775,504 (≈ 11.4%); and for EoMT, roughly 12 million of 320 million (≈ 3.8%). Notably, Mask2Former - the architecture with the smallest fine-tuned fraction - shows the strongest efficiency gain, while EoMT - with the largest absolute number of fine-tuned parameters and a comparatively larger fine-tuned share than Mask2Former - shows the weakest outcome. This suggests that as the fine-tuned fraction grows, the relative savings from adapting only a small subset of parameters are increasingly offset by the fixed energy cost of the full-rank forward and backward passes through the frozen backbone.

This aligns with observations made in the original LoRA paper Hu et al. (2022). The authors emphasize that LoRA’s primary benefit lies in reducing the number of trainable parameters and the GPU memory required during fine-tuning, rather than eliminating the computational overhead of the adapters themselves. They explicitly note that the extra computation introduced by the low-rank matrices cannot be bypassed - it can only be minimized by constraining the adapter rank. Critically, the full pre-trained weights must still be loaded at deployment time, meaning LoRA does not reduce the model’s footprint during inference so much as it adds a smaller number of parameters on top of it. Their reported efficiency gains are concentrated in memory storage and I/O, rather than raw compute: the only training-time improvement they document is a 25% reduction for GPT-3 Brown, Mann, Ryder, Subbiah, Kaplan, Dhariwal, Neelakantan, Shyam, Sastry, Askell et al. [12], and no FLOPs figures are provided for LoRA itself, beyond the general recommendation to keep the rank small in order to limit additional compute.

This is further corroborated by Ko [58], who show that LoRA does not universally accelerate training across all model architectures and training configurations. In particular, they demonstrate that LoRA can be slower than full fine-tuning on an NVIDIA A100 GPU¹⁵, with the effect depending on batch size and sequence length. They attribute this slowdown to how computation is scheduled on the GPU: since GPUs are optimized to execute a single kernel at a time, the additional low-rank adapter layers introduced by LoRA must be processed sequentially rather than fused into the base computation, creating bottleneck that can offset the training efficiency gains LoRA is otherwise expected to provide.

Cherniuk, Mikhalev and Oseledets (2025) similarly observe that existing LoRA implementations rely on a fixed computation graph regardless of the underlying layer configuration, making training unnecessarily inefficient. They note that computational cost has been largely overlooked in the low-rank adaptation literature, with prior work focusing almost exclusively on parameter efficiency. To address this, they propose selecting the optimal forward and backward computation graph based on the linear layer’s dimensions, the input dimensionality, and the LoRA rank, guided by FLOPs and runtime estimates; achieving faster training without sacrificing accuracy.

Finally for the LoRA ensemble Halbheер et al. (2024), the authors have also mentioned that their approach achieved the same training time compared to the Explicit ensemble. Hence, Our efficiency results are therefore consistent with findings in the literature that LoRA’s advantage is parameter, storage efficiency, and I/O efficiency, not a clear reduction in training-time computation.

Statistical Significance and Results Discussion

We have compared the average performance of ST-LoRA and FRE across multiple perspectives. To strengthen our findings, we additionally perform paired statistical significance tests comparing both approaches seed-to-seed. We report the paired $t$-test Student [95], a bootstrap confidence interval (CI) Efron [30], the Wilcoxon signed-rank test Wilcoxon [106] as a non-parametric alternative, and Cohen’s $d_z$ as a paired effect size measure Cohen (1985)¹⁶. We use 5 seeds to represent the number of samples (4 degrees of freedom for $t$-student) and two-tailed for $t$-test.

As shown in Table 5, the efficiency gains favour ST-LoRA for Mask2Former, are on par for SegFormer-B2, and favour FRE for EoMT. Overall, ST-LoRA consistently reduces the number of fine-tuned parameters, inference time, and storage requirements; however, for training time, energy, and emissions, further analysis is needed to investigate the relationship between the proportion of fine-tuned parameters relative to the total model size and the role of the frozen base model in driving these costs.

Table 6 shows a similarly mixed picture: ST-LoRA is broadly on par with FRE for calibration (ECE), while for mIoU it excels between outperforming FRE (SegFormer-B2), matching it (Mask2Former), and underperforming it (EoMT). Hence, our approach emerges as a strong competitor to the FRE approach.

Arch	Metric	Dir	FRE	ST-LoRA	%$\Delta$	95% CI	$p$ ($t/W$)	$d_z$
Mask2Former	mIoU	$\uparrow$	0.6882 $\pm$ 0.02	0.6731 $\pm$ 0.0072	−2.2%	[−0.0293, −0.0009]	0.144/0.188	−0.81
Mask2Former	ECE	$\downarrow$	0.00316 $\pm$ 0.0012	0.003349 $\pm$ 0.00035	+6.0%	[−0.0006, +0.0010]	0.708/1.000	+0.18
SegFormer	mIoU	$\uparrow$	0.6717 $\pm$ 0.004	0.6784 $\pm$ 0.002	+1.0%	[+0.0029, +0.0097]	0.025/0.125	+1.57
SegFormer	ECE	$\downarrow$	0.004362 $\pm$ 0.00015	0.003005 $\pm$ 0.00045	−31.1%	[−0.0018, −0.0010]	0.003/0.062	−2.95
EoMT	mIoU	$\uparrow$	0.6873 $\pm$ 0.0025	0.6753 $\pm$ 0.0024	−1.7%	[−0.0146, −0.0098]	0.001/0.062	−3.86
EoMT	ECE	$\downarrow$	0.003285 $\pm$ 0.0012	0.003195 $\pm$ 0.00067	−2.7%	[−0.0009, +0.0006]	0.842/1.000	−0.10

While ensembling enhances calibration, it comes at the cost of reduced segmentation accuracy. Both effects stem from the same underlying mechanism, which is directly visible in the reliability diagrams of Figure 5. All single models exhibit overconfident predicted probabilities consistently exceed empirical accuracy because individual models tend to concentrate predictions in high-confidence bins regardless of whether those predictions are correct Guo et al. [41]. Ensemble averaging softens this behavior; since the averaged probability across members is almost always lower than any individual member’s maximum, predictions are redistributed from overconfident high-probability bins into more moderate bins where confidence and accuracy are better aligned. This redistribution moves the reliability curve upward toward the perfectly calibrated diagonal, reducing ECE. The same softening, however, occasionally overrides confident correct predictions, causing the marginal accuracy drop observed in mIoU.

We attribute the limited magnitude of both effects to insufficient ensemble diversity as members share architecture, initialization, and augmentation strategy, with only learning rate scheduling as a source of variation. This restricts inter-member disagreement, limiting both the calibration gains and the accuracy cost¹⁷. As ST-LoRA’s rank constraint implicitly restricts optimization to a low-dimensional subspace where good solutions are more densely concentrated, which is also supported by Halbheer et al. [42]. This subspace retains sufficient capacity. Within it, learning rate variation may be enough to drive members toward solutions that are both competitive and complementary. In contrast, full-rank optimization spans a much larger space where the same perturbation may have proportionally less influence on the training trajectory, potentially causing members to converge to similar solutions. We leave empirical/theoretical validation of this hypothesis to future work.

As conclusion:

Smaller Everywhere, Faster Only Sometimes

Overall, ST-LoRA emerges as a strong competitor to FRE, consistently reducing storage requirements, the number of trainable parameters, and I/O bottlenecks across all three architectures. However, this efficiency advantage does not extend uniformly to training time: ST-LoRA’s advantages over FRE are architecture-dependent, clearly outperforming on Mask2Former, performing on par on SegFormer-B2, and underperforming on EoMT. This indicates that LoRA hyperparameters and the target architecture jointly determine training efficiency, and that LoRA cannot guarantee faster training by design; identifying which architectural properties predict this behaviour is an important direction for future work.

Having established that ST-LoRA is a viable and efficient alternative to full-rank ensemble, a natural question follows: is the baseline configuration optimal, or can targeted hyperparameter choices unlock further gains? We investigate this in the next Experiment.

Experiment 2: Ablation Study of ST-LoRA’s Hyperparameters

We conduct an extensive ablation study to identify the hyperparameters that most critically govern performance, stability, and parameter efficiency under ST-LoRA-based ensemble fine-tuning. Given the architectural heterogeneity of our benchmark SgF-B2 on GrowliFlower-L versus M2F-Swin/B on BUP20, we treat each dataset/architecture pair independently, as hyperparameter sensitivity may differ substantially across them¹⁸. Our analysis focuses on three factors: rank $r$, target components, and scaling factor $\alpha$, which together govern the capacity, scope, and magnitude of the low-rank adaptation¹⁹. Each experiment varies a single factor while holding all remaining settings fixed at the baseline configuration as mentioned in Table 24, allowing the effect of each choice to be isolated cleanly. The baseline is deliberately chosen as a reproducible standard reference rather than a pre-optimized configuration, so that improvements over it reflect genuine sensitivity rather than tuning artifacts. Interactions between hyperparameters are not explored, as exhaustive combinatorial search falls outside the scope of this study.

Setting	SegFormer-B2	Mask2Former-Swin/B
Parameter Groups
ST-LoRA-adapted	$q_e$, $k_e$, $v_e$, MLP (encoder)	$q_d$, $k_d$, CLS, MLP (encoder)
Fully fine-tuned	Decoder + segmentation head	—
Frozen	Remaining encoder	Remaining encoder parameters and pixel decoder
ST-LoRA Configuration
Rank $r$	32	8
Scaling $\alpha$	64	8
$A$ init	Random	Random
$B$ init	Zero	Zero
Bias	No	No
Training
Optimizer	Adam	Adam
Learning rate	$2 \times10^{-2}$–$2 \times10^{-4}$	$2 \times10^{-3}$–$2 \times10^{-5}$
Batch size	32	4
Epochs	100	110
LR schedule	CosineAnnealingWarmRestarts ($T_0 = 20$, $T_{\mathrm{mult}} = 1$)
Ensemble size	5 checkpoints
Evaluation
Seeds	5
Reported stats	Mean $\pm$ std

LoRA Rank ($r$)

Across both architectures, we observe a consistent trend: increasing the LoRA rank improves performance in terms of both segmentation accuracy and calibration as shown in Table 19 and Table 20. For SegFormer-B2, rank $r = 32$ emerges as a sweet spot between parameter efficiency and robust performance; higher ranks yield marginal further gains at the cost of a proportionally larger number of trainable parameters. Mask2Former-Swin/B exhibits a qualitatively similar trend but with a notably different sensitivity profile even at rank $r = 2$, the model achieves reasonable segmentation performance with mIoU reaching 65.8%, suggesting that a low-dimensional subspace already captures much of the task-relevant variation. We hypothesize that this difference is not solely attributable to rank, but is also shaped by the substantially different adaptation strategies employed across the two models. SegFormer-B2 applies LoRA to both the self-attention projections and the position-wise feed-forward layers of the encoder, while the decoder is fully fine-tuned. In contrast, Mask2Former-Swin/B forgoes full fine-tuning entirely: the encoder feed-forward layers are adapted via LoRA, and the decoder is handled through low-rank injection into the query and key projections along with the class predictor. This contrasts full decoder fine-tuning versus LoRA-only decoder adaptation introduces a confounding factor beyond rank alone, and may explain why the two models exhibit different calibration sensitivity profiles as rank increases. We next investigate the effect of the targeted components directly.

Category	Setting	mIoU $\uparrow$	FNR $\downarrow$	ECE $\downarrow$	MECE $\downarrow$	Trainable Parameters (%) $\downarrow$
LoRA rank - $r$	2	0.3151 $\pm$ 0.3863	0.6574 $\pm$ 0.4205	0.0646 $\pm$ 0.0303	0.1367 $\pm$ 0.0863	4.64
	4	0.5553 $\pm$ 0.3191	0.4009 $\pm$ 0.3532	0.1021 $\pm$ 0.0338	0.2905 $\pm$ 0.1190	5.11
	8	0.7285 $\pm$ 0.1081	0.2151 $\pm$ 0.1460	0.0744 $\pm$ 0.0688	0.2864 $\pm$ 0.1483	6.03
	16	0.6277 $\pm$ 0.3154	0.3257 $\pm$ 0.3409	0.0529 $\pm$ 0.0350	0.1748 $\pm$ 0.0640	7.81
	32 (baseline)	0.8178 $\pm$ 0.0295	0.1093 $\pm$ 0.0410	0.0126 $\pm$ 0.0166	0.1367 $\pm$ 0.0198	11.19
	64	0.8134 $\pm$ 0.0194	0.0973 $\pm$ 0.0442	0.0329 $\pm$ 0.0338	0.2165 $\pm$ 0.0835	17.26
	128	0.8231 $\pm$ 0.0110	0.0879 $\pm$ 0.0180	0.0062 $\pm$ 0.0041	0.1607 $\pm$ 0.0750	27.20
Components	$q$, $k_v$, $f_{c1}$, $f_{c2}$ (baseline)	0.8178 $\pm$ 0.0295	0.1093 $\pm$ 0.0410	0.0126 $\pm$ 0.0166	0.1367 $\pm$ 0.0198	11.19
	$q$, $f_{c1}$, $f_{c2}$	0.8469 $\pm$ 0.0021	0.0726 $\pm$ 0.0039	0.0041 $\pm$ 0.0008	0.1561 $\pm$ 0.0348	9.87
	$q$, $k_v$, $f_{c1}$	0.8429 $\pm$ 0.0095	0.0719 $\pm$ 0.0097	0.0038 $\pm$ 0.0018	0.1280 $\pm$ 0.0601	8.97
	$q$, $k_v$, $f_{c2}$	0.7505 $\pm$ 0.1439	0.1810 $\pm$ 0.1765	0.0358 $\pm$ 0.0604	0.1916 $\pm$ 0.1460	8.97
	$q$, $k_v$	0.8158 $\pm$ 0.0273	0.1300 $\pm$ 0.0394	0.0037 $\pm$ 0.0013	0.0999 $\pm$ 0.0290	6.63
	$f_{c1}$, $f_{c2}$	0.8509 $\pm$ 0.0023	0.0718 $\pm$ 0.0045	0.0034 $\pm$ 0.0003	0.1231 $\pm$ 0.0124	8.97
	$k_v$	0.8311 $\pm$ 0.0128	0.1064 $\pm$ 0.0145	0.0025 $\pm$ 0.0018	0.0705 $\pm$ 0.0430	8.97
	$q$	0.8551 $\pm$ 0.0012	0.0557 $\pm$ 0.0016	0.0035 $\pm$ 0.0002	0.1163 $\pm$ 0.0057	8.97
	$f_{c2}$	0.8498 $\pm$ 0.0046	0.0715 $\pm$ 0.0053	0.0035 $\pm$ 0.0002	0.1367 $\pm$ 0.0075	6.63
	$f_{c1}$	0.8613 $\pm$ 0.0018	0.0513 $\pm$ 0.0021	0.0042 $\pm$ 0.0003	0.1686 $\pm$ 0.0079	6.63
LoRA scaling factor - $\alpha$	8	0.8622 $\pm$ 0.0080	0.0498 $\pm$ 0.0109	0.0132 $\pm$ 0.0173	0.1719 $\pm$ 0.0334	11.19
	16	0.8531 $\pm$ 0.0042	0.0551 $\pm$ 0.0034	0.0047 $\pm$ 0.0007	0.1760 $\pm$ 0.0190	11.19
	32	0.8290 $\pm$ 0.0163	0.0709 $\pm$ 0.0104	0.0091 $\pm$ 0.0085	0.2005 $\pm$ 0.0804	11.19
	64 (baseline)	0.8178 $\pm$ 0.0295	0.1093 $\pm$ 0.0410	0.0126 $\pm$ 0.0166	0.1367 $\pm$ 0.0198	11.19
	128	0.4645 $\pm$ 0.3796	0.5072 $\pm$ 0.4031	0.0492 $\pm$ 0.0379	0.1184 $\pm$ 0.1015	11.19
Initializer	EVA	0.7795 $\pm$ 0.0455	0.1557 $\pm$ 0.0580	0.0130 $\pm$ 0.0164	0.1256 $\pm$ 0.0574	11.19
	OLoRA	0.7872 $\pm$ 0.0850	0.1244 $\pm$ 0.1202	0.0261 $\pm$ 0.0350	0.2331 $\pm$ 0.0929	11.19
	PISSA	0.8097 $\pm$ 0.0172	0.1224 $\pm$ 0.0291	0.0299 $\pm$ 0.0353	0.1527 $\pm$ 0.0888	11.19
	None (baseline)	0.8178 $\pm$ 0.0295	0.1093 $\pm$ 0.0410	0.0126 $\pm$ 0.0166	0.1367 $\pm$ 0.0198	11.19
Bias Setting	All	0.4348 $\pm$ 0.3574	0.5515 $\pm$ 0.3690	0.0703 $\pm$ 0.0377	0.2537 $\pm$ 0.0958	11.45
	LoRA-only	0.8103 $\pm$ 0.0232	0.1143 $\pm$ 0.0456	0.0142 $\pm$ 0.0160	0.1599 $\pm$ 0.0479	11.31
	None (baseline)	0.8178 $\pm$ 0.0295	0.1093 $\pm$ 0.0410	0.0126 $\pm$ 0.0166	0.1367 $\pm$ 0.0198	11.19
Batch Size	8	0.8254 $\pm$ 0.0098	0.0899 $\pm$ 0.0162	0.0050 $\pm$ 0.0015	0.1405 $\pm$ 0.0518	11.19
	16	0.8106 $\pm$ 0.0212	0.1124 $\pm$ 0.0489	0.0408 $\pm$ 0.0421	0.2222 $\pm$ 0.0916	11.19
	32 (baseline)	0.8178 $\pm$ 0.0295	0.1093 $\pm$ 0.0410	0.0126 $\pm$ 0.0166	0.1367 $\pm$ 0.0198	11.19
Learning Rate	$2 \times10^{-1}$	0.0000 $\pm$ 0.0000	1.0000 $\pm$ 0.0000	0.0820 $\pm$ 0.0148	0.0820 $\pm$ 0.0148	11.19
	$2 \times10^{-2}$ (baseline)	0.8178 $\pm$ 0.0295	0.1093 $\pm$ 0.0410	0.0126 $\pm$ 0.0166	0.1367 $\pm$ 0.0198	11.19
	$2 \times10^{-3}$	0.8860 $\pm$ 0.0018	0.0254 $\pm$ 0.0013	0.0050 $\pm$ 0.0003	0.2876 $\pm$ 0.0138	11.19
	$2 \times10^{-4}$	0.8802 $\pm$ 0.0023	0.0274 $\pm$ 0.0049	0.0057 $\pm$ 0.0005	0.2590 $\pm$ 0.0427	11.19
$\eta_{\min}$	$2 \times10^{-3}$	0.8135 $\pm$ 0.0021	0.1011 $\pm$ 0.0161	0.0248 $\pm$ 0.0188	0.2021 $\pm$ 0.0764	11.19
	$2 \times10^{-4}$ (baseline)	0.8178 $\pm$ 0.0295	0.1093 $\pm$ 0.0410	0.0126 $\pm$ 0.0166	0.1367 $\pm$ 0.0198	11.19
	$2 \times10^{-5}$	0.8096 $\pm$ 0.0212	0.1132 $\pm$ 0.0438	0.0310 $\pm$ 0.0330	0.2046 $\pm$ 0.0881	11.19
	$2 \times10^{-6}$	0.8148 $\pm$ 0.0304	0.0983 $\pm$ 0.0543	0.0240 $\pm$ 0.0341	0.2168 $\pm$ 0.0735	11.19

Category	Setting	mIoU $\uparrow$	ECE $\downarrow$	MECE $\downarrow$	ACE $\downarrow$	MACE $\downarrow$
ST-LoRA rank - $r$	2	0.6582 $\pm$ 0.0170	0.0143 $\pm$ 0.0058	0.1507 $\pm$ 0.0410	0.0646 $\pm$ 0.0331	0.0961 $\pm$ 0.0377
	4	0.6734 $\pm$ 0.0223	0.0089 $\pm$ 0.0049	0.0833 $\pm$ 0.0361	0.0306 $\pm$ 0.0222	0.0510 $\pm$ 0.0246
	8 (baseline)	0.6809 $\pm$ 0.0171	0.0028 $\pm$ 0.0017	0.0939 $\pm$ 0.0612	0.0041 $\pm$ 0.0020	0.0092 $\pm$ 0.0060
	16	0.6791 $\pm$ 0.0292	0.0036 $\pm$ 0.0019	0.1672 $\pm$ 0.0320	0.0024 $\pm$ 0.0009	0.0091 $\pm$ 0.0059
	32	0.6801 $\pm$ 0.0239	0.0030 $\pm$ 0.0003	0.0798 $\pm$ 0.0653	0.0018 $\pm$ 0.0008	0.0061 $\pm$ 0.0094
	64	0.6942 $\pm$ 0.0122	0.0045 $\pm$ 0.0039	0.1053 $\pm$ 0.0799	0.0019 $\pm$ 0.0025	0.0099 $\pm$ 0.0155
ST-LoRA scaling - $\alpha$	2	0.6763 $\pm$ 0.0097	0.0161 $\pm$ 0.0043	0.2020 $\pm$ 0.0945	0.0505 $\pm$ 0.0202	0.0759 $\pm$ 0.0260
	4	0.6773 $\pm$ 0.0230	0.0062 $\pm$ 0.0015	0.0678 $\pm$ 0.0147	0.0137 $\pm$ 0.0045	0.0245 $\pm$ 0.0060
	8 (baseline)	0.6809 $\pm$ 0.0171	0.0028 $\pm$ 0.0017	0.0939 $\pm$ 0.0612	0.0041 $\pm$ 0.0020	0.0092 $\pm$ 0.0060
	16	0.6831 $\pm$ 0.0140	0.0026 $\pm$ 0.0006	0.1033 $\pm$ 0.0968	0.0026 $\pm$ 0.0006	0.0074 $\pm$ 0.0044
	32	0.6766 $\pm$ 0.0145	0.0030 $\pm$ 0.0020	0.0746 $\pm$ 0.0402	0.0022 $\pm$ 0.0005	0.0087 $\pm$ 0.0029
ST-LoRA dropout	0.10 (baseline)	0.6809 $\pm$ 0.0171	0.0028 $\pm$ 0.0017	0.0939 $\pm$ 0.0612	0.0041 $\pm$ 0.0020	0.0092 $\pm$ 0.0060
	0.15	0.6734 $\pm$ 0.0165	0.0037 $\pm$ 0.0020	0.1277 $\pm$ 0.0551	0.0048 $\pm$ 0.0010	0.0094 $\pm$ 0.0039
	0.20	0.6908 $\pm$ 0.0153	0.0044 $\pm$ 0.0019	0.0752 $\pm$ 0.0342	0.0047 $\pm$ 0.0022	0.0098 $\pm$ 0.0060
	0.25	0.6825 $\pm$ 0.0209	0.0035 $\pm$ 0.0024	0.0711 $\pm$ 0.0301	0.0053 $\pm$ 0.0020	0.0105 $\pm$ 0.0054
	0.30	0.6752 $\pm$ 0.0144	0.0035 $\pm$ 0.0017	0.0622 $\pm$ 0.0428	0.0041 $\pm$ 0.0024	0.0081 $\pm$ 0.0071
Components	$q_d$, $k_d$, cls, dense$_e$, $q_e$	0.6870 $\pm$ 0.0199	0.0030 $\pm$ 0.0013	0.0665 $\pm$ 0.0285	0.0042 $\pm$ 0.0018	0.0093 $\pm$ 0.0040
	$q_d$, $k_d$, cls, dense$_e$, $k_e$	0.6976 $\pm$ 0.0099	0.0038 $\pm$ 0.0013	0.0796 $\pm$ 0.0518	0.0049 $\pm$ 0.0031	0.0098 $\pm$ 0.0051
	$q_d$, $k_d$, cls, dense$_e$, $q_e$, $k_e$	0.6599 $\pm$ 0.0248	0.0030 $\pm$ 0.0011	0.0574 $\pm$ 0.0408	0.0040 $\pm$ 0.0019	0.0090 $\pm$ 0.0039
	$q_d$, $k_d$, cls, dense$_e$, $q_e$	0.6760 $\pm$ 0.0152	0.0027 $\pm$ 0.0013	0.0704 $\pm$ 0.0424	0.0032 $\pm$ 0.0010	0.0075 $\pm$ 0.0022
	$q_d$, $k_d$, cls, dense$_e$, $k_e$, $v_e$	0.6828 $\pm$ 0.0154	0.0041 $\pm$ 0.0026	0.0699 $\pm$ 0.0473	0.0043 $\pm$ 0.0017	0.0115 $\pm$ 0.0072
	$q_d$, $k_d$, cls, dense$_e$, $q_e$, $k_e$, $v_e$	0.6972 $\pm$ 0.0198	0.0040 $\pm$ 0.0012	0.0825 $\pm$ 0.0231	0.0036 $\pm$ 0.0015	0.0076 $\pm$ 0.0037
	$q_d$, $k_d$, cls, dense$_e$, $q_e$, $k_e$, $v_e$	0.6767 $\pm$ 0.0313	0.0032 $\pm$ 0.0018	0.0990 $\pm$ 0.0296	0.0040 $\pm$ 0.0019	0.0098 $\pm$ 0.0049

LoRA Target Components

We investigate whether adapting the decoder alone is sufficient for segmentation. Keeping the encoder completely frozen, we inject LoRA modules exclusively into the transformer decoder of Mask2Former-Swin/B and vary the rank. Across all rank values, no reasonable performance is observed and the overall results are substantially worse than those obtained when the encoder is also adapted as shown in Table 21. This is expected: the encoder is responsible for extracting task-relevant feature representations, and without adapting it to the target domain, the decoder operates on generic pre-trained embeddings that are ill-suited for fine-grained plant segmentation. Under these conditions, the decoder merely attempts to upsample frozen representations into a segmentation mask, offering little capacity to recover the necessary discriminative structure.

Category	Setting	mIoU ↑	ECE ↓	MECE ↓	ACE ↓	MACE ↓
ST-LoRA rank – $r$	2	0.2270 ± 0.0140	0.0720 ± 0.0253	0.3203 ± 0.0357	0.1619 ± 0.0347	0.2489 ± 0.0230
	4	0.2528 ± 0.0070	0.0480 ± 0.0043	0.2466 ± 0.0247	0.0933 ± 0.0199	0.1819 ± 0.0162
	8	0.2831 ± 0.0068	0.0487 ± 0.0014	0.2032 ± 0.0155	0.0533 ± 0.0160	0.1531 ± 0.0169
	16	0.3002 ± 0.0107	0.0467 ± 0.0038	0.1782 ± 0.0076	0.0386 ± 0.0096	0.1352 ± 0.0095
	32	0.3192 ± 0.0103	0.0445 ± 0.0035	0.1676 ± 0.0129	0.0363 ± 0.0066	0.1270 ± 0.0107
	64	0.3203 ± 0.0181	0.0478 ± 0.0040	0.1824 ± 0.0161	0.0316 ± 0.0093	0.1275 ± 0.0157
Components	$q_d$, $k_d$, cls, $q_e$	0.5818 ± 0.0073	0.0148 ± 0.0050	0.2164 ± 0.0900	0.0109 ± 0.0026	0.0416 ± 0.0126
	$q_d$, $k_d$, cls, $k_e$	0.5594 ± 0.0256	0.0141 ± 0.0014	0.2528 ± 0.3065	0.0114 ± 0.0055	0.0407 ± 0.0076
	$q_d$, $k_d$, cls, $v_e$	0.6310 ± 0.0235	0.0127 ± 0.0021	0.1133 ± 0.0115	0.0094 ± 0.0016	0.0383 ± 0.0033
	$q_d$, $k_d$, cls, dense$_e$	0.6839 ± 0.0174	0.0032 ± 0.0012	0.0619 ± 0.0258	0.0032 ± 0.0008	0.0057 ± 0.0010
	$q_d$, $k_d$, cls, $q_e$, $k_e$	0.5982 ± 0.0086	0.0095 ± 0.0037	0.1035 ± 0.0154	0.0094 ± 0.0059	0.0271 ± 0.0133
	$q_d$, $k_d$, cls, $q_e$, $v_e$	0.6692 ± 0.0153	0.0102 ± 0.0035	0.1146 ± 0.0488	0.0088 ± 0.0048	0.0295 ± 0.0091
	$q_d$, $k_d$, cls, $q_e$, dense$_e$	0.6833 ± 0.0199	0.0035 ± 0.0014	0.0851 ± 0.0573	0.0043 ± 0.0018	0.0092 ± 0.0038
	$q_d$, $k_d$, cls, $k_e$, $v_e$	0.6647 ± 0.0142	0.0088 ± 0.0018	0.0921 ± 0.0135	0.0096 ± 0.0056	0.0258 ± 0.0059
	$q_d$, $k_d$, cls, $k_e$, dense$_e$	0.6873 ± 0.0114	0.0033 ± 0.0012	0.0920 ± 0.0729	0.0048 ± 0.0018	0.0091 ± 0.0041
	$q_d$, $k_d$, cls, $v_e$, dense$_e$	0.6833 ± 0.0248	0.0042 ± 0.0011	0.0774 ± 0.0610	0.0047 ± 0.0020	0.0106 ± 0.0036
	$q_d$, $k_d$, cls, $q_e$, $k_e$, $v_e$	0.6704 ± 0.0237	0.0073 ± 0.0022	0.1206 ± 0.0461	0.0072 ± 0.0022	0.0213 ± 0.0077
	$q_d$, $k_d$, cls, $q_e$, $k_e$, dense$_e$	0.6927 ± 0.0308	0.0036 ± 0.0006	0.0943 ± 0.0284	0.0043 ± 0.0009	0.0085 ± 0.0030
	$q_d$, $k_d$, cls, $q_e$, $v_e$, dense$_e$	0.6844 ± 0.0209	0.0041 ± 0.0013	0.0871 ± 0.0478	0.0036 ± 0.0024	0.0075 ± 0.0043
	$q_d$, $k_d$, cls, $k_e$, $v_e$, dense$_e$	0.6867 ± 0.0214	0.0043 ± 0.0015	0.1151 ± 0.0459	0.0037 ± 0.0010	0.0093 ± 0.0019
	$q_d$, $k_d$, cls, $q_e$, $k_e$, $v_e$, dense$_e$	0.6859 ± 0.0175	0.0030 ± 0.0010	0.0741 ± 0.0430	0.0042 ± 0.0010	0.0076 ± 0.0021

We then ask the complementary question: can the decoder be frozen entirely while adapting only the encoder? For Mask2Former-Swin/B, we freeze the decoder completely and evaluate all component configurations previously tested with decoder adaptation. The results reveal a consistent asymmetry: while a subset of configurations yields marginal mIoU improvements, nearly all exhibit severe calibration degradation across ECE, ACE, and MACE as illustrated in Table 22. This is attributable to the frozen decoder applying a fixed mapping optimized for the source domain, which is misaligned with the target domain’s feature distribution regardless of how well the encoder is adapted. The decoder therefore acts as a calibration bottleneck; accurate enough to segment, but insufficiently adapted to produce reliable confidence estimates. These findings motivate the hybrid strategy adopted in our final configuration: LoRA-only adaptation of selected encoder components, with the decoder adapted via LoRA rather than frozen or fully fine-tuned.

Category	Setting	Δ mIoU (%)	Δ ECE (%)	Δ MECE (%)	Δ ACE (%)	Δ MACE (%)
Rank – $r$	2	−1.7	+198.6	+97.3	+107.6	+86.4
	4	−0.6	+21.3	+44.8	+5.6	−0.6
	8 (baseline)	+1.0	+42.9	+12.7	+82.9	+42.4
	16	+0.6	−27.8	−16.0	+62.5	−35.2
	32	+0.2	−23.3	−11.4	+77.8	−25.5
	64	−1.9	+46.7	−14.5	+100.0	+81.8
Scaling – $\alpha$	2	−3.3	+16.1	+75.2	+65.9	+48.7
	4	−2.5	+27.4	+78.3	+60.6	+51.0
	8 (baseline)	+1.0	+42.9	+12.7	+82.9	+42.4
	16	−1.6	−7.7	−10.1	+30.8	+2.7
	32	+0.2	−6.7	+22.7	+118.2	+43.7
Dropout – $p$	0.10 (baseline)	+1.0	+42.9	+12.7	+82.9	+42.4
	0.15	+0.1	+35.1	−52.2	+112.5	+115.9
	0.20	−1.2	+13.6	+9.8	+100.0	+60.2
	0.25	−0.7	+42.9	+53.3	+84.9	+63.8
	0.30	+0.3	+37.1	−9.3	+107.3	+83.9
Components	dns, cls (baseline)	+1.0	+42.9	+12.7	+82.9	+42.4
	dns, cls, $q_e$	−1.8	+53.3	+46.8	+123.8	+76.3
	dns, cls, $k_e$	−0.6	+42.1	+32.9	+114.3	+83.7
	dns, cls, $v_e$	+1.6	+20.0	+38.3	+95.0	+60.0
	dns, cls, $q_e$, $k_e$	+2.5	+63.0	+25.7	+165.6	+96.0
	dns, cls, $q_e$, $v_e$	−1.9	−17.1	+10.9	+69.8	−12.2
	dns, cls, $k_e$, $v_e$	+0.1	+60.0	+8.0	+130.6	+117.1
	dns, cls, $q_e$, $k_e$, $v_e$	+1.0	+18.8	−15.7	+87.5	+38.8

We then follow the conventional LoRA adaptation strategy of targeting only the self-attention projections Hu et al. (2022), injecting low-rank modules into $Q$, $K$, and $V$. Contrary to the gains reported in LLM settings, this configuration yields worse performance compared to the respective baselines for both architectures, as reflected by both mIoU and ECE. This suggests that attention-only adaptation, while effective for generic image classification and language tasks [75, 8], is insufficient for dense prediction in agricultural imagery, where spatial detail and pixel-level calibration impose stricter representational demands on the adapted layers.

We now turn to an intensive component ablation, examining how each encoder component contributes to segmentation quality and uncertainty calibration. Our analysis focuses on two families of components: the self-attention projections and the position-wise Multi-Layer Perception (MLP) layers, as these constitute the two primary sites of learnable computation within a transformer encoder block.

Adapting the query projection alone yields mixed results: for Mask2Former-Swin/B, performance remains comparable to the baseline despite the added parameters, whereas for SegFormer-B2 it surpasses both the baseline and the conventional $(q,k,v)$ configuration. Extending adaptation to the key and value projections jointly improves over query-only across both architectures, again outperforming the baseline and the conventional attention-only setting. However, the most pronounced gains are observed when the MLP layers are included. Across both models and all component combinations tested, incorporating MLP layers consistently yields the strongest improvements in both mIoU and calibration metrics while having the smaller number of parameters, indicating that these layers carry the dominant share of task-relevant adaptability in the context of agricultural semantic segmentation.

This observation is further supported by a converging body of evidence from both the text and vision literature. In the text domain, Dong, Cordonnier and Loukas [26] showed that transformer networks without skip connections or MLP layers converge doubly exponentially to a rank-1 matrix, rendering them unable to learn, establishing MLP layers as structurally necessary rather than merely beneficial. Geva, Schuster, Berant and Levy [37] further demonstrated that MLP layers act as key-value memories that store and retrieve factual knowledge, with deeper layers encoding increasingly rich semantic content. From an adaptation perspective, He, Zhou, Ma, Berg-Kirkpatrick and Neubig found that under a restricted parameter budget, MLP layers yield better adaptation than attention projections, and Zhang, Chen, Bukharin, He, Cheng, Chen and Zhao confirmed through extensive experimentation that MLP layers consistently outperform attention at all ranks, matching full fine-tuning at higher ranks.

In the vision domain, the evidence is equally compelling. Melas-Kyriazi [70] constructed a vision transformer using only MLP layers and achieved performance comparable to the full model, while the attention-only counterpart degraded severely. Zhang, Liu, Lin and He [113] showed that attention is not needed at every layer, with selective removal reducing computational cost while improving performance. Wang, Zhao, Tang, Luo and Zeng [101] replaced self-attention entirely with a mean-shift operation and obtained competitive results, and Venkataramanan, Ghodrati, Asano, Porikli and Habibian demonstrated that skipping attention modules via skip connections incurs minimal performance loss. Complementing this, Zhang, Lin, Liu, Li, Sun and Zhou [114] revealed that MLP layers exhibit a Mixture-Of-Experts (MOE) behaviour, with only a sparse subset of neurons activating per input, suggesting that their representational capacity is both rich and highly structured.

To verify empirically and align with the theoretical and empirical findings in the literature, we train all three architectures using two configurations: adapting the feed-forward layers only, and adapting both the feed-forward layers and the attention mechanism jointly.²⁰ As shown in Table 7, removing attention adaptation leaves Mask2Former largely unaffected, with only marginal and non-significant changes in mIoU and ECE, while computational efficiency remains essentially unchanged while having a lower number of parameters. For SegFormer-B2, we observe a small, non-significant drop in mIoU; however, FFN-only adaptation brings significant improvements in both ECE and computational efficiency, leading to reduced GPU energy, CO$_2$ emissions, and wall-clock time.

Arch	Metric	Mean $d$	95% CI	$p$ ($t/W$)	$d_z$
SegFormer	mIoU $\uparrow$	+0.0010	[$-0.0019,+0.0047$]	0.614/0.625	+0.24
SegFormer	ECE $\downarrow$	$-0.0072$	[$-0.0118,-0.0017$]	0.065/0.125	$-1.13$
SegFormer	GPU energy (kWh) $\downarrow$	$-0.0022$	[$-0.0030,-0.0014$]	0.010/0.062	$-2.04$
SegFormer	CO$_2$e (kg) $\downarrow$	$-0.0013$	[$-0.0017,-0.0008$]	0.009/0.062	$-2.14$
SegFormer	Wall-clock (min) $\downarrow$	$-0.7841$	[$-1.1030,-0.4652$]	0.014/0.062	$-1.86$
EoMT	mIoU $\uparrow$	$-0.0030$	[$-0.0047,-0.0015$]	0.032/0.062	$-1.44$
EoMT	ECE $\downarrow$	+0.0007	[$-0.0003,+0.0020$]	0.354/0.438	+0.47
EoMT	GPU energy (kWh) $\downarrow$	$-0.0054$	[$-0.0087,-0.0022$]	0.043/0.062	$-1.31$
EoMT	CO$_2$e (kg) $\downarrow$	$-0.0009$	[$-0.0032,+0.0014$]	0.533/0.625	$-0.30$
EoMT	Wall-clock (min) $\downarrow$	$-1.2636$	[$-3.0206,+0.2225$]	0.241/0.625	$-0.62$
Mask2Former	mIoU $\uparrow$	$-0.0036$	[$-0.0111,+0.0044$]	0.471/0.438	$-0.36$
Mask2Former	ECE $\downarrow$	+0.0005	[$-0.0001,+0.0011$]	0.252/0.312	+0.60
Mask2Former	GPU energy (kWh) $\downarrow$	$-0.0007$	[$-0.0077,+0.0076$]	0.871/0.625	$-0.08$
Mask2Former	CO$_2$e (kg) $\downarrow$	$-0.0056$	[$-0.0135,+0.0007$]	0.252/0.312	$-0.60$

EoMT, however, stands out as the exception. FFN-only adaptation results in a significant drop in mIoU, no meaningful change in ECE, and notably, a significant reduction in GPU energy despite comparable wall-clock time. This indicates that adaptation feed forward layers only can be beneficial for computational efficiency while affecting the predictive power slightly.

Taken together, this points to the same conclusion: MLP layers are the dominant site of task-relevant computation and adaptation in transformer architectures, in both text and vision settings. Our ablation results and significance align with and extend this finding to the dense agricultural prediction regime, where attention-only adaptation consistently underperforms MLP-inclusive configurations across the three architectures studied with more computational power needed.

LoRA Scaling Factor ($\alpha$)

Having established rank and target components as the two most influential hyperparameters, we turn to the scaling factor $\alpha$. Recalling the LoRA adaptation equation but with the previously omitted $\delta_r=\alpha/r$ for simplicity,

The effective magnitude of the low-rank update is governed by $\delta_r=\alpha/r$, the ratio of the scaling factor to the rank. Rather than treating $\alpha$ in isolation, this formulation makes clear that it is $\delta_r$ that controls how strongly the adapter output is blended into the pre-trained representation. We therefore ablate across $\alpha$ values at fixed rank to characterize the sensitivity of both segmentation accuracy and calibration to this ratio.

For SegFormer-B2, we observe a clear degradation in performance as $\alpha$ increases, manifesting in both mIoU and calibration metrics. Since the effective update magnitude is governed by $\delta_r=\alpha/r$, a large $\alpha$ at fixed rank amplifies the low-rank update $\Delta\theta=\delta_rBA$ to a degree that distorts the pre-trained feature representations, effectively overriding the visual prior encoded during pre-training rather than refining it as shown in Table 19. Mask2Former-Swin/B exhibits the opposite trend: performance improves as $\alpha$ increases, suggesting that a stronger update signal is beneficial in this setting. Nevertheless, this gain saturates and reverses once the ratio $\delta_r$ grows large enough that $\Delta\theta\gg\theta_0$, at which point the adapter update dominates the frozen weights and degrades learning, consistent with the representation distortion observed for SegFormer-B2. Notably, Mask2Former-Swin/B appears considerably less sensitive to $\alpha$ overall, implying that the scaling factor does not act in isolation but interacts with rank and the choice of adapted components as shown in Table 20.

The original LoRA formulation Hu et al. (2022) acknowledges this sensitivity and recommends treating $\alpha$ as a regularization-like hyperparameter, noting that setting $\alpha=r$ provides a neutral starting point where the adapter contributes without overwhelming the frozen weights. For high ranks $r\gg128$ Biderman, Ortiz, Portes, Paul, Greengard, Jennings, King, Havens, Chiley, Frankle, Blakeney and Cunningham [10] suggest using $\alpha=2r$ but this is in text domain and needs more verification for dense prediction in vision, this reinforces the broader observation that LoRA hyperparameters should not be tuned independently, and that the relative magnitude of the adapter update with respect to the pre-trained weights is a more principled diagnostic than any single hyperparameter alone Hu et al. (2022); Büyükakyüz [13].

Finally, The relative insensitivity of Mask2Former-Swin/B to $\alpha$ likely reflects two compounding factors. First, unlike SegFormer-B2 where the fully fine-tuned decoder creates a rigid dependency on the quality of the adapted encoder representations, Mask2Former-Swin/B distributes adaptation entirely through LoRA modules across both encoder and decoder. Second, the adapted components in Mask2Former-Swin/B are predominantly MLP layers, which may be intrinsically less sensitive to scaling perturbations than the lower-dimensional attention projections targeted in SegFormer-B2. We note that this remains a hypothesis, as disentangling the individual contributions of architecture, component choice, and scaling interactions would require controlled experiments beyond the scope of this ablation. Our results confirm this intuition in the vision segmentation regime and suggest that conservative scaling is particularly important when adapting pre-trained encoders to fine-grained, data-limited agricultural datasets. Hence we provide a practitioner’s guide according to the previous experiments for ST-LoRA as a starting point:

However, a well-configured model is a necessary but not sufficient condition for reliable deployment as agricultural imaging conditions vary substantially across seasons,

lighting, and sensing conditions, making robustness under distribution shift equally critical. We therefore turn to two complementary questions: does ensemble member diversity affect calibration stability when the input distribution shifts, and do the component choices identified above retain their advantage beyond the training distribution?

Experiment 3: Ensemble Diversity and Architectural Sensitivity under Distribution Shift

We evaluate robustness under covariate shift through two complementary analyses. The first examines how ensemble member diversity affects robustness, using SegFormer-B2 on GrowliFlower-L. The second investigates the sensitivity of different LoRA component configurations to distribution shift, using Mask2Former-Swin/B on BUP20. In both cases, covariate shift is simulated by applying controlled corruptions to the test set, the full experimental setup for both analyses is summarized in Table 25.

Variants	Details	Exp. 1: Ensemble Diversity	Exp. 2: Component Sensitivity
		SegFormer-B2 / GrowliFlower-L	Mask2Former-Swin/B / BUP20
		Homogeneous ensemble (LR schedule only)<br>Heterogeneous ensemble (LR + augmentation)	ST-LoRA hyperparameters robustness study
Photometric	Brightness	Factor $\in[1.0, 2.0]$, 20 levels ($\Delta= 0.05$)	Factor $\in\{0.50, 0.65, 0.80\}$
Photometric	Contrast	Factor $\in[1.0, 2.0]$, 20 levels ($\Delta= 0.05$)	Factor $\in\{0.50, 0.65, 0.80\}$
Photometric	Gaussian noise	$\sigma\in[0, 0.285]$, 20 levels ($\Delta= 0.015$)	$\sigma\in\{0.05, 0.10, 0.20\}$
Photometric	Gaussian blur	—	$\sigma\in\{1, 2, 3\}$ px
Geometric	Rotation	$\theta\in[0^\circ, 85.5^\circ]$, 20 levels ($\Delta= 4.5^\circ$)	—
Geometric	Zoom	Scale $\in[1.0\times, 1.475\times]$, 20 levels ($\Delta= 0.025$)	Scale $\in\{1.10\times, 1.25\times, 1.50\times\}$
Geometric	Horizontal flip	—	Binary (single level)
Geometric	Translation	—	$\{5\%, 10\%, 15\%\}$ of image width
	Total shift conditions	$4 \times20 = 80$	$7 \times3 - 2 = 19$
	Mask handling	Geometric: nearest-neighbour interp. on mask	Geometric: nearest-neighbour interp. on mask
	Metrics	mIoU, and ECE (mean $\pm$ std over 5 seeds)

Ensemble diversity under distribution shift

We compare two ensemble construction strategies for both FRE and ST-LoRA to assess the impact of member diversity on calibration robustness. Homogeneous ensembles rely solely on learning rate scheduling as a source of diversity 20, with no data augmentation applied during training; members therefore differ only in their optimization trajectories. Heterogeneous ensembles additionally introduce per-member data augmentation policies, incorporating random rotation (uniformly sampled from $\{0^\circ, 25^\circ\}$) and horizontal flipping (probability 0.5), applied consistently to both images and segmentation masks. This introduces diversity at both the optimization and data levels, which leads to more robust ensemble predictions and better-calibrated uncertainty estimates under distribution shift.

Across all distribution shifts as shown in Figure 6, ST-LoRA achieves performance competitive with FRE, while exhibiting greater robustness to increasing shift severity maintaining lower ECE values with only a 3–5% reduction in mIoU. For FRE, homogeneous ensembles achieve higher mIoU but substantially worse calibration compared to their heterogeneous counterparts. The heterogeneous strategy is therefore critical for calibration quality in full-rank ensembles. ST-LoRA, by contrast, benefits from heterogeneity in both metrics simultaneously: heterogeneous ST-LoRA outperforms its homogeneous counterpart in mIoU and ECE alike, achieving the best calibration across all shift conditions while maintaining segmentation quality competitive with FRE variants.

We attribute this asymmetry to the reduced parameter space of LoRA adapters: the constrained adaptation subspace acts as an implicit regularizer, allowing ST-LoRA members to leverage augmentation-induced diversity more efficiently than full-rank models operating under the same limited training budget. Furthermore, we observe that SgF-B2 performance is deteriorating under high brightness/contrast shift compared to other shifts. This is due to high severity which completely distorts the input image to the model, however around this shift level of 10, the model still achieves 64% mIoU, and this robustness is due to the design of SegFormer which has been mentioned in Xie et al. [107].

Component Sensitivity under Distribution Shift

We evaluate how ST-LoRA hyperparameter choices - specifically rank, target components, scaling factor, and dropout - affect calibration robustness when the input distribution shifts. For each hyperparameter, we compare how varying a single setting relative to the baseline configuration affects robustness across the 19 corruption conditions described in Table 25.

Rank. Across all corruption types and severity levels, a consistent pattern emerges: low-rank configurations are brittle under distribution shift, exhibiting substantially worse calibration alongside degraded segmentation quality as shown in Figure 7. This instability is most pronounced at $r = 2$ as shown in Figure 8, where calibration metrics deteriorate sharply even under mild photometric perturbations, suggesting that a low-dimensional adaptation subspace lacks sufficient capacity to maintain well-calibrated predictions beyond the training distribution. Increasing the rank to $r = 16$ yields a notable improvement in both robustness and calibration stability across shift types, establishing it as a practical lower bound for reliable deployment under covariate shift. However, continuing to increase the rank beyond this point introduces diminishing returns: while segmentation accuracy may marginally improve, calibration under distribution shift does not benefit proportionally and can even degrade at higher ranks as shown in Figure 8 which has been also shown by Halbh... et al. (2024). The additional computational cost of high-rank adaptation therefore does not translate into a robustness advantage, reinforcing $r \in$ [8, 16] as the recommended operating range for ST-LoRA on Mask2Former-Swin/B under distribution shift.

Target components. Ranking configurations by mIoU and ECE robustness under distribution shift reveals a persistent tension between segmentation accuracy and calibration as shown in Figure 20 and Figure 21: the configurations that best preserve accuracy under shift are not the same as those that best preserve calibration. Specifically, base + $k_e$ ranks second on mIoU but drops to sixth on calibration, while base + $q_e$ ranks third on mIoU yet last on calibration. The configuration achieving the strongest mIoU robustness overall is base + $k_e + v_e$, whereas base + $q_e + k_e$ leads on calibration robustness. The baseline configuration (FF layers, Class predictor) occupies a rank of approximately five on both objectives, placing it consistently in an intermediate position that avoids the extremes of either. This trade-off mirrors the accuracy/calibration tension observed in the in-distribution ablation and suggests that no single component configuration simultaneously dominates both objectives under covariate shift.

Furthermore, Cross-referencing with the in-distribution ablation results reveals that the baseline and the base + $q_e + k_e$ configuration yield comparable performance across both mIoU and ECE relative to the best-performing configurations for both SegFormer-B2 and Mask2Former-Swin/B. This consistency across architectures, datasets, and evaluation regimes - in-distribution and under shift - suggests that adapting query and key projections alongside the encoder MLP layers provides a robust and transferable operating point. baseline + $q_e + k_e$ therefore represents the most principled choice, offering a stable balance between segmentation quality and uncertainty reliability without requiring architecture-specific tuning.

Scaling factor $\alpha$. The effect of $\alpha$ on robustness under distribution shift exhibits a clear asymmetry between segmentation accuracy and calibration. For mIoU, increasing $\alpha$ initially improves robustness, suggesting that a stronger adapter signal helps the model maintain discriminative representations under input perturbations. However, this benefit saturates and reverses beyond a certain threshold, where the amplified low-rank update begins to distort the pretrained prior rather than refine it as shown in Figure 20 and Figure 21, consistent with the in-distribution degradation observed at high $\delta_r = \alpha/r$ values. For calibration under distribution shift, the relationship is qualitatively different: increasing $\alpha$ consistently yields better calibrated predictions, with ECE improving monotonically across shift types and severity levels. This suggests that a stronger adapter update enhance robustness when the input deviates from the training distribution. The two objectives therefore pull in opposite directions as $\alpha$ increases, reinforcing the broader observation that accuracy robustness and calibration robustness are not jointly optimizable through a single hyperparameter, and that the choice of $\alpha$ should be guided by the deployment priority - higher $\alpha$ for better-calibrated uncertainty, lower to moderate $\alpha$ for stronger segmentation accuracy under shift but we leave the empirical validation for future work.

LoRA Dropout $p$. The effect of LoRA dropout on robustness under distribution shift follows a non-monotonic pattern for segmentation accuracy: both the lowest ($p = 0.1$) and highest ($p = 0.3$) dropout values yield degraded mIoU under shift, while intermediate values $p \in[0.2, 0.25]$ achieve the strongest robustness as shown in Figure 20 and Figure 21. Small dropout allows the adapter to overfit to in-distribution features that do not generalize under shift, while too much dropout disrupts the adapter’s capacity to maintain discriminative representations altogether. A moderate dropout rate therefore strikes the best operating point between preventing overfitting and preserving representational capacity.

For calibration, the picture is more nuanced and shift-type selective. Under photometric corruptions, $p = 0.3$ yields the best calibration robustness, suggesting that aggressive dropout encourages the model to be robust when pixel statistics deviate from the training distribution. Under geometric corruptions, however, $p = 0.3$ underperforms relative to lower dropout values, which instead produce more stable calibration. This shift-type selectivity of dropout echoes the broader finding that photometric and geometric corruptions engage different aspects of the model’s learned representations, and reinforces the recommendation to treat calibration robustness under the two shift families as distinct design objectives.

For statistical significance, as shown in Figure 9, the low rank/alpha values ($r \le4$, $\alpha\le4$) are hurting the segmentation quality and calibration. While for larger values the gains are significant. For the targeted components we first notice that as an overview there are no big differences regarding the attention components being adapted with the FF layers. However we observe that MLP+$q_e+k_e$ or MLP+$k_e+v_e$ enhances the calibration with respect to the other configurations but for less fine tuned parameters we could start by MLP+$q_e$.

Finally, Guo et al. [41] highlight an inherent tension between minimizing classification error and achieving well-calibrated predictions. We observe a similar trade-off in ST-LoRA: accuracy and calibration emerge as competing objectives, and we hypothesize that this competition has a low-rank geometric interpretation. Drawing on the notion of intrinsic dimensionality Aghajanyan et al. [3], the low-rank subspace, sufficient for optimizing the segmentation objective, may not be expressive enough to simultaneously ensure reliable calibration; particularly under distribution shift, where the adapter must generalize beyond the fine-tuning data manifold. This suggests that accuracy and calibration place structurally different demands on the adaptation subspace, and that jointly optimizing both may require explicit calibration-aware training objective.

According to the previous observations, we give a recommendation for hyperparameters selection under covariate shift:

Having established ST-LoRA’s advantages over full-rank ensemble and characterized its behaviour under distribution shift, we turn to a broader question: how does ST-LoRA compare against established efficient SOTA uncertainty quantification methods? We benchmark against three approaches - Snapshot Ensemble, MC Dropout, and DDU - spanning deterministic, and stochastic uncertainty estimation paradigms, and six post-hoc calibration methods.

Experiment 4: Benchmarking ST-LoRA Against Efficient Uncertainty Methods

We compare ST-LoRA against three efficient uncertainty estimation baselines whose configurations are detailed in Table 26. Integrated ensemble methods are excluded due to architectural and compatibility constraints discussed in § 3. To ensure a fair comparison, all methods share an identical training setup: the same cosine annealing learning rate schedule and the same heterogeneous augmentation pipeline, isolating the effect of the uncertainty estimation strategy itself.

Method	Ensemble	Inference	Settings
FRE	Yes	($M = 4$) forward passes	Following the similar Full Rank Ensembles from previous experiments.
MC Dropout	No	$T = 4$ stochastic passes	We dropout layer before the class predictor using a dropout probability of $p = 0.5$ following Gal and Ghahramani [35]; Srivastava et al. [94].
DDU	No	Single forward pass	SN has been applied to all linear and convolutional layers. A Gaussian Mixture Model ($K=2$ components per class, 8 classes total) is fitted to these features using the EM algorithm for 1000 iterations. Pixels are treated as spatially independent data points, yielding 1280 $\times$ 720 data points per image.
ST-LoRA (ours)	Yes	($M = 4$) forward passes	LoRA adaptation with frozen decoder. Rank $r=32$, scaling $\alpha=32$, dropout $p=0.1$. Target modules: decoder $q_d$, $k_d$, $c_l$; encoder MLP, $k_e$, $v_e$.

The augmentation pipeline consists of two families of transformations. Geometric augmentations, applied jointly to image and mask, include random horizontal flipping ($p=0.5$) and lateral translation (up to $10\%$ of image width, $p=0.5$). Photometric augmentations, applied to the image only, include colour jitter (brightness $\pm0.3$, contrast $\pm0.3$, saturation $\pm0.1$), Gaussian blur ($\sigma\in[0.5,1.5]$, $p=0.3$), Gaussian noise ($\sigma=0.02$, $p=0.2$), spatially non-uniform depth-of-field blur ($p=0.3$) simulating centre-sharp optics via a Gaussian spatial weight mask, and random erasing of up to three patches per image ($p=0.3$, patch area $\in[2\%,15\%]$ of image) to simulate leaf occlusion and canopy overlap. Additionally, CutMix Yun, Han, Oh, Chun, Choe and Yoo [110] ($p=0.5$) is applied at the dataset level, randomly replacing a rectangular region of one image and its corresponding mask labels with those from another training sample, encouraging the model to learn from partially occluded scenes. ²³

For ensemble methods, five snapshots are collected per training trajectory for both FRE and ST-LoRA by saving the model weights blindly at the end of each cosine annealing cycle, without any validation-based selection. For MC Dropout and DDU, a single checkpoint is saved at the end of training. This blind collection strategy is applied uniformly across all methods: tracking the best validation loss would introduce a selection advantage that is incompatible with the snapshot ensembling design, where checkpoints are taken at fixed cycle endpoints regardless of their individual validation performance. Keeping this consistent ensures that observed differences in performance reflect the uncertainty estimation strategy rather than checkpoint selection policy.

Finally, to position ST-LoRA within the broader calibration literature, we additionally compare it against six established post-hoc calibration methods: Temperature Scaling (TS) Guo et al. [41], Logistic scaling Platt [83], Dirichlet scaling Kull, Perello-Nieto, Kängsepp, Silva Filho, Song and Flach [59], Meta-Cal Ma and Blaschko (2021), Local Temperature Scaling (LTS) Ding, Han, Liu and Niethammer [25], and the Selective scaling method of

Wang et al. [100] for semantic segmentation calibration.²⁴ To calibrate these methods, we randomly split the original validation data (63 samples) into two subsets: subset A, the new validation set (33 samples), and subset B, the calibration set (30 samples).

We additionally report Negative Log-Likelihood (NLL) and the Brier Score Brier [11] alongside ECE/ACE. As, ECE alone has several known limitations: a low ECE does not necessarily correspond to high accuracy, its binning procedure introduces a bias/variance trade-off, and it considers only the maximum predicted probability rather than the full predictive distribution Pavlovic [82].²⁵

In-Distribution Performance and Robustness Under Distribution Shift

We follow the same two-stage evaluation protocol as previous experiments: in-distribution performance followed by robustness under distribution shift.

In-distribution performance. Table 8 shows that ST-LoRA achieves mIoU competitive with the best-performing method (FRE) and remains highly competitive in calibration, ranking among the top three on most metrics. Specifically, on ECE, DDU performs best, followed by Dirichlet scaling, with ST-LoRA ranking third. On Brier score, FRE leads, followed by LTS, with ST-LoRA again placing third. On NLL, however, most methods outperform ST-LoRA, with the exception of Meta-Cal and Selective Scaling. For the remaining calibration metrics, ST-LoRA ranks second on ACE and MACE, while achieving the best performance on MECE.

Method	mIoU $\uparrow$	ECE $\downarrow$	MECE $\downarrow$	ACE $\downarrow$	MACE $\downarrow$	Brier $\downarrow$	NLL $\downarrow$
Uncertainty estimation methods
FRE	0.6990 ± 0.0259	0.0068 ± 0.0068	0.1792 ± 0.1613	0.0047 ± 0.0037	0.0209 ± 0.0242	0.0171 ± 0.0003	0.0389 ± 0.0024
MC Dropout	0.6505 ± 0.0391	0.0049 ± 0.0049	0.0686 ± 0.0302	0.0062 ± 0.0090	0.0177 ± 0.0126	0.0193 ± 0.0014	0.0440 ± 0.0062
DDU	0.6553 ± 0.0197	0.0019 ± 0.0010	0.1171 ± 0.1069	0.0016 ± 0.0008	0.0106 ± 0.0062	0.0197 ± 0.0004	0.0419 ± 0.0007
ST-LoRA (ours)	0.6920 ± 0.0147	0.0026 ± 0.0018	0.0901 ± 0.0311	0.0027 ± 0.0011	0.0077 ± 0.0059	0.0186 ± 0.0004	0.0518 ± 0.0044
Post-hoc calibrators (applied to single model)
Uncalibrated	0.6692 ± 0.0289	0.0049 ± 0.0026	0.1794 ± 0.0463	0.0058 ± 0.0019	0.0369 ± 0.0127	0.0192 ± 0.0021	0.0476 ± 0.0069
TS	0.6692 ± 0.0289	0.0052 ± 0.0039	0.1572 ± 0.0581	0.0055 ± 0.0023	0.0364 ± 0.0174	0.0193 ± 0.0023	0.0467 ± 0.0077
Logistic	0.6717 ± 0.0276	0.0042 ± 0.0020	0.1414 ± 0.0588	0.0048 ± 0.0019	0.0302 ± 0.0152	0.0189 ± 0.0016	0.0454 ± 0.0059
Dirichlet	0.6432 ± 0.0196	0.0023 ± 0.0008	0.1073 ± 0.0763	0.0043 ± 0.0015	0.0313 ± 0.0146	0.0203 ± 0.0008	0.0443 ± 0.0031
LTS	0.6692 ± 0.0289	0.0044 ± 0.0011	0.1761 ± 0.0707	0.0043 ± 0.0010	0.0346 ± 0.0111	0.0185 ± 0.0010	0.0410 ± 0.0022
Meta-Cal	0.3921 ± 0.0694	0.0284 ± 0.0131	0.3510 ± 0.1015	0.0201 ± 0.0073	0.1794 ± 0.0721	0.0520 ± 0.0108	0.1302 ± 0.0266
Selective	0.4156 ± 0.1006	0.0718 ± 0.1233	0.3820 ± 0.2166	0.0810 ± 0.1485	0.1971 ± 0.2980	0.0967 ± 0.1309	0.2328 ± 0.3098

Among all methods, DDU offers the lowest computational cost and calibration gains (see Table 9), as uncertainty is estimated in a single forward pass without sampling. However, this efficiency comes at the cost of segmentation accuracy. The observed mIoU drop for DDU is attributed to the widespread application of spectral normalization, which constrains the feature space and reduces discriminative capacity, consistent with the accuracy/robustness trade-off reported in [51, 39].

Approach	Flops (GMAC)	Train Params. (M)
M2F (base)	229,72	108.77
FRE	1148	534
MC-Dropout	1148	108.77
DDU	320	108.8
ST-LoRA (ours)	1170	116

Furthermore, applying SN may inherently limit ViT performance. As Dong et al. [26] demonstrate, MLP layers play a critical role in preserving representational capacity by counteracting the rank collapse induced by self-attention and increases the Lipschitz constant of the network. SN directly opposes this dynamic by constraining the Lipschitz constant to a fixed upper bound, creating a counter force between both. Consequently, how to enforce a meaningful Lipschitz bound in transformers without sacrificing representational capacity remains an open question, with promising directions explored in Newhouse, Hess, Cesista, Zahorodnii, Bernstein and Isola [77]. Finally, MC Dropout, performance sensitivity likely reflects the joint effect of the targeted dropout layer and the number of stochastic forward passes in addition to other hyperparameters as mentioned in Srivastava, Hinton, Krizhevsky, Sutskever and Salakhutdinov [94].

ST-LoRA exhibits the lowest standard deviation across all metrics among all methods, indicating stable performance regardless of the random seed a practically important property for reliable deployment, in addition to reducing the need for architectural changes like in the DDU case or

the need for more sample like in the case of the calibration methods. Taken together, these results position ST-LoRA as a strong candidate that balances calibration quality, segmentation accuracy, and computational efficiency - consistently competitive without being the top performer on any single metric in isolation.²⁶

However, in the next section we show that the results obtained for in-distribution experiment does not necessarily transfer to distribution-shifted settings.

Robustness under distribution shift

Robustness under distribution shift. We evaluate all methods on BUP20 under a systematic corruption grid of seven corruption types at five severity levels each, yielding 35 shift conditions. All severity levels are deliberately chosen to lie outside the training augmentation range, ensuring exposure to unseen conditions. Photometric corruptions, applied to the image only, include Gaussian blur ($\sigma\in\{1.5, 3, 5, 7, 10\}$ px, versus training $\sigma\in[0.5, 1.5]$ px), additive Gaussian noise ($\sigma\in\{0.04, 0.08, 0.12, 0.18, 0.25\}$, versus training $\sigma= 0.02$), brightness reduction (factor $\in\{0.60, 0.40, 0.25, 0.10, 0.05\}$, versus training $[0.7, 1.3]$), contrast reduction (same factor range), and saturation reduction (factor $\in\{0.80, 0.60, 0.40, 0.15, 0.00\}$, versus training $[0.9, 1.1]$). Geometric corruptions include lateral translation ($\{20, 30, 40, 50, 60\}\%$ of image width, versus training $\pm10\%$) and rotation (vertical flip, $90^\circ$, $180^\circ$, $270^\circ$, and $90^\circ$+vertical flip, versus training horizontal flip only).

Method	bg ↑	pepper red ↑	pepper yellow ↑	pepper green ↑	mixed red ↑	mixed yellow ↑	macro-mIoU ↑
Uncertainty estimation methods
DDU	0.990	0.605	0.494	0.750	0.550	0.544	$0.655 \pm0.018$
MC-Dropout	0.990	0.641	0.418	0.765	0.582	0.516	$0.652 \pm0.035$
FRE	0.991	0.672	0.545	0.774	0.611	0.620	$0.702 \pm0.016$
ST-LoRA (ours)	0.991	0.645	0.536	0.769	0.602	0.627	$0.695 \pm0.015$
Post-hoc calibrators (applied to single model)
Uncalibrated	0.991	0.617	0.499	0.780	0.545	0.583	$0.669 \pm0.029$
TS	0.991	0.617	0.499	0.780	0.545	0.583	$0.669 \pm0.029$
Logistic	0.991	0.623	0.498	0.780	0.551	0.588	$0.672 \pm0.028$
Dirichlet	0.989	0.509	0.503	0.760	0.510	0.588	$0.643 \pm0.020$
LTS	0.991	0.617	0.499	0.780	0.545	0.583	$0.669 \pm0.029$
Meta-Cal	0.977	0.155	0.130	0.567	0.276	0.247	$0.392 \pm0.069$
Selective	0.984	0.196	0.243	0.744	0.227	0.306	$0.450 \pm0.126$

Method	| Per-Class NLL ↓ |							| Per-Class Brier ↓ |
	bg	red	yel	grn	m-red	m-yel	macro	bg	red	yel	grn	m-red	m-yel	macro
Uncertainty estimation methods
DDU	0.013	0.877	0.644	0.537	1.494	1.002	0.761 ± 0.053	0.007	0.407	0.357	0.238	0.560	0.471	0.340 ± 0.015
MC-Dropout	0.018	0.474	0.870	0.491	1.209	0.968	0.672 ± 0.024	0.008	0.220	0.474	0.210	0.536	0.489	0.323 ± 0.007
FRE	0.016	0.467	0.661	0.481	0.873	0.744	0.540 ± 0.019	0.006	0.225	0.381	0.227	0.459	0.395	0.282 ± 0.011
ST-LoRA (ours)	0.030	0.583	0.709	0.458	0.943	0.874	0.600 ± 0.015	0.007	0.272	0.379	0.220	0.495	0.465	0.306 ± 0.009
Post-hoc calibrators (applied to single model)
Uncalibrated	0.014	0.528	1.078	0.525	1.983	1.548	0.946 ± 0.132	0.007	0.241	0.494	0.212	0.572	0.483	0.335 ± 0.043
TS	0.015	0.521	1.041	0.501	1.866	1.464	0.901 ± 0.120	0.007	0.241	0.491	0.211	0.567	0.479	0.333 ± 0.042
Logistic	0.014	0.585	1.105	0.485	1.835	1.448	0.912 ± 0.118	0.007	0.264	0.503	0.206	0.559	0.470	0.335 ± 0.045
Dirichlet	0.018	1.388	1.131	0.387	1.007	1.206	0.856 ± 0.076	0.010	0.639	0.502	0.165	0.385	0.430	0.355 ± 0.040
LTS	0.013	0.654	0.969	0.498	1.339	1.079	0.759 ± 0.053	0.006	0.285	0.464	0.219	0.528	0.458	0.327 ± 0.035
Meta-Cal	0.075	1.824	1.973	1.070	2.057	2.060	1.510 ± 0.143	0.031	0.748	0.817	0.416	0.727	0.729	0.578 ± 0.046
Selective	0.499	1.107	2.085	0.812	2.148	2.087	1.456 ± 0.368	0.210	0.482	0.896	0.344	0.756	0.750	0.573 ± 0.159

As shown in Figure 10, across nearly all shift types and severity levels, ST-LoRA achieves the strongest calibration and segmentation robustness among all methods. FRE follows as the second most robust, ahead of MC Dropout, while DDU degrades most severely under distribution shift. Furthermore, We compare the four methods across the seven corruption types introduced in Experiment 3, reporting performance averaged across corruption type and severity level, with mean and standard deviation over five seeds. As shown in Figure 11, ST-LoRA maintains strong calibration robustness under distribution shift, consistent with our earlier findings in Figure 10. At low shift severity, FRE performs slightly better; however, ST-LoRA becomes increasingly competitive as shift severity increases, ultimately matching or exceeding FRE under the most severe corruptions. Finally, we extend this comparison to include the post-hoc calibration baselines, providing a complete picture of how all evaluated methods degrade under shift. Figure 12 demonstrates that the Bayesian approaches (MC Dropout, FRE, ST-LoRA) generally outperform post-hoc calibration methods under distribution shift

Although DDU is the most computationally efficient method at inference time, carries the smallest memory footprint, and nearly best calibrated at in-distribution analysis, its calibration collapses rapidly under shift which is a well-documented limitation of deterministic uncertainty methods, not exclusive to DDU Postels et al. [85]. The poor robustness of MC-Dropout is consistent with our earlier hypothesis that both the choice of dropout layer and the sampling rate critically affect performance under shift, and needs further investigation to identify configurations that generalize more reliably beyond the training distribution. For post-hoc calibration methods our findings align with Ovadia, Fertig, Ren, Nado, Sculley, Nowozin, Dillon, Lakshminarayanan and Snoek [80], who show that strong in-distribution calibration does not guarantee calibration or robustness under distributional shift.

Consistent with our earlier observation, FRE and ST-LoRA remain the two best-performing methods overall: FRE is stronger at low shift severity, while ST-LoRA outperforms all methods - including FRE - under higher shift severity.

Out of Distribution (OoD) detection

We evaluate our uncertainty quantification methods under two distribution-shift regimes: a far-OoD condition and a near-OoD condition, illustrated in Figure 13.

Far-OoD. We use GrowliFlower-L, which represents markedly different capture conditions - open-field, UAV-acquired imagery- relative to BUP20, the dataset on which our models are trained. From BUP20 we use 93 images (the full test set), while from GrowliFlower-L we use 1,500 samples from its validation set, after removing soil-only images corresponding to the early growth stage, all cauliflower plant’s pixels are grouped into a single foreground (anomalous) class, with the background assigned as the normal class.

Near-OoD. We select the BUtom2l tomato dataset Halstead, Guclu, Farag, Pallotta, Hund, Roscher, Bennewitz, Gall, Stachniss and McCool [43]²⁷ as a well-suited candidate, captured in the University of Bonn’s commercial-style glasshouse at Campus Klein-Altendorf (CKA). Although the glasshouse hosts multiple crops across separate cells, we use only the Sweeterno tomato cell, which contains no other fruit or vegetable species. This dataset forms a natural near-OoD counterpart to BUP20, sharing the same glasshouse environment and camera setup, with tomato classes that are visually similar in color to the sweet pepper classes in BUP20. We evaluate on the dataset’s validation split, comprising approximately 4,500 images at 1280×720 resolution and spanning five ripeness classes: green, orange, mixed-orange, mixed-red, and red. For our purposes, all tomato classes are grouped into a single foreground (anomalous) class, with the background assigned as the normal class.

Image-level OoD detection. We assess each method’s ability to separate ID and OoD samples at the image level, without exploiting local spatial structure. Uncertainty scores are computed per pixel and aggregated by mean across each image. For ensemble and MC Dropout methods, we use predictive entropy and mutual information Shaker and Hüllermeier [90] as scoring functions; for DDU, we use the GMM log-density²⁸. Finally, we evaluate ID/OoD separability qualitatively via score histograms and quantitatively via the Area Under the Receiver Operating Characteristic curve (AUROC) Hendrycks and Gimpel [46] and False Positive Rate@95% (FPR@95).

Pixel-level anomaly segmentation We are the first to apply such evaluation scheme - following the protocol of Chan, Lis, Uhlemeyer, Blum, Honari, Siegwart, Fua, Salzmann and Rottmann [14]; Sodano, Magistri, Behley and Stachniss [93] - to close-range glasshouse crop segmentation, assessing the sensitivity of uncertainty estimation methods to cross-crop OoD objects at the pixel level. Pixel-level anomaly segmentation using ranking metrics (Area Under the Precision-Recall Curve (AUPR), False Positive Rate at 95% True Positive Rate (FPR95)), and localization metrics (segment-wise Intersection over Union (sIoU), Positive Predictive Value (PPV), and mean F1 score (MeanF1)) has not, to our knowledge, been evaluated in this setting.

Table 10 summarizes OoD detection performance across all methods. On the image level, ST-LoRA achieves the best AUROC/FPR@95 between ID and OoD images on both datasets; followed by FRE, MC Dropout, and DDU. Critically, ST-LoRA also exhibits the lowest standard deviation across both scoring functions among all methods, confirming that its OoD sensitivity is stable and seed-independent, a practically important property for deployment.

Method	Scorer	AUROC ↑	FPR95 ↓	AUROC ↑	AUPR ↑	FPR95 ↓	sIoU ↑	PPV ↑	MeanF1 ↑
		Image-level		Pixel-level
Far-OoD (GrowliFlower-L)
FRE	Ent.	0.9282 ± 0.0497	0.4129 ± 0.2723	0.5828 ± 0.0896	0.9021 ± 0.0266	0.9185 ± 0.0436	0.4121 ± 0.0184	0.1986 ± 0.0848	0.2074 ± 0.0856
	MI	0.7886 ± 0.1241	0.7505 ± 0.1989	0.5845 ± 0.0767	0.9016 ± 0.0242	0.9256 ± 0.0370	0.4133 ± 0.0225	0.1820 ± 0.0718	0.1927 ± 0.0734
MC Dropout	Ent.	0.5874 ± 0.1549	0.9161 ± 0.1579	0.5901 ± 0.0606	0.9058 ± 0.0165	0.9094 ± 0.0451	0.4029 ± 0.0250	0.2364 ± 0.0894	0.2425 ± 0.0844
	MI	0.6914 ± 0.0486	0.8323 ± 0.1052	0.5274 ± 0.0238	0.8830 ± 0.0073	0.9436 ± 0.0189	0.4121 ± 0.0124	0.2322 ± 0.0462	0.2455 ± 0.0432
DDU	Den.	0.3860 ± 0.3335	0.8220 ± 0.3930	0.5475 ± 0.0458	0.8910 ± 0.0155	0.9426 ± 0.0190	0.3405 ± 0.0192	0.0036 ± 0.0001	0.0065 ± 0.0008
ST-LoRA (ours)	Ent.	0.9981 ± 0.0025	0.0065 ± 0.0096	0.4069 ± 0.1043	0.8309 ± 0.0357	0.9719 ± 0.0230	0.4022 ± 0.0338	0.3013 ± 0.0792	0.3088 ± 0.0765
	MI	0.5877 ± 0.2110	0.8903 ± 0.1001	0.4044 ± 0.0862	0.8329 ± 0.0278	0.9794 ± 0.0127	0.3938 ± 0.0325	0.2746 ± 0.0727	0.2845 ± 0.0707
Near-OoD (BUTom21)
FRE	Ent.	0.8340 ± 0.1381	0.5441 ± 0.1821	0.9734 ± 0.0053	0.5826 ± 0.0310	0.0675 ± 0.0165	0.5745 ± 0.0096	0.2514 ± 0.0117	0.3157 ± 0.0119
	MI	0.8884 ± 0.1424	0.3183 ± 0.2546	0.9709 ± 0.0065	0.6179 ± 0.0575	0.0801 ± 0.0244	0.5715 ± 0.0097	0.1591 ± 0.0173	0.2312 ± 0.0199
MC Dropout	Ent.	0.7177 ± 0.1050	0.8258 ± 0.1111	0.9474 ± 0.0185	0.4525 ± 0.0929	0.2201 ± 0.1774	0.5349 ± 0.0055	0.1835 ± 0.0596	0.2420 ± 0.0681
	MI	0.5777 ± 0.0443	0.9054 ± 0.0439	0.8309 ± 0.1032	0.2950 ± 0.1155	0.6763 ± 0.1729	0.5116 ± 0.0675	0.1167 ± 0.0317	0.1504 ± 0.0446
DDU	Den.	0.8660 ± 0.025	0.5660 ± 0.1660	0.9365 ± 0.0343	0.5073 ± 0.0609	0.3020 ± 0.2299	0.5484 ± 0.0088	0.1354 ± 0.0146	0.1861 ± 0.0127
ST-LoRA (ours)	Ent.	0.8919 ± 0.0871	0.3871 ± 0.1647	0.9752 ± 0.0040	0.5295 ± 0.0374	0.0445 ± 0.015	0.5693 ± 0.0055	0.2946 ± 0.0154	0.3568 ± 0.0111
	MI	0.9594 ± 0.0304	0.2043 ± 0.1498	0.9737 ± 0.0045	0.5995 ± 0.0707	0.0486 ± 0.0032	0.5704 ± 0.0154	0.2039 ± 0.0205	0.2780 ± 0.0200

Across all bayesian approximation methods, mutual information consistently underperforms predictive entropy as an OoD scoring function on GrowliFlower-L and vice versa on the BUTom21 dataset except for MC-Dropout approach where the observation is the opposite²⁹. This is not a failure of the ensemble approach: Hüllermeier [52] have shown that very low MI values reflect a fundamental limitation of the measure itself under OoD conditions. For the mismatch of how mutual information behaves on different datasets³⁰, Hüllermeier [52] mention that the additivity of uncertainties in DL models remain subtle. This questions the assumption itself and remains a research hotspot. Furthermore, Fellaji and Pennerath [33] explores the epistemic uncertainty hole which is an epistemic uncertainty collapse phenomenon that occurs when we train on small dataset or having large models. DDU shows the highest standard deviation across both of its scoring functions, making it the least reliable method for OoD detection on Mask2Former, consistent with the architectural incompatibility concerns discussed in the previous experiment³¹.

Figure 14 shows that ST-LoRA achieves the clearest separation between ID and OoD distributions under entropy scoring, exhibiting the largest gap between distribution means. Across all methods, OoD samples consistently produce higher entropy/lower density than ID samples; the expected and desirable behaviour, confirming that uncertainty estimates are well-aligned with the novelty of the input.

Among the baselines, DDU achieves strong separation using GMM density scores but is highly unstable across seeds, making it unreliable in practice. MC Dropout collapses a large fraction of OoD samples into the ID score range a known failure mode under far OoD conditions Lakshminarayanan et al. [60], where the method produces overconfident, low-variance predictions on inputs far from the training distribution. FRE ranks second overall, offering reliable but less consistent separation than ST-LoRA while performing better on mutual information.

At the pixel level on the GrowliFlower-L dataset, no single winning approach emerges, as most methods perform close to the random model (AUROC=0.5) on several metrics. Specifically, on three out of six metrics, ST-LoRA performs the worst, while MC-Dropout achieves the best performance in terms of AUROC, AUPR, and FPR@95 compared to all other approaches. On sIoU, ST-LoRA ranks as the third-best approach; however, for PPV and MeanF1, ST-LoRA achieves the best performance. DDU achieves near-zero PPV and MeanF1, confirming that its density-based scores do not translate into reliable pixel-level anomaly localization in this setting.

For BUTom21, ST-LoRA outperforms FRE on four out of six metrics. Specifically, ST-LoRA achieves slightly better AUROC, while ranking second after FRE on AUPR. Our approach performs best in terms of FPR@95. For sIoU, FRE takes the lead, albeit by a narrow margin. Finally, ST-LoRA delivers the best performance on both PPV and MeanF1. Overall, ST-LoRA provides the most consistent and reliable OoD detection profile: the best image-level separation, the best precision and F1 at the pixel level, and the lowest variance across seeds; making it the most dependable choice for uncertainty-aware deployment in agricultural environments where OoD inputs must be identified reliably.

Statistical Significance and Results Discussion

To draw more rigorous conclusions, we performed paired significance testing across all evaluation protocols used in this study. As shown in Table 11, ST-LoRA emerges as a strong competitor to FRE, underscoring the favorable efficiency-performance trade-off it offers.

Setting	Metric	FRE	ST-LoRA	$d$	$p$ ($t/W$)	$d_z$	95% CI
In-domain (clean)
Clean	mIoU ↑	0.7023 ± 0.0182	0.6942 ± 0.0176	−0.0081	0.227/0.312	−0.64	[−0.018, +0.002]
Clean	ECE ↓	0.0068 ± 0.0035	0.0187 ± 0.0046	+0.0120	0.019/0.062	+1.70	[+0.007, +0.017]
Clean	ACE ↓	0.0067 ± 0.0035	0.0187 ± 0.0047	+0.0120	0.019/0.062	+1.70	[+0.007, +0.017]
Clean	Brier ↓	0.0171 ± 0.0004	0.0186 ± 0.0004	+0.0014	0.003/0.062	+2.87	[+0.001, +0.002]
Clean	NLL ↓	0.0389 ± 0.0027	0.0518 ± 0.0050	+0.0130	0.013/0.062	+1.92	[+0.008, +0.018]
Distribution shift (overall mean across shifts, 35 conditions)
Overall	mIoU ↑	0.4558 ± 0.0090	0.4641 ± 0.0096	+0.0083	0.291/0.312	+0.54	[−0.003, +0.019]
Overall	ECE ↓	0.0196 ± 0.0029	0.0295 ± 0.0050	+0.0099	0.015/0.062	+1.85	[+0.006, +0.015]
Overall	ACE ↓	0.0200 ± 0.0023	0.0295 ± 0.0051	+0.0095	0.014/0.062	+1.87	[+0.006, +0.014]
Overall	Brier ↓	0.0567 ± 0.0014	0.0563 ± 0.0017	−0.0003	0.774/0.625	−0.14	[−0.002, +0.002]
Overall	NLL ↓	0.1671 ± 0.0050	0.1621 ± 0.0086	−0.0050	0.404/0.438	−0.42	[−0.013, +0.005]
Image-level OoD (entropy)
BUTom21 (near)	AUROC ↑	0.8340 ± 0.1381	0.8919 ± 0.0871	+0.0579	0.111/0.125	+0.91	[+0.013, +0.108]
BUTom21 (near)	AUPR ↑	0.9945 ± 0.0064	0.9965 ± 0.0038	+0.0020	0.188/0.188	+0.71	[+0.000, +0.005]
BUTom21 (near)	FPR95 ↓	0.5441 ± 0.1821	0.3871 ± 0.1647	−0.1570	0.055/0.125	−1.20	[−0.256, −0.056]
GrowliFlower-L (far)	AUROC ↑	0.9283 ± 0.0555	0.9981 ± 0.0025	+0.0699	0.049/0.062	+1.25	[+0.030, +0.119]
GrowliFlower-L (far)	AUPR ↑	0.9962 ± 0.0032	0.9999 ± 0.0001	+0.0037	0.060/0.062	+1.17	[+0.002, +0.007]
GrowliFlower-L (far)	FPR95 ↓	0.4129 ± 0.2723	0.0065 ± 0.0096	−0.4065	0.030/0.062	−1.48	[−0.607, −0.170]
Pixel-level OoD (entropy)
BUTom21 (near)	AUROC ↑	0.9734 ± 0.0053	0.9752 ± 0.0040	+0.0019	0.640/0.438	+0.23	[−0.005, +0.009]
BUTom21 (near)	AUPR ↑	0.5826 ± 0.0310	0.5629 ± 0.0374	−0.0197	0.190/0.312	−0.70	[−0.041, +0.005]
BUTom21 (near)	FPR95 ↓	0.0675 ± 0.0165	0.0445 ± 0.0015	−0.0231	0.038/0.062	−1.36	[−0.038, −0.011]
BUTom21 (near)	sIoU ↑	0.5745 ± 0.0096	0.5693 ± 0.0055	−0.0053	0.336/0.625	−0.49	[−0.014, +0.003]
BUTom21 (near)	PPV ↑	0.2514 ± 0.0117	0.2946 ± 0.0154	+0.0432	0.007/0.062	+2.27	[+0.029, +0.058]
BUTom21 (near)	MeanF1 ↑	0.3157 ± 0.0119	0.3568 ± 0.0111	+0.0411	0.007/0.062	+2.24	[+0.028, +0.055]
GrowliFlower-L (far)	AUROC ↑	0.6881 ± 0.1250	0.4335 ± 0.1051	−0.2546	0.037/0.062	−1.38	[−0.420, −0.147]
GrowliFlower-L (far)	AUPR ↑	0.4148 ± 0.1425	0.1603 ± 0.0269	−0.2545	0.017/0.062	−1.77	[−0.383, −0.152]
GrowliFlower-L (far)	FPR95 ↓	0.8567 ± 0.1384	0.9702 ± 0.0261	+0.1135	0.174/0.062	+0.74	[−0.019, +0.251]
GrowliFlower-L (far)	sIoU ↑	0.4263 ± 0.1333	0.1889 ± 0.0093	−0.2374	0.016/0.062	−1.78	[−0.342, −0.122]
GrowliFlower-L (far)	PPV ↑	0.0641 ± 0.0533	0.0722 ± 0.0370	+0.0081	0.736/1.000	+0.16	[−0.030, +0.046]
GrowliFlower-L (far)	MeanF1 ↑	0.0920 ± 0.0634	0.0696 ± 0.0301	−0.0224	0.460/0.812	−0.36	[−0.072, +0.024]

On in-domain performance, segmentation quality (mIoU) shows no significant difference between the two methods, while FRE achieves significantly better calibration. Under distribution shift (covariate shift only, in our setting), mIoU, Brier score, and NLL again show no significant difference, whereas FRE achieves significantly better ECE and ACE.

For image-level OoD detection, ST-LoRA proves highly competitive: performance is statistically tied on BUTom21 (near-OoD), while ST-LoRA significantly outperforms FRE on GrowliFlower-L (far-OoD). At the pixel level, ST-LoRA significantly outperforms FRE on three metrics for the near-OoD BUTom21 dataset, with no significant difference on the remaining metrics. On the far-OoD GrowliFlower-L dataset, however, this pattern reverses: FRE significantly outperforms ST-LoRA on three metrics, with no significant difference on the rest.

Taken together, these results demonstrate that ST-LoRA offers a compelling trade-off between efficiency and performance. It reduces training time compared to deep ensemble, I/O inference latency, and checkpoint size -and therefore storage requirements compared to FRE- while matching FRE on several evaluation protocols and outperforming it in specific scenarios, most notably far-OoD image-level detection and near-OoD pixel-level localization.

Final Remarks

Comparison with Related Work

ST-LoRA establishes a strong, scalable ensembling framework that achieves a compelling balance between computational efficiency and uncertainty quantification quality, reducing training time compared to deep ensemble, inference I/O latency, trainable parameter count, and memory footprint simultaneously while remaining competitive compared to established SOTA methods across all evaluated tasks.

Table 12 situates our work within the existing LoRA ensemble literature across four dimensions: uncertainty evaluation schemes, ablation scope, target components, and application domain. The closest related work is Halbhe er et al. (2024)³², which is the only prior study to evaluate both distribution shift and OoD detection. However, their analysis stops short of investigating how LoRA hyperparameters interact with covariate shift, a gap our work directly addresses. They also do not report whether ensemble members were trained with augmentation, nor examine its effect on robustness, whereas we show that heterogeneous augmentation is critical for calibration stability under shift.

On the ablation side, Halbhe er et al. [42] focus exclusively on rank and target component selection, reporting that FF layer adaptation improves accuracy but degrades calibration, a conclusion our results contradict, and which a growing body of evidence from both text and vision literature supports in our favour He et al.; Zhang et al.; Melas-Kyriazi [70]. Critically, they overlook the scaling factor $\delta_{r}=\alpha/r$ entirely, which we identify as a key driver of the accuracy/calibration trade-off and could be further investigated in future work as a training-free deployment-time calibration knob. Their focus on attention-only adaptation - using up to 16 ensemble members - also makes FF adaptation appear costly, whereas in our setting of $M=4$ members, FF layers provide the strongest performance at the lowest parameter count.

Beyond Halbheer et al. [42], remaining related works are either restricted to specific vision tasks such as depth estimation, or operate exclusively in the text domain, with no ablation studies and no evaluation under distribution shift. None address dense prediction, which imposes substantially different representational demands than image classification.

ST-LoRA is therefore the first LoRA ensemble framework to jointly address parameter efficiency, calibration robustness under distribution shift, near/far OoD detection, and dense agricultural prediction, providing both empirical evidence and principled guidance for practitioners deploying uncertainty-aware models in resource-constrained real-world environments.

Limitations of ST-LoRA and Future Work

While ST-LoRA has demonstrated strong performance, several limitations and open directions remain.

1. Hyperparameter sensitivity. ST-LoRA inherits the large hyperparameter search space characteristic of LoRA-based methods, making configuration time-consuming; a limitation shared by most uncertainty quantification approaches.

Promising directions: Broader studies across datasets, tasks, and architectures will help identify which hyperparameters matter most and under what conditions. Neural Architecture Search (NAS) White, Safari, Sukthanker, Ru, Elsken, Zela, Dey and Hutter [105] is a natural complement, enabling principled adapter placement within the pre-trained encoder rather than searching for architectures from scratch.

2. Miscalibration. Although ST-LoRA improves calibration over strong baselines, residual miscalibration persists; a challenge shared across all uncertainty quantification methods evaluated.

Promising directions: Post-hoc calibration methods Wang, Gong and Wang [99] are a low-cost remedy that leave the trained model intact. More specifically, we propose for future work training with $\alpha=r$ and subsequently fine-tuning $\alpha$ as a post-hoc calibration knob could simultaneously reduce training complexity and improve deployment-time calibration.

3. Data, task, and model dependency. LoRA adaptation remains sensitive to the specific model, dataset, and task, meaning conclusions from one setting cannot be assumed to transfer directly to another; a known challenge across the LoRA literature.

Promising directions: Extending ablation studies to account for hyperparameter interactions across more diverse tasks would yield more transferable guidelines. Additionally, theoretical analysis of LoRA in the vision domain Xu, Min, MacDonald, Luo, Tarmoun, Mallada and Vidal (2025); Kim, Kim and Ryu [56] remains significantly underdeveloped relative to the text domain and represents an important open research direction.

4. Deployment efficiency. Despite its parameter efficiency, The frozen backbone continues to dominate the overall computation, while the adapters introduce additional FLOPs and sometimes training speed reduction. Furthermore, maintaining multiple adapter sets in memory remains a practical bottleneck, particularly on memory-constrained edge devices.

Promising directions: Post-training pruning Cheng, Zhang and Shi [18] of LoRA modules Zhang, Chen, Shen, Yang, Ou, Yu and Zhuang [111] can reduce adapter size further, while quantization Gholami, Kim, Dong, Maehoney and Keutzer [38] - as demonstrated by QLoRA Dettmers, Pagnoni, Holtzman and Zettlemoyer [23] - can substantially reduce memory footprint for edge deployment.

Ethical Implications

The computational efficiency of ST-LoRA carries implications that extend beyond benchmark performance. By restricting fine-tuning to less model parameters and eliminating the need for $M$ independent training runs, ST-LoRA reduces GPU hours compared to the deep ensemble required to train and deploy uncertainty-aware segmentation models. This reduction in compute demand has a direct environmental consequence: lower energy consumption and, by extension, lower CO$_2$ emissions and water usage during both training and inference. As large-scale deep learning increasingly contributes to the carbon footprint of the technology sector Fernandez, Na, Tiwari, Bisk, Luccioni and Strubell [34], parameter-efficient methods such as ST-LoRA represent a step toward more environmentally responsible machine learning practice.

We believe that the research community has a responsibility to account for these downstream consequences when designing and reporting computational methods, and that efficiency is not only a technical virtue but an ethical one. In addition, these efficiency gains align with broader global sustainability objectives, particularly the United Nations Sustainable Development Goals (SDGs), including SDG 2: Zero Hunger. Climate change remains a critical factor affecting agricultural productivity, land usability, and food security worldwide. By reducing energy consumption and associated emissions, more efficient machine learning methods may contribute, in aggregate, to mitigating climate-related risks that threaten food systems. While the impact of any single methodological improvement is necessarily limited, such efforts collectively support a transition toward more sustainable technological practices that are better aligned with long-term environmental stability and global food security.

Method	Uncertainty Evaluation: OoD	Uncertainty Evaluation: Dist. Shift	Ablation Study: Rank Only	Ablation Study: Layer Selection	Component: Attn. Only	Component: FF Layers	Domain: Generic	Domain: Agri.
Halbheer et al. [42]	✓	✗	✓	✗	✓	✗	✓	✗
Balabanov and Linander [8]	✗	✗	✗	✗	✓	✗	✓	✗
Paul et al. (2024)	✗	✗	✗	✗	✓	✗	✓	✗
Onal, Flöge, Caldwell, Sheverdin and Fortuin [79]	✗	✗	✗	✗	✓	✗	✓	✗
ST-LoRA (Ours)	✓	✓	✓	✓	✓	✓	✗	✓

Conclusion

We introduced ST-LoRA, a parameter-efficient ensemble framework for uncertainty-aware semantic segmentation that combines Low-Rank Adaptation with snapshot ensemble. By sharing a frozen backbone across ensemble members and restricting each member to lightweight low-rank adapters, ST-LoRA reduces training time by $1/M$, trainable parameters, and checkpoint size; while demonstrating competitive performance compared to full-rank ensembles in segmentation quality, calibration, robustness under covariate shift, and far/near OoD performance.

Two findings stand out from our ablation study with one that has been proved empirically with support from literature while the second is left for a future work. We have shown that feed-forward layers are the critical LoRA target for dense prediction, consistently outperforming attention-only adaptation in both accuracy and calibration - contrary to the convention inherited from language model fine-tuning.

For future work, The scaling ratio $\delta_r = \alpha/r$ governs the accuracy/calibration trade-off under distribution shift and could serve as a training-free deployment-time calibration knob, allowing practitioners to adjust the model’s uncertainty behaviour after fine-tuning without retraining. This property has no equivalent in full-rank fine-tuning.

Across two crops, three architectures, and three evaluation schemes, ST-LoRA emerges as a competitive approach compared to Snapshot Ensemble, MC Dropout, and DDU in calibration stability under distribution shift, while achieving strong performance in image- and pixel-level near/far out-of-distribution detection. Its lowest cross-seed variance among all methods confirms robust, seed-independent performance.

More broadly, the efficiency gains of ST-LoRA reduce the computational and environmental cost of deploying uncertainty-aware models in agriculture; a domain where the sustainability of the technology should align with the sustainability goals it serves.

Acknowledgements

This work has partially been funded by the Deutsche Forschungsgemeinschaft (DFG, German Research Foundation) under Germany’s Excellence Strategy, EXC-2070 - 390732324 - PhenoRob. Further, it has been funded by the Deutsche Forschungsgemeinschaft (DFG, German Research Foundation) - RO 4839/6-1 - 459376902 - AID4Crops. The authors gratefully acknowledge the access to the Marvin cluster of the University of Bonn.

CRediT authorship contribution statement

Mohamed Farag: Conceptualization, Methodology, Formal analysis, Ablation-Study ,Software, Validation, Investigation, Writing - original draft, Writing - review & editing, Visualization. Genc Hoxha: Methodology, Investigation, Formal analysis, Visualization, Validation, Writing-review & editing. Yahya Maleki: Ablation-study, Software, Writing. Chris McCool: Methodology, Investigation, Writing - review & editing, Visualization. Ribana Roscher: Methodology, Investigation, Writing - review & editing, Visualization, Supervision, Project administration, Funding acquisition.

Complementary Analysis

BUP20 Dataset Per Class Analysis

We evaluate per-class mIoU and calibration performance³³ to determine whether ST-LoRA’s improvements are concentrated in majority classes - background and green peppers - or distributed consistently across all classes. As shown in Table 13, FRE achieves the best overall per-class performance, followed by ST-LoRA, while MC Dropout and DDU degrade performance relative to the uncalibrated model. Most post-hoc calibration methods leave per-class mIoU largely unchanged, as calibration is expected to preserve the predicted class while smoothing the output distribution; Dirichlet, Meta-Cal, and selective scaling are exceptions, each degrading performance.

For per-class calibration (Table 14), FRE again achieves the best performance on both NLL and Brier score, followed by ST-LoRA, with LTS ranking third and showing strong in-distribution calibration gains. These results confirm that ST-LoRA remains highly competitive with both FRE and the post-hoc calibration methods at the per-class level, indicating that its improvements are not limited to majority classes.

Number of Ensemble Members

We ablate across the number of ensemble members for FRE and ST-LoRA to understand how the number of members affect the performance of both approaches. We keep the same checkpoint (shots) sampling as it is with a shot every 20 epochs and we increase the number of the training epochs to 200. All of the training settings remain the same³⁴. As shown in Figure 15, with increasing the number of the ensemble members the performance improves for both approaches with a gap between FRE and ST-LoRA. However, with increasing the number of members beyond 5 we notice that on mIoU the change in performance is minimal while the calibration metrics keep improving with a linear trend.

Furthermore, in this paper we trim the first member as it represents the training region where the model has not converged or in other words the member at the first cycle is underfitting. Hence we want to check the effect of trimming the first three cycles/members from the ensemble on the performance. For mIoU, the performance of both methods degrades while using only the last seven shots, however we can see the effect is more obvious in the case of FRE compared to ST-LoRA. On calibrations metrics, the gains are more for ST-LoRA compared to FRE while the performance improves slightly for FRE relative to the case where we use the 10 members, the performance for ST-LoRA improves significantly with lower standard deviation compared to FRE on NLL while higher standard deviation on ECE compared to FRE.

Metric	| Foreground (gt>0) |			| All valid pixels |			| Macro (mean over classes) |
	FRE	ST-LoRA	Δ	FRE	ST-LoRA	Δ	FRE	ST-LoRA	Δ
Pairwise argmax Disag.	0.0889 ± 0.0024	0.0685 ± 0.0047	−0.0204	0.0063 ± 0.0001	0.0050 ± 0.0002	−0.0013	0.1059 ± 0.0032	0.0881 ± 0.0131	−0.0178
KL Symm.	0.2598 ± 0.0079	0.1774 ± 0.0277	−0.0824	0.0184 ± 0.0007	0.0175 ± 0.0012	−0.0009	0.3501 ± 0.0175	0.2799 ± 0.0558	−0.0702
Mutual Info.	0.0561 ± 0.0012	0.0479 ± 0.0072	−0.0082	0.0041 ± 0.0001	0.0050 ± 0.0003	+0.0009	0.0745 ± 0.0018	0.0757 ± 0.0143	+0.0011
Fraction of Pixels with Disag.	0.1611 ± 0.0047	0.1242 ± 0.0079	−0.0369	0.0115 ± 0.0001	0.0091 ± 0.0003	−0.0023	0.1901 ± 0.0479	0.1598 ± 0.0230	−0.0302
Expected Entropy	0.1744 ± 0.0101	0.2633 ± 0.0181	+0.0889	0.0192 ± 0.0006	0.0424 ± 0.0031	+0.0232	0.1847 ± 0.0172	0.3197 ± 0.0272	+0.1350
Shannon Entropy	0.2304 ± 0.0093	0.3111 ± 0.0243	+0.0807	0.0233 ± 0.0006	0.0474 ± 0.0031	+0.0241	0.2954 ± 0.0166	0.3954 ± 0.3833	+0.1361

Metric	| Foreground (gt>0) |			| All valid pixels |			| Macro (mean over classes) |
	FRE	ST-LoRA	Δ	FRE	ST-LoRA	Δ	FRE	ST-LoRA	Δ
Pairwise argmax Disag.	0.1355 ± 0.0171	0.1021 ± 0.0059	−0.0334	0.0091 ± 0.0009	0.0076 ± 0.0006	−0.0015	0.1876 ± 0.0400	0.1394 ± 0.0130	−0.0482
KL Symm.	0.2557 ± 0.0532	0.2699 ± 0.0921	+0.0143	0.0264 ± 0.0109	0.0238 ± 0.0048	−0.0025	0.2846 ± 0.0400	0.3398 ± 0.0196	+0.0552
Mutual Info.	0.0744 ± 0.0144	0.0791 ± 0.0238	+0.0048	0.0077 ± 0.0030	0.0072 ± 0.0014	−0.0005	0.0860 ± 0.0119	0.1020 ± 0.0239	+0.0159
Fraction of Pixels with Disag.	0.2444 ± 0.0330	0.1845 ± 0.0094	−0.0599	0.0164 ± 0.0018	0.0139 ± 0.0012	−0.0025	0.3350 ± 0.0742	0.2520 ± 0.0234	−0.0830
Expected Entropy	0.4400 ± 0.0892	0.4463 ± 0.0290	+0.0062	0.0763 ± 0.0160	0.0690 ± 0.0095	−0.0073	0.5243 ± 0.0637	0.5069 ± 0.0132	−0.0174
Shannon Entropy	0.5144 ± 0.1024	0.5254 ± 0.0520	+0.0110	0.0840 ± 0.0187	0.0762 ± 0.0104	−0.0078	0.6103 ± 0.0565	0.6089 ± 0.0490	−0.0015

Diversity Analysis

To compare the diversity between FRE and ST-LoRA³⁵ members—and thereby understand how ensembling low-rank matrices differs from full-rank ensembling—we isolate two specific effects: (1) the learning rate scheduler (cosine annealing) and (2) the augmentation pipeline. We quantify disagreement using pairwise hard-label disagreement, symmetric Kullback-Leibler (KL) divergence - Jeffreys divergence, disagreement on fraction on all valid pixels, and mutual information. Furthermore, we report expected entropy (aleatoric component) and total entropy (total uncertainty) to gain insight into how members estimate these two components under different configurations.³⁶

Table 15 compares the disagreement between FRE and ST-LoRA under three experimental conditions: (a) when both the scheduler and augmentation are absent, (b) when only augmentation is absent, and (c) when both augmentation and the scheduler are present. The comparison is evaluated across three scopes: foreground pixels only, all pixels, and macro averaging.

Metric	| Foreground (gt>0) |			| All valid pixels |			| Macro (mean over classes) |
	FRE	ST-LoRA	Δ	FRE	ST-LoRA	Δ	FRE	ST-LoRA	Δ
Pairwise argmax Disag.	0.1176 ± 0.0072	0.0677 ± 0.0088	−0.0499	0.0080 ± 0.0005	0.0049 ± 0.0005	−0.0031	0.1467 ± 0.0142	0.0887 ± 0.0147	−0.0579
KL Symm.	0.3755 ± 0.0307	0.1308 ± 0.0178	−0.2447	0.0245 ± 0.0020	0.0113 ± 0.0014	−0.0132	0.4670 ± 0.0654	0.1988 ± 0.0286	−0.2682
Mutual Info.	0.0857 ± 0.0077	0.0370 ± 0.0045	−0.0487	0.0058 ± 0.0005	0.0033 ± 0.0004	−0.0024	0.1087 ± 0.0126	0.0560 ± 0.0071	−0.0527
Fraction of Pixels with Disag.	0.2117 ± 0.0145	0.1225 ± 0.0163	−0.0892	0.0144 ± 0.0010	0.0088 ± 0.0009	−0.0056	0.2607 ± 0.0256	0.1614 ± 0.0276	−0.0993
Expected Entropy	0.2894 ± 0.0367	0.2675 ± 0.0111	−0.0218	0.0263 ± 0.0028	0.0378 ± 0.0054	+0.0115	0.3235 ± 0.0435	0.2930 ± 0.0090	−0.0305
Shannon Entropy	0.3751 ± 0.0403	0.3046 ± 0.0135	−0.0705	0.0320 ± 0.0030	0.0411 ± 0.0056	+0.0091	0.4322 ± 0.0443	0.3490 ± 0.0103	−0.0831

For condition (a), across all diversity metrics, ST-LoRA consistently exhibits lower diversity than FRE. We hypothesize that this reduced diversity stems from the absence of the scheduler; since the underlying models already occupy a proper position in the hypothesis space, a constant learning rate limits diversity, particularly given the constrained low-rank space explored for fine-tuning. In contrast, FRE operates in a full-rank space, which enlarges the hypothesis space and thus increases the likelihood of obtaining more diverse ensemble members.

When the scheduler is introduced, the effect differs across the two methods: for ST-LoRA, we observe a clear increase in diversity compared to the scheduler-free case, whereas for FRE, diversity decreases. We hypothesize that this reduction for FRE is not a failure of the scheduler itself, but rather a consequence of the specific learning rate values chosen for the cycles. Given the expanded search space associated with full-rank fine-tuning, we suspect that higher learning rates than those used for ST-LoRA may be required to achieve comparable diversity³⁷. We leave a systematic investigation of this hypothesis to future work.

When augmentation is introduced, we observe a clear boost in diversity for both methods, which further underscores the importance of heterogeneous ensemble members over homogeneous ones. Furthermore, ST-LoRA’s diversity increases and approaches that of the full-rank ensemble; in particular, mutual information among ST-LoRA members is higher than for FRE, which may help explain its strong performance on OoD tasks.

Finally, we measure the mutual information per class in Table 16 and observe that, while both approaches agree on the major classes, the mutual information is higher for ST-LoRA across almost all remaining classes. This indicates higher disagreement among LoRA members compared to FRE members. Collectively, these findings underscore the critical roles of both the scheduler and augmentation in achieving effective diversity.

Class	FRE	ST-LoRA	$\Delta$
Background	0.0047 $\pm$ 0.0026	0.0040 $\pm$ 0.0009	$-0.0007$
Pepper red	0.0928 $\pm$ 0.0345	0.1272 $\pm$ 0.0286	+0.0344
Pepper yellow	0.1111 $\pm$ 0.0373	0.1230 $\pm$ 0.0431	+0.0119
Pepper green	0.0574 $\pm$ 0.0274	0.0530 $\pm$ 0.0233	$-0.0044$
Pepper mixed-red	0.1462 $\pm$ 0.0250	0.1633 $\pm$ 0.0419	+0.0171
Pepper mixed-yellow	0.1039 $\pm$ 0.0251	0.1413 $\pm$ 0.0390	+0.0373

Qualitative Results

Success and Failure Scenarios. We present two qualitative cases the first represents ID analysis and OoD analysis where for both of the we show success and failure cases for our approach. As shown in Figure 16 and according to the per class analysis our approach is segmenting green peppers correctly and confidently compared to the mixed red class which is second worst class the approach is performing on.

Figure 17 shows that for near OoD tomato dataset the mutual information which is used to quantify epistemic uncertainty is useful to localize the regions that could be informative for the model for the case of Active Learning (AL) Li, Wang, Chen, Jiang, Ding and Okumura (2024) to enhance its performance on the new environment, compared to the cauliflower plants case where the mutual information and entropy fail to localize the plant.

Aleatoric and Epistemic Decomposition. Figure 19 reports the three uncertainty components on the in-domain test set and on the near- and far-OoD datasets.³⁸ ST-LoRA exhibits consistently higher aleatoric and total predictive uncertainty across all three domains, with a median increase in the aleatoric term of +0.028, +0.046 and +0.147 nats on BUP20, BUTom21 and GrowliFlower-L. The epistemic term behaves differently. In-domain, ST-LoRA members disagree slightly less than FRE members ($-0.0007$ nats), and this remains the case on near-OoD data ($-0.0068$ nats) although the gap narrows. On the far-OoD GrowliFlower-L set the ordering reverses: the disagreement between FRE members collapses, while ST-LoRA maintains a monotonic increase in mutual information with shift severity against FRE. Since only ST-LoRA’s epistemic estimate tracks the severity of the shift, this could offer a direct explanation for its stronger image-level OoD detection performance. The same asymmetry holds for the total uncertainty, which increases only marginally for FRE as the shift grows but rises sharply for ST-LoRA as both components contribute.

We attribute the aleatoric gap to a difference in how the two adaptation schemes displace the pretrained representation rather than to a difference in fitted data noise. Under the entropy decomposition, the aleatoric term is the expected entropy of the individual members, which is a property of the fitted models and not of the underlying data. Full fine-tuning is free to move all members in the same direction away from the pretrained solution, so on inputs far from the training distribution they inherit the same distorted features and remain jointly confident; the low-rank constraint keeps each ST-LoRA member closer to the pretrained representation, where differences between members are still expressible on unfamiliar inputs.

Figure 18 shows how these differences appear in the pixel-wise maps. For Frame 11 of the BUP20 test set, the ST-LoRA aleatoric map assigns elevated uncertainty over broader regions than FRE, while the epistemic maps of the two methods are comparable, yet ST-LoRA highlights more regions.

LoRA Hyperparameters

We ablate four additional hyperparameter configurations in this section, split into two parts: (A) rank and scaling ablations for SegFormer-B2 and EoMT, with SegFormer’s decoder also adapted via LoRA; and (B) remaining hyperparameter ablations for SegFormer-B2 only.

SegFormer and EoMT rank/$\alpha$ Ablation

In Experiment 2, we used a heterogeneous setup between Mask2Former and SegFormer to assess whether LoRA adaptation could be performed using arbitrary hyperparameters without prior knowledge, comparing Mask2Former with an adapted decoder against SegFormer with a fully fine-tuned decoder across two datasets.

We extend this analysis by unifying the decoder adaptation strategy -applying LoRA to the decoder for both architectures- swhile also incorporating the EoMT architecture and restricting the evaluation to a single dataset (BUP20). We ablate rank ($r$) and scaling ($\alpha$) under a consistent experimental protocol: a baseline of $r=8$ and $\alpha=8$ is established, and each hyperparameter is varied independently while the other remains fixed. In all cases, the adapted modules include the feed-forward layers and attention mechanisms in the encoder.

For the rank ablation, as shown in Table 17, Table 18, and Table 19, similar patterns emerge for SegFormer-B2 and Mask2Former. Both models achieve mIoU greater than 65% with a rank as low as 2, and performance improves as the rank increases. In contrast, EoMT exhibits stable performance that remains largely unaffected by rank variation. Regarding calibration, SegFormer-B2 benefits from lower ranks (2–4), whereas EoMT and Mask2Former show better calibration with higher ranks (8–16).

Category	Setting	mIoU ↑	ECE ↓	ACE ↓	Brier ↓	NLL ↓
LoRA rank – $r$ ($\alpha$=8)	2	0.6757 ± 0.0020	0.0030 ± 0.0005	0.0041 ± 0.0005	0.0188 ± 0.0002	0.0407 ± 0.0015
LoRA rank – $r$ ($\alpha$=8)	4	0.6780 ± 0.0016	0.0029 ± 0.0013	0.0034 ± 0.0007	0.0185 ± 0.0001	0.0413 ± 0.0021
LoRA rank – $r$ ($\alpha$=8)	8 (baseline)	0.6753 ± 0.0024	0.0032 ± 0.0007	0.0041 ± 0.0005	0.0185 ± 0.0004	0.0403 ± 0.0006
LoRA rank – $r$ ($\alpha$=8)	16	0.6766 ± 0.0024	0.0027 ± 0.0010	0.0038 ± 0.0008	0.0185 ± 0.0003	0.0404 ± 0.0013
LoRA rank – $r$ ($\alpha$=8)	32	0.6762 ± 0.0039	0.0032 ± 0.0006	0.0040 ± 0.0002	0.0187 ± 0.0004	0.0411 ± 0.0010
LoRA rank – $r$ ($\alpha$=8)	64	0.6750 ± 0.0032	0.0030 ± 0.0006	0.0039 ± 0.0005	0.0187 ± 0.0005	0.0416 ± 0.0017
LoRA scaling – $\alpha$ ($r$=8)	2	0.6694 ± 0.0016	0.0048 ± 0.0031	0.0050 ± 0.0012	0.0192 ± 0.0003	0.0442 ± 0.0030
LoRA scaling – $\alpha$ ($r$=8)	4	0.6710 ± 0.0051	0.0023 ± 0.0004	0.0038 ± 0.0008	0.0188 ± 0.0003	0.0418 ± 0.0011
LoRA scaling – $\alpha$ ($r$=8)	8 (baseline)	0.6753 ± 0.0024	0.0032 ± 0.0007	0.0041 ± 0.0005	0.0185 ± 0.0004	0.0403 ± 0.0006
LoRA scaling – $\alpha$ ($r$=8)	16	0.6813 ± 0.0034	0.0029 ± 0.0012	0.0033 ± 0.0010	0.0180 ± 0.0004	0.0389 ± 0.0010
LoRA scaling – $\alpha$ ($r$=8)	32	0.6835 ± 0.0034	0.0039 ± 0.0007	0.0040 ± 0.0005	0.0177 ± 0.0005	0.0384 ± 0.0014
LoRA scaling – $\alpha$ ($r$=8)	64	0.6882 ± 0.0051	0.0033 ± 0.0007	0.0034 ± 0.0006	0.0172 ± 0.0005	0.0366 ± 0.0012

Category	Setting	mIoU ↑	ECE ↓	ACE ↓	MECE ↓	MACE ↓	Brier ↓	NLL ↓
LoRA rank – $r$ ($\alpha$=8)	2	0.6678 ± 0.0071	0.0025 ± 0.0009	0.0030 ± 0.0006	0.0477 ± 0.0105	0.0243 ± 0.0053	0.0190 ± 0.0005	0.0411 ± 0.0016
LoRA rank – $r$ ($\alpha$=8)	4	0.6699 ± 0.0035	0.0022 ± 0.0007	0.0030 ± 0.0006	0.0658 ± 0.0307	0.0243 ± 0.0053	0.0185 ± 0.0003	0.0398 ± 0.0008
LoRA rank – $r$ ($\alpha$=8)	8 (baseline)	0.6731 ± 0.0043	0.0023 ± 0.0004	0.0036 ± 0.0004	0.0800 ± 0.0670	0.0289 ± 0.0044	0.0186 ± 0.0004	0.0421 ± 0.0013
LoRA rank – $r$ ($\alpha$=8)	16	0.6762 ± 0.0037	0.0025 ± 0.0005	0.0039 ± 0.0004	0.0653 ± 0.0193	0.0317 ± 0.0041	0.0186 ± 0.0002	0.0422 ± 0.0010
LoRA rank – $r$ ($\alpha$=8)	32	0.6752 ± 0.0078	0.0025 ± 0.0007	0.0038 ± 0.0005	0.0639 ± 0.0227	0.0309 ± 0.0052	0.0185 ± 0.0005	0.0425 ± 0.0011
LoRA rank – $r$ ($\alpha$=8)	64	0.6738 ± 0.0036	0.0026 ± 0.0003	0.0037 ± 0.0001	0.0836 ± 0.0604	0.0293 ± 0.0017	0.0186 ± 0.0002	0.0417 ± 0.0010
LoRA scaling – $\alpha$ ($r$=8)	2	0.6597 ± 0.0045	0.0026 ± 0.0005	0.0033 ± 0.0004	0.0604 ± 0.0261	0.0261 ± 0.0040	0.0193 ± 0.0004	0.0415 ± 0.0013
LoRA scaling – $\alpha$ ($r$=8)	4	0.6658 ± 0.0037	0.0028 ± 0.0002	0.0037 ± 0.0004	0.0804 ± 0.0574	0.0306 ± 0.0044	0.0190 ± 0.0002	0.0417 ± 0.0007
LoRA scaling – $\alpha$ ($r$=8)	8 (baseline)	0.6731 ± 0.0043	0.0023 ± 0.0004	0.0036 ± 0.0004	0.0800 ± 0.0670	0.0289 ± 0.0044	0.0186 ± 0.0004	0.0421 ± 0.0013
LoRA scaling – $\alpha$ ($r$=8)	16	0.6808 ± 0.0081	0.0020 ± 0.0003	0.0033 ± 0.0003	0.0503 ± 0.0093	0.0264 ± 0.0031	0.0182 ± 0.0004	0.0410 ± 0.0012
LoRA scaling – $\alpha$ ($r$=8)	32	0.6668 ± 0.0373	0.0125 ± 0.0155	0.0097 ± 0.0095	0.1112 ± 0.0778	0.0500 ± 0.0402	0.0202 ± 0.0043	0.0469 ± 0.0112
LoRA scaling – $\alpha$ ($r$=8)	64	0.0166 ± 0.0371	0.0336 ± 0.0459	0.0215 ± 0.0194	0.1950 ± 0.2773	0.0778 ± 0.1036	0.0747 ± 0.0125	0.1813 ± 0.0441

For the $\alpha$ ablation, increasing the scaling value generally improves mIoU; SegFormer-B2 and Mask2Former perform best around $\alpha= 2r$, while EoMT continues to improve beyond this range. In terms of calibration, larger $\alpha$ values lead to better calibration across all three models. However, we observe a collapse for SegFormer-B2 at $\alpha= 64$, whereas Mask2Former and EoMT remain stable and exhibit similar performance without such degradation.³⁹

ST-LoRA Initializer

We also evaluate the impact of different ST-LoRA weight initialization strategies, including EVA, OST-LoRA, and PISSA Paischer, Hauzenberger, Schmied, Alkin, Deisenroth and Hochreiter [81]; Büyükakyüz [13]; Meng, Wang and Zhang [71]. As shown in Table 19, introducing

these specialized initializers does not lead to consistent performance improvements. The baseline configuration without any dedicated ST-LoRA initializer achieves the best overall performance, with a mean IoU of 0.8178 ± 0.0295, outperforming EVA (0.7795 ± 0.0455), OLoRA (0.7872 ± 0.0850), and PISSA (0.8097 ± 0.0172).

While some initializers yield competitive results, without specialized intializer option provide a clear advantage over the default initialization. This observation suggests that, for the considered segmentation task, explicit ST-LoRA initialization strategies may not be necessary. However, further experiments on different datasets and architectures would be required to confirm whether this behavior generalizes beyond the current setting.

Bias Setting

We further investigate the impact of different bias training strategies, including fine-tuning all biases (Backbone and ST-LoRA), fine-tuning only ST-LoRA biases, and freezing all biases. The results indicate that the performance gap between the None and ST-LoRA-only bias settings is negligible. In contrast, fine-tuning all biases leads to a substantial degradation in performance, resulting in a mean IoU of $0.4348 \pm0.3574$ as shown in Table 19.

This significant drop in performance may be related to the fact that the backbone has been pre-trained and already encodes useful representations through transfer learning. Allowing the backbone biases to be updated can perturb these learned representations, particularly given the depth of the network. Properly adapting such pre-trained components typically requires larger training datasets and longer training to avoid destabilizing previously learned features. Under the current experimental setup, where data and training duration are limited, fine-tuning all biases appears to introduce instability rather than improving adaptation.

Batch Size and Learning Rate

While varying the batch size does not appear to have a noticeable effect on performance, the learning rate plays a crucial role in model behavior and stability.

Ablation on the learning rate reveals two extreme regimes. When the learning rate is set to a large value ($2 \times10^{-1}$), the model completely fails to learn meaningful representations as shown in Table 19, resulting in a mean IoU of $0.0$ and a False Negative Rate of $1.0$. This indicates that the model collapses to predicting only the background class. Such behavior is expected, as an excessively high learning rate can destabilize optimization, causing the model to overshoot minima and converge to degenerate solutions. Specifically in Biderman et al. [10] they mention that LoRA adapters need smaller learning rate compared to full rank fine tuning.

At the other extreme, smaller learning rates such as $2 \times10^{-3}$ and $2 \times10^{-4}$ yield the highest mean IoU scores in our experiments ($0.8860 \pm0.0018$ and $0.8802 \pm0.0023$, respectively). Although these results might initially suggest that very small learning rates are optimal. Very small learning rates might limit ST-LoRA diversity as we expect that with small learning rate the approach lose an aspect of diversification which leads to a degenerate solution reached by different checkpoints.

LoRA Dropout and Frozen Encoder/Decoder

As shown in Table 20, fine tuning the dropout value enhances the performance. The baseline with $p = 0.10$ achieves mIoU of 68.09% while the best result achieved is 69.0% with $p = 0.20$. However the calibration metrics for the former dropout value is smaller compared to the later one, which reflects the tension between optimizing for segmentation quality versus miscalibration reduction. Table 20 shows that with a frozen encoder and increasing the rank monotonically, mIoU and calibration metrics have shown the worst performance compared to the case where only one component from the encoder to be adapted by LoRA.

For Decoder, we tried to compare the role of fine tuning the decoder compared to having it frozen during encoder fine-tuning. Table 22 shows a clear trend, for nearly all cases the frozen decoder affects not only the segmentation quality but degrades the calibration heavily.

Complementary Information: Experimental Setup Settings

Hardware Settings and Reproducibility. All experiments are conducted on NVIDIA A100-PCIE-80GB GPUs from the Marvin cluster, using up to four GPUs per node for training and a single GPU for inference. RAM allocation varies between 128 GB and 512 GB depending on dataset and image size. All results are reported across five random seeds (42, 123, 456, 789, 1337) for statistical significance.

Uncertainty Methods Settings For snapshot-based methods (ST-LoRA and FRE), we retain the last four checkpoints of each training trajectory, discarding the first snapshot as the model is still in an early underfitting phase at that point and including it would distort the ensemble evaluation. For DDU, we fit the GMM on features extracted from the last layer of the pixel decoder, using per-class pixel frequency in the training set as a prior.

All methods share the same augmentation policy to ensure a fair comparison. For both FRE and ST-LoRA, ensemble diversity is not enforced through fixed, member-specific augmentation schemes; instead, we apply highly randomized augmentation throughout the entire training trajectory, with most augmentations set to a probability close to 0.5. Diversity across members therefore emerges naturally from this stochasticity rather than from an explicitly assigned per-member policy.

Post-hoc Calibration Methods Settings. We keep the trained Mask2Former completely frozen and calibrate its outputs. For each image, the model emits two heads -class query logits and mask query logits- which, by architectural design, collapse into a per-pixel semantic probability map. We then drop the no-object column, apply softmax to the class logits and sigmoid to the mask logits, and combine them via element-wise multiplication. Finally, we take the logarithm of the combined output to obtain per-pixel logits, which reside in a well-behaved logit space suitable for our six post-hoc calibrators.

Each calibrator is fit on a small, fixed held-out calibration split -specifically, a deterministic subset of 30 samples carved from the validation set, while the remaining 33 validation samples and the full 93 test samples remain untouched for tuning. Fitting is performed by minimizing pixel-wise NLL and cross-entropy against the ground-truth labels using AdamW Loshchilov and Hutter (2019) for approximately 40 epochs, with the backbone kept frozen. Calibration is conducted at a low resolution (320x180),⁴⁰ and the learned transform is subsequently applied at native resolution (1270x720). At evaluation, we simply pass the frozen model’s logits through the fitted calibrator, apply softmax, and recompute mIoU and accuracy alongside the calibration metrics (ECE, ACE, MECE, MACE, Brier, and NLL).

We acknowledge that the validation split used for calibrating the post-hoc methods differs from the validation set used for the four uncertainty baselines and the main experiments. However, we believe this does not compromise the fairness of our comparison, as we adopt the same training set and hyperparameters throughout. Crucially, we do not select model weights based on validation loss or accuracy; instead, we use the final checkpoint from the training trajectory, corresponding to the last snapshot. This choice effectively isolates the influence of the validation set and ensures a consistent basis for comparison. Had we instead used the checkpoint selected by best validation performance, we would have been forced to retrain all baseline methods under identical conditions to align with the post-hoc calibrators’ underlying models, which would introduce additional complexity and potential variability.

Architectures and LoRA software. We use Hugging Face’s (HF) tool PEFT (Parameter-Efficient Fine-Tuning) 40 to adapt the LoRA modules to the different architectures in our study. Mask2Former 41, and EoMT 42 architectures have been used from models library in HF. While for SegFormer we use the same model either from HF 43 or segmentation models 44.