ST-LoRA: Single Trajectory LoRA Ensemble for Uncertainty Aware Agricultural Segmentation

Highlights

• ST-LoRA builds diverse ensembles using less model parameters

• Feed-forward layers, not attention, are the most important LoRA target for dense prediction

• ST-LoRA matches full-rank ensemble quality at significantly smaller checkpoint size

• ST-LoRA achieves competitive performance for calibration robustness under distribution shift

Introduction

Digital agriculture powered by sensor data and Machine Learning (ML) offers promising tools to optimize current practices and explore innovative solutions for future food production. Deep Neural Networks (DNN), in particular, excels at processing large sensor-based datasets to uncover meaningful patterns [6], [24], [23], [1], [2], and building foundation models for enhancing the performance on various downstream tasks by using multi-modal sensory inputs [1].

As ML models become more integrated into agricultural workflows, evaluating the quality of their predictions becomes essential. Uncertainty quantification (UQ) plays a key role by indicating when predictions are trustworthy and when human oversight is needed. Unlike point estimates, UQ enables probabilistic predictions that help diagnose issues arising from the data, the model, or both. In supervised learning, models generalize from labelled data to unseen examples, an inductive process that introduces epistemic uncertainty (i.e., model uncertainty) due to limited knowledge about the underlying data-generating process. In contrast, aleatoric uncertainty (i.e., data uncertainty) stems from inherent noise and variability in the data. Unlike epistemic uncertainty which can in principle be reduced with more data or improved models, aleatoric uncertainty is irreducible, regardless of model capacity or training set size. [28]; [44].

Among existing UQ methods, Deep Ensembles [49] are widely regarded as the most commonly used method due to their strong calibration, predictive performance, and ability to decompose total uncertainty into its epistemic and aleatoric components. However, they are computationally expensive, multiplying both training time and memory footprint by the ensemble size, an increasingly critical limitation as modern architectures scale to billions of parameters and Vision Transformers [?] become the dominant backbone for dense prediction tasks. Snapshot ensembling [41] addresses the training cost by collecting $M$ checkpoints from a single cosine-annealing trajectory, reducing training time by a factor of $1/M$. However, at deployment time, the snapshot ensembling is affected by large inference latency and memory footprint due to the need of maintaining $M$ full model replicas.

The same scaling pressure that makes deep ensembles prohibitive also affects standard fine-tuning. As pretrained vision and language models grow larger, adapting them to downstream tasks becomes increasingly costly in both compute and storage. Parameter-efficient fine-tuning offers a principled solution to this bottleneck. Low-Rank Adaptation (LoRA) [40], originally developed for large language models, injects trainable low-rank matrices into the frozen layers of a pretrained backbone, dramatically reducing the number of trainable parameters and training time.

This naturally has raised the question of whether LoRA adapters can serve as ensemble members in their own right.

Recent work has begun to explore this direction for uncertainty quantification [34]; [8], freezing the shared backbone and constructing diverse members through different low-rank initializations. However, existing efforts have been limited classification tasks at image level, with narrow ablation studies that leave the effect of key hyperparameters largely unexplored. Dense prediction tasks such as semantic segmentation, which impose stricter demands on spatial representations and pixel-level calibration, remain entirely unaddressed.

To fill this gap, we introduce and explore Single Trajectory LoRA (ST-LoRA), a lightweight ensemble-based extension of LoRA ensemble method for pixel-wise uncertainty quantification that constructs the members from a shared pretrained vision backbone. Illustrated in Figure 1, our method not only outperforms the Full Rank Ensemble (FRE)⁵ on calibration and segmentation quality but also with significantly lower computational overhead.

Our main contributions are:

Contributions

• We introduce ST-LoRA, which combines LoRA adapters with snapshot ensembling to reduce training time, parameters, and inference latency while matching or outperforms full-rank ensemble’s calibration and segmentation quality.

• We verify the efficiency claims on a real edge hardware, which supports our findings and claims.

• We show through extensive ablation that feed-forward layers are important LoRA target in vision transformers for dense prediction, contrary to LLM practice.

• We provide a practitioner’s guide for configuring ST-LoRA across datasets and architectures, grounded in ablation findings on two crop types and two backbones.

• ST-LoRA achieves strong competitive performance with the low cross-seed variance across in-distribution evaluation, distribution shift, and near/far out-of-distribution detection against Snapshot Ensemble, MC Dropout, and DDU baselines.

The rest of this article is organized as follows. sec:2 reviews research gaps in uncertainty estimation for agricultural machine learning. sec:3 introduces the mathematical background and presents the proposed ST-LoRA approach. sec:4 describes the experimental setup while sec:5 shows the experimental results, organized into four evaluation settings: 1) benchmarking ST-LoRA against FRE in terms of segmentation quality, calibration, and computational efficiency (Section 5.1); 2) an ablation study on the most influential hyperparameters (Section 5.2); 3) an assessment of calibration robustness under distribution shift (Section 5.3); and 4) a comparison with state-of-the-art methods covering in-distribution performance, covariate shift, and out-of-distribution (OoD) detection (Section 5.4). sec:6 presents the final remarks and sec:7 concludes the paper. Additional analysis and implementation details are provided in the Appendix A.

Through this extensive evaluation, we demonstrate that ST-LoRA achieves competitive uncertainty quantification and segmentation quality relative to both full-rank and efficient SOTA methods, while offering substantial advantages in computational efficiency, storage footprint, and deployment flexibility.⁶

Related Work

In this section, We begin with a general overview of uncertainty in machine learning, then follow the uncertainty pipeline through its two building blocks — representation, and quantification — before reviewing the limitations of existing approaches that motivate this work.

Uncertainty in ML can be addressed from two main perspectives. One common approach assumes that predictive uncertainty can be decomposed into epistemic (model) and aleatoric (data) components, each estimated independently Shaker and Hüllermeier (2021). An alternative perspective challenges the validity of this decomposition Hüllermeier (2022), focusing instead on estimating total predictive uncertainty to ensure model reliability [5]. As noted by [32], the origin and categorization of uncertainty in ML remain ambiguous, with additional factors such as missing data or deployment in dynamic environments contributing to a more complex uncertainty landscape.

Uncertainty Evaluation. Evaluating UQ methods is non-trivial, as no ground-truth uncertainty labels exist for model predictions. Instead, evaluation relies on indirect schemes that probe whether uncertainty estimates are meaningful and actionable in practice. Following [38], four paradigms have been established: calibration under distribution shift Moreno-Torres, Raeder, Alaiz-Rodríguez, Chawla and Herrera (2012) assesses whether confidence scores remain reliable when imaging conditions change — critical in agricultural settings where lighting and weather vary; OoD detection Salehi, Mirzaei, Hendrycks, Li, Rohban and Sabokrou (2021) identifies semantically novel inputs, flagging them for human review rather than producing overconfident predictions; selective prediction Xin, Tang, Yu and Lin (2021) extends this by allowing abstention on any low-confidence sample, preferring a partial prediction over a confident wrong one; and active learning [7] guides annotation by prioritizing uncertain samples, though this is orthogonal to our deployment-oriented focus. This work evaluates the first three schemes, as they directly assess prediction reliability in real-world agricultural deployment.

Uncertainty Representation for UQ falls into two primary categories: stochastic and deterministic. Stochastic representation captures epistemic uncertainty by modeling a second-order distribution over model parameters or outputs [39]. In contrast, deterministic represents uncertainty directly through distributions over latent representations Postels, Segù, Sun, Sieber, Van Gool, Yu and Tombari (2022).⁷

Uncertainty Quantification Methods. Stochastic methods approximate Bayesian Neural Networks (BNNs), tackling the intractability of posterior inference through techniques like Monte Carlo Dropout (MC-Dropout) [27] (2015), Ensemble Learning [49] et al. (2017), and Flipout Wen, Vicol, Ba, Tran and Grosse (2018). Deterministic methods - sampling free approaches, estimate uncertainty in a single forward pass by treating model weights as fixed. Recent advances in Deterministic Uncertainty Modeling (DUMs) Mukhoti, Kirsch, van Amersfoort, Torr and Gal (2023); [4] (2020); Liu, Lin, Padhy, Tran, Bedrax-Weiss and Lakshminarayanan (2020); [9] (2024) achieve reliable estimates by regularizing latent representations for smoothness and bi-Lipschitz continuity, while avoiding feature collapse. These approaches offer lower computational overhead however, DUMs can suffer under distribution shifts and may negatively impact predictive accuracy due to the smoothness constraints imposed on the learned representations Postels, Segu, Sun, Gool, Yu and Tombari (2021). Hence more efforts focused on enhancing and exploring more efficient ensemble strategies.

Efficient Ensembles. Post-hoc ensemble methods like Checkpoint Ensembles (CE) [13] (2017) and Snapshot Ensembles (Snap-E) [41] (2017) reduce training cost by saving intermediate model states during a single run. In contrast, integrated ensemble methods, including Packed Ensembles (PE) Laurent, Lafage, Tartaglione, Daniel, Martinez, Bursuc and Franchi, Multi-Input Multi-Output (MIMO) Havasi, Jenatton, Fort, Liu, Snoek, Lakshminarayanan, Dai and Tran, and Batch Ensembles (BE) Wen, Tran and Ba, embed ensembling into the architecture itself. For instance, PE uses group convolutions for efficient computation, MIMO extracts sub-networks from a shared backbone, and BE represents weights using a shared matrix modulated by rank-one factors. Finally, MC-Dropout provides a lightweight stochastic alternative by activating dropout at inference time.

While these methods improve the efficiency of uncertainty estimation, several challenges remain. Post-hoc ensembles reduce training time but do not lower inference cost or memory usage, as each ensemble member must be evaluated independently. Integrated ensemble approaches often require architectural modifications, which can be complex and potentially degrade the base model’s performance; for instance, the group convolutions used in Packed Ensembles (PE) are incompatible with Vision Transformer (ViT) architectures Dosovitskiy, Beyer, Kolesnikov, Weissenborn, Zhai, Unterthiner, Dehghani, Minderer, Heigold, Gelly et al.. MC-Dropout tends to be miscalibrated on in-distribution data and overly confident on out-of-distribution inputs, limiting its applicability in safety-critical settings [?] et al. (2017).

In the text domain, LoRA [40] et al. has been particularly impactful, enabling fine-tuning on downstream tasks at a fraction of the computational cost of full fine-tuning. This efficiency has motivated its extension to ensemble construction: rather than training $M$ independent full models, LoRA adapters can serve as lightweight ensemble members over a shared frozen backbone, preserving ensemble diversity.

Closely related to our work, Paul, Quercia, Fortuin, Nöh and Scharr (2024) explored adapter-based techniques combined with Bayesian inference for uncertainty estimation in monocular depth estimation, while Onal, Flöge, Caldwell, Sheverdin and Fortuin applied a Bayesian framework with LoRA to quantify uncertainty in large language models. Recent efforts to investigate the suitability of LoRA ensembles for uncertainty estimation have been presented in [34] (2024) and [8] (2024). However, these studies share several limitations that motivate our work. First, they are restricted to image classification and Natural Language Processing (NLP) benchmarks, leaving open whether LoRA ensemble uncertainty estimation transfers to dense prediction tasks where uncertainty is a spatial field of pixel-wise distributions requiring class-aware calibration metrics, region-level OoD detection — none of which have been evaluated. Second, while [34] (2024) showed that LoRA Ensemble performs well against other uncertainty the investigation of LoRA hyperparameters has been limited which heavily affects the performance. Third, existing work retains M-fold training cost through independent member initialization, treating snapshot ensembling as a weaker alternative rather than a complementary mechanism. Finally, while prior layer placement studies compare coarse blocks such as attention versus attention with MLP, fine-grained component-level ablation remains absent, and the optimal configuration in classification settings does not transfer to vision transformer encoders used for segmentation. Finally [25] (2025) have shown that DEs and MC-Dropout are the most commonly used methods for uncertainty estimation in agricultural segmentation. This reveals a significant gap in exploring alternative methods that offer a better trade-off between computational efficiency and predictive reliability. In conclusion we observe the following gaps:

Research Gaps

• LoRA ensembles are untested in vision UQ. Existing work is confined to image classification and NLP benchmarks, with no evaluation on dense prediction tasks or agricultural datasets.

• Critical deployment properties are neglected. No prior LoRA-based UQ method evaluates calibration under distribution shift for dense prediction tasks which is essential for safe real-world deployment.

• Layer selection is an open question. All prior work applies LoRA exclusively to attention projections following LLM conventions, without investigating whether feed-forward layers are more effective in vision transformers.

• Agricultural UQ lacks efficient alternatives. Uncertainty estimation in agricultural segmentation relies solely on expensive methods [25], with no calibration-aware, resource-efficient solution available.

Methodology

In this section we begin by establishing notation, then briefly review the uncertainty representations used in our evaluation, before introducing the ST-LoRA formulation and its integration into semantic segmentation architectures.

Problem Formulation: Pixel-Wise Classification

Let $F_{\theta}: \mathbb{R}^{H \times W \times3} \to\mathbb{R}^{H \times W \times L}$ be a pre-trained segmentation model mapping an input image $X \in\mathbb{R}^{H \times W \times3}$ to pixel-wise logits over $L$ classes, where $H$ and $W$ denote image height and width. For each pixel $(i,j)$, the predicted class probability vector is obtained by applying a softmax:

where $\Delta^{L-1}$ is the $(L-1)$-dimensional probability simplex. The predicted class label is then:

The ground-truth segmentation mask $Y \in\{1,\ldots,L\}^{H \times W}$ assigns a single class label to each pixel. The model is trained by minimizing the cross-entropy loss over all pixels:

Under this formulation, uncertainty quantification amounts to estimating, for each pixel $(i,j)$, how reliable $\hat{y}_{ij}$ is — distinguishing pixels where the model is confidently correct from those where the prediction should be treated with

caution or abstained from entirely. For clarity, we adopt the following notation throughout this work. Scalars are denoted by lowercase letters (e.g., $r$, $\tau$), vectors by bold lowercase letters (e.g., $\mathbf{y}$), and matrices by bold uppercase letters (e.g., $X$). Model parameters are denoted by $\theta$, with $\theta_0$ indicating frozen pre-trained weights. Probability distribution is represented by $\mathcal{P}(\cdot)$ and Gaussian Mixture Model (GMM) [17] is represented as $\mathcal{Q}(\cdot)$, and functions or models by calligraphic letters (e.g., $\mathcal{F}$, $\mathcal{G}$, $\mathcal{H}$). Sets and spaces are denoted by blackboard bold letters (e.g., $\mathbb{R}$, $\mathbb{M}$). A summary of the main symbols is provided in Table 1.

Symbol	Description
Data and Labels
$X$	Input image
$\mathbf{y}_l$	Label for class $l$
$D$, $L$, $H$, $W$	Dataset, classes, height, width
Model and Parameters
$\theta_0$	Frozen pre-trained weights
$\Delta\theta_m$	Low-rank update for member $m$
$A_m$, $B_m$	ST-LoRA decomposition matrices
$r$, $\alpha$, $M$	Rank, scaling factor, ensemble size
$F_\theta$	Segmentation model
$\mathbb{M}_r$	Low-rank subspace
Uncertainty
TU, EU, AU	Total, epistemic, aleatoric uncertainty
$\mathcal{P}(Y \mid X)$	Predictive posterior
$\mathcal{P}(\theta_m \mid D)$	Posterior weight for member $m$
$\psi(X)$
DDU-specific
$Z$	Latent feature vector
$\mathcal{Q}(Z \mid l)$	Class-conditional GMM density
$\mu_l$, $\Sigma_l$	GMM mean and covariance
$\Pi(l)$	Class prior probability

Uncertainty Representation and Quantification

In this work, we consider two uncertainty representation frameworks: 1) stochastic representation (e.g. second-Order Distributions) which explicitly model uncertainty over predictions and 2) latent space distributions which represent uncertainty directly through distributions over latent space (i.e., feature space). We briefly describe their mathematical formulation in the following subsections.

Second-Order Distributions

Stochastic methods represent epistemic uncertainty by modeling a distribution over model parameters. The total predictive uncertainty (TU) is decomposed as $\mathrm{TU} \approx\mathrm{AU} + \mathrm{EU}$, where AU and EU denote aleatoric and epistemic uncertainty respectively. Using Shannon entropy [38] (2025b):

where $\mathcal{P}(y_l \mid X_{\mathrm{test}})$ is the probability of class $l$ under the predictive posterior. The aleatoric component is the expected entropy across ensemble members:

and epistemic uncertainty follows as $\mathrm{EU} \approx\mathrm{TU} - \mathrm{AU}$.

Latent Space Distributions

Deterministic methods such as DDU [?] estimate uncertainty from the density of the input’s latent representation, without requiring multiple forward passes. A GMM is fitted per class on penultimate-layer features $Z = F_{\theta}(X)$:

Epistemic uncertainty is then the negative log-likelihood under the class-marginalized mixture:

where a high value indicates the sample lies far from the training manifold. Aleatoric uncertainty is estimated as the categorical entropy of the softmax output, consistent with (1).

ST-LoRA: Efficient Ensemble Learning via Low-Rank Adaptation

We start by injecting LoRA trainable low-rank matrices into frozen layers. This constrains members’ diversity to a learned task-relevant subspace, analogous to how Principal Component Analysis (PCA) [?] identifies low-dimensional structure — but unlike PCA, ST-LoRA learns this weight subspace end-to-end via gradient descent, aligning it with downstream objectives. For a linear layer with weight $\theta_0 \in\mathbb{R}^{d \times k}$, the adapted output is:

where $A \in\mathbb{R}^{r \times k}$ and $B \in\mathbb{R}^{d \times r}$ with $r \ll\min(d,k)$, reducing trainable parameters from $dk$ to $r(d+k)$.

This design is theoretically motivated by two observations. First, Aghajanyan, Gupta and Zettlemoyer [3] show that pre-trained models have a low intrinsic dimension, suggesting low-rank reparameterizations can match full-rank fine-tuning. Second, Hu et al. [40] demonstrate that $\Delta\theta$ is strongly correlated with $\theta_0$, implying the low-rank subspace retains sufficient task-relevant information for meaningful uncertainty estimation. An illustration is shown in Figure 2.

Deep Ensembles [49, ?] approximate Bayesian inference by treating independently trained models as samples from the parameter posterior $\mathcal{P}(\theta\mid D)$. The ensemble predictive posterior is:

While effective, this requires training and storing $M$ full model replicas, making it prohibitive for large vision transformers.

Hence, we use snapshot approach to construct an ensemble from a single training trajectory, reducing the effective training cost by a factor of $1/M$ compared to training $M$ independent models. A cosine annealing learning rate scheduler introduces cyclic variation in the learning rate, allowing the model to converge to distinct local minima within a single run. At the end of each cycle, the current model weights are saved as a snapshot, and the final ensemble averages predictions across all $M$ collected checkpoints [41]. Figure 4 illustrates the learning rate schedule and the corresponding snapshot collection points.

Each ensemble member $m \in\{1,\ldots,M\}$ in ST-LoRA for Semantic Segmentation is defined by a distinct adapter initialized from a different seed, yielding the uniform mixture:

from which TU, AU, and EU are computed pixel-wise using Equations (1)–(3).

Experimental Setup

Datasets and Pre-Processing

BUP20⁸ sweet pepper phenotyping dataset [?] , which contains high-resolution (1280 $\times$ 720) video frames captured by an Unmanned Ground Vehicle (UGV) in a commercial greenhouse. A subset of 279 non-overlapping frames is manually annotated for both semantic and instance segmentation across six classes: background, red, yellow, green, mixed-red, and mixed-yellow peppers. The dataset poses real-world challenges, including occlusions, lighting variation, background clutter, and class overlap, making it a strong benchmark for uncertainty-aware models in agriculture. All images are normalized using the mean [0.485, 0.456, 0.406] and the standard deviation [0.229, 0.224, 0.225], consistent with the ADE20K dataset [?] .

GrowliFlower⁹ [46] is an

open-source dataset acquired in 2020 and 2021 by flying a UAV, It contains 14K samples of cauliflower plants with multiple modalities available, including RGB and multi-spectral images. The dataset also includes a comprehensive range of annotations, making it useful for computer vision tasks such as classification and segmentation. All images are resized to $256 \times256$¹⁰ and normalized to be in range $[0, 1]$. No augmentation has been used while it helps the ensembles to get more diverse, however we isolate its effect and investigate more its effect at Section 5.3. Furthermore, we filter out samples that include only soil at the early growth stages. Samples from both datasets are shown in Figure 3.

LoRA Adaptation for Segmentation Models

In this work, we focus on Vision Transformer (ViT)–based segmentation models for two reasons. First, our aim is to study the effect of low-rank adaptation in a setting that reflects current practice, and recent advances in semantic segmentation have shifted heavily toward transformer-based architectures, such as SegFormer, DPT, MaskFormer, Mask2Former, EoMT, and SAM [?, 14, 15, ?, 48, 45]. Evaluating LoRA in this context therefore aligns our analysis with the predominant direction of modern segmentation research. Second, although LoRA can in principle be applied to convolutional networks, its formulation and most empirical validations have been developed around transformer blocks.

We adopt SegFormer and Mask2Former as our two primary models due to their distinct design philosophies. SegFormer follows a per-pixel classification paradigm, combining a ViT-based encoder with a lightweight MLP decoder. In contrast, Mask2Former frames segmentation as a mask-classification problem, using a shared transformer-based¹¹ encoder and two decoders: a convolutional decoder for pixel-wise upsampling and a transformer decoder for predicting mask embeddings. Lastly, SegFormer is a task-specific (non-universal) segmentor tailored primarily for semantic segmentation, whereas Mask2Former is a universal segmentor capable of handling semantic, instance, and panoptic segmentation within a unified framework.

To enable efficient and scalable uncertainty quantification, we integrate LoRA into selected modules of both architectures. We first choose the target components (e.g., MLP layers or Multi-Head Attention blocks) and specify a low rank (e.g., $r=8$). The remaining model parameters can either remain frozen or be partially fine-tuned alongside the LoRA adapters, depending on the configuration. During inference, only the LoRA adapter and partly fine tuned weights are loaded, enabling faster execution and reduced memory usage. This design supports the construction of lightweight ensembles with significantly lower computational cost, making the approach well suited for resource-constrained agricultural robotics.

Evaluation Schemes and Metrics

We organize our evaluation metrics into three categories aligned with the evaluation schemes described above: predictive performance, computational efficiency, and uncertainty quantification.

Predictive Performance and Computational Efficiency

We assess segmentation quality using mean Intersection over Union (mIoU). The mIoU measures the average overlap between predicted and ground-truth segments across all classes. We evaluate computational cost through multiple dimensions: (1) number of trainable parameters, and (2) FLOPs (floating-point operations per forward pass). These metrics are essential for assessing deployment feasibility on resource-constrained agricultural platforms.

Uncertainty Method Quality Assessment

Robustness under Distribution Shift. Dataset shift occurs when the joint distribution of training and test data differ, formally expressed as $P_{\mathrm{train}}(X,\mathbf{y}) \ne P_{\mathrm{test}}(X,\mathbf{y})$ [?]. Multiple shift types exist, including prior probability shift, concept shift, and covariate shift. This work focuses on covariate shift, defined as:

where the conditional relationship between inputs $X$ and labels $\mathbf{y}$ remains invariant, but the marginal input distribution shifts between training and test conditions. This scenario is particularly relevant for agricultural applications, where imaging conditions vary across seasons, weather, and lighting.

To evaluate calibration robustness, we adopt the concept of breakdown point from robust statistics [?]. The breakdown point quantifies the maximum proportion of corrupted observations an estimator can tolerate before producing arbitrarily incorrect results. In our context, we assess how calibration quality (measured by ECE) [33] computed with both $\ell_1$ and $\ell_\infty$ norms to capture average and worst-case miscalibration respectively and segmentation performance (measured by mIoU) degrade as the severity of distribution shift increases. A robust UQ method should maintain relatively stable calibration and performance across varying shift intensities.

Out of Distribution (OoD) detection An essential method for evaluating the quality of an uncertainty quantification technique is to observe its response to these OoD samples. This response reflects the model’s ignorance expressed as high uncertainty about these points.

The problem of OoD detection can be formulated as follows:

When $X_{n+1}$ is declared , the OoD detector assigns a score to the sample by using the $\psi$ which is the scoring function, if it is greater than or equals $\tau$ it is considered ID, otherwise it is rejected and classified as OoD. For scoring we can use confidence, entropy, or any function under the condition that it is able to differentiate between the classes or to rank them differently. Hence, OoD detection can be treated as a binary classification problem, the only difference is that we lack the reference values for the OoD samples. For out-of-distribution detection, we measure discrimination performance using the area under the receiver operating characteristic curve (AUCROC), which quantifies the method’s ability to separate in-distribution from out-of-distribution samples across all possible confidence thresholds.

Design rationale

Our evaluation spans three dimensions - predictive performance, uncertainty quantification, and computational efficiency - each assessed with appropriate metrics. The experimental stages are deliberately ordered so that each is motivated by the findings of the previous one, building toward a complete picture of ST-LoRA’s deployment readiness.

We begin with in-distribution performance (Section 5.1), addressing the most fundamental question: can low-rank adapters substitute full model replicas without meaningful loss in accuracy or calibration? Establishing this is a prerequisite for everything that follows; if ST-LoRA fails here, its efficiency advantages are irrelevant.

Having confirmed in-distribution reliability, we turn to the ablation study (Section 5.2), which asks how ST-LoRA should be configured. Since prior work applies LoRA exclusively to attention projections [40]; Mühlematter, Halbheer, Becker, Narnhofer, Aasen, Schindler and Turkoglu (2024), we systematically investigate target components, rank, scaling factor, and dropout to explore the hyperparameter landscape for vision transformer segmentation. All subsequent experiments use the configuration identified here.

With a validated and well-configured model, we assess its robustness under distribution shift (Section 5.3). Agricultural imaging conditions vary across seasons, weather, and lighting - a model that is well calibrated in-distribution but degrades rapidly under shift offers limited practical value. This stage examines whether ST-LoRA’s calibration advantage is stable or fragile, and identifies which hyperparameter choices confer the greatest robustness.

Finally, comparison against efficient baselines (Section 5.4) situates ST-LoRA within the broader landscape of uncertainty quantification methods, asking whether it matches or outperforms established alternatives - Snapshot Ensemble, MC Dropout, and DDU¹² - across In distribution performance, robustness under distribution shift , and out-of-distribution detection. Together, these four stages demonstrate that ST-LoRA’s uncertainty estimates are not only accurate in controlled conditions but practically reliable under the challenging and variable conditions of real agricultural deployments.

Experiments and Results

In Distribution (ID) comparative analysis

We evaluate three model architectures (SegFormer-B2, SegFormer-B4, and Mask2Former Swin-Base¹³) with three uncertainty quantification approaches (Softmax baseline, Full-Rank Ensemble, and ST-LoRA). Predictive performance is assessed via mIoU, calibration via ECE, and computational efficiency via FLOPs and parameter count. Additionally, we quantify the deployment I/O overhead of each ensemble strategy by measuring checkpoint loading latency on both GPU available by Google Colab and edge hardware (Raspberry Pi 4B) ¹⁴, comparing per-switch disk reads against in-memory adapter switching. Training settings can be shown in Table 2.

Category	Hyperparameter	Pixel wise binary classification — Baseline	Pixel wise binary classification — FRE / ST-LoRA	Pixel wise multiclass classification — Baseline / FRE, and ST-LoRA
Dataset	Name	GrowliFlower-L		BUP20
Dataset	Train	1381		124
Dataset	Validation	293		63
Dataset	Test	296		92
Pre-processing	Image size	256 × 256		1280 × 720
Pre-processing	Normalization	[0, 1]		$\mu=[0.485, 0.456, 0.406],\ \sigma=[0.229, 0.224, 0.225]$
Pre-processing	Augmentation	None		None
Model	Architecture	SegFormer (B2 / B4)		Mask2Former (Swin-B)
Model	Pre-trained weights	ImageNet		ADE20K
Model	LoRA rank	—	$r=8$	— / $r=8$
Training	Epochs	100	100	100
Training	Optimizer	Adam	Adam	Adam
Training	Learning rate	$1 \times10^{-4}$	$[2 \times10^{-4}, 2 \times10^{-2}]$	$2 \times10^{-4}$ / $[2 \times10^{-5}, 2 \times10^{-3}]$
Training	LR schedule	Constant	Cosine annealing w/ warm restarts	Constant / Cosine annealing
Training	Cycle length	—	20 epochs	20 epochs
Training	Ensemble members	—	5	— / 5
Training	Batch size	32	32	4
Training	Early stopping	Patience = 100	—	—
Training	Loss	Cross-entropy		Cross-entropy
Training	Weight decay	0.0		0.0
Training	Hardware	NVIDIA A100 (80 GB)		NVIDIA A100 (80 GB)

Table 3 reveals a consistent pattern across both datasets: ensembling reduces mIoU on GrowliFlower-L but improves it on BUP20 dataset while improving calibration on both reflected by lower ECE compared to single-model counterparts. This trade-off is expected, as ensemble averaging smooths overconfident predictions.

Model	Baseline		FRE				ST-LoRA (ours)
	mIoU(%) ↑	ECE ↓	mIoU(%) ↑	ΔmIoU(%)	ECE ↓	ΔECE(%)	mIoU(%) ↑	ΔmIoU(%)	ECE ↓	ΔECE(%)
GrowliFlower-L (Cauliflower)
SgF-B4	88.6	0.0052	83.9	−5.3	0.0039	−25.0	84.5	−4.6	0.0027	−48.1
SgF-B2	87.8	0.0061	83.9	−4.4	0.0034	−44.3	84.7	−3.5	0.0052	−14.8
BUP20 (Sweet Pepper)
M2F	67.8	0.0112	67.1	−1.0	0.0030	−73.2	68.1	+0.4	0.0043	−61.6
SgF-B2	66.1	0.0028	70.0	+6.0	0.0048	+71.4	69.3	+5	0.0021	+25

Comparing ST-LoRA directly against FRE, ST-LoRA achieves competitive performance across both datasets and architectures. On BUP20, ST-LoRA achieves the best mIoU among all methods with only a marginal ECE increase relative to FRE. On GrowliFlower-L with SegFormer-B4, ST-LoRA achieves the best ECE while also outperforming FRE in segmentation accuracy, demonstrating that low-rank adapters can effectively substitute full model replicas without meaningful performance loss.

Furthermore, we use reliability diagrams [20] to assess model calibration by plotting prediction confidence on the x-axis against empirical accuracy on the y-axis. A perfectly calibrated model follows the diagonal line $y = x$, where accuracy equals confidence at all prediction levels. Deviations below the diagonal indicate overconfidence (predictions are more confident than accurate), while deviations above indicate underconfidence. Figure 5 reveals the substantial impact of ensembling on calibration. Both the baseline and single ST-LoRA models exhibit severe overconfidence, with their reliability curves falling significantly below the diagonal across most confidence bins. Ensembling significantly improves calibration for both approaches: FRE and ST-LoRA curves shift closer to the ideal diagonal, reducing the gap between predicted confidence and actual accuracy. FRE demonstrates marginally better calibration than ST-LoRA across most confidence bins, particularly in the 0.4–0.8 range. However, FRE exhibit underconfidence in the highest confidence bin (0.8–1.0), where the curves rise above the diagonal. This suggests that when the ensembles are most correct, they are actually too conservative in their predictions. This under-confidence at high confidence levels represents a different calibration failure mode that may require targeted post-processing techniques as mentioned in [?].

Beyond predictive quality, ST-LoRA is substantially more computationally efficient than FRE, as shown in Table 4. It reduces trainable parameters, memory footprint and inference latency, while maintaining comparable FLOPs and a cost shared by both methods due to the ensemble averaging step.¹⁵

Approach	FLOPs (GMAC)	Train Params. (Million)
Sgf-B4 (base)	12.54	61.37
FRE	62.70	306.84
ST-LoRA (ours)	64.78	70.84
SgF-B2 (base)	5.34	24.72
FRE	26.64	123.61
ST-LoRA (ours)	27.64	29.85
M2F (base)	229.72	108.77
FRE	1148	534
ST-LoRA (ours)	1170	116

The latency reduction stems from the size asymmetry between LoRA adapters and full model replicas: since each adapter is significantly smaller than a full checkpoint, the I/O overhead that typically bottlenecks sequential model loading becomes negligible. To verify this, we benchmark two loading strategies for Mask2Former checkpoints under both FRE and ST-LoRA configurations, on Google colab (network-mounted filesystem) and an edge device (Raspberry Pi 4B, local ext4, 449 MB/s). The first strategy loads each checkpoint from disk before every forward pass, incurring full I/O cost per ensemble member. The second strategy preloads all checkpoints into RAM once at startup, then switches between them in memory during inference, eliminating repeated disk reads entirely.

Table 5 reports ensemble switching latency under sequential and preloaded strategies on server and edge hardware. In the sequential setting, ST-LoRA reduces per-model loading time by $10\times$ on the server and $127\times$ on the edge device, a direct consequence of its $178\times$ smaller checkpoint size. Under the preloaded strategy, ST-LoRA’s one-time setup cost is $2.9\times$ lower on the server and $163\times$ lower on the edge, reducing startup overhead from 4080 ms to just ends_mid=1 to similar solutions. We leave empirical validation of this hypothesis to future work.

Device	Strategy	FRE	ST-LoRA (ours)
Colab	Sequential (ms)	1704.7	172.2
Colab	Setup† (ms)	6402.0	2237.4
Colab	Per switch (ms)	0.001	8.0
Rasp. Pi	Sequential (ms)	1017.0	8.0
Rasp. Pi	Setup† (ms)	4080.0	25.0
Rasp. Pi	Per switch (ms)	0.001	0.001

† One-time preloading cost

Sizes: FRE 4×431.8 MB, ST-LoRA 4×5.5 MB (78× smaller).

25 ms on the Raspberry Pi 4B. Once preloaded, per-switch latency is negligible on both methods and both devices with an advantage for ST-LoRA as the memory is much smaller. These results demonstrate that ST-LoRA’s parameter efficiency translates directly into deployment efficiency, making ensemble inference practical on resource-constrained hardware where full-rank ensemble loading would impose prohibitive I/O overhead.

While ensembling significantly enhances calibration, it comes at the cost of reduced segmentation accuracy. Both effects stem from the same underlying mechanism, which is directly visible in the reliability diagrams of Figure 5. All single models exhibit overconfidence predicted probabilities consistently exceed empirical accuracy because individual models tend to concentrate predictions in high-confidence bins regardless of whether those predictions are correct [33]. Ensemble averaging softens this behavior; since the averaged probability across members is almost always lower than any individual member’s maximum, predictions are redistributed from overconfident high-probability bins into more moderate bins where confidence and accuracy are better aligned. This redistribution moves the reliability curve upward toward the perfectly calibrated diagonal, reducing ECE. The same softening, however, occasionally overrides confident correct predictions, causing the marginal accuracy drop observed in mIoU.

We attribute the limited magnitude of both effects to insufficient ensemble diversity as members share architecture, initialization, and augmentation strategy, with only learning rate scheduling as a source of variation. This restricts inter-member disagreement, limiting both the calibration gains and the accuracy cost.

This might be due to ST-LoRA’s rank constraint implicitly restricts optimization to a low-dimensional subspace where good solutions are more densely concentrated, which is also supported by [34]. This subspace retains sufficient capacity. Within it, learning rate variation may be enough to drive members toward solutions that are both competitive and complementary. In contrast, full-rank optimization spans a much larger space where the same perturbation may have proportionally less influence on the training trajectory, potentially causing members to converge

Less is More: ST-LoRA Match Full-Rank Quality

ST-LoRA matches or exceeds FRE in segmentation accuracy and calibration across both datasets and architectures, while reducing trainable parameters, inference latency, and memory footprint confirming that low-rank adapters are effective ensemble components for uncertainty-aware segmentation.

Having established that ST-LoRA is a viable and efficient alternative to full-rank ensembling, a natural question follows: is the baseline configuration optimal, or can targeted hyperparameter choices unlock further gains? We investigate this in the next subsection.

Ablation Study of ST-LoRA’s Hyperparameters

We conduct an extensive ablation study to identify the hyperparameters that most critically govern performance, stability, and parameter efficiency under ST-LoRA-based ensemble fine-tuning. Given the architectural heterogeneity of our benchmark SgF-B2 on GrowliFlower-L versus M2F-Swin/B on BUP20, we treat each dataset/architecture pair independently, as hyperparameter sensitivity may differ substantially across them. Our analysis focuses on three factors: rank $r$, target components, and scaling factor $\alpha$, which together govern the capacity, scope, and magnitude of the low-rank adaptation ¹⁶. Each experiment varies a single factor while holding all remaining settings fixed at the baseline configuration as mentioned in Table 6, allowing the effect of each choice to be isolated cleanly. The baseline is deliberately chosen as a reproducible standard reference rather than a pre-optimized configuration, so that improvements over it reflect genuine sensitivity rather than tuning artifacts. Interactions between hyperparameters are not explored, as exhaustive combinatorial search falls outside the scope of this study.

Setting	SegFormer-B2	Mask2Former-Swin/B
Parameter Groups
ST-LoRA-adapted	$q_e$, $k_e$, $v_e$, MLP	$q_d$, $k_d$, CLS, MLP
Fully fine-tuned	Decoder + segmentation head	—
Frozen	Remaining encoder	Remaining encoder parameters and pixel decoder
ST-LoRA Configuration
Rank $r$	32	8
Scaling $\alpha$	64	8
$A$ init	Random	Random
$B$ init	Zero	Zero
Bias	No	No
Training
Optimizer	Adam	Adam
Learning rate	$2 \times10^{-2}$–$2 \times10^{-4}$	$2 \times10^{-3}$–$2 \times10^{-5}$
Batch size	32	4
Epochs	100	110
LR schedule	CosineAnnealingWarmRestarts ($T_0 = 20$, $T_{\mathrm{mult}} = 1$)
Ensemble size	5 checkpoints
Evaluation
Seeds	5
Reported stats	Mean $\pm$ std

LoRA Rank ($r$)

Across both architectures, we observe a consistent trend: increasing the LoRA rank improves performance in terms of both segmentation accuracy and calibration as shown in Table 15 and Table 16. For SegFormer-B2, rank $r = 32$ emerges as a sweet spot between parameter efficiency and robust performance; higher ranks yield marginal further gains at the cost of a proportionally larger number of trainable parameters. Mask2Former-Swin/B exhibits a qualitatively similar trend but with a notably different sensitivity profile even at rank $r = 2$, the model achieves reasonable segmentation performance with mIoU reaching 65.8%, suggesting that a low-dimensional subspace already captures much of the task-relevant variation. We hypothesize that this difference is not solely attributable to rank, but is also shaped by the substantially different adaptation strategies employed across the two models. SegFormer-B2 applies LoRA to both the self-attention projections and the position-wise feed-forward layers of the encoder, while the decoder is fully fine-tuned. In contrast, Mask2Former-Swin/B forgoes full fine-tuning entirely: the encoder feed-forward layers are adapted via LoRA, and the decoder is handled through low-rank injection into the query and key projections along with the class predictor. This contrasts full decoder fine-tuning versus LoRA-only decoder adaptation introduces a confounding factor beyond rank alone, and may explain why the two models exhibit different calibration sensitivity profiles as rank increases. We next investigate the effect of the targeted components directly.

Effect of ST-LoRA hyperparameters and architectural choices on Mask2Former segmentation performance on BUP20. $q_d$, $k_d$, and cls denote the transformer decoder’s query projection, key projection, and class predictor, respectively. $q_e$, $k_e$, $v_e$, and $\mathrm{dense}_e$ refer to the Swin-B encoder’s query, key, value projections and dense (MLP) layers. The rank sweep uses $q_d$, $k_d$, cls as target modules. Best result per category is in bold. We ablate the rank here with the encoder completely frozen.

Category	Setting	mIoU ↑	ECE ↓	MECE ↓	ACE ↓	MACE ↓
ST-LoRA rank – $r$	2	0.2270 ± 0.0140	0.0720 ± 0.0253	0.3203 ± 0.0357	0.1619 ± 0.0347	0.2489 ± 0.0230
	4	0.2528 ± 0.0070	0.0480 ± 0.0043	0.2466 ± 0.0247	0.0933 ± 0.0199	0.1819 ± 0.0162
	8	0.2831 ± 0.0068	0.0487 ± 0.0014	0.2032 ± 0.0155	0.0533 ± 0.0160	0.1531 ± 0.0169
	16	0.3002 ± 0.0107	0.0467 ± 0.0038	0.1782 ± 0.0076	0.0386 ± 0.0096	0.1352 ± 0.0095
	32	0.3192 ± 0.0103	0.0445 ± 0.0035	0.1676 ± 0.0129	0.0363 ± 0.0066	0.1270 ± 0.0107
	64	0.3203 ± 0.0181	0.0478 ± 0.0040	0.1824 ± 0.0161	0.0316 ± 0.0093	0.1275 ± 0.0157
Components	$q_d$, $k_d$, cls, $q_e$	0.5818 ± 0.0073	0.0148 ± 0.0050	0.2164 ± 0.0900	0.0109 ± 0.0026	0.0416 ± 0.0126
	$q_d$, $k_d$, cls, $k_e$	0.5594 ± 0.0256	0.0141 ± 0.0014	0.2528 ± 0.3065	0.0114 ± 0.0055	0.0407 ± 0.0076
	$q_d$, $k_d$, cls, $v_e$	0.6310 ± 0.0235	0.0127 ± 0.0021	0.1133 ± 0.0115	0.0094 ± 0.0016	0.0383 ± 0.0033
	$q_d$, $k_d$, cls, $\mathrm{dense}_e$	0.6839 ± 0.0174	0.0032 ± 0.0012	0.0619 ± 0.0258	0.0032 ± 0.0008	0.0057 ± 0.0010
	$q_d$, $k_d$, cls, $q_e$, $k_e$	0.5982 ± 0.0086	0.0095 ± 0.0037	0.1035 ± 0.0154	0.0094 ± 0.0059	0.0271 ± 0.0133
	$q_d$, $k_d$, cls, $q_e$, $v_e$	0.6692 ± 0.0153	0.0102 ± 0.0035	0.1146 ± 0.0488	0.0088 ± 0.0048	0.0295 ± 0.0091
	$q_d$, $k_d$, cls, $q_e$, $\mathrm{dense}_e$	0.6833 ± 0.0199	0.0035 ± 0.0014	0.0851 ± 0.0573	0.0043 ± 0.0018	0.0092 ± 0.0038
	$q_d$, $k_d$, cls, $k_e$, $v_e$	0.6647 ± 0.0142	0.0088 ± 0.0018	0.0921 ± 0.0135	0.0096 ± 0.0056	0.0258 ± 0.0059
	$q_d$, $k_d$, cls, $k_e$, $\mathrm{dense}_e$	0.6873 ± 0.0114	0.0033 ± 0.0012	0.0920 ± 0.0729	0.0048 ± 0.0018	0.0091 ± 0.0041
	$q_d$, $k_d$, cls, $v_e$, $\mathrm{dense}_e$	0.6833 ± 0.0248	0.0042 ± 0.0011	0.0774 ± 0.0610	0.0047 ± 0.0020	0.0106 ± 0.0036
	$q_d$, $k_d$, cls, $q_e$, $k_e$, $\mathrm{dense}_e$	0.6927 ± 0.0308	0.0036 ± 0.0006	0.0943 ± 0.0284	0.0043 ± 0.0009	0.0085 ± 0.0030
	$q_d$, $k_d$, cls, $q_e$, $v_e$, $\mathrm{dense}_e$	0.6844 ± 0.0209	0.0041 ± 0.0013	0.0871 ± 0.0478	0.0036 ± 0.0024	0.0075 ± 0.0043
	$q_d$, $k_d$, cls, $q_e$, $k_e$, $v_e$, $\mathrm{dense}_e$	0.6867 ± 0.0214	0.0043 ± 0.0015	0.1151 ± 0.0459	0.0037 ± 0.0010	0.0093 ± 0.0019
	$q_d$, $k_d$, cls, $q_e$, $k_e$, $v_e$, $\mathrm{dense}_e$	0.6859 ± 0.0175	0.0030 ± 0.0010	0.0741 ± 0.0430	0.0042 ± 0.0010	0.0076 ± 0.0021

Category	Setting	$\Delta$ mIoU (%)	$\Delta$ ECE (%)	$\Delta$ MECE (%)	$\Delta$ ACE (%)	$\Delta$ MACE (%)
Rank – $r$	2	−1.7	+198.6	+97.3	+107.6	+86.4
	4	−0.6	+21.3	+44.8	+5.6	−0.6
	8 (baseline)	+1.0	+42.9	+12.7	+82.9	+42.4
	16	+0.6	−27.8	−16.0	+62.5	−35.2
	32	+0.2	−23.3	−11.4	+77.8	−25.5
	64	−1.9	+46.7	−14.5	+100.0	+81.8
Scaling – $\alpha$	2	−3.3	+16.1	+75.2	+65.9	+48.7
	4	−2.5	+27.4	+78.3	+60.6	+51.0
	8 (baseline)	+1.0	+42.9	+12.7	+82.9	+42.4
	16	−1.6	−7.7	−10.1	+30.8	+2.7
	32	+0.2	−6.7	+22.7	+118.2	+43.7
Dropout – $p$	0.10 (baseline)	+1.0	+42.9	+12.7	+82.9	+42.4
	0.15	+0.1	+35.1	−52.2	+112.5	+115.9
	0.20	−1.2	+13.6	+9.8	+100.0	+60.2
	0.25	−0.7	+42.9	+53.3	+84.9	+63.8
	0.30	+0.3	+37.1	−9.3	+107.3	+83.9
Components	dns, cls (baseline)	+1.0	+42.9	+12.7	+82.9	+42.4
	dns, cls, $q_e$	−1.8	+53.3	+46.8	+123.8	+76.3
	dns, cls, $k_e$	−0.6	+42.1	+32.9	+114.3	+83.7
	dns, cls, $v_e$	+1.6	+20.0	+38.3	+95.0	+60.0
	dns, cls, $q_e$, $k_e$	+2.5	+63.0	+25.7	+165.6	+96.0
	dns, cls, $q_e$, $v_e$	−1.9	−17.1	+10.9	+69.8	−12.2
	dns, cls, $k_e$, $v_e$	+0.1	+60.0	+8.0	+130.6	+117.1
	dns, cls, $q_e$, $k_e$, $v_e$	+1.0	+18.8	−15.7	+87.5	+38.8

LoRA Target Components

We investigate whether adapting the decoder alone is sufficient for segmentation. Keeping the encoder completely frozen, we inject LoRA modules exclusively into the transformer decoder of Mask2Former-Swin/B and vary the rank. Across all rank values, no reasonable performance is observed and the overall results are substantially worse than those obtained when the encoder is also adapted as shown in tbl:17. This is expected: the encoder is responsible for extracting task-relevant feature representations, and without adapting it to the target domain, the decoder operates on generic pre-trained embeddings that are ill-suited for fine-grained plant segmentation. Under these conditions, the decoder merely attempts to upsample frozen representations into a segmentation mask, offering little capacity to recover the necessary discriminative structure.

We then ask the complementary question: can the decoder be frozen entirely while adapting only the encoder? For Mask2Former-Swin/B, we freeze the decoder completely and evaluate all component configurations previously tested with decoder adaptation. The results reveal a consistent asymmetry: while a subset of configurations yields marginal mIoU improvements, nearly all exhibit severe calibration degradation across ECE, ACE, and MACE as illustrated in tbl:18. This is attributable to the frozen decoder applying a fixed mapping optimized for the source domain, which is misaligned with the target domain’s feature distribution regardless of how well the encoder is adapted. The decoder therefore acts as a calibration bottleneck; accurate enough to segment, but insufficiently adapted to produce reliable confidence estimates. These findings motivate the hybrid strategy adopted in our final configuration: LoRA-only adaptation of selected encoder components, with the decoder adapted via LoRA rather than frozen or fully fine-tuned.

We then follow the conventional LoRA adaptation strategy of targeting only the self-attention projections [40], injecting low-rank modules into $Q$, $K$, and $V$. Contrary to the gains reported in LLM settings, this configuration yields worse performance compared to the respective baselines for both architectures, as reflected by both mIoU and ECE. This suggests that attention-only adaptation, while effective for generic image classification and language tasks [34], [8], is insufficient for dense prediction in agricultural imagery, where spatial detail and pixel-level calibration impose stricter representational demands on the adapted layers.

We now turn to an intensive component ablation, examining how each encoder component contributes to segmentation quality and uncertainty calibration. Our analysis focuses on two families of components: the self-attention projections and the position-wise Multi-Layer Perception (MLP) layers, as these constitute the two primary sites of learnable computation within a transformer encoder block.

Adapting the query projection alone yields mixed results: for Mask2Former-Swin/B, performance remains comparable to the baseline despite the added parameters, whereas for SegFormer-B2 it surpasses both the baseline and the conventional $(q, k, v)$ configuration. Extending adaptation to the key and value projections jointly improves over query-only across both architectures, again outperforming the baseline and the conventional attention-only setting. However, the most pronounced gains are observed when the MLP layers are included. Across both models and all component combinations tested, incorporating MLP layers consistently yields the strongest improvements in both mIoU and calibration metrics while having the smaller number of parameters, indicating that these layers carry the dominant share of task-relevant adaptability in the context of agricultural semantic segmentation.

This observation is further supported by a converging body of evidence from both the text and vision literature. In the text domain, [21] showed that transformer networks without skip connections or MLP layers converge doubly exponentially to a rank-1 matrix, rendering them unable to learn, establishing MLP layers as structurally necessary rather than merely beneficial. [29] further demonstrated that MLP layers act as key-value memories that store and retrieve factual knowledge, with deeper layers encoding increasingly rich semantic content. From an adaptation perspective, [36] found that under a restricted parameter budget, MLP layers yield better adaptation than attention projections, and Zhang, Chen, Bukharin, He, Cheng, Chen and Zhao confirmed through extensive experimentation that MLP layers consistently outperform attention at all ranks, matching full fine-tuning at higher ranks.

In the vision domain, the evidence is equally compelling. [?] constructed a vision transformer using only MLP layers and achieved performance comparable to the full model, while the attention-only counterpart degraded severely. [?] showed that attention is not needed at every layer, with selective removal reducing computational cost while improving performance. [?] replaced self-attention entirely with a mean-shift operation and obtained competitive results, and [?] demonstrated that skipping attention modules via skip connections incurs minimal performance loss. Complementing this, [?] revealed that MLP layers exhibit a Mixture-Of-Experts (MOE) behaviour, with only a sparse subset of neurons activating per input, suggesting that their representational capacity is both rich and highly structured.

Taken together, this points to the same conclusion: MLP layers are the dominant site of task-relevant computation and adaptation in transformer architectures, in both text and vision settings. Our ablation results align with and extend this finding to the dense agricultural prediction regime, where attention-only adaptation consistently underperforms MLP-inclusive configurations across both architectures studied.

LoRA Scaling Factor ($\alpha$)

Having established rank and target components as the two most influential hyperparameters, we turn to the scaling factor $\alpha$. Recalling the LoRA adaptation equation but with the previously omitted $\delta_r = \alpha/r$ for simplicity,

The effective magnitude of the low-rank update is governed by $\delta_r = \alpha/r$, the ratio of the scaling factor to the rank. Rather than treating $\alpha$ in isolation, this formulation makes clear that it is $\delta_r$ that controls how strongly the adapter output is blended into the pre-trained representation. We therefore ablate across $\alpha$ values at fixed rank to characterize the sensitivity of both segmentation accuracy and calibration to this ratio.

For SegFormer-B2, we observe a clear degradation in performance as $\alpha$ increases, manifesting in both mIoU and calibration metrics. Since the effective update magnitude is governed by $\delta_r = \alpha/r$, a large $\alpha$ at fixed rank amplifies the low-rank update $\Delta\theta= \delta_r BA$ to a degree that distorts the pre-trained feature representations, effectively overriding the visual prior encoded during pre-training rather than refining it as shown in Table 15. Mask2Former-Swin/B exhibits the opposite trend: performance improves as $\alpha$ increases, suggesting that a stronger update signal is beneficial in this setting. Nevertheless, this gain saturates and reverses once the ratio $\delta_r$ grows large enough that $\Delta\theta\gg\theta_0$, at which point the adapter update dominates the frozen weights and degrades learning, consistent with the representation distortion observed for SegFormer-B2. Notably, Mask2Former-Swin/B appears considerably less sensitive to $\alpha$ overall, implying that the scaling factor does not act in isolation but interacts with rank and the choice of adapted components as shown in Table 16.

The original LoRA formulation [40] acknowledges this sensitivity and recommends treating $\alpha$ as a regularization-like hyperparameter, noting that setting $\alpha= r$ provides a neutral starting point where the adapter contributes without overwhelming the frozen weights. For high ranks $r \gg128$ [10] suggest using $\alpha= 2r$ but this is in text domain and needs more verification for dense prediction in vision, this reinforces the broader observation that LoRA hyperparameters should not be tuned independently, and that the relative magnitude of the adapter update with respect to the pre-trained weights is a more principled diagnostic than any single hyperparameter alone [40]; [11].

Finally, The relative insensitivity of Mask2Former-Swin/B to $\alpha$ likely reflects two compounding factors. First, unlike SegFormer-B2 where the fully fine-tuned decoder creates a rigid dependency on the quality of the adapted encoder representations, Mask2Former-Swin/B distributes adaptation entirely through LoRA modules across both encoder and decoder. Second, the adapted components in Mask2Former-Swin/B are predominantly MLP layers, which may be intrinsically less sensitive to scaling perturbations than the lower-dimensional attention projections targeted in SegFormer-B2. We note that this remains a hypothesis, as disentangling the individual contributions of architecture, component choice, and scaling interactions would require controlled experiments beyond the scope of this ablation. Our results confirm this intuition in the vision segmentation regime and suggest that conservative scaling is particularly important when adapting pre-trained encoders to fine-grained, data-limited agricultural datasets. Hence we provide a practitioner’s guide for ST-LoRA as a starting point:

Configuring ST-LoRA Always adapt the encoder — freezing it entirely produces near-random performance regardless of rank. Target feed-forward layers first: contrary to standard LoRA practice, FF layers consistently outperform attention-only adaptation for dense prediction, with fewer parameters. Do not freeze the decoder entirely, as it acts as a calibration bottleneck regardless of encoder quality; adapt it via LoRA instead. For rank $r \in[8, 32]$ is recommended. Finally, keep $\delta_r = \alpha/r$ conservative: setting $\alpha= r$ is a safe default, and large $\delta_r$ degrades both accuracy and calibration.

However, a well-configured model is a necessary but not sufficient condition for reliable deployment as agricultural imaging conditions vary substantially across seasons, lighting, and sensing conditions, making robustness under distribution shift equally critical. We therefore turn to two complementary questions: does ensemble member diversity affect calibration stability when the input distribution shifts, and do the component choices identified above retain their advantage beyond the training distribution?

Distribution Shift: Ensemble Diversity and Architectural Sensitivity

We evaluate robustness under covariate shift through two complementary analyses. The first examines how ensemble member diversity affects robustness, using SegFormer-B2 on GrowliFlower-L. The second investigates the sensitivity of different LoRA component configurations to distribution shift, using Mask2Former-Swin/B on BUP20. In both cases, covariate shift is simulated by applying controlled corruptions to the test set, the full experimental setup for both analyses is summarized in Table 7.

Variants	Details	Exp. 1: Ensemble Diversity — SegFormer-B2 / GrowliFlower-L	Exp. 2: Component Sensitivity — Mask2Former-Swin/B / BUP20
Photometric	Brightness	Homogeneous ensemble (LR schedule only); Heterogeneous ensemble (LR + augmentation)	Factor $\in[1.0, 2.0]$, 20 levels ($\Delta= 0.05$); Factor $\in\{0.50, 0.65, 0.80\}$
Photometric	Contrast	Factor $\in[1.0, 2.0]$, 20 levels ($\Delta= 0.05$)	Factor $\in\{0.50, 0.65, 0.80\}$
Photometric	Gaussian noise	$\sigma\in[0, 0.285]$, 20 levels ($\Delta= 0.015$)	$\sigma\in\{0.05, 0.10, 0.20\}$
Photometric	Gaussian blur	—	$\sigma\in\{1, 2, 3\}$ px
Geometric	Rotation	$\theta\in[0^\circ, 85.5^\circ]$, 20 levels ($\Delta= 4.5^\circ$)	—
Geometric	Zoom	Scale $\in[1.0\times, 1.475\times]$, 20 levels ($\Delta= 0.025$)	Scale $\in\{1.10\times, 1.25\times, 1.50\times\}$
Geometric	Horizontal flip	—	Binary (single level)
Geometric	Translation	—	$\{5\%, 10\%, 15\%\}$ of image width
Total shift conditions		$4 \times20 = 80$	$7 \times3 - 2 = 19$
Mask handling		Geometric: nearest-neighbour interp. on mask	Geometric: nearest-neighbour interp. on mask
Metrics		mIoU, and ECE (mean $\pm$ std over 5 seeds)

Ensemble diversity under distribution shift.

We compare two ensemble construction strategies for both FRE and ST-LoRA to assess the impact of member diversity on calibration robustness. Homogeneous ensembles rely solely on learning rate scheduling as a source of diversity, with no data augmentation applied during training; members therefore differ only in their optimization trajectories. Heterogeneous ensembles additionally introduce per-member data augmentation policies, incorporating random rotation (uniformly sampled from $\{0^\circ, 25^\circ\}$) and horizontal flipping (probability $0.5$), applied consistently to both images and segmentation masks. This introduces diversity at both the optimization and data levels, which leads to more robust ensemble predictions and better-calibrated uncertainty estimates under distribution shift.

Across all distribution shifts as shown in Figure 6, ST-LoRA achieves performance competitive with FRE, while exhibiting greater robustness to increasing shift severity maintaining lower ECE values with only a 3–5% reduction in mIoU. For FRE, homogeneous ensembles achieve higher mIoU but substantially worse calibration compared to their heterogeneous counterparts. The heterogeneous strategy is therefore critical for calibration quality in full-rank ensembles. ST-LoRA, by contrast, benefits from heterogeneity in both metrics simultaneously: heterogeneous ST-LoRA outperforms homogeneous counterpart in mIoU and ECE alike, achieving the best calibration across all shift conditions while maintaining segmentation quality competitive with FRE variants.

We attribute this asymmetry to the reduced parameter space of LoRA adapters: the constrained adaptation subspace acts as an implicit regularizer, allowing ST-LoRA members to leverage augmentation-induced diversity more efficiently than full-rank models operating under the same limited training budget. Furthermore, we notice that SgF-B2 performance is deteriorating under brightness/contrast shift compared to other shifts. This is due to high shift severity which completely distorts the input image to the model, as around shift level of 10, the model still achieves 64%, and this robustness is due to the design of SegFormer which has been mentioned in [?].

Component Sensitivity under Distribution Shift.

We evaluate how ST-LoRA hyperparameter choices - specifically rank, target components, scaling factor, and dropout - affect calibration robustness when the input distribution shifts. For each hyperparameter, we compare how varying a single setting relative to the baseline configuration affects robustness across the 19 corruption conditions described in Table 7.

Rank. Across all corruption types and severity levels, a consistent pattern emerges: low-rank configurations are brittle under distribution shift, exhibiting substantially worse calibration alongside degraded segmentation quality as shown in Figure 7. This instability is most pronounced at $r = 2$ as shown in Figure 8, where calibration metrics deteriorate sharply even under mild photometric perturbations, suggesting that a low-dimensional adaptation subspace lacks sufficient capacity to maintain well-calibrated predictions beyond the training distribution. Increasing the rank to $r = 16$ yields a notable improvement in both robustness and calibration stability across shift types, establishing it as a practical lower bound for reliable deployment under covariate shift. However, continuing to increase the rank beyond this point introduces diminishing returns: while segmentation accuracy may marginally improve, calibration under

distribution shift does not benefit proportionally and can even degrade at higher ranks as shown in Figure 8 which has been also shown by [34]. The additional computational cost of high-rank adaptation therefore does not translate into a robustness advantage, reinforcing $r \in[8, 16]$ as the recommended operating range for ST-LoRA on Mask2Former-Swin/B under distribution shift.

Target components. Ranking configurations by mIoU and ECE robustness under distribution shift reveals a persistent tension between segmentation accuracy and calibration as shown in Figure 12 and Figure 12: the configurations that best preserve accuracy under shift are not the same as those that best preserve calibration. Specifically, base + $k_e$¹⁷ ranks second on mIoU but drops to sixth on calibration, while base + $q_e$ ranks third on mIoU yet last on calibration. The configuration achieving the strongest mIoU robustness overall is base + $k_e + v_e$, whereas base + $q_e + k_e$ leads on calibration robustness. The baseline configuration (MLP, cls) occupies a rank of approximately five on both objectives, placing it consistently in an intermediate position that avoids the extremes of either. This trade-off mirrors the accuracy/calibration tension observed in the in-distribution ablation and suggests that no single component configuration simultaneously dominates both objectives under covariate shift.

Furthermore, Cross-referencing with the in-distribution ablation results reveals that the baseline and the base + $q_e + k_e$ configuration yield comparable performance across both mIoU and ECE relative to the best-performing configurations for both SegFormer-B2 and Mask2Former-Swin/B. This consistency across architectures, datasets, and evaluation regimes - in-distribution and under shift - suggests that adapting query and key projections alongside the encoder MLP layers provides a robust and transferable operating point. baseline + $q_e + k_e$ therefore represents the most principled choice, offering a stable balance between segmentation quality and uncertainty reliability without requiring architecture-specific tuning.

Scaling factor $\alpha$. The effect of $\alpha$ on robustness under distribution shift exhibits a clear asymmetry between segmentation accuracy and calibration. For mIoU, increasing $\alpha$ initially improves robustness, suggesting that a stronger adapter signal helps the model maintain discriminative representations under input perturbations. However, this benefit saturates and reverses beyond a certain threshold, where the amplified low-rank update begins to distort the pre-trained prior rather than refine it as shown in Figure 11 and Figure 12, consistent with the in-distribution degradation observed at high $\delta_r = \alpha/r$ values. For calibration under distribution shift, the relationship is qualitatively different: increasing $\alpha$ consistently yields better calibrated predictions, with ECE improving monotonically across shift types and severity levels. This suggests that a stronger adapter update enhance robustness when the input deviates from the training distribution. The two objectives therefore pull in opposite directions as $\alpha$ increases, reinforcing the broader observation that accuracy robustness and calibration robustness are not jointly optimisable through a single hyperparameter, and that the choice of $\alpha$ should be guided by the deployment priority - higher $\alpha$ for better-calibrated uncertainty, lower to moderate $\alpha$ for stronger segmentation accuracy under shift.

LoRA Dropout $p$. The effect of LoRA dropout on robustness under distribution shift follows a non-monotonic pattern for segmentation accuracy: both the lowest ($p = 0.1$) and highest ($p = 0.3$) dropout values yield degraded mIoU under shift, while intermediate values $p \in[0.2, 0.25]$ achieve the strongest robustness as shown in Figure 11 and Figure 12. Small dropout allows the adapter to overfit to in-distribution features that do not generalize under shift, while too much dropout disrupts the adapter’s capacity to maintain discriminative representations altogether. A moderate dropout rate therefore strikes the best operating point between preventing overfitting and preserving representational capacity.

For calibration, the picture is more nuanced and shift-type selective. Under photometric corruptions, $p = 0.3$ yields the best calibration robustness, suggesting that aggressive dropout encourages the model to be robust when pixel statistics deviate from the training distribution. Under geometric corruptions, however, $p = 0.3$ underperforms relative to lower dropout values, which instead produce more stable calibration. This shift-type selectivity of dropout echoes the broader finding that photometric and geometric corruptions engage different aspects of the model’s learned representations, and reinforces the recommendation to treat calibration robustness under the two shift families as distinct design objectives.

Finally, [33] highlight an inherent tension between minimizing classification error and achieving well-calibrated predictions. We observe a similar trade-off in ST-LoRA: accuracy and calibration emerge as competing objectives, and we hypothesize that this competition has a low-rank geometric interpretation. Drawing on the notion of intrinsic dimensionality [3], the low-rank subspace, sufficient for optimizing the segmentation objective, may not be expressive enough to simultaneously ensure reliable calibration; particularly under distribution shift, where the adapter must generalize beyond the fine-tuning data manifold. This suggests that accuracy and calibration place structurally different demands on the adaptation subspace, and that jointly optimizing both may require explicit calibration-aware training objective.

ST-LoRA Under Distribution Shift Start with $r\in[8, 16]$: low ranks ($r\le4$) are brittle under all shift types; higher ranks add cost without calibration benefit. Prefer $\mathrm{MLP}+q_e+k_e$ for the most consistent accuracy–calibration balance across architectures and shift types. Tune $\delta_r$: lower $\alpha$ when calibration is the priority, raise it when accuracy matters more; $\alpha=r$ is the safe default. For dropout, $p\in[0.2, 0.25]$ maximizes accuracy robustness; $p=0.3$ is better when photometric shift dominates. Accuracy and calibration under shift are competing objectives; no single configuration wins on both.

Having established ST-LoRA’s advantages over full-rank ensemble and characterized its behaviour under distribution shift, we turn to a broader question: how does ST-LoRA compare against established efficient uncertainty quantification methods? We benchmark against three complementary approaches - Snapshot Ensemble, MC Dropout, and DDU - spanning deterministic, and stochastic uncertainty estimation paradigms.

ST-LoRA Against Efficient Uncertainty Methods

We compare ST-LoRA against three efficient uncertainty estimation baselines whose configurations are detailed in Table 8. Integrated ensemble methods are excluded due to architectural and compatibility constraints discussed in Section 2. To ensure a fair comparison, all methods share an identical training setup: the same cosine annealing learning rate schedule and the same heterogeneous augmentation pipeline, isolating the effect of the uncertainty estimation strategy itself.

Method	Ensemble	Inference	Settings
FRE	Yes	$(M = 4)$ forward passes	Following the similar Full Rank Ensembles from previous experiments.
MC Dropout	No	$T = 4$ stochastic passes	We dropout layer before the class predictor using a dropout probability of $p = 0.5$ following [27]; Srivastava et al. (2014).
DDU	No	Single forward pass	SN has been applied to all linear and convolutional layers. A Gaussian Mixture Model ($K=2$ components per class, 8 classes total) is fitted to these features using the EM algorithm for 1000 iterations. Pixels are treated as spatially independent data points, yielding $1280 \times720$ data points per image.
ST-LoRA (ours)	Yes	$(M = 4)$ forward passes	LoRA adaptation with frozen decoder. Rank $r=32$, scaling $\alpha=32$, dropout $p=0.1$. Target modules: decoder $q_d$, $k_d$, cls; encoder MLP, $k_e$, $v_e$.

The augmentation pipeline consists of two families of transformations. Geometric augmentations, applied jointly to image and mask, include random horizontal flipping ($p=0.5$) and lateral translation (up to 10% of image width, $p=0.5$). Photometric augmentations, applied to the image only, include colour jitter (brightness $\pm0.3$, contrast $\pm0.3$, saturation $\pm0.1$), Gaussian blur ($\sigma\in[0.5,1.5]$, $p=0.3$), Gaussian noise ($\sigma=0.02$, $p=0.2$), spatially non-uniform depth-of-field blur ($p=0.3$) simulating centre-sharp optics via a Gaussian spatial weight mask, and random erasing of up to three patches per image ($p=0.3$, patch area $\in[2\%,15\%]$ of image) to simulate leaf occlusion and canopy overlap. Additionally, CutMix [?] ($p=0.5$) is applied at the dataset level, randomly replacing a rectangular region of one image and its corresponding mask labels with those from another training sample, encouraging the model to learn from partially occluded scenes. ¹⁸

For ensemble methods, five snapshots are collected per training trajectory for both FRE and ST-LoRA by saving the model weights blindly at the end of each cosine annealing cycle, without any validation-based selection. For MC Dropout and DDU, a single checkpoint is saved at the end of training. This blind collection strategy is applied uniformly across all methods: tracking the best validation loss would introduce a selection advantage that is incompatible with the snapshot ensembling design, where checkpoints are taken at fixed cycle endpoints regardless of their individual validation performance. Keeping this consistent ensures that observed differences in performance reflect the uncertainty estimation strategy rather than checkpoint selection policy.

In-Distribution Performance and Robustness Under Distribution Shift

We follow the same two-stage evaluation protocol as previous experiments: in-distribution performance followed by robustness under distribution shift.

In-distribution performance. Table 9 shows that ST-LoRA achieves mIoU competitive with the best-performing method on this metric (FRE), while matching DDU on calibration and outperforming it on MACE with a margin of nearly 4% mIoU. Notably, ST-LoRA exhibits the lowest standard deviation across all metrics among all methods, indicating stable performance regardless of the random seed—a practically important property for reliable deployment.

Method	mIoU $\uparrow$	ECE $\downarrow$	MECE $\downarrow$	ACE $\downarrow$	MACE $\downarrow$
FRE	0.6990 $\pm$ 0.0259	0.0068 $\pm$ 0.0068	0.1792 $\pm$ 0.1613	0.0047 $\pm$ 0.0037	0.0209 $\pm$ 0.0242
MC Dropout	0.6505 $\pm$ 0.0391	0.0049 $\pm$ 0.0049	0.0686 $\pm$ 0.0302	0.0062 $\pm$ 0.0090	0.0177 $\pm$ 0.0126
DDU	0.6553 $\pm$ 0.0197	0.0019 $\pm$ 0.0010	0.1171 $\pm$ 0.1069	0.0016 $\pm$ 0.0008	0.0106 $\pm$ 0.0062
ST-LoRA (ours)	0.6920 $\pm$ 0.0147	0.0026 $\pm$ 0.0018	0.0901 $\pm$ 0.0311	0.0027 $\pm$ 0.0011	0.0077 $\pm$ 0.0059

Table 8

Configuration summary for all uncertainty quantification methods evaluated on Mask2Former-Swin/B (BUP20). SN: Spectral Normalization Miyato, Kataoka, Koyama and Yoshida (2018). GMM: Gaussian Mixture Model fitted via EM [18]. ST-LoRA target modules follow the notation of Table 6.

Table 9

In-distribution (ID) performance of ST-LoRA against efficient uncertainty quantification baselines on Mask2Former-Swin/B (BUP20, mean $\pm$ std over 5 seeds). FRE refers to a snapshot ensemble built on top of a fully fine-tuned model. MC Dropout uses $p = 0.5$. DDU: Deep Deterministic Uncertainty Mukhoti et al. (2023). $\uparrow$: higher is better; $\downarrow$: lower is better,and best result per metric is in bold.

Table 10

Approach	Flops (GMAC)	Train Params. (Million)
M2F (base)	229,72	108.77
FRE	1148	534
MC-Dropout	1148	108.77
DDU	320	108.8
ST-LoRA (ours)	1170	116

Computational efficiency comparison across approaches.

Among all methods, DDU offers the lowest computational cost (see Table 10), as uncertainty is estimated in a single forward pass without sampling, though this efficiency comes at the cost of segmentation accuracy. The mIoU drop for DDU is attributed to the widespread application of spectral normalization, which constrains the feature space and reduces discriminative capacity, consistent with the accuracy/robustness trade-off reported in [42]; [31].

Furthermore, applying SN may inherently limit ViT performance. As [21] demonstrate, MLP layers play a critical role in preserving representational capacity by counteracting the rank collapse induced by self-attention

and increases the Lipschitz constant of the network. SN directly opposes this dynamic by constraining the Lipschitz constant to a fixed upper bound, creating a counter force between both. Consequently, how to enforce a meaningful Lipschitz bound in transformers without sacrificing representational capacity remains an open question, with promising directions explored in Newhouse, Hess, Cesista, Zahordnii, Bernstein and Isola (2025). Finally, MC Dropout, performance sensitivity likely reflects the joint effect of the targeted dropout layer and the number of stochastic forward passes in addition to other hyperparameters as mentioned in Srivastava et al. (2014).

Robustness under distribution shift

We evaluate all methods on BUP20 under a systematic corruption grid of seven corruption types at five severity levels each, yielding 35 shift conditions. All severity levels are deliberately chosen to lie outside the training augmentation range, ensuring exposure to unseen conditions . Photometric corruptions, applied to the image only, include Gaussian blur ($\sigma\in\{1.5, 3, 5, 7, 10\}$ px, versus training $\sigma\in[0.5, 1.5]$ px), additive Gaussian noise ($\sigma\in\{0.04, 0.08, 0.12, 0.18, 0.25\}$), versus training $\sigma= 0.02$, brightness reduction (factor $\in\{0.60, 0.40, 0.25, 0.10, 0.05\}$), versus training $[0.7, 1.3]$, contrast reduction (same factor range), and saturation reduction (factor $\in\{0.80, 0.60, 0.40, 0.15, 0.00\}$, versus training

footer: Farag et al.: Preprint submitted to Elsevier $[0.9, 1.1])$. Geometric corruptions include lateral translation ({20, 30, 40, 50, 60}% of image width, versus training $\pm10$%) and rotation (vertical flip, $90^\circ$, $180^\circ$, $270^\circ$, and $90^\circ$+vertical flip, versus training horizontal flip only).

As shown in Figure 9, across nearly all shift types and severity levels, ST-LoRA achieves the strongest calibration and segmentation robustness among all methods. FRE follows as the second most robust, ahead of MC Dropout, while DDU degrades most severely under distribution shift. Although DDU is the most computationally efficient method at inference time and carries the smallest memory footprint, its calibration collapses rapidly under shift which is a well-documented limitation of deterministic uncertainty methods, not exclusive to DDU [?]. The poor robustness of MC-Dropout is consistent with our earlier hypothesis that both the choice of dropout layer and the sampling rate critically affect performance under shift, and needs further investigation to identify configurations that generalize more reliably beyond the training distribution.

Out of Distribution (OoD) detection

We use BUP20/GrowliFlower-L as ID/far OoD datasets respectively. Cauliflower plants in open fields differ from sweet peppers in glasshouses at every level; crop morphology, background, lighting, and sensing conditions. OoD evaluation proceeds at two levels of granularity.

Image-level OoD detection. We assess each method’s ability to separate ID and OoD samples at the image level, without exploiting local spatial structure. Uncertainty scores are computed per pixel and aggregated by mean across each image. For ensemble and MC Dropout methods, we use predictive entropy and mutual information [?] as scoring functions; for DDU, we use the GMM log-density and predictive entropy. Finally, we evaluate separability qualitatively via score histograms and quantitatively via the Area Under the Receiver Operating Characteristic curve (AUROC) [37].

Pixel-level anomaly segmentation We are the first to apply such evaluation scheme - following the protocol of [12]; Sodano, Magistri, Behley and Stachniss (2024) - to close-range glasshouse crop segmentation, assessing the sensitivity of uncertainty estimation methods to cross-crop OoD objects at the pixel level. Pixel-level anomaly segmentation using ranking metrics (Area Under the Precision-Recall Curve (AUPR), False Positive Rate at 95% True Positive Rate (FPR95)), and localization metrics (segment-wise Intersection over Union (sIoU), Positive Predictive Value (PPV), and mean F1 score (MeanF1)) has not, to our knowledge, been evaluated in this setting.

Table 13 summarizes OoD detection performance across all methods. At the image level, ST-LoRA achieves the best entropy-based AUROC (0.998 $\pm$ 0.003) nearly perfect separation between ID and OoD images; followed by FRE, MC Dropout, and DDU. Critically, ST-LoRA also exhibits the lowest standard deviation across both scoring functions among all methods, confirming that its OoD sensitivity is stable and seed-independent, a practically important property for deployment.

Category	Setting	mIoU $\uparrow$	FNR $\downarrow$	ECE $\downarrow$	MECE $\downarrow$	Trainable Parameters (%) $\downarrow$
LoRA rank - $r$	2	0.3151 $\pm$ 0.3863	0.6574 $\pm$ 0.4205	0.0646 $\pm$ 0.0303	0.1367 $\pm$ 0.0863	4.64
LoRA rank - $r$	4	0.5553 $\pm$ 0.3191	0.4009 $\pm$ 0.3532	0.1021 $\pm$ 0.0338	0.2905 $\pm$ 0.1190	5.11
LoRA rank - $r$	8	0.7285 $\pm$ 0.1081	0.2151 $\pm$ 0.1460	0.0744 $\pm$ 0.0688	0.2864 $\pm$ 0.1483	6.03
LoRA rank - $r$	16	0.6277 $\pm$ 0.3154	0.3257 $\pm$ 0.3409	0.0529 $\pm$ 0.0350	0.1748 $\pm$ 0.0640	7.81
LoRA rank - $r$	32 (baseline)	0.8178 $\pm$ 0.0295	0.1093 $\pm$ 0.0410	0.0126 $\pm$ 0.0166	0.1367 $\pm$ 0.0198	11.19
LoRA rank - $r$	64	0.8134 $\pm$ 0.0194	0.0973 $\pm$ 0.0442	0.0329 $\pm$ 0.0338	0.2165 $\pm$ 0.0835	17.26
LoRA rank - $r$	128	0.8231 $\pm$ 0.0110	0.0879 $\pm$ 0.0180	0.0062 $\pm$ 0.0041	0.1607 $\pm$ 0.0750	27.20
Components	$q$, $k_v$, $f_{c1}$, $f_{c2}$ (baseline)	0.8178 $\pm$ 0.0295	0.1093 $\pm$ 0.0410	0.0126 $\pm$ 0.0166	0.1367 $\pm$ 0.0198	11.19
Components	$q$, $f_{c1}$, $f_{c2}$	0.8469 $\pm$ 0.0021	0.0726 $\pm$ 0.0039	0.0041 $\pm$ 0.0008	0.1561 $\pm$ 0.0348	9.87
Components	$q$, $k_v$, $f_{c1}$	0.8429 $\pm$ 0.0095	0.0719 $\pm$ 0.0097	0.0038 $\pm$ 0.0018	0.1280 $\pm$ 0.0601	8.97
Components	$q$, $k_v$, $f_{c2}$	0.7505 $\pm$ 0.1439	0.1810 $\pm$ 0.1765	0.0358 $\pm$ 0.0604	0.1916 $\pm$ 0.1460	8.97
Components	$q$, $k_v$	0.8158 $\pm$ 0.0273	0.1300 $\pm$ 0.0394	0.0037 $\pm$ 0.0013	0.0999 $\pm$ 0.0290	6.63
Components	$f_{c1}$, $f_{c2}$	0.8509 $\pm$ 0.0023	0.0718 $\pm$ 0.0045	0.0034 $\pm$ 0.0003	0.1231 $\pm$ 0.0124	8.97
Components	$k_v$	0.8311 $\pm$ 0.0128	0.1064 $\pm$ 0.0145	0.0025 $\pm$ 0.0018	0.0705 $\pm$ 0.0430	8.97
Components	$q$	0.8551 $\pm$ 0.0012	0.0557 $\pm$ 0.0016	0.0035 $\pm$ 0.0002	0.1163 $\pm$ 0.0057	8.97
Components	$f_{c2}$	0.8498 $\pm$ 0.0046	0.0715 $\pm$ 0.0053	0.0035 $\pm$ 0.0002	0.1367 $\pm$ 0.0075	6.63
Components	$f_{c1}$	0.8613 $\pm$ 0.0018	0.0513 $\pm$ 0.0021	0.0042 $\pm$ 0.0003	0.1686 $\pm$ 0.0079	6.63
LoRA scaling factor - $\alpha$	8	0.8622 $\pm$ 0.0080	0.0498 $\pm$ 0.0109	0.0132 $\pm$ 0.0173	0.1719 $\pm$ 0.0334	11.19
LoRA scaling factor - $\alpha$	16	0.8531 $\pm$ 0.0042	0.0551 $\pm$ 0.0034	0.0047 $\pm$ 0.0007	0.1760 $\pm$ 0.0190	11.19
LoRA scaling factor - $\alpha$	32	0.8290 $\pm$ 0.0163	0.0709 $\pm$ 0.0104	0.0091 $\pm$ 0.0085	0.2005 $\pm$ 0.0804	11.19
LoRA scaling factor - $\alpha$	64 (baseline)	0.8178 $\pm$ 0.0295	0.1093 $\pm$ 0.0410	0.0126 $\pm$ 0.0166	0.1367 $\pm$ 0.0198	11.19
LoRA scaling factor - $\alpha$	128	0.4645 $\pm$ 0.3796	0.5072 $\pm$ 0.4031	0.0492 $\pm$ 0.0379	0.1184 $\pm$ 0.1015	11.19
Initializer	EVA	0.7795 $\pm$ 0.0455	0.1557 $\pm$ 0.0580	0.0130 $\pm$ 0.0164	0.1256 $\pm$ 0.0574	11.19
Initializer	OLoRA	0.7872 $\pm$ 0.0850	0.1244 $\pm$ 0.1202	0.0261 $\pm$ 0.0350	0.2331 $\pm$ 0.0929	11.19
Initializer	PISSA	0.8097 $\pm$ 0.0172	0.1224 $\pm$ 0.0291	0.0299 $\pm$ 0.0353	0.1527 $\pm$ 0.0888	11.19
Initializer	None (baseline)	0.8178 $\pm$ 0.0295	0.1093 $\pm$ 0.0410	0.0126 $\pm$ 0.0166	0.1367 $\pm$ 0.0198	11.19
Bias Setting	All	0.4348 $\pm$ 0.3574	0.5515 $\pm$ 0.3690	0.0703 $\pm$ 0.0377	0.2537 $\pm$ 0.0958	11.45
Bias Setting	LoRA-only	0.8103 $\pm$ 0.0232	0.1143 $\pm$ 0.0456	0.0142 $\pm$ 0.0160	0.1599 $\pm$ 0.0479	11.31
Bias Setting	None (baseline)	0.8178 $\pm$ 0.0295	0.1093 $\pm$ 0.0410	0.0126 $\pm$ 0.0166	0.1367 $\pm$ 0.0198	11.19
Batch Size	8	0.8254 $\pm$ 0.0098	0.0899 $\pm$ 0.0162	0.0050 $\pm$ 0.0015	0.1405 $\pm$ 0.0518	11.19
Batch Size	16	0.8106 $\pm$ 0.0212	0.1124 $\pm$ 0.0489	0.0408 $\pm$ 0.0421	0.2222 $\pm$ 0.0916	11.19
Batch Size	32 (baseline)	0.8178 $\pm$ 0.0295	0.1093 $\pm$ 0.0410	0.0126 $\pm$ 0.0166	0.1367 $\pm$ 0.0198	11.19
Learning Rate	$2 \times10^{-1}$	0.0000 $\pm$ 0.0000	1.0000 $\pm$ 0.0000	0.0820 $\pm$ 0.0148	0.0820 $\pm$ 0.0148	11.19
Learning Rate	$2 \times10^{-2}$ (baseline)	0.8178 $\pm$ 0.0295	0.1093 $\pm$ 0.0410	0.0126 $\pm$ 0.0166	0.1367 $\pm$ 0.0198	11.19
Learning Rate	$2 \times10^{-3}$	0.8860 $\pm$ 0.0018	0.0254 $\pm$ 0.0013	0.0050 $\pm$ 0.0003	0.2876 $\pm$ 0.0138	11.19
Learning Rate	$2 \times10^{-4}$	0.8802 $\pm$ 0.0023	0.0274 $\pm$ 0.0049	0.0057 $\pm$ 0.0005	0.2590 $\pm$ 0.0427	11.19
$\eta_{\min}$	$2 \times10^{-3}$	0.8135 $\pm$ 0.0021	0.1011 $\pm$ 0.0161	0.0248 $\pm$ 0.0188	0.2021 $\pm$ 0.0764	11.19
$\eta_{\min}$	$2 \times10^{-4}$ (baseline)	0.8178 $\pm$ 0.0295	0.1093 $\pm$ 0.0410	0.0126 $\pm$ 0.0166	0.1367 $\pm$ 0.0198	11.19
$\eta_{\min}$	$2 \times10^{-5}$	0.8096 $\pm$ 0.0212	0.1132 $\pm$ 0.0438	0.0310 $\pm$ 0.0330	0.2046 $\pm$ 0.0881	11.19
$\eta_{\min}$	$2 \times10^{-6}$	0.8148 $\pm$ 0.0304	0.0983 $\pm$ 0.0543	0.0240 $\pm$ 0.0341	0.2168 $\pm$ 0.0735	11.19

Method	Scorer	Image-level AUROC ↑	Pixel-level AUROC ↑	AUPR ↑	FPR95 ↓	sIoU ↑	PPV ↑	MeanF1 ↑
FRE	Ent.	0.9282 ± 0.0497	0.5828 ± 0.0896	0.9021 ± 0.0266	0.9185 ± 0.0436	0.4121 ± 0.0184	0.1986 ± 0.0848	0.2074 ± 0.0856
	MI	0.7886 ± 0.1241	0.5845 ± 0.0767	0.9016 ± 0.0242	0.9256 ± 0.0370	0.4133 ± 0.0225	0.1820 ± 0.0718	0.1927 ± 0.0734
MC Dropout	Ent.	0.5874 ± 0.1549	0.5901 ± 0.0606	0.9058 ± 0.0165	0.9094 ± 0.0451	0.4029 ± 0.0250	0.2364 ± 0.0894	0.2425 ± 0.0844
	MI	0.6914 ± 0.0486	0.5274 ± 0.0238	0.8830 ± 0.0073	0.9436 ± 0.0189	0.4121 ± 0.0124	0.2322 ± 0.0462	0.2455 ± 0.0432
DDU	Ent.	0.4691 ± 0.2589	0.5578 ± 0.0359	0.8982 ± 0.0102	0.9585 ± 0.0204	0.3832 ± 0.0074	0.0054 ± 0.0013	0.0100 ± 0.0021
	Den.	0.3860 ± 0.3335	0.5475 ± 0.0458	0.8910 ± 0.0155	0.9426 ± 0.0190	0.3405 ± 0.0192	0.0036 ± 0.0001	0.0065 ± 0.0008
ST-LoRA (ours)	Ent.	0.9981 ± 0.0025	0.4069 ± 0.1043	0.8309 ± 0.0357	0.9719 ± 0.0230	0.4022 ± 0.0338	0.3013 ± 0.0792	0.3088 ± 0.0765
	MI	0.5877 ± 0.2110	0.4044 ± 0.0862	0.8329 ± 0.0278	0.9794 ± 0.0127	0.3938 ± 0.0325	0.2746 ± 0.0727	0.2845 ± 0.0707

Method	Uncertainty Evaluation OoD	Uncertainty Evaluation Dist. Shift	Ablation Study Rank Only	Ablation Study Layer Selection	Component Attn. Only	Component FF Layers	Domain Generic	Domain Agri.
Halbheer et al. (2024)	✓	✗	✓	✗	✓	✗	✓	✗
Balabanov and Linander (2024)	✗	✗	✗	✗	✓	✗	✓	✗
Paul et al. (2024)	✗	✗	✗	✗	✓	✗	✓	✗
Onal et al.	✗	✗	✗	✗	✓	✗	✓	✗
ST-LoRA (Ours)	✓	✓	✓	✓	✓	✓	✗	✓

Across all methods, mutual information consistently underperforms predictive entropy as an OoD scoring function. This is not a failure of the ensemble approach: [43] have shown that very low MI values reflect a fundamental limitation of the measure itself under far OoD condition. DDU shows the highest standard deviation across both of its scoring functions, making it the least reliable method for OoD detection on Mask2Former, consistent with the architectural incompatibility concerns discussed in the previous experiment.¹⁹

At the pixel level, ST-LoRA achieves the best PPV and MeanF1 among all methods, indicating that its uncertainty maps are the most precise at identifying anomalous plant pixels when a threshold is applied. On sIoU, ST-LoRA is highly competitive with FRE, the best-performing method on this metric. The trade-off is on AUPR and pixel-level AUROC, where MC Dropout leads and achieves the best performance. DDU achieves near-zero PPV and MeanF1, confirming that its density-based scores do not translate into reliable pixel-level anomaly localization in this setting.

Figure 10 shows that ST-LoRA achieves the clearest separation between ID and OoD distributions under entropy scoring, exhibiting the largest gap between distribution means. Across all methods, OoD samples consistently produce higher entropy than ID samples; the expected and desirable behaviour, confirming that uncertainty estimates are well-aligned with the novelty of the input.

Among the baselines, DDU achieves strong separation using GMM density scores but is highly unstable across seeds, making it unreliable in practice. MC Dropout collapses a large fraction of OoD samples into the ID score range a known failure mode under far OoD conditions [49], where the method produces overconfident, low-variance predictions on inputs far from the training distribution. FRE ranks second overall, offering reliable but less consistent separation than ST-LoRA while performing better on mutual information.

Overall, ST-LoRA provides the most consistent and reliable OoD detection profile: the best image-level separation, the best precision and F1 at the pixel level, and the lowest variance across seeds; making it the most dependable choice for uncertainty-aware deployment in agricultural environments where OoD inputs must be flagged reliably.

Final Remarks

Comparison with Related Work

ST-LoRA establishes a strong, scalable ensembling framework that achieves a compelling balance between computational efficiency and uncertainty quantification quality, reducing training time, inference latency, parameter count, and memory footprint simultaneously while remaining competitive with or outperforming established baselines across all evaluated tasks.

Table 14 situates our work within the existing LoRA ensemble literature across four dimensions: uncertainty evaluation schemes, ablation scope, target components, and application domain. The closest related work is [34]²⁰, which is the only prior study to evaluate both distribution shift and OoD detection. However, their analysis stops short of investigating how LoRA hyperparameters interact with covariate shift, a gap our work directly addresses. They also do not report whether ensemble members were trained with augmentation, nor examine its effect on robustness, whereas we show that heterogeneous augmentation is critical for calibration stability under shift.

On the ablation side, [34] focus exclusively on rank and target component selection, reporting that FF layer adaptation improves accuracy but degrades calibration, a conclusion our results contradict, and which a growing body of evidence from both text and vision literature supports in our favour [36, ?, ?]. Critically, they overlook the scaling factor $\delta_r = \alpha/r$ entirely, which we identify as a key driver of the accuracy–calibration trade-off and a training-free deployment-time calibration knob. Their focus on attention-only adaptation - using up to 16 ensemble members - also makes FF adaptation appear costly, whereas in our setting of $M=4$ members, FF layers provide the strongest performance at the lowest parameter count.

Beyond [34], remaining related works are either restricted to specific vision tasks such as depth estimation, or operate exclusively in the text domain, with no ablation studies and no evaluation under distribution shift. None address dense prediction, which imposes substantially different representational demands than image classification.

ST-LoRA is therefore the first LoRA ensemble framework to jointly address parameter efficiency, calibration robustness under distribution shift, OoD detection, and dense agricultural prediction, providing both empirical evidence and principled guidance for practitioners deploying uncertainty-aware models in resource-constrained real-world environments.

Limitations of ST-LoRA and Future Work

While ST-LoRA has demonstrated strong performance, several limitations and open directions remain.

1. Hyperparameter sensitivity. ST-LoRA inherits the large hyperparameter search space characteristic of LoRA-based methods, making configuration time-consuming; a limitation shared by most uncertainty quantification approaches.

Promising directions: Broader studies across datasets, tasks, and architectures will help identify which hyperparameters matter most and under what conditions. Neural Architecture Search (NAS) [?] is a natural complement, enabling principled adapter placement within the pre-trained encoder rather than searching for architectures from scratch.

2. Miscalibration. Although ST-LoRA improves calibration over strong baselines, residual miscalibration persists; a challenge shared across all uncertainty quantification methods evaluated.

Promising directions: Post-hoc calibration methods [?] are a low-cost remedy that leave the trained model intact. More specifically, we propose training with $\alpha=r$ and subsequently fine-tuning $\alpha$ as a post-hoc calibration knob could simultaneously reduce training complexity and improve deployment-time calibration.

3. Data, task, and model dependency. LoRA adaptation remains sensitive to the specific model, dataset, and task, meaning conclusions from one setting cannot be assumed to transfer directly to another; a known challenge across the LoRA literature.

Promising directions: Extending ablation studies to account for hyperparameter interactions across more diverse tasks would yield more transferable guidelines. Additionally, theoretical analysis of LoRA in the vision domain [?, ?, ?, ?, 47] remains significantly underdeveloped relative to the text domain and represents an important open research direction.

4. Deployment efficiency. Despite its parameter efficiency, ST-LoRA introduces additional FLOPs per forward pass, and maintaining multiple adapter sets on memory-constrained edge devices remains a practical bottleneck.

Promising directions: Post-training pruning [16] of LoRA modules [?] can reduce adapter size further, while quantization [30] - as demonstrated by QLoRA [19] - can substantially reduce memory footprint for edge deployment.

Ethical Implications

The computational efficiency of ST-LoRA carries implications that extend beyond benchmark performance. By restricting fine-tuning to less than 10% of model parameters and eliminating the need for $M$ independent training runs, ST-LoRA substantially reduces GPU hours required to train and deploy uncertainty-aware segmentation models. This reduction in compute demand has a direct environmental consequence: lower energy consumption and, by extension, lower CO$_2$ emissions and water usage during both training and inference. As large-scale deep learning increasingly contributes to the carbon footprint of the technology sector [26], parameter-efficient methods such as ST-LoRA represent a step toward more environmentally responsible machine learning practice.

We believe that the research community has a responsibility to account for these downstream consequences when designing and reporting computational methods, and that efficiency is not only a technical virtue but an ethical one. In addition, these efficiency gains align with broader global sustainability objectives, particularly the United Nations Sustainable Development Goals (SDGs), including SDG 2: Zero Hunger. Climate change remains a critical factor affecting agricultural productivity, land usability, and food security worldwide. By reducing energy consumption and associated emissions, more efficient machine learning methods may contribute, in aggregate, to mitigating climate-related risks that threaten food systems. While the impact of any single methodological improvement is necessarily limited, such efforts collectively support a transition toward more sustainable technological practices that are better aligned with long-term environmental stability and global food security.

Conclusion

We introduced ST-LoRA, a parameter-efficient ensemble framework for uncertainty-aware semantic segmentation that combines Low-Rank Adaptation with snapshot ensembling. By sharing a frozen backbone across ensemble members and restricting each member to lightweight low-rank adapters, ST-LoRA reduces training time by $1/M$, trainable parameters, and checkpoint size; while matching or exceeding full-rank ensembles in segmentation quality and calibration.

Two findings stand out from our ablation study with one that has been proved empirically with support from literature while the second is left for a future work. We have shown that Feed-forward layers are the critical LoRA target for dense prediction, consistently outperforming attention-only adaptation in both accuracy and calibration - contrary to the convention inherited from language model fine-tuning.

For future work, The scaling ratio $\delta_r = \alpha/r$ governs the accuracy/calibration trade-off under distribution shift and could serve as a training-free deployment-time calibration knob, allowing practitioners to adjust the model’s uncertainty behaviour after fine-tuning without retraining. This property has no equivalent in full-rank fine-tuning.

Across two crops, two architectures, and three evaluation schemes, ST-LoRA consistently matched or outperformed Snapshot Ensemble, MC Dropout, and DDU in calibration stability under distribution shift and achieved a perfect image-level AUROC of 0.99 for far out-of-distribution detection. Its lowest cross-seed variance among all methods confirms robust, seed-independent performance.

More broadly, the efficiency gains of ST-LoRA reduce the computational and environmental cost of deploying uncertainty-aware models in agriculture; a domain where the sustainability of the technology should match the sustainability goals it serves.

Acknowledgements

This work has partially been funded by the Deutsche Forschungsgemeinschaft (DFG, German Research Foundation) under Germany’s Excellence Strategy, EXC-2070 - 390732324 - PhenoRob. Further, it has been funded by the Deutsche Forschungsgemeinschaft (DFG, German Research Foundation) - RO 4839/6-1 - 459376902 - AID4Crops. The authors gratefully acknowledge the access to the Marvin cluster of the University of Bonn.

CRediT authorship contribution statement

Mohamed Farag: Conceptualization, Methodology, Formal analysis, Ablation-Study, Software, Validation, Investigation, Writing - original draft, Writing - review & editing, Visualization. Genc Hoxha: Methodology, Investigation, Formal analysis, Visualization, Validation, Writing-review & editing. Yahya Maleki: Ablation-study, Software, Writing. Chris McCool: Methodology, Investigation, Writing - review & editing, Visualization. Ribana Roscher: Methodology, Investigation, Writing - review & editing, Visualization, Supervision, Project administration, Funding acquisition.

References

Appendix A: Further Analysis of The Ablation Study

ST-LoRA Initializer

We also evaluate the impact of different ST-LoRA weight initialization strategies, including EVA, OST-LoRA, and PISSA [?]; [11]; [?]. As shown in Table 15, introducing these specialized initializers does not lead to consistent performance improvements. The baseline configuration without any dedicated ST-LoRA initializer achieves the best overall performance, with a mean IoU of 0.8178 $\pm$ 0.0295, outperforming EVA (0.7795 $\pm$ 0.0455), OLoRA (0.7872 $\pm$ 0.0850), and PISSA (0.8097 $\pm$ 0.0172).

While some initializers yield competitive results, without specialized initializer option provide a clear advantage over the default initialization. This observation suggests that, for the considered segmentation task, explicit ST-LoRA initialization strategies may not be necessary. However, further experiments on different datasets and architectures would be required to confirm whether this behavior generalizes beyond the current setting.

Bias Setting

We further investigate the impact of different bias training strategies, including fine-tuning all biases (Backbone and ST-LoRA), fine-tuning only ST-LoRA biases, and freezing all biases. The results indicate that the performance gap between the None and ST-LoRA-only bias settings is negligible. In contrast, fine-tuning all biases leads to a substantial degradation in performance, resulting in a mean IoU of 0.4348 $\pm$ 0.3574 as shown in Table 15.

This significant drop in performance may be related to the fact that the backbone has been pre-trained and already encodes useful representations through transfer learning. Allowing the backbone biases to be updated can perturb these learned representations, particularly given the depth of the network. Properly adapting such pre-trained components typically requires larger training datasets and longer training to avoid destabilizing previously learned features. Under the current experimental setup, where data and training duration are limited, fine-tuning all biases appears to introduce instability rather than improving adaptation.

Batch Size and Learning Rate

While varying the batch size does not appear to have a noticeable effect on performance, the learning rate plays a crucial role in model behavior and stability.

Ablation on the learning rate reveals two extreme regimes. When the learning rate is set to a large value ($2 \times10^{-1}$), the model completely fails to learn meaningful representations as shown in Table 15, resulting in a mean IoU of 0.0 and a False Negative Rate of 1.0. This indicates that the model collapses to predicting only the background class. Such behavior is expected, as an excessively high learning rate can destabilize optimization, causing the model to overshoot minima and converge to degenerate solutions. Specifically in [10] they mention that LoRA adapters need smaller learning rate compared to full rank fine tuning.

At the other extreme, smaller learning rates such as $2 \times10^{-3}$ and $2 \times10^{-4}$ yield the highest mean IoU scores in our experiments (0.8860 $\pm$ 0.0018 and 0.8802 $\pm$ 0.0023, respectively). Although these results might initially suggest that very small learning rates are optimal. Very small learning rates might limit ST-LoRA diversity as we expect that with small learning rate the approach lose an aspect of diversification which leads to a degenerate solution reached by different checkpoints.

LoRA Dropout and Frozen Encoder/Decoder

As shown in Table 16, fine tuning the dropout value enhances performance. The baseline with $p = 0.10$ achieves mIoU of 68.09% while the best result achieved is 69.0% with $p = 0.20$. However the calibration metrics for the former dropout value is smaller compared to the later one, which reflects the tension between optimizing for segmentation quality versus miscalibration reduction. Table 16 shows that with a frozen encoder and increasing the rank monotonically, mIoU and calibration metrics have shown the worst performance compared to the case where only one component from the encoder to be adapted by LoRA.

For Decoder, we tried to compare the role of fine tuning the decoder compared to having it frozen during encoder fine-tuning. tbl:18 shows a clear trend, for nearly all cases the frozen decoder affects not only the segmentation quality but degrades the calibration heavily.

Further Information: Experimental Setup Settings

All experiments are conducted on NVIDIA A100-PCIE-80GB GPUs from the Marvin cluster, using up to four GPUs per node for training and a single GPU for inference. RAM allocation varies between 128 GB and 512 GB depending on dataset and image size. All results are reported across five random seeds (42, 123, 456, 789, 1337) for statistical significance.

For snapshot-based methods (ST-LoRA and FRE), we retain the last four checkpoints of each training trajectory, discarding the first snapshot as the model is still in an early underfitting phase at that point and including it would distort the ensemble evaluation.

For DDU, we use features from the last layer of the pixel decoder to fit our GMM model. We use the number of pixels for each class in the dataset as a prior by counting the number of pixels for each class per dataset and dividing by the total number of pixels.

Effect of ST-LoRA hyperparameters and architectural choices on Mask2Former segmentation performance on BUP20. $q_d$, $k_d$, and cls denote the transformer decoder’s query projection, key projection, and class predictor, respectively. $q_e$, $k_e$, $v_e$, and $\mathrm{dense}_e$ refer to the Swin-B encoder’s query, key, value projections and dense (MLP) layers. The rank, $\alpha$, and dropout sweeps use $q_d$, $k_d$, cls, $\mathrm{dense}_e$ as target modules. ↑: higher is better; ↓: lower is better. All results report mean ± std over 5 seeds. and best result per metric in bold.

Category	Setting	mIoU ↑	ECE ↓	MECE ↓	ACE ↓	MACE ↓
ST-LoRA rank – $r$	2	0.6582 ± 0.0170	0.0143 ± 0.0058	0.1507 ± 0.0410	0.0646 ± 0.0331	0.0961 ± 0.0377
	4	0.6734 ± 0.0223	0.0089 ± 0.0049	0.0833 ± 0.0361	0.0306 ± 0.0222	0.0510 ± 0.0246
	8 (baseline)	0.6809 ± 0.0171	0.0028 ± 0.0017	0.0939 ± 0.0612	0.0041 ± 0.0020	0.0092 ± 0.0060
	16	0.6791 ± 0.0292	0.0036 ± 0.0019	0.1672 ± 0.0320	0.0024 ± 0.0009	0.0091 ± 0.0059
	32	0.6801 ± 0.0239	0.0030 ± 0.0003	0.0798 ± 0.0653	0.0018 ± 0.0008	0.0106 ± 0.0094
	64	0.6942 ± 0.0122	0.0045 ± 0.0039	0.1053 ± 0.0799	0.0019 ± 0.0025	0.0099 ± 0.0155
ST-LoRA scaling – $\alpha$	2	0.6763 ± 0.0097	0.0161 ± 0.0043	0.2020 ± 0.0945	0.0505 ± 0.0202	0.0759 ± 0.0260
	4	0.6773 ± 0.0230	0.0062 ± 0.0015	0.0678 ± 0.0147	0.0137 ± 0.0045	0.0245 ± 0.0060
	8 (baseline)	0.6809 ± 0.0171	0.0028 ± 0.0017	0.0939 ± 0.0612	0.0041 ± 0.0020	0.0092 ± 0.0060
	16	0.6831 ± 0.0140	0.0026 ± 0.0006	0.1033 ± 0.0968	0.0026 ± 0.0006	0.0074 ± 0.0044
	32	0.6766 ± 0.0145	0.0030 ± 0.0020	0.0746 ± 0.0402	0.0022 ± 0.0005	0.0087 ± 0.0029
ST-LoRA dropout	0.10 (baseline)	0.6809 ± 0.0171	0.0028 ± 0.0017	0.0939 ± 0.0612	0.0041 ± 0.0020	0.0092 ± 0.0060
	0.15	0.6734 ± 0.0165	0.0037 ± 0.0020	0.1277 ± 0.0551	0.0048 ± 0.0010	0.0094 ± 0.0039
	0.20	0.6908 ± 0.0153	0.0044 ± 0.0019	0.0752 ± 0.0342	0.0047 ± 0.0022	0.0098 ± 0.0060
	0.25	0.6825 ± 0.0209	0.0035 ± 0.0024	0.0711 ± 0.0301	0.0053 ± 0.0020	0.0105 ± 0.0054
	0.30	0.6752 ± 0.0144	0.0035 ± 0.0017	0.0622 ± 0.0428	0.0041 ± 0.0024	0.0081 ± 0.0071
Components	$q_d$, $k_d$, cls, $\mathrm{dense}_e$, $q_e$	0.6870 ± 0.0199	0.0030 ± 0.0013	0.0665 ± 0.0285	0.0042 ± 0.0018	0.0093 ± 0.0040
	$q_d$, $k_d$, cls, $\mathrm{dense}_e$, $k_e$	0.6976 ± 0.0099	0.0038 ± 0.0013	0.0796 ± 0.0518	0.0049 ± 0.0031	0.0098 ± 0.0051
	$q_d$, $k_d$, cls, $\mathrm{dense}_e$, $v_e$	0.6599 ± 0.0248	0.0030 ± 0.0011	0.0574 ± 0.0408	0.0040 ± 0.0019	0.0090 ± 0.0039
	$q_d$, $k_d$, cls, $\mathrm{dense}_e$, $q_e$, $k_e$	0.6760 ± 0.0152	0.0027 ± 0.0013	0.0704 ± 0.0424	0.0032 ± 0.0010	0.0075 ± 0.0022
	$q_d$, $k_d$, cls, $\mathrm{dense}_e$, $q_e$, $v_e$	0.6828 ± 0.0154	0.0041 ± 0.0026	0.0699 ± 0.0473	0.0043 ± 0.0017	0.0115 ± 0.0072
	$q_d$, $k_d$, cls, $\mathrm{dense}_e$, $k_e$, $v_e$	0.6972 ± 0.0198	0.0040 ± 0.0012	0.0825 ± 0.0231	0.0036 ± 0.0015	0.0076 ± 0.0037
	$q_d$, $k_d$, cls, $\mathrm{dense}_e$, $q_e$, $k_e$, $v_e$	0.6767 ± 0.0313	0.0032 ± 0.0018	0.0990 ± 0.0296	0.0040 ± 0.0019	0.0098 ± 0.0049