betaFeedback

WorksComputer Science

PACE-MoE: Adaptive Domain Sampling and Domain-Specific LoRA for Medical Visual Grounding

Abstract

Medical visual grounding requires models to align clinical descriptions with spatial evidence across heterogeneous imaging domains. However, existing multi-domain post-training strategies typically rely on fixed domain sampling and shared parameter updates, overlooking the need to adapt both which domains receive training and where their updates are applied. To address this issue, we propose PACE-MoE, a policy-guided framework that jointly performs adaptive domain sampling and domain-specific adaptation for medical visual grounding. PACE dynamically assesses the training value of each imaging domain from current-policy feedback and progressively adjusts domain coverage through domain admission, replay, and final all-domain balancing. Meanwhile, domain-specific LoRA adapters route updates to the corresponding imaging domains while preserving the pretrained backbone and visual encoder. This coordinated design aligns training-data allocation with parameter adaptation without modifying the original reward formulation or Group Relative Policy Optimization (GRPO) objective. Experiments across seven medical imaging domains show that PACE-MoE consistently improves grounding performance over fixed-domain training, achieving a 3.64% relative improvement in mean box IoU over the Vanilla baseline. Under matched domain-specific adapter architectures, adaptive sampling alone further improves mean box IoU by 2.07%, demonstrating the benefit of policy-guided domain allocation. These results establish coordinated data-parameter routing as an effective strategy for multi-domain medical visual grounding.

Introduction

Medical visual grounding requires a model to connect a clinical description with the corresponding image region. A query may refer to a pathology, an anatomical relation, or a diagnostic clue without naming the target directly. The model must therefore combine language understanding with spatial localization. This requirement extends across CT, MRI, radiography, ultrasound, endoscopy, dermoscopy, and mammography, despite their different appearances and acquisition processes.

MedReasoner [1] addresses this problem with a clinical reasoning model that emits a reasoning trace, a box, and two points; frozen MedSAM [2] converts the prompts into a mask. Because format and spatial overlap are directly verifiable, the reasoner can be optimized with Group Relative Policy Optimization (GRPO) [3, 4]. However, improvements from post-training also depend on how heterogeneous training data are used. We treat each imaging modality as a domain with its own data distribution [5]. All seven domains and their labels are available during both training and evaluation. The setting is thus multi-domain learning, rather than generalization to unseen domains.

A useful training mixture should reflect the current model, not only the number of examples in each domain. In GRPO, responses to the same query are compared through their rewards. Malformed responses or nearly identical rewards can provide little useful spatial feedback even when many examples are available. As the model changes, the domains that supply usable feedback can also change. Fixed sampling does not capture this dependence on the learning state.

Data allocation is only one part of the problem. A shared LoRA adapter receives updates from every imaging domain, although the domains may require different adjustments to the pretrained model. Domain-specific adapters retain the common backbone while giving each domain its own trainable parameters. Yet this separation does not determine how often an adapter should be trained. Conversely, adaptive sampling alone does not change which parameters are shared. Figure 1 motivates considering these two decisions together: allocating queries across domains and assigning their updates to the corresponding adapters.

Figure 1. Motivation for domain-aware post-training. (a) A fixed data mixture does not respond to the current model, and one shared LoRA adapter receives updates from all domains. (b) PACE-MoE uses policy feedback to adapt the sampling distribution ps(g∣St)p_s(g \mid S_t). The selected domain label gg also identifies the adapter to update. The pretrained backbone remains shared and frozen.

We propose PACE-MoE to coordinate domain sampling and domain-specific adaptation using the current policy as feedback. A fixed probe set estimates whether each domain provides valid responses and useful reward contrast. The resulting curriculum adjusts domain coverage over training instead of committing to an initial ordering. Replay retains earlier domains, and a final all-domain stage gives additional attention to domains with lower probe reward.

The sampled domain label also selects the LoRA adapter used to generate and learn from its responses. Thus, the curriculum determines the training frequency of each adapter, and the adapted model supplies the next round of feedback. Building on established domain-specific adaptation [6], this design uses the known labels without learning another routing network. The dense backbone, visual encoder, and MedSAM2 remain frozen; the reward and GRPO objective are unchanged. The framework adapts how training data and domain-specific parameters are used while retaining the released grounding pipeline.

The contributions are summarized as follows:

  • We introduce a domain-aware post-training framework that links training-data allocation to domain-specific model updates. It retains shared pretrained representations while assigning each domain’s responses to its corresponding adapter.

  • We develop a policy-guided curriculum that adapts domain coverage to the current model. It combines feedback-based admission with replay and final balancing, so that prioritizing useful feedback does not remove earlier domains from subsequent training.

  • We evaluate the complete framework and its sampling component separately. The full system improves mean box IoU over Vanilla by 3.64% in relative terms. Within-architecture controls assess adaptive sampling across eight shared-adapter seeds and five domain-specific-adapter seeds; ablations examine the roles of replay and final balancing.

Multi-Domain Visual Learning

A domain is a data distribution associated with an acquisition setting, environment, or population. Unlike domain generalization to unseen target domains [5], our setting contains the same seven imaging domains during training and evaluation. Multi-domain methods often combine shared features with domain-specific components [6]. We study this design together with policy-guided domain sampling for GRPO grounding.

Reasoning-Based Medical Visual Grounding

Medical MLLMs extend vision-language modeling to clinical terminology and heterogeneous imaging. MPMA jointly models medical image–text reconstruction and cross-modal alignment to learn representations with domain knowledge [7]. LLaVA-Med uses biomedical visual instruction tuning [8], while Lingshu targets unified medical understanding and reasoning [9]. Language-guided segmentation methods include LISA [10], Seg-Zero [11], and MedGround-R1 [12]. MedReasoner separates reasoning from mask decoding: a trainable reasoner predicts spatial prompts and a frozen segmentation expert produces the mask [1]. We retain this output and reward pipeline while changing domain sampling and the assignment of LoRA adapters.

Spatial Representations and Grounded Visual Reasoning

General visual segmentation offers complementary approaches to maintaining spatial evidence. Isogenous memory sampling with frame-relation mining addresses memory matching and temporal context [13]. Multi-frame context memory captures information across adjacent frames [14], while adaptive sparse memory selects informative past frames and retrieves relevant local features [15]. Video Decoupling Networks separate scene, motion, and instance cues [16], and Fast Track Anything uses sparse spatio-temporal propagation for unified video segmentation [17]. These studies examine visual representations and temporal propagation; our focus is the allocation of domain queries and adapter updates during medical post-training.

Visual-language reasoning also depends on selecting evidence and preserving semantic consistency. REVU retrieves external object-action memory to support long-term video understanding [18]. Hierarchical multimodal consistency uses multi-level alignment to reduce video-language hallucinations [19]. Fact generates faithful, concise, and transferable rationales through verifiable visual programming [20]. Diff-LMM [21] and PhysMLLMs [22] further investigate teacher-guided spatial representation learning for general visual understanding.

Reinforcement Learning for Medical Grounding

GRPO estimates a query-specific baseline from multiple responses, avoiding a separate critic [3], [4]. It normalizes rewards within a group but does not determine domain sampling or parameter sharing. QoQ-Med addresses heterogeneity through Domain-aware Relative Policy Optimization (DRPO), which rescales advantages using domain and cluster statistics [23]. PACE instead changes which domains supply queries before responses are generated. Sampling and post-generation reweighting therefore address different decisions.

Adaptive Data Selection and Curriculum Learning

Curriculum learning organizes examples rather than sampling from an unchanged distribution [24]; self-paced learning adapts that organization to the learner [25]. Recent work has brought adaptive data selection into RL-based LLM post-training. DUMP combines policy advantages with an upper-confidence-bound sampler across data distributions [26], whereas SEC treats problem categories as a non-stationary bandit and uses absolute advantage as the learning signal [27]. Prioritized Replay updates problem priorities from empirical success rates [28], while Actor-Curator learns a neural curator that selects problems to maximize policy improvement [29]. PACE scores known imaging domains on a fixed, gradient-free probe using spatial reward, within-query dispersion, and output validity. It connects staged domain coverage with domain-specific LoRA selection for medical grounding.

Medical curricula use task structure, anatomical similarity, or reasoning difficulty [30, 31, 32]. MedLoc-R1 adapts reward strictness from online performance [33], while CX-Mind uses a predefined closed-to-open curriculum for chest X-ray GRPO [34]. PACE leaves reward strictness unchanged and repeatedly updates the sampling order across seven imaging domains.

Mixture of Experts and Low-Rank Adaptation

Med-MoE learns a router over domain-specific experts for supervised medical VQA and classification [35]. We instead use known modality labels to select low-rank FFN adapters during GRPO grounding. MoE-LLaVA extends sparse expert routing to large vision-language models while keeping only a subset of experts active for each input [36]. More generally, mixture-of-experts models route inputs through specialized parameter subsets [37]; sparse routing has also been used to scale visual representations [38]. Replicating full experts is expensive for a 7B multimodal model. LoRA represents an update with two low-rank matrices while freezing the pretrained weight [39]. MoCLE routes vision-language instructions to cluster-conditional LoRA experts [40]. Our Domain-MoE differs in using an observed medical imaging-domain label, rather than an instruction cluster or a learned router, as a hard Top-1 selection during GRPO. It provides domain-specific adaptation without copying the complete reasoner or adding a load-balancing objective.

Problem Formulation

Multi-Domain Medical Grounding

Let the multi-domain training set be

D=⋃g=1GDg,(1)\mathcal{D} = \bigcup_{g=1}^{G} \mathcal{D}_g, \tag*{(1)}

where gg is the known imaging-domain label. Each example x=(I,Q,Y,g)x = (I,Q,Y,g) contains an image II, a clinical query QQ, and spatial supervision Y=(B∗,P1∗,P2∗,M∗)Y = (B^{*},P_1^{*},P_2^{*},M^{*}). Following MedReasoner, the policy produces

o=(T,B^,P^1,P^2)∼πθ0,ϕg(⋅∣I,Q),(2)o = (T,\hat{B},\hat{P}_1,\hat{P}_2) \sim\pi_{\theta_0,\phi_g}(\cdot\mid I,Q), \tag*{(2)}

where θ0\theta_0 is the frozen shared backbone and ϕg\phi_g is the LoRA adapter for domain gg. A frozen segmenter returns

M^=Fseg(I,B^,P^1,P^2).(3)\hat{M} = F_{\mathrm{seg}}(I,\hat{B},\hat{P}_1,\hat{P}_2). \tag*{(3)}

Group-Relative Policy Optimization

All experiments use the released MedReasoner Soft reward, which combines format checks with smoothed box-IoU and point-overlap terms and penalizes invalid outputs [1]. Our method introduces neither a domain-specific reward nor an additional reward coefficient. PACE uses the same scalar reward for domain scoring and GRPO optimization.

For a query q=(I,Q)q=(I,Q), GRPO samples KK responses. Let rq,kr_{q,k} be the verifiable reward of response kk. Its group-relative advantage is

A^q,k=rq,k−μqσq+ϵ,μq=1K∑j=1Krq,j,(4)\hat{A}_{q,k} = \frac{r_{q,k}-\mu_q}{\sigma_q+\epsilon}, \qquad\mu_q = \frac{1}{K}\sum_{j=1}^{K}r_{q,j}, \tag*{(4)}

where σq\sigma_q is the within-group reward standard deviation. The clipped objective can be written as

J(ϕ;θ0)=Eq,k,t[min⁡{ρq,k,tA^q,k, clip⁡(ρq,k,t,1−ϵ,1+ϵ)A^q,k}−βDKL(πθ0,ϕ∥πref)].(5)\mathcal{J}(\phi;\theta_0) = \mathbb{E}_{q,k,t}\left[\min\left\{\rho_{q,k,t}\hat{A}_{q,k},\ \operatorname{clip}(\rho_{q,k,t},1-\epsilon,1+\epsilon)\hat{A}_{q,k}\right\} - \beta D_{\mathrm{KL}}(\pi_{\theta_0,\phi}\Vert\pi_{\mathrm{ref}})\right]. \tag*{(5)}

Here, ρq,k,t\rho_{q,k,t} is the token-level probability ratio between the current and rollout policies, ϵ\epsilon is the clipping threshold, and β\beta weights regularization toward the fixed reference policy πref\pi_{\mathrm{ref}}.

Domain-Aware Update Allocation

Equations (4)–(5) describe what happens after a query is selected. Standard continuation draws from fixed p(g)p(g) and writes all low-rank updates to one shared adapter. PACE-MoE instead uses a stage-dependent ps(g)p_s(g) and selects the matching adapter ϕg\phi_g. Response format, reward, advantage calculation, and clipped loss remain unchanged. The scheduler is motivated by Eq. (4). A domain with low output validity does not yet provide reliable spatial feedback. A domain with nearly identical response rewards provides weak within-group contrast. PACE uses a small probe set to estimate the current training value of each domain. The resulting score is policy-dependent; it is not a permanent difficulty label.

PACE-MoE

Coordinating Sampling and Adaptation

Figure 2 shows the two components of PACE-MoE. Let St\mathcal{S}_t denote the latest probe statistics. For query group bb at update tt, the scheduler samples gt,b∼ps(g∣St)g_{t,b} \sim p_s(g \mid\mathcal{S}_t) and then draws qt,b∼Dgt,bq_{t,b} \sim\mathcal{D}_{g_{t,b}}. The known label gt,bg_{t,b} also selects ϕgt,b\phi_{g_{t,b}} for rollout generation and optimization. The sampling distribution changes with the current policy, while adapter selection follows the observed domain label. Several adapters may be updated in one optimizer step, but each query group updates only its matching adapter. The sampling schedule determines which adapters receive updates, and the updated policy supplies the next probe statistics. This feedback loop links data allocation and adaptation throughout training.

Overview of PACE-MoE architecture and optimization pipeline

Figure 2. Overview of PACE-MoE. (a) Architecture of PACE-MoE. Current-policy rollouts on a fixed seven-domain probe buffer estimate reward, within-query dispersion, and output validity. These statistics update the domain sampling distribution through admission, replay, and a final all-domain stage. The sampled label gt,bg_{t,b} selects a query from Dg\mathcal{D}_g and activates the corresponding rank-16 FFN LoRA adapter ϕg\phi_g. (b) Optimization pipeline. The selected adapter is updated using the shared spatial reward and GRPO objective, while the MedReasoner backbone and MedSAM2 remain frozen. At each probe event, the updated policy is evaluated again to determine the next domain sampling distribution.

Estimating Domain Training Value

We keep probe queries fixed and regenerate their responses with the current policy to compare domain-level feedback over training. The statistics below are proxies for training value, not direct estimates of the improvement from an additional update. Let Vg\mathcal{V}_g be the fixed probe-query subset for domain gg. Every HH optimizer updates, the current policy generates Kp=2K_p = 2 responses for each query without gradient computation and with the training-rollout decoding settings. We record the domain mean reward rˉg(s)\bar{r}_g^{(s)}, mean within-query reward dispersion ug(s)u_g^{(s)}, and valid output rate vg(s)v_g^{(s)} at probe stage ss:

rˉg(s)=1∣Vg∣Kp∑q∈Vg∑k=1Kprq,k,(6)\bar{r}_g^{(s)} = \frac{1}{|\mathcal{V}_g|K_p}\sum_{q\in\mathcal{V}_g}\sum_{k=1}^{K_p} r_{q,k}, \tag*{(6)}
ug(s)=1∣Vg∣∑q∈VgStd⁡k(rq,k),(7)u_g^{(s)} = \frac{1}{|\mathcal{V}_g|}\sum_{q\in\mathcal{V}_g}\operatorname{Std}_k(r_{q,k}), \tag*{(7)}
vg(s)=1∣Vg∣Kp∑q∈Vg∑k=1KpI[oq,k is valid].(8)v_g^{(s)} = \frac{1}{|\mathcal{V}_g|K_p}\sum_{q\in\mathcal{V}_g}\sum_{k=1}^{K_p}\mathbb{I}[o_{q,k}\ \text{is valid}]. \tag*{(8)}

Here, Std⁡k\operatorname{Std}_k denotes the sample standard deviation across the KpK_p responses. An output is valid when it contains the required reasoning and answer blocks and finite spatial coordinates in the prescribed order.

We first smooth each raw statistic with

z^g(s)=ηz^g(s−1)+(1−η)zg(s),z∈{rˉ,u,v},η=0.60.(9)\hat{z}_g^{(s)} = \eta\hat{z}_g^{(s-1)} + (1-\eta)z_g^{(s)},\quad z\in\{\bar{r},u,v\},\quad\eta=0.60. \tag*{(9)}

We then min–max normalize the smoothed statistics across the GG domains. The raw stage score is

ℓg(s)=λrr~g(s)+λuu~g(s)+λvv~g(s),λr+λu+λv=1.(10)\ell_g^{(s)} = \lambda_r\tilde{r}_g^{(s)}+\lambda_u\tilde{u}_g^{(s)}+\lambda_v\tilde{v}_g^{(s)},\quad\lambda_r+\lambda_u+\lambda_v=1. \tag*{(10)}

Each tilde denotes the normalized version of the corresponding smoothed statistic. Finally, the score itself is smoothed before ranking:

ℓ^g(s)=ηℓ^g(s−1)+(1−η)ℓg(s).(11)\hat{\ell}_g^{(s)} = \eta\hat{\ell}_g^{(s-1)} + (1-\eta)\ell_g^{(s)}. \tag*{(11)}

Domains are ranked by ℓ^g(s)\hat{\ell}_g^{(s)}. The normalized progress measure is Cg(s)=clip⁡(rˉg(s)/Rmax⁡,0,1)C_g^{(s)}=\operatorname{clip}(\bar{r}_g^{(s)}/R_{\max},0,1) with Rmax⁡=4R_{\max}=4.

Mean reward indicates competence, while validity discounts frequent malformed responses, and dispersion measures the reward contrast available to GRPO. Admission therefore favors domains that currently provide usable feedback, rather than simply the hardest examples. Lower-reward domains remain part of the curriculum: all domains become active in the final stage, where the deficit term in Eq. (14) increases their sampling weight.

Scheduling Domain Coverage

PACE converts the probe scores into a curriculum with three functions: admitting domains that currently provide usable feedback, revisiting earlier domains, and ending with coverage of all seven domains.

Progressive Domain Admission

1) Progressive Domain Admission: PACE maintains an active domain set AsA_s and an inactive set LsL_s. At initialization, the m0m_0 domains with the highest priority scores form A0A_0. The remaining domains are ranked again at each probe event rather than assigned a fixed order.

We summarize progress on the active set by

Cs=1∣As∣∑g∈Asc^g(s).(12)C_s = \frac{1}{\lvert A_s\rvert}\sum_{g\in A_s}\hat{c}_g^{(s)}. \tag*{(12)}

Here, c^g(s)\hat{c}_g^{(s)} denotes the smoothed normalized reward used by the competence gate. After at least one probe in the current stage, the next mm locked domains are admitted if either competence is sufficient or progress has remained on a plateau:

Cs≥τc ∨ ⋂j=0P−1{∣Cs−j−Cs−j−1∣<τp}.(13)C_s \ge\tau_c \ \lor\ \bigcap_{j=0}^{P-1}\{\lvert C_{s-j}-C_{s-j-1}\rvert< \tau_p\}. \tag*{(13)}

The first condition waits until the policy reaches a minimum level on the active domains. The second condition prevents training from stalling when progress reaches a plateau. When either condition is met, PACE admits the highest-ranked domains in LsL_s according to the latest probe.

Replay of Earlier Domains

2) Replay of Earlier Domains: Let NsN_s contain the domains newly admitted at stage ss. Let Rs=As∖NsR_s=A_s\setminus N_s contain the earlier active domains. PACE assigns probability mass 1−ρ1-\rho to NsN_s and replay mass ρ\rho to RsR_s. The probability is uniform within each set. Replay reduces the abrupt sampling shift caused by domain admission. It does not add examples or change the loss. If no earlier domain exists, the initial active set receives all probability mass.

Final All-domain Balancing

3) Final All-domain Balancing: The final checkpoint is evaluated on all domains. PACE therefore reserves the last fraction of updates for a stage in which every domain is active. The sampling distribution is

pall(s)(g)=(1−α)1G+αexp⁡(δg(s)/τ)∑j=1Gexp⁡(δj(s)/τ).(14)p_{\mathrm{all}}^{(s)}(g)=(1-\alpha)\frac{1}{G}+\alpha\frac{\exp(\delta_g^{(s)}/\tau)}{\sum_{j=1}^{G}\exp(\delta_j^{(s)}/\tau)}. \tag*{(14)}

with normalized deficit

δg(s)=max⁡jr^j(s)−r^g(s)max⁡jr^j(s)−min⁡jr^j(s)+ϵ.(15)\delta_g^{(s)}=\frac{\max_j\hat{r}_j^{(s)}-\hat{r}_g^{(s)}}{\max_j\hat{r}_j^{(s)}-\min_j\hat{r}_j^{(s)}+\epsilon}. \tag*{(15)}

The uniform component gives every domain non-zero probability. The deficit component assigns more probability to domains with lower probe reward. This stage changes only the sampling probability; it does not rescale advantages.

Domain-Specific Low-Rank Adaptation

We replace the single shared LoRA adapter with a bank of lightweight domain-specific adapters. For FFN projection

MethodAdaptive samplingDomain-specific adapterReward changedReplayAdvantage reweighting
Vanilla GRPO×××××
Balanced GRPO×××××
Fixed curriculum×××✓×
DRPO-style××××✓
Domain-MoE✓✓×××
PACE-GRPO (Ours)✓××✓×
PACE-MoE✓✓×✓×

Table I. Comparison of domain sampling and parameter-sharing strategies.

with frozen weight Wl∈Rdo×diW_l\in\mathbb{R}^{d_o\times d_i}, domain gg owns factors Al,g∈Rr×diA_{l,g}\in\mathbb{R}^{r\times d_i} and Bl,g∈Rdo×rB_{l,g}\in\mathbb{R}^{d_o\times r}. Its forward map is

hl′=Wlhl+γrBl,gAl,ghl,g=label⁡(x).(16)h'_l=W_lh_l+\frac{\gamma}{r}B_{l,g}A_{l,g}h_l,\qquad g=\operatorname{label}(x). \tag*{(16)}

We attach the adapters to gate_proj, up_proj, and down_proj in every language-transformer FFN while excluding the visual tower. There are seven adapters, one per imaging domain, with rank r=16r=16 and scaling γ=16\gamma=16. Standard LoRA initialization makes the initial residual zero, preserving the released model function before RL updates.

The known modality label stored with each example selects exactly one adapter. No learned gate or load-balancing loss is required. At layer ll, all adapters require Gr(di+do)G_r(d_i+d_o) stored parameters, whereas a sample evaluates only r(di+do)r(d_i+d_o) low-rank operations. This is substantially smaller than copying GG dense 7B reasoners and has the active compute order of one ordinary LoRA adapter.

Domain-Conditioned GRPO Updates

At update tt, the scheduler independently samples a domain label gt,bg_{t,b} for each query group b∈{1,…,B}b\in\{1,\ldots,B\}. The query is drawn from Dgt,bD_{g_{t,b}}, and the same label activates ϕgt,b\phi_{g_{t,b}} during rollout generation, old-log-probability computation, and the actor update. Consequently,

∇ϕgJt=∑b:gt,b=g∇ϕgJt,b;(17)\nabla_{\phi_g}\mathcal{J}_t=\sum_{b:g_{t,b}=g}\nabla_{\phi_g}\mathcal{J}_{t,b}; \tag*{(17)}

several adapters may receive gradients in one optimizer step. However, each query group updates only the adapter selected by its domain label. Replay and the final all-domain stage also revisit adapters that were updated earlier in training.

Experimental Setup

Evaluation Protocol

We evaluate whether policy-guided domain allocation improves grounding with domain-specific adapters and whether the sampling benefit also holds with a shared adapter. The domain-specific comparison uses five seeds (1, 18, 42, 123, and 456); the shared-adapter comparison uses eight seeds. Each comparison holds the adapter architecture fixed. We then examine alternative sampling strategies, domain-wise performance, and replay and final-balancing ablations over seeds 42, 1, and 18. A separate 200-update study tests the effect of a larger training budget. Cross-architecture differences provide system-level context because the seed sets, adapter placement, and trainable capacity differ. All adapted models start from the released MedReasoner-Soft-7B checkpoint. The main setting uses 25 optimizer updates; Section VI-G examines 200 updates. We match optimizer updates and gradient-bearing rollouts, not total generation or wall-clock cost. A 25-update run contains 200 gradient-bearing responses. PACE additionally probes at initialization and after updates 5, 10, 15, and 20. Each event evaluates 56 held-out queries with two responses per query, totaling 560 no-gradient responses. A representative PACE run takes about 65–66 minutes on four GPUs; this timing alone does not establish a cost advantage over Vanilla.

Dataset and Splits

We use the released category-shift protocol derived from U-MRG [1]. We denote its 6,590-example gradient-bearing update split as U-MRG-6K; the probe and test sets are separate held-out partitions and are not included in the “6K” name. The update split contains 2,717 CT, 252 dermoscopy, 134 digital mammography, 417 endoscopy, 2,023 MRI, 254 ultrasound, and 793 X-ray cases. The fixed probe buffer contains 56 examples, eight per modality. Probe examples guide domain sampling but do not contribute gradient-bearing responses. The held-out test split contains 1,297 unique samples and is used for evaluation, not domain scoring.

Implementation Details

All runs start from the downloaded released MedReasoner-Soft-7B checkpoint; the reference policy and MedSAM2 remain frozen. Each 25-update run uses four queries per update and two responses per query. Maximum prompt/response lengths are 2,048/256 tokens, learning rate is 10−510^{-5}, and rollouts use temperature 1.0 and top-pp 1.0 with top-kk disabled. Shared-adapter Vanilla/PACE runs use the original rank-64 attention LoRA on query, key, value, and output projections. Domain-MoE runs use seven rank-16 domain-specific FFN LoRA adapters on gate, up, and down projections. Training uses four GPUs on one node. All optimizer and GRPO settings not listed below are kept identical to the released MedReasoner configuration across compared methods.

PACE probes at five-update intervals. It starts with two domains, admits at most two domains at an expansion event, uses replay ratio ρ=0.20\rho= 0.20, and reserves the final 20% of updates for all-domain sampling. The deficit mixture weight and temperature in Eq. (14) are 0.50 and 0.25. The weights on mean reward, reward dispersion, and validity in Eq. (10) are 0.45, 0.30, and 0.25, and the EMA decay in Eq. (9) is 0.60. (Eq. (13)), we set τc=0.60\tau_c = 0.60 and τp=0.01\tau_p = 0.01. A plateau is declared when ∣Cs−Cs−1∣<τp|C_s - C_{s-1}| < \tau_p for P=2P = 2 consecutive probe events. These values were fixed before the multi-seed comparisons and kept unchanged across methods and seeds. A systematic sensitivity analysis is left for future work. Table II summarizes the experimental configuration.

ItemValue
InitializationReleased MedReasoner-Soft-7B
Reasoner backbone / segmenterLingshu-7B / frozen MedSAM2
RL implementationveRL + vLLM
Updates / queries per update25 / 4
Responses per query2
Probe responses per query KpK_p2
Maximum prompt / response length2048 / 256
Rollout temperature / top-p / top-k1.0 / 1.0 / disabled
Learning rate1×10−51 \times10^{-5}
Shared LoRA rank / targets64 / attention q,k,v,oq,k,v,o
Domain-specific LoRA rank / targets16 / FFN gate, up, down
Number of adapters / active adapters7 / Top-1
Probe interval5 updates
Probe events / no-gradient responses5 / 560
Initial / maximum newly admitted domains2 / 2
Replay ratio ρ\rho0.20
All-domain fraction / deficit weight α\alpha0.20 / 0.50
Deficit temperature / EMA decay0.25 / 0.60
Priority-score weights (λr,λu,λv)(\lambda_r,\lambda_u,\lambda_v)(0.45, 0.30, 0.25)
Reward maximum Rmax⁡R_{\max}4.0
Competence threshold τc\tau_c / plateau threshold τp\tau_p0.60 / 0.01
Plateau consecutive events PP2
Hardware4 GPUs, one node

Table II. Configuration of the 25-update experiments.

Baselines and Ablations

a) Released Soft-7B: The public checkpoint is evaluated without additional updates and serves as a zero-update reference.

b) Vanilla-25: The released checkpoint is continued for 25 GRPO updates using the original fixed training mixture. This is the primary control.

c) Balanced-25: The seven domains are sampled with equal probability throughout training. This tests whether a static correction of domain frequency explains the result.

d) Fixed-25: Domains follow the initial probe order: dermoscopy, endoscopy, ultrasound, digital mammography, X-ray, MRI, and CT. The ranking is frozen, while admission, replay, and final all-domain balancing remain identical to PACE-25. This isolates adaptive re-ranking from a fixed ordering.

e) DRPO-style-25: Domain-aware scaling is applied after group-relative normalization. The query batch is smaller than in QoQ-Med; we therefore describe this run as DRPO-style rather than as an exact reproduction of its large-batch clustering regime.

f) PACE-NoReplay-25: The policy-adaptive domain order is retained, but earlier active domains are not sampled while the curriculum focuses on newly admitted domains.

g) PACE-NoFinalBalance-25: Adaptive admission and replay are retained, and all remaining domains are activated at the reserved final stage. The deficit-weighted distribution in Eq. (14) is disabled. This run isolates the final balancing strategy while preserving all-domain coverage.

h) PACE-25: The full sampler uses adaptive admission, replay, and final all-domain sampling with one shared LoRA adapter.

i) MoE-Vanilla-25: The fixed training mixture is retained, but every sample activates its rank-16 domain-specific LoRA adapter. This measures the domain-specific adapter system without adaptive sampling.

j)j) PACE-MoE-25: This is the complete method. Probe statistics determine the domain sampling distribution, and each known domain label selects its matching LoRA adapter.

k)k) Vanilla-200, PACE-200, and PACE-MoE-200: These seed-42 runs extend post-training to 200 updates from the same released initialization and are used to compare final grounding performance under the larger update budget.

Evaluation Metrics

Box IoU is the primary localization metric. We also report two point-overlap scores. The released MedReasoner evaluator implements

pDicerel=2∣Op∩Og∣∣Op∪Og∣(18)\mathrm{pDice}_{\mathrm{rel}} = \frac{2|\mathcal{O}_p \cap\mathcal{O}_g|}{|\mathcal{O}_p \cup\mathcal{O}_g|} \tag*{(18)}

where Op\mathcal{O}_p and Og\mathcal{O}_g are the predicted and reference point regions. We retain this released-evaluator score for direct comparability. We also report the conventional Dice form for the same point regions:

pDicestd=2∣Op∩Og∣∣Op∣+∣Og∣(19)\mathrm{pDice}_{\mathrm{std}} = \frac{2|\mathcal{O}_p \cap\mathcal{O}_g|}{|\mathcal{O}_p| + |\mathcal{O}_g|} \tag*{(19)}

The two scores use different denominators and are reported separately. Grounding scores are percentages. Relative gains are 100(Smethod−Sbase)/Sbase100(S_{\mathrm{method}} - S_{\mathrm{base}})/S_{\mathrm{base}}, using the named baseline’s reported mean Sbase>0S_{\mathrm{base}} > 0. Paired changes and standard deviations use one common study-level Vanilla mean. For zero-baseline qualitative cases, only IoU scores are reported.

The evaluator deduplicates retries by sample identifier. It also verifies the required reasoning and answer blocks, JSON keys, bounding-box ordering, finite coordinates, and distinct points before computing spatial metrics. Evaluation uses deterministic decoding (temperature 0, top-p 1.0, maximum 256 generated tokens). Reported test results cover all 1,297 unique identifiers. For the primary eight-seed study, we report the sample standard deviation and two-sided paired t-test p-values.

Results and Analysis

Grounding Accuracy with PACE-MoE

Table III compares the complete framework with its controls after 25 post-training updates. PACE-MoE achieves the highest mean score on all three grounding metrics. Its mean box IoU is (36.1182±0.1341)%(36.1182 \pm0.1341)\%, compared with (34.8485±0.1502)%(34.8485 \pm0.1502)\% for Vanilla-25, a relative improvement of 3.64%. This comparison evaluates the complete design: adaptive domain sampling with domain-specific LoRA versus fixed sampling with a shared LoRA adapter.

To assess the sampling contribution separately, we compare PACE-MoE with MoE-Vanilla-25 using the same seven rank-16 FFN adapters and five seeds. Adaptive sampling raises mean box IoU from (35.3871±0.1226)%(35.3871 \pm0.1226)\% to (36.1182±0.1341)%(36.1182 \pm0.1341)\%, a relative gain of 2.07%. The corresponding relative gains in pDicerel\mathrm{pDice}_{\mathrm{rel}} and pDicestd\mathrm{pDice}_{\mathrm{std}} are 2.32% and 1.14%. Thus, adaptive sampling improves grounding beyond using domain-specific adapters with a fixed data mixture.

These comparisons answer different questions. The 3.64% system-level gain includes adapter capacity and placement changes and compares eight-seed with five-seed means; it does not isolate either component. The 2.07% within-architecture gain holds adapters and seeds fixed to assess sampling. The shared-adapter study below tests the same sampling strategy with a single adapter.

Qualitative Comparison Across Domains

Figure 3 illustrates the spatial changes in the predictions across all seven domains. The clearest differences occur in the CT and chest X-ray examples. In the CT case (a), the Vanilla-25 prediction falls on a different kidney region and has no overlap with the annotated right kidney. PACE-MoE localizes the target with an IoU of 68.38%. In the X-ray case (c), Vanilla-25 predicts the opposite lung, whereas PACE-MoE overlaps the annotated left lung with an IoU of 84.74%. In both examples, the change concerns identifying the correct anatomical target, rather than only adjusting the boundary of an already overlapping box. The remaining five columns show increased overlap in MRI, ultrasound, endoscopy, dermoscopy, and mammography. Together, these comparisons provide visual context for the full-test-set evaluation in Tables III and IV.

Qualitative comparison across seven imaging domains

Figure 3. Qualitative comparison across seven imaging domains using the same test image and query for each pair. Top: Vanilla-25. Bottom: PACE-MoE. Solid green and dashed orange boxes denote ground truth and predictions, respectively. CT (a) and X-ray (c) retain the full image; the other columns use identical crops for both methods that retain all compared boxes. IoU is reported as a percentage and computed in the original image coordinates.

Consistency of Adaptive Sampling

Table IV isolates adaptive domain sampling. Relative to fixed-mixture training across eight seeds, PACE improves mean box IoU by 1.09% (p=0.0023p = 0.0023), pDicerel\mathrm{pDice}_{\mathrm{rel}} by 0.93% (p=0.0023p = 0.0023), and pDicestd\mathrm{pDice}_{\mathrm{std}} by 1.06% (p=0.0003p = 0.0003). All three remain significant after Holm correction across the reported metrics (adjusted p≤0.0046p \le0.0046). Relative to the released checkpoint, the relative gains in mean IoU are 0.50% for Vanilla-25 and 1.60% for PACE-25. These latter differences are contextual because the released checkpoint is a single, zero-update model rather than a paired training condition.

PACE improves box IoU in all eight paired runs (Figure 4). Normalizing each paired change by the common Vanilla mean of 34.8485% gives gains from 0.33% to 2.34%, with a mean of 1.09% and a sample standard deviation of 0.66%.

Paired seed-level changes from fixed to adaptive domain sampling

Figure 4. Paired seed-level changes from fixed to adaptive domain sampling. Scores are in %. Each line connects runs with the same seed and initialization. All 24 metric–seed pairs improve with adaptive sampling.

Why Adapt the Domain Schedule?

The seed-42 controls answer three narrower questions. First, equal domain sampling does not reproduce the PACE result: PACE improves box IoU by 2.62% relative to Balanced-25. Second, a fixed domain order is also insufficient; PACE improves by 2.97% relative to Fixed-25. Third, advantage reweighting is not equivalent to changing which domains are sampled: PACE improves by 3.32% relative to DRPO-style-25.

Taken together, the matched seed-42 controls show that neither static domain balancing, fixed ordering, nor post-sampling advantage scaling reproduces the full PACE checkpoint.

Domain-Wise Grounding Performance

The aggregate score can conceal whether scheduling benefits a narrow subset of modalities or shifts error from one domain to another. We therefore decompose the matched seed-42 predictions by the seven test domains. The same 1,297 sample identifiers and the same evaluator are used for both checkpoints. PACE improves box IoU in six of the seven domains (Table VI) and Figure 5. The largest relative gains occur on mammography (10.21%) and ultrasound (4.63%), followed by CT (4.34%), endoscopy (2.59%), X-ray (2.25%), and dermoscopy (1.90%). MRI is the only exception, changing from 36.6831% to 36.6709%, a relative decrease of 0.03%. The macro-average score improves by 3.09% relative to the Vanilla macro average, compared with a 2.33% improvement in the sample-weighted score. Some of the largest gains occur in smaller domains. At the same time, CT contains 528 test cases and improves by 4.34%, so the aggregate result is not produced solely by rare modalities. These values are descriptive results from the matched seed-42 pair and are not used for domain-level significance claims.

Seed-42 relative box-IoU gains by imaging domain

Figure 5. Seed-42 relative box-IoU gains by imaging domain, using each domain’s Vanilla-25 score as the baseline. Six domains improve; MRI changes by −0.03%.

MethodNNBox IoU↑\uparrowpDicerel↑pDice_{\mathrm{rel}}\uparrowpDicestd↑pDice_{\mathrm{std}}\uparrow
Released Soft-7B [1]–34.673430.428221.8461
Vanilla-25 [1]834.8485 ± 0.150230.5631 ± 0.063221.8746 ± 0.0565
PACE-25835.2297 ± 0.364530.8478 ± 0.207622.1075 ± 0.1303
MoE-Vanilla-25535.3871 ± 0.122630.9523 ± 0.250221.9571 ± 0.0899
PACE-MoE-25 (Ours)536.1182 ± 0.134131.6718 ± 0.287722.2067 ± 0.0929

Table 3. Multi-seed system comparison and sampling controls after 25 updates. Shared-adapter rows report eight seeds; domain-specific rows report five seeds (1, 18, 42, 123, and 456). Grounding scores are mean ± sample standard deviation, in %. Cross-architecture rows compare complete systems; within-architecture pairs assess adaptive sampling.

MethodBox IoU↑\uparrowpDicerel↑pDice_{\mathrm{rel}}\uparrowpDicestd↑pDice_{\mathrm{std}}\uparrow
Released Soft-7B [1]34.673430.428221.8461
Vanilla-25 [1]34.8485 ± 0.150230.5631 ± 0.063221.8746 ± 0.0565
PACE-25 (Ours)35.2297 ± 0.364530.8478 ± 0.207622.1075 ± 0.1303
Relative gain (%)+1.09+0.93+1.06
Paired tt-test pp0.00230.00230.0003

Table 4. Shared-adapter results after 25 updates on 1,297 test cases. Scores (%): mean ± sample standard deviation over 8 seeds (1, 18, 36, 42, 123, 456, 999, 2026). Gains are relative to Vanilla-25; paired tests use original scores.

MethodBox IoU↑\uparrowpDicerel↑pDice_{\mathrm{rel}}\uparrowpDicestd↑pDice_{\mathrm{std}}\uparrow
Released Soft-7B [1]34.673430.428221.8461
Vanilla-25 [1]35.012930.593321.9745
Balanced-2534.913530.681822.0257
Fixed-2534.796230.702722.0253
DRPO-style-25 [23]34.677430.770521.8747
PACE-NoReplay-2534.887130.869021.9824
PACE-NoFinalBalance-2534.764730.586321.9337
PACE-25 (Ours)35.829831.083222.3856

Table 5. Seed-42 control study. Grounding scores are in %. Every adapted method starts from the same checkpoint and uses 25 optimizer updates.

Contribution of Replay and Final Balancing

Across three random seeds (42, 1, 18), we repeat the ablation study to assess the contribution of replay and final balancing. Table VII and Figure 6 summarize the results. Across seeds 42, 1, and 18, full PACE improves mean box IoU over Vanilla-25 by 1.59%. Normalizing paired changes by the common Vanilla mean gives a range of 1.20%–2.34%. PACE-NoReplay obtains a mean IoU of 34.9364%, and PACE-NoFinalBalance obtains 34.9302%. Both values are close to the Vanilla-25 mean of 34.9320%. The ablation pattern is not identical in every seed.

Paired ablations across seeds 42, 1, and 18

Figure 6. Paired ablations across seeds 42, 1, and 18: mean ±1 sample standard deviation of paired box-IoU changes, both normalized by the common Vanilla-25 mean (34.9320%) and expressed as percentages.

Imaging domainNNVanilla-25PACE-25Gain (%)
CT52825.519226.6257+4.34
Dermoscopy5155.775456.8357+1.90
Mammography2322.392824.6799+10.21
Endoscopy8544.506245.6580+2.59
MRI39736.683136.6709−0.03
Ultrasound5336.156037.8294+4.63
X-ray16051.969853.1400+2.25
Macro average–39.000440.2056+3.09
Overall1,29735.012935.8298+2.33

Table 6. Seed-42 domain-level box IoU (%) on the 1,297-case test split. Gain is the relative change from Vanilla-25, in %. The macro average weights the seven domains equally, whereas the overall result is sample-weighted.

MethodBox IoU↑pDicerel_{\mathrm{rel}} ↑pDicestd_{\mathrm{std}} ↑
Vanilla34.9320 ± 0.106930.5628 ± 0.061221.8899 ± 0.0774
w/o replay34.9364 ± 0.097030.7735 ± 0.083721.8937 ± 0.0775
w/o final balance34.9302 ± 0.169230.6962 ± 0.112021.8361 ± 0.0855
PACE (full)35.4867 ± 0.308830.9714 ± 0.099322.2168 ± 0.1527

Table 7. PACE ablations after 25 updates across seeds 42, 1, and 18. Grounding scores are mean ± standard deviation, in %.

Both variants slightly exceed Vanilla-25 in seeds 1 and 18 but fall below it in seed 42. The magnitude of the component effect therefore depends on the seed, while full PACE remains above both variants in all three matched comparisons. This consistent ordering supports retaining replay and final balancing together in the tested schedule.

Effect of Additional Post-Training Updates

MethodIoU↑pDicerel_{\mathrm{rel}} ↑pDicestd_{\mathrm{std}} ↑
Released Soft-7B [1]34.673430.428221.8461
Vanilla-200 [1]34.413830.254122.0571
PACE-20034.586730.373121.8042
PACE-MoE-20034.669130.382221.9208

Table 8. Seed-42 extended-training analysis after 200 post-training updates. Grounding scores are in %.

At 200 updates, PACE-MoE achieves the highest final box IoU among the three continuation methods (Table VII). Its relative gains are 0.24% over PACE-200 and 0.74% over Vanilla-200. Its box IoU of 34.6691% is close to the released checkpoint at 34.6734%, whereas the other continuation methods finish further below that reference. This endpoint does not improve on the released model and is below the five-seed mean observed after 25 updates. The 200-update result uses only seed 42, so the two summaries do not form a paired training curve. The result shows that the reported gains cannot be assumed to persist with more updates; it does not establish a convergence rate or a forgetting mechanism.

Discussion

What Does Policy Feedback Contribute?

The full-system comparison evaluates the combined design, whereas the within-architecture comparisons assess adaptive sampling without changing adapter placement or capacity. The latter show a positive mean sampling gain in both adapter architectures. The fixed-order control retains admission, replay, and final balancing, but freezes the initial ranking. Its lower seed-42 result suggests that updating the order matters beyond merely dividing training into stages. The ablations also show that prioritization alone is insufficient in the tested setting. Replay keeps earlier domains in the sampling distribution, while final balancing increases the probability of lower-reward domains. Removing either component brings the three-seed mean close to Vanilla-25. The practical role of the complete schedule is therefore to combine selective training with continued domain coverage. This observation supports the design without identifying catastrophic forgetting as the cause of an ablation’s lower score.

Domain-Specific Adaptation and Training Cost

Sampling controls how often each domain is trained; the known label selects the parameters that receive its updates. Neither decision requires changing the GRPO advantage or reward. The design shares one frozen backbone and activates one adapter per sample, although all seven adapters must be stored. Probe generation also adds computation. Consequently, the experiments support better grounding at a fixed optimizer-update budget, not a reduction in total generation or wall-clock cost.

Limitations

The evidence covers one benchmark and one backbone. The shared-adapter and domain-specific comparisons use eight and five seeds, respectively, but do not provide a capacity- and placement-matched estimate of the adapter effect. The domain-wise aggregate analysis is available for the shared-adapter pair, not for the complete PACE-MoE system. The probe size and scheduler hyperparameters also lack systematic sensitivity analysis. Hard routing assumes correct imaging-domain labels; missing, mixed, and noisy labels are not evaluated. The experiments do not include compute-matched controls or direct implementations of recent bandit and prioritized-replay samplers. Finally, box and point grounding do not measure final mask quality or clinical utility. Broader validation should examine these factors and training budgets beyond the repeated-seed 25-update setting.

Conclusion

We presented PACE-MoE for domain-aware medical visual grounding. Its core idea is to use policy feedback to allocate training queries and the known domain label to assign their updates to domain-specific adapters. This coordinates data allocation and model adaptation while preserving shared pretrained representations and the original GRPO objective. Across seven imaging domains, the complete method reaches 36.12% mean box IoU after 25 updates, a 3.64% relative improvement over Vanilla with fixed sampling and a shared adapter. With the domain-specific adapter architecture held fixed, adaptive sampling provides a 2.07% relative gain over fixed sampling; a positive sampling gain also holds with shared adapters. Ablations support combining adaptive admission with replay and final balancing. These findings motivate further study of coordinated domain sampling and adaptation across backbones, datasets, and training budgets.

References

  1. [1]Z. Yan et al., “MedReasoner: Reinforcement learning drives reasoning grounding from clinical thought to pixel-level precision,” in Proc. AAAI Conf. Artif. Intell., vol. 40, no. 14, 2026, pp. 11577–11585.
  2. [2]J. Ma et al., “SegSAM2: Segment anything in 3D medical images and videos,” arXiv:2504.03600, 2025.arxiv.org/abs/2504.03600
  3. [3]Z. Shao et al., “DeepSeekMath: Pushing the limits of mathematical reasoning in open language models,” arXiv:2402.03300, 2024.arxiv.org/abs/2402.03300
  4. [4]D. Guo et al., “DeepSeek-R1: Incentivizing reasoning capability in LLMs via reinforcement learning,” arXiv:2501.12948, 2025.DOI
  5. [5]K. Zhou et al., “Domain generalization: A survey,” IEEE Trans. Pattern Anal. Mach. Intell., vol. 45, no. 4, pp. 4396–4415, 2023.DOI
  6. [6]K. Omi, J. Kimata, and T. Tamaki, “Model-agnostic multi-domain learning with domain-specific adapters for action recognition,” arXiv:2204.07270, 2022.DOI
  7. [7]K. Zhang, Y. Yang, J. Yu, H. Jiang, J. Fan, Q. Huang, and W. Han, “Multi-task paired masking with alignment modeling for medical vision-language pre-training,” IEEE Trans. Multimedia, vol. 26, pp. 4706–4721, 2024, doi: 10.1109/TMM.2023.3325965.DOI
  8. [8]C. Li et al., “LLaVA-Med: Training a large language-and-vision assistant for biomedicine in one day,” in Adv. Neural Inf. Process. Syst., vol. 36, 2023, pp. 28541–28564.arxiv.org/abs/2306.00890
  9. [9]W. Xu et al., “Lingshu: A generalist foundation model for unified multimodal medical understanding and reasoning,” arXiv:2506.07044, 2025.arxiv.org/abs/2506.07044
  10. [10]X. Lai et al., “LISA: Reasoning segmentation via large language model,” in Proc. IEEE/CVF Conf. Comput. Vis. Pattern Recognit., 2024, pp. 9579–9589.arxiv.org/abs/2308.00692
  11. [11]Y. Liu et al., “Seg-Zero: Reasoning-chain guided segmentation via cognitive reinforcement,” arXiv:2503.06520, 2025.arxiv.org/abs/2503.06520
  12. [12]H. Xu et al., “MedGround-R1: Advancing medical image grounding via spatial-semantic rewarded group relative policy optimization,” in Proc. MICCAI, 2025.arxiv.org/abs/2507.02994
  13. [13]J. Dang, H. Zheng, J. Lai, X. Yan, and Y. Guo, “Efficient and robust video object segmentation through isogenous memory sampling and frame relation mining,” IEEE Trans. Image Process., vol. 32, pp. 3924–3938, 2023, doi: 10.1109/TIP.2023.3280389.DOI
  14. [14]J. Dang, H. Zheng, X. Xu, L. Wang, and Y. Guo, “Beyond appearance: Multi-frame spatio-temporal context memory networks for efficient and robust video object segmentation,” IEEE Trans. Image Process., vol. 33, pp. 4853–4866, 2024, doi: 10.1109/TIP.2024.3432390.DOI
  15. [15]J. Dang, H. Zheng, X. Xu, L. Wang, Q. Hu, and Y. Guo, “Adaptive sparse memory networks for efficient and robust video object segmentation,” IEEE Trans. Neural Netw. Learn. Syst., vol. 36, no. 2, pp. 3820–3833, 2025, doi: 10.1109/TNNLS.2024.3357118.DOI
  16. [16]J. Dang, H. Zheng, Y. Guo, J. Lai, B. Hu, and T.-S. Chua, “Video decoupling networks for accurate, efficient, generalizable, and robust video object segmentation,” IEEE Trans. Image Process., vol. 35, pp. 1218–1230, 2026, doi: 10.1109/TIP.2025.3649360.DOI
  17. [17]J. Dang, H. Zheng, Z. Chen, Z. Li, Y. Guo, and T.-S. Chua, “Fast track anything with sparse spatio-temporal propagation for unified video segmentation,” IEEE Trans. Image Process., vol. 35, pp. 955–969, 2026, doi: 10.1109/TIP.2025.3649365.DOI
  18. [18]J. Dang et al., “External memory matters: Generalizable object-action memory for retrieval-augmented long-term video understanding,” in Proc. Int. Joint Conf. Artif. Intell., 2025, pp. 864–872, doi: 10.24963/ijcai.2025/97.DOI
  19. [19]J. Dang et al., “Hallucination reduction in video-language models via hierarchical multimodal consistency,” in Proc. Int. Joint Conf. Artif. Intell., 2025, pp. 9167–9175, doi: 10.24963/ijcai.2025/1019.DOI
  20. [20]M. Gao, S. Chen, L. Pang, Y. Yao, J. Dang, W. Zhang, J. Li, S. Tang, Y. Zhuang, and T.-S. Chua, “Fact: Teaching MLLMs with faithful, concise and transferable rationales,” in Proc. ACM Int. Conf. Multimedia, 2024, pp. 846–855, doi: 10.1145/3664647.3681552.DOI
  21. [21]J. Dang et al., “Diff-LMM: Diffusion teacher-guided spatio-temporal perception for video large multimodal models,” in Proc. Int. Joint Conf. Artif. Intell., 2025, pp. 873–881, doi: 10.24963/ijcai.2025/98.
  22. [22]S. Yan et al., “PhysMLLMs: Spatial priors for unified referring segmentation and grounded reasoning of images and videos,” arXiv:2608.24574, 2026.arxiv.org/abs/2608.24574
  23. [23]W. Dai, P. Chen, C. Ekbote, and P. P. Liang, “OoQ-Med: Building multimodal clinical foundation models with domain-aware GRPO training,” arXiv:2506.00711, 2025.arxiv.org/abs/2506.00711
  24. [24]Y. Bengio, J. Louradour, R. Collobert, and J. Weston, “Curriculum learning,” in Proc. Int. Conf. Mach. Learn., 2009, pp. 41–48.
  25. [25]M. P. Kumar, B. Packer, and D. Koller, “Self-paced learning for latent variable models,” in Adv. Neural Inf. Process. Syst., vol. 23, 2010.
  26. [26]Z. Wang, G. Cui, Y.-J. Li, K. Wan, and W. Zhao, “DUMP: Automated distribution-level curriculum learning for RL-based LLM post-training,” arXiv:2504.09710, 2025.arxiv.org/abs/2504.09710
  27. [27]X. Chen et al., “Self-evolving curriculum for LLM reasoning,” arXiv:2505.14970, 2025.arxiv.org/abs/2505.14970
  28. [28]M. Fatemi, “Prioritized replay for RL post-training,” arXiv:2601.02648, 2025.arxiv.org/abs/2601.02648
  29. [29]Z. Gu et al., “Actor-Curator: Co-adaptive curriculum learning via policy-improvement bandits for RL post-training,” arXiv:2602.20532, 2026.arxiv.org/abs/2602.20532
  30. [30]S. Rui, K. Chen, W. Ma, and X. Wang, “Improving medical reasoning with curriculum-aware reinforcement learning,” arXiv:2505.19213, 2025.arxiv.org/abs/2505.19213
  31. [31]Z. Song et al., “Anatomy-R1: Enhancing anatomy reasoning in multi-modal large language models via anatomical similarity curriculum and group diversity augmentation,” arXiv:2512.19512, 2025.arxiv.org/abs/2512.19512
  32. [32]S. Y. Kim, S. Cho, V.-D. Yun, and G. Hwang, “MedCLM: Learning to localize and reason via a CoT-curriculum in medical vision-language models,” arXiv:2510.04477, 2025.arxiv.org/abs/2510.04477
  33. [33]G. Yang et al., “MedLoc-R1: Performance-aware curriculum reward scheduling for GRPO-based medical visual grounding,” arXiv:2603.28120, 2026.arxiv.org/abs/2603.28120
  34. [34]W. Li et al., “CX-Mind: A pioneering multimodal large language model for interleaved reasoning in chest X-ray via curriculum-guided reinforcement learning,” Inf. Fusion, vol. 129, Art. no. 104027, 2026.arxiv.org/abs/2508.03733
  35. [35]S. Jiang, T. Zheng, Y. Zhang, Y. Jin, L. Yuan, and Z. Liu, “Med-MoE: Mixture of domain-specific experts for lightweight medical vision-language models,” in Findings Assoc. Comput. Linguistics: EMNLP, 2024, pp. 3843–3860.arxiv.org/abs/2404.10237
  36. [36]B. Lin, Z. Tang, Y. J. Huang, J. Zhang, Y. Pang, P. Jin, M. Ning, J. Luo, and L. Yuan, “MoE-LLaVA: Mixture of experts for large vision-language models,” IEEE Trans. Multimedia, vol. 28, pp. 4408–4419, 2026, doi: 10.1109/TMM.2026.3654458.DOI
  37. [37]N. Shazeer et al., “Outrageously large neural networks: The sparsely-gated mixture-of-experts layer,” in Proc. Int. Conf. Learn. Represent., 2017.arxiv.org/abs/1701.06538
  38. [38]C. Riquelme et al., “Scaling vision with sparse mixture of experts,” in Adv. Neural Inf. Process. Syst., vol. 34, 2021, pp. 8583–8595.arxiv.org/abs/2106.05974
  39. [39]E. J. Hu et al., “LoRA: Low-Rank adaptation of large language models,” in Proc. Int. Conf. Learn. Represent., 2022.arxiv.org/abs/2106.09685
  40. [40]Y. Gou et al., “Mixture of cluster-conditional LoRA experts for vision-language instruction tuning,” arXiv:2312.12379, 2023.arxiv.org/abs/2312.12379

Paper details

vlnc:W_qm926n4u
Version
v1 · latest
Published
Fresh — added to the Hub within the last 30 days
Added
License
CC0-1.0
Usage

views · 0

downloads · 0

A view counts once per browser per day, a download each time the PDF is requested.

  • Computer Science

Not recorded for this paper

View statement
Looking for related papers…
Contents