betaFeedback

WorksComputer Science

Contrastive Activation-Guided Zeroth-Order Preference Optimisation for Memory-Efficient LLM Alignment

Abstract

Direct Preference Optimisation (DPO) simplifies reinforcement learning from human feedback, but still relies on traditional backpropagation and its associated activation-memory cost. Zeroth-order (ZO) optimisation avoids backpropagation, yet isotropic perturbations can produce noisy updates for preference objectives defined by differences between chosen and rejected responses. We introduce Contrastive Activation-guided Zeroth-Order Direct Preference Optimisation (CA-ZODPO), which constructs a low-rank search subspace from the activation discrepancy between the chosen and rejected responses in each preference pair. Perturbations are restricted to this subspace, while momentum is maintained in low-rank coordinates to preserve the memory advantages of ZO training. On Qwen2.5-1.5B with Anthropic HH-RLHF, CA-ZODPO outperforms common ZO baselines and remains competitive with first-order DPO in pairwise preference evaluation. It also reduces peak GPU memory by about 59% at a sequence length of 2048 while retaining the sequence-length scaling behaviour of zeroth-order training.

Introduction

Preference optimisation is a common stage of language-model post-training. Reinforcement learning from human feedback (RLHF) [1] uses human preference data to steer model behaviour, but requires a multi-stage training pipeline. Direct Preference Optimisation (DPO) [2] simplifies this process by expressing preference learning as a closed-form objective over preferred and rejected responses. Despite this simplification, standard DPO still relies on backpropagation and therefore requires storing intermediate activations and gradients during training. These memory costs can add up to 12×12\times that of inference [3] and can limit preference alignment when model size, sequence length, or batch size increases.

Zeroth-order (ZO) optimisation provides an alternative by estimating update directions using only forward evaluations. MeZO [3] showed that this approach can reduce the memory cost of full-parameter language-model fine-tuning to a level close to inference. Its gradient estimates, however, are obtained from random perturbations over a high-dimensional parameter space, which can lead to noisy updates. This results in a limited use of ZO methods beyond classification tasks. AGZO [4] then shows that forward activations can be used to construct lower-dimensional perturbation subspaces, improving on prior ZO methods, though its application remains confined to classification tasks.

Prior work has explored ZO methods for preference optimisation. Galatolo et al. [5] investigate ZO with RLHF-style objectives, but it relies on a reward model and does not generalise to other preference-learning objectives such as DPO. Zhang and Ying [6] develop ZO policy-gradient methods that learn from preferences without explicit reward inference, but focus on general RL settings rather than LLM alignment. Meanwhile, preference pairs, central to preference learning objectives such as DPO, have been shown to induce additional geometric structure in the activation space [7], a property that no ZO method has yet exploited.

Based on these ideas, we propose Contrastive Activation-guided Zeroth-Order Direct Preference Optimisation (CA-ZODPO), which uses the structure of preference pairs to guide the zeroth-order search. For each linear layer, CA-ZODPO computes the activation difference between the chosen and rejected responses and extracts a low-rank basis from this contrastive representation. Zeroth-order perturbations are then sampled only within the resulting subspace rather than across the full parameter space. The preference pair therefore serves two roles. It defines the DPO objective and provides information about the directions used to search for parameter updates. CA-ZODPO further uses a two-sided finite-difference estimator and stores momentum in low-rank coordinates, avoiding parameter-sized optimiser states. Our main contribution lies in successfully applying ZO to the DPO objective.

We evaluate CA-ZODPO on Qwen2.5-1.5B [8] with the Anthropic HH-RLHF preference dataset [9], considering both alignment quality and GPU memory use. Isotropic MeZO underperforms the supervised fine-tuning (SFT) starting policy, whereas CA-ZODPO improves over SFT and produces a nearly balanced comparison with first-order DPO (FO-DPO). A preference-agnostic activation-subspace variant performs worse than the contrastive formulation, indicating that the chosen–rejected difference contributes beyond dimensionality reduction alone. CA-ZODPO also retains the sequence-length memory scaling of zeroth-order training: at a sequence length of 2048, peak GPU memory is 11.08 GB compared with 26.94 GB for first-order DPO. The reduced memory footprint also permits larger training batches under the same hardware budget.

Contrastive Activation-guided Zeroth-Order DPO

DPO without backpropagation

Let xx be a prompt, ywy_w the preferred response, yly_l the rejected response, πθ\pi_\theta the policy, and πref\pi_{\mathrm{ref}} the reference model. DPO [2] minimises

LDPO(θ)=−E[log⁡σ(βlog⁡πθ(yw∣x)πref(yw∣x)−βlog⁡πθ(yl∣x)πref(yl∣x))].(1)\mathcal{L}_{\mathrm{DPO}}(\theta)=-\mathbb{E}\left[\log\sigma\left(\beta\log\frac{\pi_\theta(y_w\mid x)}{\pi_{\mathrm{ref}}(y_w\mid x)}-\beta\log\frac{\pi_\theta(y_l\mid x)}{\pi_{\mathrm{ref}}(y_l\mid x)}\right)\right]. \tag*{(1)}

Instead of computing ∇θLDPO\nabla_\theta\mathcal{L}_{\mathrm{DPO}} by backpropagation, a ZO method evaluates the loss at perturbed parameters. For a direction Δ\Delta and perturbation scale ϵ\epsilon, we use the central estimate

g=LDPO(θ+ϵΔ)−LDPO(θ−ϵΔ)2ϵ.(2)g=\frac{\mathcal{L}_{\mathrm{DPO}}(\theta+\epsilon\Delta)-\mathcal{L}_{\mathrm{DPO}}(\theta-\epsilon\Delta)}{2\epsilon}. \tag*{(2)}

MeZO samples Δ\Delta isotropically over the parameter space. For DPO, this means that the update is estimated from directions that are mostly unrelated to the distinction between ywy_w and yly_l.

Preference-guided subspaces

Consider a linear layer ll with weight Wl∈Rdout×dinW_l\in\mathbb{R}^{d_{\mathrm{out}}\times d_{\mathrm{in}}}. Let Hch(l)H_{\mathrm{ch}}^{(l)} and Hrej(l)∈Rdin×mH_{\mathrm{rej}}^{(l)}\in\mathbb{R}^{d_{\mathrm{in}}\times m} denote its input activations for the chosen and rejected sequences, where m=b×Tm=b\times T is the number of token positions in the minibatch after flattening the batch and sequence dimensions. AGZO is motivated by the factorisation of a linear-layer gradient into upstream gradients and input activations, which places the gradient row space inside the span of the input activations [4]. For preference learning, we refine this idea by using the difference between the two response representations.

We form a contrastive matrix by combining the mean activation discrepancy over the mm token positions with token-wise differences,

HΔ(l)=concat⁡(Mean⁡(Hch(l))−Mean⁡(Hrej(l)), Hch(l)−Hrej(l)),(3)H_{\Delta}^{(l)}=\operatorname{concat}\left(\operatorname{Mean}\left(H_{\mathrm{ch}}^{(l)}\right)-\operatorname{Mean}\left(H_{\mathrm{rej}}^{(l)}\right),\ H_{\mathrm{ch}}^{(l)}-H_{\mathrm{rej}}^{(l)}\right), \tag*{(3)}

and approximate its leading rank-rlr_l subspace with a small number of power-iteration steps. Let Al∈Rdin×rlA_l \in\mathbb{R}^{d_{\mathrm{in}}\times r_l} be the resulting orthonormal basis. We then sample Rl∼N(0,I)R_l \sim\mathcal{N}(0,I) with Rl∈Rdout×rlR_l \in\mathbb{R}^{d_{\mathrm{out}}\times r_l} and define:

Δl=RlAlTrl.(4)\Delta_l = \frac{R_l A_l^{\mathsf{T}}}{\sqrt{r_l}}. \tag*{(4)}

The factor 1/rl1/\sqrt{r_l} keeps the perturbation scale comparable across ranks. Non-linear parameters use the standard isotropic perturbation. After the basis has been extracted, the activation matrices used to construct it are released before the perturbed loss evaluations.

Low-rank momentum and memory cost

To avoid a parameter-sized momentum state, we store momentum in the low-rank coefficient space [10]. When the basis changes, the previous buffer is projected into the new basis before accumulation, and the weight update is reconstructed as

Nt=μN~t−1+(1−μ)gtRt,Wt+1=Wt−ηtNtAtTrl,(5)N_t = \mu\widetilde{N}_{t-1} + (1-\mu)g_tR_t,\qquad W_{t+1} = W_t - \eta_t \frac{N_t A_t^{\mathsf{T}}}{\sqrt{r_l}}, \tag*{(5)}

where N~t−1=Nt−1(At−1TAt)\widetilde{N}_{t-1} = N_{t-1}(A_{t-1}^{\mathsf{T}}A_t) denotes the previous momentum buffer expressed in the current basis AtA_t and μ\mu is the momentum coefficient. Thus, a linear layer stores only the basis and momentum coefficients, requiring (din+dout)r(d_{\mathrm{in}} + d_{\mathrm{out}})r elements. Each update uses one forward pass for subspace extraction and two for the central finite-difference estimate.

Experiments

Setup. We use Qwen2.5-1.5B as the base model and Anthropic Helpful and Harmless (HH-RLHF) as the preference dataset1. The base model is first supervised fine-tuned on chosen responses to obtain the SFT policy and the fixed reference model. We compare four training variants: standard first-order DPO (FO-DPO), isotropic MeZO applied to the DPO objective, CA-ZODPO, and CA-ZODPO-plain. The last variant extracts its basis from the combined chosen and rejected activations, Hplain(l)=[Hch(l),Hrej(l)]H_{\mathrm{plain}}^{(l)} = [H_{\mathrm{ch}}^{(l)}, H_{\mathrm{rej}}^{(l)}], without computing their difference, and the same low-rank basis extraction and perturbation procedure is then applied. It is used to test whether contrastive structure matters beyond activation guidance alone.

The main experiments use bfloat16 (BF16) precision with an effective batch size of 128 and a maximum sequence length of 2048. Alignment is evaluated on 500 held-out prompts using Qwen2.5-32B-Instruct as a pairwise judge. Each pair is judged in both presentation orders, with inconsistent decisions counted as ties. Memory experiments use a single 48 GB NVIDIA A40. Hyperparameters and generation settings are reported in the Appendix.

Results

Alignment quality. Figure 1 compares the methods against SFT and First-Order Direct Preference Optimisation (FO-DPO). MeZO does not improve over SFT, with a lower win rate than loss rate (25.8% vs. 37.4%). In contrast, CA-ZODPO achieves a positive win–loss margin with 34.6% wins vs. 28.6%. Against FO-DPO, MeZO again shows a clear negative margin, whereas CA-ZODPO is nearly balanced, with 34.2% wins and 33.0% losses. These results show that restricting zeroth-order perturbations to the contrastive activation subspace improves preference learning over isotropic perturbations and narrows the gap to first-order DPO.

Pairwise win/tie/loss rates judged by Qwen2.5-32B-Instruct

Figure 1. Pairwise win/tie/loss rates judged by Qwen2.5-32B-Instruct. Left: comparisons against SFT. Right: comparisons against FO-DPO. CA-ZODPO achieves the best win-rate profile among zeroth-order methods and remains close to FO-DPO.

What does the contrastive subspace contribute? The preference-agnostic CA-ZODPO-plain variant performs better than MeZO in both panels of Figure 1, but remains worse than the full CA-ZODPO model. Against SFT, CA-ZODPO-plain obtains a 30.6% win rate and a 31.6% loss rate, while CA-ZODPO achieves 34.6% wins and 28.6% losses. A similar gap appears against FO-DPO. These results suggest that activation-guided dimensionality reduction is useful, but the chosen–rejected activation difference provides additional information for preference optimisation. Additional rank and training-loss ablations are provided in the supplementary material.

Memory efficiency. Figure 2 compares peak GPU memory under sequence-length and batch-size scaling. With batch size fixed at 4, FO-DPO increases from 17.27 GB at 128 tokens to 26.94 GB at 2048 tokens because backpropagation retains intermediate activations. MeZO stays near 10.65 GB at 2048 tokens, while CA-ZODPO uses 11.08 GB. The difference between the two ZO methods is small relative to the gap to FO-DPO and corresponds to the stored low-rank basis, momentum buffers, and temporary subspace-extraction matrices.

Peak GPU memory on Qwen2.5-1.5B

Figure 2. Peak GPU memory on Qwen2.5-1.5B. ZO methods avoid the sequence-length-dependent activation cache of FO-DPO. At batch size 32 and sequence length 256, FO-DPO is out of memory on a 48 GB A40 while CA-ZODPO uses 29.35 GB.

The batch-size experiment shows a practical consequence of this memory profile. At sequence length 256, FO-DPO uses 26.18 GB at batch size 16 and runs out of memory at 32, whereas CA-ZODPO uses 17.77 GB and 29.35 GB, respectively. This matters for ZO preference learning because small batches increase the noise of the loss-difference estimator. In this setting, memory saved by removing backpropagation can be spent on larger batches rather than only reducing hardware requirements.

Limitations and conclusion

Preference pairs can define the search geometry as well as the loss: contrastive guidance improves over isotropic ZO and a preference-agnostic activation subspace while retaining ZO memory scaling. CA-ZODPO uses three forward passes per update, so lower memory does not imply faster training; evaluation is also limited to one 1.5B model, one main preference dataset, and an LLM judge. Within this scope, preference-guided perturbations narrow the quality gap between ZO and first-order DPO.

Author Contributions

Weiyun Huang developed the proposed CA-ZODPO method, designed and implemented the algorithms, conducted all experiments and evaluations, analysed the results, and wrote the manuscript.

Alessio Galatolo initiated the research direction of combining zeroth-order optimisation with DPO, provided conceptual insights that informed the proposed approach; supervised the research, contributed to experimental design, and reviewed and revised the manuscript.

Additional Experimental Details

Training configuration

Table 1 reports the hyperparameter search space used for the BF16 zeroth-order experiments. Both MeZO and CA-ZODPO use an effective batch size of 128. CA-ZODPO additionally searches over the rank of the contrastive subspace and the momentum coefficient.

The models are trained in BF16 with a maximum sequence length of 2048 tokens, of which up to 1024 tokens are allocated to the prompt context. The effective batch size of 128 is obtained through gradient accumulation. We use a cosine learning-rate schedule with a warmup ratio of 0.1.

The perturbation scale is important for the finite-difference estimate g=(ℓ+−ℓ−)/(2ϵ)g = (\ell_{+} - \ell_{-})/(2\epsilon). In BF16, very small values of ϵ\epsilon can make the loss difference difficult to resolve numerically, while large values evaluate the objective farther from the current parameter point. We therefore search the perturbation scales listed in Table 1 rather than fixing a single value.

HyperparameterValues
MeZO
Learning rate η\eta{2×10−6, 5×10−6, 1×10−5}\{2 \times10^{-6},\ 5 \times10^{-6},\ 1 \times10^{-5}\}
DPO coefficient βDPO\beta_{\mathrm{DPO}}0.1
Perturbation scale ϵ\epsilon{5×10−5, 1×10−4, 1×10−3}\{5 \times10^{-5},\ 1 \times10^{-4},\ 1 \times10^{-3}\}
Effective batch size128
CA-ZODPO
Learning rate η\eta{2×10−6, 5×10−6, 1×10−5}\{2 \times10^{-6},\ 5 \times10^{-6},\ 1 \times10^{-5}\}
DPO coefficient βDPO\beta_{\mathrm{DPO}}0.1
Perturbation scale ϵ\epsilon{5×10−5, 1×10−4, 1×10−3}\{5 \times10^{-5},\ 1 \times10^{-4},\ 1 \times10^{-3}\}
Effective batch size128
Subspace rank rr{1,2,4,8}\{1, 2, 4, 8\}
Momentum coefficient μ\mu{0.0,0.3,0.5,0.7}\{0.0, 0.3, 0.5, 0.7\}

Table 1. Hyperparameter search space for the BF16 experiments.

Generation and judging protocol

For evaluation, we sample 500 prompts from the HH-RLHF test split using a fixed random seed. All evaluated policies use the same decoding settings: temperature 0.7, top-pp 0.9, and a maximum of 512 generated tokens.

Pairwise comparisons are evaluated using Qwen2.5-32B-Instruct in the main paper. We additionally evaluate the same comparisons with Qwen2.5-7B-Instruct as a smaller judge. To reduce position bias, each response pair is judged twice. In the first evaluation, the responses are presented as A and B; in the second, their positions are swapped. A model is counted as the winner only if it is preferred in both orderings. If the two orderings give inconsistent preferences, the comparison is counted as a tie.

Judge prompt

The following system prompt is used for both judge models:

You are an expert evaluator of AI assistant responses. Your task is
to compare two responses to the same human query and determine which
one is better.
For the Anthropic-HH dataset, you must evaluate based on two core
criteria:
1. Helpfulness: Does the response fully and clearly address the
user’s request?
2. Harmlessness: Is the response safe, ethical, and free of
toxicity or bias? (This is strictly required).
FIRST, provide a brief comparison of the two responses, explaining
their strengths and weaknesses. SECOND, on a new line, state your
final verdict.

Your response MUST perfectly match this format: Comparison: <your brief analysis> Verdict: <"A", "B", or "tie">

Additional Alignment Results

Results with two judge sizes

Table 2 reports the complete pairwise results for both Qwen2.5-7B-Instruct and Qwen2.5-32B-Instruct. The two judges show the same overall pattern: MeZO has a negative win–loss margin against SFT and FO-DPO, while CA-ZODPO improves over MeZO and gives a more balanced comparison with FO-DPO.

JudgeComparisonWinTieLose
Qwen2.5-7B-InstructMeZO vs. SFT25.039.835.2
Qwen2.5-7B-InstructCA-ZODPO vs. SFT34.239.426.4
Qwen2.5-7B-InstructFO-DPO vs. SFT21.661.417.0
Qwen2.5-7B-InstructMeZO vs. FO-DPO23.642.234.2
Qwen2.5-7B-InstructCA-ZODPO vs. FO-DPO31.641.227.2
Qwen2.5-7B-InstructCA-ZODPO vs. MeZO36.038.225.8
Qwen2.5-32B-InstructMeZO vs. SFT25.836.837.4
Qwen2.5-32B-InstructCA-ZODPO vs. SFT34.636.828.6
Qwen2.5-32B-InstructFO-DPO vs. SFT26.655.018.4
Qwen2.5-32B-InstructMeZO vs. FO-DPO23.437.639.0
Qwen2.5-32B-InstructCA-ZODPO vs. FO-DPO34.232.833.0
Qwen2.5-32B-InstructCA-ZODPO vs. MeZO40.832.027.2

Table 2. Pairwise win/tie/loss rates (%) on 500 HH-RLHF test prompts. The first method in each comparison is the method being evaluated.

Direct comparison of the preference-agnostic ablation

The CA-ZODPO-plain variant uses an activation-guided low-rank subspace but does not take the difference between chosen and rejected activations. Table 3 provides the direct comparisons omitted from the main figure. CA-ZODPO-plain improves over isotropic MeZO, but the full CA-ZODPO model is preferred more often in their direct comparison.

ComparisonWinTieLose
CA-ZODPO-plain vs. SFT30.637.831.6
CA-ZODPO-plain vs. FO-DPO27.837.634.6
CA-ZODPO-plain vs. MeZO37.234.228.6
CA-ZODPO-plain vs. CA-ZODPO31.832.835.4

Table 3. Pairwise results (%) for CA-ZODPO-plain using Qwen2.5-32B-Instruct.

Training Dynamics and Ablations

Training-loss stability

Figure 3 compares the DPO training-loss trajectories of CA-ZODPO and MeZO. MeZO reaches slightly lower loss values at some steps but shows larger step-to-step fluctuations. CA-ZODPO follows a more bounded trajectory. Together with the held-out generation results in the main paper, this shows that a lower training loss for MeZO does not correspond to better pairwise generation performance.

Training DPO loss trajectories for CA-ZODPO and MeZO

Figure 3. Training DPO loss trajectories for CA-ZODPO and MeZO.

Momentum and subspace rank

We further examine the two CA-ZODPO-specific hyperparameters: the momentum coefficient and the rank of the contrastive subspace.

Figure 4a compares training with no momentum (μ=0.0\mu= 0.0) and with μ=0.3\mu= 0.3. The two trajectories are similar, while the momentum variant shows a modest reduction in the later loss fluctuations. This suggests that momentum provides a limited stabilising effect rather than being the main source of the performance difference.

Figure 4b shows a clearer dependence on subspace rank. Rank r=1r = 1 produces the least stable trajectory, and r=2r = 2 partially reduces the upward drift. Ranks r=4r = 4 and r=8r = 8 remain more stable over the observed training steps. These results indicate that restricting the search space is useful, but a very low-dimensional subspace can discard directions needed for preference optimisation.

Momentum coefficient

(a) Momentum coefficient.

Subspace rank

(b) Subspace rank.

Figure 4. CA-ZODPO training-loss ablations. Left: momentum. Right: contrastive subspace rank.

References

  1. [1]Long Ouyang, Jeffrey Wu, Xu Jiang, Diogo Almeida, Carroll Wainwright, Pamela Mishkin, Chong Zhang, Sandhini Agarwal, Katarina Slama, Alex Ray, John Schulman, Jacob Hilton, Fraser Kelton, Luke Miller, Maddie Simens, Amanda Askell, Peter Welinder, Paul F Christiano, Jan Leike, and Ryan Lowe. Training language models to follow instructions with human feedback. In S. Koyejo, S. Mohamed, A. Agarwal, D. Belgrave, K. Cho, and A. Oh, editors, Advances in Neural Information Processing Systems, volume 35, pages 27730–27744. Curran Associates, Inc., 2022.arxiv.org/abs/2203.02155
  2. [2]Rafael Rafailov, Archit Sharma, Eric Mitchell, Christopher D Manning, Stefano Ermon, and Chelsea Finn. Direct preference optimization: Your language model is secretly a reward model. In A. Oh, T. Naumann, A. Globerson, K. Saenko, M. Hardt, and S. Levine, editors, Advances in Neural Information Processing Systems, volume 36, pages 53728–53741. Curran Associates, Inc., 2023.arxiv.org/abs/2305.18290
  3. [3]Sadhika Malladi, Tianyu Gao, Eshaan Nichani, Alex Damian, Jason D Lee, Danqi Chen, and Sanjeev Arora. Fine-tuning language models with just forward passes. In A. Oh, T. Naumann, A. Globerson, K. Saenko, M. Hardt, and S. Levine, editors, Advances in Neural Information Processing Systems, volume 36, pages 53038–53075. Curran Associates, Inc., 2023.arxiv.org/abs/2305.17333
  4. [4]Wei Lin, Yining Jiang, Qingyu Song, Qiao Xiang, and Hong Xu. AGZO: Activation-guided zeroth-order optimization for LLM fine-tuning. In Forty-third International Conference on Machine Learning, 2026.arxiv.org/abs/2601.17261
  5. [5]Alessio Galatolo, Zhenbang Dai, Katie Winkle, and Meriem Beloucif. Visualising policy-reward interplay to inform zeroth-order preference optimisation of large language models. In Wanxiang Che, Joyce Nabende, Ekaterina Shutova, and Mohammad Taher Pilehvar, editors, Findings of the Association for Computational Linguistics: ACL 2025, pages 17446–17461, Vienna, Austria, July 2025. Association for Computational Linguistics.
  6. [6]Qining Zhang and Lei Ying. Zeroth-order policy gradient for reinforcement learning from human feedback without reward inference. In Y. Yue, A. Garg, N. Peng, F. Sha, and R. Yu, editors, International Conference on Learning Representations, volume 2025, pages 99911–99925, 2025.arxiv.org/abs/2409.17401
  7. [7]Alessio Galatolo and Meriem Beloucif. Low-resource preference adaptation of LLMs via activation-based label propagation. In Proceedings of the 2026 Conference on Empirical Methods in Natural Language Processing (EMNLP), 2026.arxiv.org/abs/2608.30902
  8. [8]Qwen Team. Qwen2.5 technical report, 2025.arxiv.org/abs/2412.15115
  9. [9]Yuntao Bai, Andy Jones, Kamal Ndousse, Amanda Askell, Anna Chen, Nova DasSarma, Dawn Drain, Stanislav Fort, Deep Ganguli, Tom Henighan, Nicholas Joseph, Saurav Kadavath, Jackson Kernion, Timo Conerly, Sheer El-Showk, Nelson Elhage, Zac Hatfield-Dodds, Danny Hernandez, Tristan Hume, Scott Johnston, Shauna Kravec, Liane Lovitt, Neel Nanda, Catherine Olsson, Dario Amodei, Tom Brown, Jack Clark, Sam McCandlish, Chris Olah, Ben Mann, and Jared Kaplan. Training a helpful and harmless assistant with reinforcement learning from human feedback, 2022.arxiv.org/abs/2204.05862
  10. [10]Yiming Chen, Yuan Zhang, Liyuan Cao, Kun Yuan, and Zaiwen Wen. Enhancing zeroth-order fine-tuning for language models with low-rank structures. In International Conference on Learning Representations, volume 2025, pages 62581–62607, 2025.arxiv.org/abs/2410.07698

Paper details

vlnc:W_tmb2e6kd
Version
v1 · latest
Published
Fresh — added to the Hub within the last 30 days
Added
License
CC BY 4.0
Usage

views · 0

downloads · 0

A view counts once per browser per day, a download each time the PDF is requested.

  • Computer Science

Not recorded for this paper

View statement
Looking for related papers…
Contents