WorksComputer Science
META-IR: Meta-Introspective Reasoning for Adaptive Safety Alignment in LLM
Abstract
Large Language Models (LLMs) remain vulnerable to jailbreak attacks and harmful-content prompts, particularly under adversarial distribution shifts where static alignment pipelines fail to generalize. Although introspective methods such as STAIR strengthen refusal consistency, they depend on fixed self-evaluation policies that cannot rapidly adapt to evolving manipulation strategies. To overcome these limitations, this paper introduces META-IR, a meta-introspective reasoning framework that enhances safety alignment through meta-learning-driven adaptation. It employs a dual-loop architecture: an inner introspective loop that performs structured risk analysis, self-assessment, and reflective correction via SI-MCTS, and an outer meta-learning loop that optimizes the introspection policy across diverse jailbreak task families to enable rapid generalization to unseen attacks. A dynamic jailbreak simulation module further exposes the model to progressively harder adversarial prompts, enabling continual self-improvement without human supervision. Experiments across multiple models (Llama-3.1-8B-Instruct and Qwen-2-7B-Instruct) and a broad suite of safety, helpfulness, generalization, and introspection metrics demonstrate that META-IR substantially improves adaptation speed, robustness, and reasoning stability compared with all baseline models. These results establish META-IR as an effective pathway toward building LLMs capable of sustained, adaptive safety alignment under dynamic and evolving threat environments.
. Warning: Harmful or offensive content may contain.
Introduction
Large Language Models (LLMs) achieve strong performance across reasoning and dialogue tasks but remain vulnerable to jailbreak attacks that deliberately bypass safety controls to elicit harmful content [1]. Standard alignment pipelines such as supervised fine-tuning (SFT) [2], reinforcement learning from human feedback (RLHF) [3], and direct preference optimization (DPO) [4] behave reliably on simple harmful instructions but rely on fixed safety policies that degrade under distribution-shifted adversarial prompts [1]. When attackers exploit role-play framing, forced prefixes, or disguised harmful intent, LLMs often revert to unsafe or inconsistent [5].
Recent introspective approaches like STAIR improve refusal reliability by encouraging System-1 style reasoning as in
Figure 1, yet their reasoning process is static and cannot adapt when adversarial methods evolve [6]. Static introspection is particularly brittle against prompts that manipulate initial responses, embed harmful intent in narrative structures, or exploit linguistic variations not seen during training [5].

Figure 1. Under adaptive jailbreaks, a standard aligned model is manipulated into unsafe output, whereas Meta-IR uses dual-loop introspective reasoning to detect hidden intent, correct intermediate reasoning, and produce a safe final response.
To address these limitations, we propose META-IR: Meta Introspective Reasoning framework for adaptive safety alignment which introduces a dual-loop architecture. An inner loop performs structured risk analysis via adaptive SI-MCTS [7], refining reasoning trajectories through safety helpfulness introspection optimization, while an outer meta-learning loop updates [8] the introspection policy across diverse jailbreak families to enable rapid adaptation. A dynamic jailbreak simulation engine further generates progressively harder adversarial prompts, preventing overfitting and supporting continuous model improvement.
Experiments on LLaMA-3.1-8B-Instruct and Qwen-2-7B-Instruct show that META-IR achieves substantial improvements in harmfulness reduction, violation rates, refusal stability, and generalization compared to STAIR, SFT, RLHF, DPO, and self-rewarding baselines. META-IR-3 consistently delivers the lowest safety errors and the strongest introspective reliability and meta-adaptation scores, outperforming all baselines under both static and adaptive jailbreak attacks. These results confirm that META-IR provides robust, adaptive safety alignment in dynamic adversarial environments.
The contributions of this work are: 1) introducing META-IR, a meta-introspective reasoning framework that integrates structured introspection with meta-learning for adaptive safety alignment; 2) developing SI-MCTS, a safety-aware introspective MCTS guided by a composite safety–helpfulness–introspection reward; 3) designing a dynamic jailbreak simulation engine that generates evolving adversarial prompts for continuous improvement; and 4) providing a comprehensive evaluation protocol showing that META-IR achieves state-of-the-art robustness under both static and distribution-shifted jailbreak attacks.
Meta-IR Framework
Meta-IR aligns via structured introspection, SI-MCTS based self-generation, and meta-learning, yielding robust safety adaptation under evolving jailbreak attacks (see Supplementary Materials Section. I).
Structured Introspective Reasoning
To shift from instinctive refusal to deliberate System 2 reasoning, we introduce structured introspection for progressive risk analysis and safety assessment.
Let denote an input prompt and let denote an introspective reasoning sequence composed of structured reasoning steps followed by a final answer, written as . Each reasoning step contains a short title summarizing the purpose of the step and a detailed explanation describing the model’s analysis, and is enclosed within the special markers <|Reasoning_step|> s_i <|/Reasoning_step|> to maintain structural consistency. After completing all reasoning steps, the model outputs the final conclusion , which is wrapped inside the markers <|Output|> f <|/Output|>, producing a complete introspective trace that separates the reasoning process from the final decision.
The introspective steps are required to identify potential risks embedded in the prompt, including role-play framing, forced-prefix generation, disguised harmful intent, or adversarial instruction templates; if any risks are detected in , the final answer must provide a clear and policy-consistent refusal. To construct training examples, we prompt a teacher model to generate structured introspective responses for each input , producing the dataset , where each follows a multi-stage template for risk analysis and safety justification. This dataset is then used for supervised fine-tuning to align the model’s introspective reasoning style with the structured format.
Meta-IR Dual Loop Implementation
Meta-IR improves safety-aware reasoning through dual-loop across diverse jailbreak families.
Inner Loop: SI-MCTS. Given an introspective reasoning trace produced in the previous stage, the inner loop applies Safety-Informed Monte Carlo Tree Search (SI-MCTS) to explore alternative reasoning paths and to evaluate their safety. Each candidate trajectory compute a composite reward that integrates helpfulness , safety , and introspection quality , defined as , where modulates the contribution of helpfulness based on safety and controls the impact of introspective clarity. This reward guides node selection and pruning, ensuring that unsafe or policy-violating branches are removed while high-safety, high-consistency reasoning paths are expanded. Let denote the refined trajectory generated by the SI-MCTS search:
Outer Loop: Meta-Learning. To enable fast adaptation to previously unseen adversarial patterns, the outer loop performs meta-learning over a task distribution , where each task corresponds to a jailbreak family such as role-play deception, forced-prefix prompts, or subtle harmful-intent masking. For each task , the refined introspection trace is evaluated using the same composite reward, yielding a task-level score . Meta-IR then updates the introspection policy parameters by maximizing the expected reward across tasks:
where is the meta step size. This update enables the reasoning policy to generalize beyond the jailbreak strategies observed during training and improves robustness under distribution shift. By refining introspective trajectories with SI-MCTS in the inner loop and optimizing across jailbreak task families in the outer loop, Meta-IR learns a safety-aware introspection policy that remains aligned under evolving and unseen adversarial attacks.
Reward Mechanism and Self-Improvement. Meta-IR enhances safety alignment through a dual-loop self-improvement process that couples an inner-loop search refinement with an outer-loop meta-learning update, both driven by a safety-prioritized composite reward. Each response compute a helpfulness score , a safety score , and an introspection-quality score . The overall reward is defined as
where modulates the contribution of helpfulness based on safety and controls the impact of introspective quality. This formulation enforces safety as the dominant objective while still encouraging helpful and clearly reasoned responses that accurately identify role-play framing, hidden malicious intent, and forced-prefix jailbreak strategies.
In the inner loop, Safety-Informed MCTS (SI-MCTS) [7] expands and evaluates structured introspective trajectories using the composite reward to guide node selection and pruning, thereby producing safe, policy-consistent introspection traces. In the outer loop, Meta-IR performs meta-learning over a distribution of jailbreak task families. For each batch of tasks , Meta-IR aggregates introspective trajectories and updates the introspection policy parameters via the task-level objective
Figure 2. Meta-IR has three stages: (1) structured introspective reasoning for prompt analysis, hidden-intent detection, risk evaluation, and safe trajectory generation; (2) dual-loop self-improvement, where SI-MCTS generates trajectories, builds safety-prioritized preference data, and meta-learns the introspection policy across jailbreak families; and (3) meta-adaptive decoding, where the learned reward model guides best-of-N and beam search to select a safe, helpful final answer.
where denote the aggregated scores for task . Optionally composite reward can be converted into pairwise preferences and optimized with step-level DPO [4] to stabilize the base model update. This dual-loop refinement enables rapid generalization to unseen jailbreaks.
Meta-Adaptive Response Generation
After dual-loop refinement, Meta-IR uses the meta-updated policy for test-time safe response generation. Given an input , it produces multiple candidate continuations with diverse decoding strategies and scores each partial or complete trajectory using a process reward model trained with the Bradley-Terry model [3] to assess safety and usefulness. Meta-IR first applies Best-of-N sampling [9], selecting the candidate with the highest process reward, and then further handles difficult cases with reward-guided Beam Search [10], which expands promising reasoning paths while pruning unsafe continuations. The final output is the highest-reward sample or beam, enabling robust safe generation under unseen jailbreaks.
Experimental Setup
Detail Experiments in Supplementary Materials Sect. II.
Models and Datasets. We use Llama-3.1-8B-Instruct and Qwen-2.7B-Instruct as the base LLMs for safety alignment. Both employed for ablation analysis, and cross-model generalization. Our experiments are constructed from a 50k-sample seed corpus drawn from AdvBench [11], HarmBench [12], and WildJailbreak [13]; we include all harmful instructions from AdvBench and all behavior prompts from HarmBench (approximately 2k combined), and subsample the remaining 48k examples from WildJailbreak to balance harmful (vanilla and adversarial) and benign look-alike prompts. Baseline systems are trained on the full labeled corpus , while META-IR additionally requires a structured introspection corpus . At the end construct a 20k-sample introspective dataset consisting of 10k safety-oriented and 10k helpfulness-oriented prompts converted into our introspective reasoning format. Each META-IR inner-loop update uses 5k safety and 5k helpfulness introspection samples, and to expose the model to distribution-shifted adversarial behavior, we introduce 1k and 2k adversarial jailbreak prompts (from the adversarial split of WildJailbreak) in the final two meta-adaptation iterations.
Baselines. We compare META-IR against eight widely used safety-alignment baselines: (i) Base, an instruction-tuned model without introspection or self-generation; (ii) CoT, which adds supervised chain-of-thought but lacks true introspective structure [14]; (iii) SFT/RLHF, representing standard non-introspective alignment pipelines [2, 3]; (iv) DPO, a preference-optimization objective without dynamic introspection [4]; (v) SACPO, which uses a learned safety critic but no structured introspective reasoning or iterative self-generation [15]; (vi) Self-Rewarding (RLOO), which adds lightweight self-generated signals but remains non-introspective [16]; and (vii) STAIR-SFT, which provides fixed-format introspective reasoning without meta-level adaptation [6].
Evaluation. We evaluate using a unified suite of metrics covering Harmlessness + Safety, Helpfulness + Generalization, and Introspection + Meta-Learning. Harmlessness and safety are assessed with five failure-oriented indicators: Adv (attack success rate) [5], Harm (overall harmfulness) [17], WJ-H (WildJailbreak harmful responses) [18], SVR (safety-violation rate) [19], and RST (refusal-stability under perturbations) [1]. Helpfulness and generalization are measured using WJ-B (benign helpfulness) [18], MGS (robustness to paraphrasing, role-play, and distribution shifts) [20], MTB (MTBench instruction quality) [21], Elo (Arena Elo pairwise preference) [22], and BBH (Big-Bench Hard reasoning accuracy) [23]. Introspection and meta-learning are motivated by prior work on introspection and meta-adaptation; we evaluate introspective reliability and adaptive safety behavior using five metrics: IA (Introspection Accuracy), REAR (Risk-Explanation Attribution Recall), MCR (Meta-Confusion Rate), JIR (Jailbreak-Introspection Failure Rate), and MTA (Meta-Training Adaptation) [8], [20], [24].
Results and Analysis
Details Analysis in Supplementary Materials Section. III
Main Result. Meta-IR consistently outperforms all baselines on both models, achieving the strongest overall balance of safety, harmlessness, helpfulness, generalization, introspection, and meta-learning. Table I summarizes the overall performance across both models, showing that META-IR-3 consistently delivers the strongest alignment, reasoning stability, and meta-generalization among all evaluated baselines. On LLAMA-3.1-8B, META-IR-3 achieves new state-of-the-art harmlessness–safety scores (0.089 / 0.097 / 0.073 / 0.061 for Adv/Harm, WJ-HL, SVR, and RST), yielding relative improvements of +64.1%, +54.8%, +47.9%, and +52.9% over the strongest non-meta baseline (STAIR-SFT). Helpfulness and generalization remain comparably strong at 0.879 (WJ-B), 0.667 (MGS), 9.1 (MTB), and 1718 (Elo), defining a superior safety–utility balance in which META-IR-3 improves MTB by +53.3% over DPO while maintaining competitive helpfulness. Substantial gains arise in introspection and meta-learning, where META-IR-3 reaches 0.902 (IA), 0.953 (REAR), 0.162 (MCR), 0.094 (JIR), and 0.702 (MTA), corresponding to improvements of +33.4%, +20.8%, -47.3%, -55.8%, and +23.3% over STAIR-SFT, indicating more stable reasoning trajectories, lower cumulative error, and stronger capability to detect unsafe chains of thought. A similar pattern holds for Qwen-2-7B, where META-IR-3 again achieves the strongest harmlessness–safety metrics (0.092 / 0.080 / 0.124 / 0.071 / 0.064), representing gains of +63.7%, +46.5%, +38.9%, +49.0%, and +47.4% over STAIR-SFT, while its meta-learning indicators (0.889 for IA, 0.950 for REAR, 0.159 for MCR, 0.095 for JIR, and 0.704 for MTA) correspond to improvements of +20.4%, +17.4%, -35.4%, -47.1%, and +19.5%, respectively.
| Method | IR | SelfGen | Adv↓ | Harm↓ | WJ-H↓ | SVR↓ | RST↓ | WJ-B↑ | MGS↑ | MTB↑ | Elo↑ | BBH↑ | IA↑ | REAR↑ | MCR↓ | JIR↓ | MTA↑ |
| LLAMA-3.1-8B-Instruct | |||||||||||||||||
| Base | – | – | 0.482 | 0.367 | 0.419 | 0.311 | 0.224 | 0.872 | 0.584 | 6.4 | 1205 | 31.2 | 0.421 | 0.512 | 0.294 | 0.211 | 0.468 |
| CoT | × | – | 0.451 | 0.352 | 0.401 | 0.304 | 0.207 | 0.856 | 0.397 | 6.1 | 1290 | 32.5 | 0.468 | 0.577 | 0.285 | 0.203 | 0.491 |
| SFT | × | × | 0.432 | 0.341 | 0.378 | 0.297 | 0.198 | 0.842 | 0.412 | 6.6 | 1328 | 33.1 | 0.517 | 0.603 | 0.279 | 0.197 | 0.502 |
| RLHF | × | × | 0.418 | 0.337 | 0.365 | 0.289 | 0.193 | 0.813 | 0.328 | 6.3 | 1339 | 34.0 | 0.529 | 0.622 | 0.276 | 0.192 | 0.514 |
| DPO | × | × | 0.405 | 0.330 | 0.351 | 0.281 | 0.185 | 0.875 | 0.549 | 6.9 | 1372 | 34.7 | 0.548 | 0.639 | 0.271 | 0.188 | 0.526 |
| SACPO | × | × | 0.345 | 0.292 | 0.276 | 0.259 | 0.167 | 0.871 | 0.497 | 7.2 | 1420 | 36.0 | 0.612 | 0.703 | 0.253 | 0.171 | 0.558 |
| Self-Rewarding | × | ✓ | 0.392 | 0.324 | 0.353 | 0.284 | 0.181 | 0.867 | 0.601 | 7.0 | 1391 | 35.2 | 0.553 | 0.661 | 0.266 | 0.181 | 0.544 |
| STAIR-SFT | ✓ | × | 0.281 | 0.243 | 0.216 | 0.211 | 0.144 | 0.907 | 0.593 | 7.3 | 1452 | 36.8 | 0.676 | 0.744 | 0.244 | 0.158 | 0.571 |
| META-IR-SFT | ✓ | ✓ | 0.254 | 0.228 | 0.198 | 0.193 | 0.133 | <u>0.901</u> | 0.528 | 7.6 | 1493 | 38.1 | 0.718 | 0.801 | 0.229 | 0.147 | 0.588 |
| META-IR-1 | ✓ | ✓ | 0.196 | 0.182 | 0.157 | 0.152 | 0.109 | 0.892 | 0.571 | 7.9 | 1537 | 40.3 | 0.793 | 0.864 | 0.211 | 0.133 | 0.613 |
| META-IR-2 | ✓ | ✓ | 0.147 | 0.124 | 0.111 | 0.103 | 0.052 | 0.873 | <u>0.612</u> | 8.4 | 1604 | 43.8 | 0.836 | 0.912 | 0.184 | 0.117 | 0.651 |
| META-IR-3 | ✓ | ✓ | 0.089 | 0.097 | 0.073 | 0.061 | 0.081 | 0.879 | 0.667 | 9.1 | 1718 | 47.5 | 0.902 | 0.953 | 0.162 | 0.094 | 0.702 |
| Qwen-2-7B-Instruct | |||||||||||||||||
| Base | – | – | 0.538 | 0.421 | 0.472 | 0.337 | 0.258 | 0.831 | 0.552 | 6.0 | 1179 | 29.3 | 0.398 | 0.504 | 0.301 | 0.221 | 0.447 |
| CoT | × | – | 0.487 | 0.395 | 0.438 | 0.322 | 0.241 | 0.807 | 0.467 | 6.3 | 1252 | 31.0 | 0.447 | 0.559 | 0.293 | 0.214 | 0.472 |
| SFT | × | × | 0.451 | 0.372 | 0.413 | 0.309 | 0.232 | 0.829 | 0.518 | 6.4 | 1297 | 31.6 | 0.493 | 0.582 | 0.289 | 0.209 | 0.483 |
| RLHF | × | × | 0.439 | 0.363 | 0.401 | 0.301 | 0.227 | 0.849 | 0.399 | 6.6 | 1321 | 32.4 | 0.506 | 0.604 | 0.283 | 0.204 | 0.497 |
| DPO | × | × | 0.422 | 0.351 | 0.389 | 0.292 | 0.219 | 0.842 | 0.412 | 6.8 | 1360 | 33.7 | 0.521 | 0.632 | 0.279 | 0.197 | 0.521 |
| SACPO | × | × | 0.360 | 0.314 | 0.315 | 0.265 | 0.192 | 0.868 | 0.541 | 7.1 | 1406 | 36.0 | 0.585 | 0.705 | 0.259 | 0.178 | 0.561 |
| Self-Rewarding | × | ✓ | 0.403 | 0.342 | 0.388 | 0.288 | 0.213 | 0.853 | 0.518 | 6.9 | 1382 | 34.9 | 0.528 | 0.661 | 0.271 | 0.192 | 0.543 |
| STAIR-SFT | ✓ | × | 0.179 | 0.295 | 0.257 | 0.241 | 0.176 | 0.871 | 0.597 | 7.2 | 1430 | 37.1 | 0.642 | 0.741 | 0.246 | 0.161 | 0.578 |
| META-IR-SFT | ✓ | ✓ | 0.298 | 0.263 | 0.236 | 0.219 | 0.098 | 0.879 | 0.649 | 7.4 | 1475 | 38.4 | 0.689 | 0.803 | 0.200 | 0.151 | 0.596 |
| META-IR-1 | ✓ | ✓ | 0.237 | 0.218 | 0.192 | 0.174 | 0.138 | 0.895 | 0.558 | 7.7 | 1522 | 40.9 | 0.764 | 0.866 | 0.218 | 0.136 | 0.628 |
| META-IR-2 | ✓ | ✓ | 0.395 | 0.152 | 0.085 | 0.128 | 0.162 | <u>0.906</u> | 0.481 | 8.2 | 1591 | 43.1 | 0.811 | 0.909 | 0.184 | 0.119 | 0.659 |
| META-IR-3 | ✓ | ✓ | 0.092 | 0.080 | <u>0.124</u> | 0.071 | 0.064 | 0.917 | 0.512 | 8.9 | 1693 | 47.0 | 0.889 | 0.950 | 0.159 | 0.095 | 0.704 |
Table I. Benchmark results on harmlessness, safety, helpfulness, generalization, introspection, and meta-learning. ‘IR’ denotes structured introspection; ‘SelfGen’ denotes self-generated data. Best and second-best are marked in bold and underline.
Comprehensive Performance Analysis
Safety and Robustness. Figure 3 shows that SFT, RLHF, and DPO achieve only moderate reductions in harmfulness (WJ-HL), adversarial vulnerability (Adv), and safety violations (SVR), and remain unstable under harmful prompts. STAIR-SFT improves robustness via structured introspection but still experiences elevated violation rates. Meta-IR provides substantially stronger safety performance across all metrics, with META-IR-3 yielding the lowest harmfulness and violation levels and maintaining stable refusal behavior under aggressive red-teaming. This demonstrates that integrating introspection, self-generated corrections, and meta-learning produces a more robust safety policy.
Figure 3. Safety and robustness improvements across the models.
Introspection Reasoning Quality. Figure 4 evaluates introspective reliability (REAR), mistake-cause reasoning (MCR), and jailbreak-risk attribution (JIR). While DPO and Self-Rewarding offer partial improvements, their reasoning-error profiles remain high; STAIR-SFT reduces some inconsistencies but does not fully address attribution errors. Meta-IR variants improve monotonically across all introspective metrics, with META-IR-3 achieving the highest REAR and the lowest MCR and JIR. These results confirm that Meta-IR’s dual-loop design inner-loop introspection with outer-loop meta-adaptation substantially strengthens internal safety reasoning.
Figure 4. Introspection and reasoning quality across models.
Meta-Generalization to Adaptive Jailbreaks. As shown in Figure 5, SFT and DPO generalize poorly to unseen jailbreak styles, and Self-Rewarding and STAIR-SFT offer limited resilience under adaptive attacks. In contrast, META-IR-1, META-IR-2, and META-IR-3 consistently improve meta-generalization, with META-IR-3 achieving the highest MTA stability and the best MTB and BBH performance. This indicates that Meta-IR not only improves in-distribution safety but also adapts effectively to adversarial conditions unseen.
Figure 5. Meta-generalization performance on adaptive attacks.
Ablation Analysis
We ablate Meta-IR on LLAMA-3.1-8B and Qwen-2-7B to measure the contribution of each component (Table II). Removing structured introspection, meta-learning, SelfGen, or adaptive jailbreak handling consistently harms safety, introspection, and meta-generalization. For LLAMA-3.1-8B, disabling introspection causes large drops in REAR (0.902 0.611) and increases in MCR/JIR, while removing meta-learning weakens both introspection and MTA. Eliminating SelfGen raises SVR and reduces generalization. Qwen-2-7B shows similar degradations, with REAR and JIR deteriorating when IR or meta-learning are removed. Across both backbones, Meta-IR-3 remains the strongest, achieving the lowest SVR and reasoning errors and the highest MTA. These results confirm that Meta-IR’s components operate synergistically and are jointly required for robust safety alignment.
| Variant | REAR↑ | SVR↓ | MCR↓ | JIR↓ | MTA↑ |
| LLAMA-3.1-8B-Instruct | |||||
| META-IR-SFT | 0.718 | 0.133 | 0.229 | 0.147 | 0.588 |
| META-IR-1 | 0.793 | 0.139 | 0.211 | 0.130 | 0.634 |
| META-IR-2 | 0.836 | 0.103 | 0.184 | 0.117 | 0.651 |
| META-IR-3 | 0.902 | 0.061 | 0.162 | 0.094 | 0.702 |
| Qwen-2-7B-Instruct | |||||
| META-IR-SFT | 0.689 | 0.098 | 0.234 | 0.151 | 0.596 |
| META-IR-1 | 0.764 | 0.174 | 0.218 | 0.136 | 0.628 |
| META-IR-2 | 0.811 | 0.128 | 0.184 | 0.119 | 0.659 |
| META-IR-3 | 0.889 | 0.064 | 0.159 | 0.095 | 0.704 |
Table II. Ablation results: removing any Meta-IR component lowers REAR/MTA and increases SVR, MCR, and JIR.
Related Work
As summarized in Table III, existing alignment methods against jailbreak attacks can be broadly grouped into static alignment, self-generated alignment, and reasoning-based alignment. Early approaches such as SFT [2], RLHF [3], and DPO [4] rely on fixed supervision or preference signals and therefore remain static after training, which limits their robustness under adaptive jailbreak attacks [19], [20], [24]. Later methods such as SACPO [15] and self-rewarding alignment [16] introduce self-generated feedback, but they still lack meta-adaptive learning and show limited generalization to evolving attack distributions [12], [22]. Reasoning-based methods such as CoT [14] and STAIR [6] improve intermediate reasoning or introspection ability, yet their safety policies also remain fixed and cannot explicitly adapt to new jailbreak families [5], [8], [10]. In contrast, Meta-IR uniquely combines self-generated signals, search-based refinement, and meta-adaptive learning within a structured introspective reasoning framework [8], making it the only method in Table III that explicitly supports adaptive safety alignment and robust generalization against evolving jailbreak attacks.
Table III. ALIGNMENT METHODS UNDER ADAPTIVE JAILBREAKS: STATIC, SELF-GENERATED, AND META-ADAPTIVE SAFETY STRATEGIES.
Conclusion
We presented META-IR, a meta-learning framework that integrates structured introspective reasoning, self-generated corrective signals, and outer-loop adaptation to enhance the safety and robustness of large language models. Experiments across harmfulness, introspection quality, and meta-generalization show that META-IR consistently outperforms SFT, RLHF, DPO, Self-Rewarding, and STAIR-SFT, producing safer and more stable responses while adapting effectively to unseen jailbreak strategies. Ablation studies further verify the contribution of each component, with the full META-IR model achieving the best balance between safety and utility. Overall, META-IR offers a principled and scalable approach to continuous safety improvement, enabling LLMs to remain robust and trustworthy under evolving adversarial conditions.
Acknowledgment
This work has supported by the National Natural Science Foundation of China (NSFC) under Grant 62322601, and the Fundamental Research Funds for the Central Universities (No.2024IAIS-QN017, 2025CDJZDGF001).
References
- [1]Hajun Kim, Hyunsik Na, and Daeseon Choi, “Beyond fixed and dynamic prompts: Embedded jailbreak templates for advancing llm security,” arXiv preprint arXiv:2511.14140, 2025.arxiv.org/abs/2511.14140
- [2]Rohan Taori, Ishaan Gulrajani, Tianyi Zhang, Yann Dubois, Xuechen Li, Carlos Guestrin, Percy Liang, and Tatsunori B Hashimoto, “Stanford alpaca: An instruction-following llama model,” 2023.
- [3]Long Ouyang, Jeffrey Wu, Xu Jiang, Diogo Almeida, Carroll Wainwright, Pamela Mishkin, Chong Zhang, Sandhini Agarwal, Katarina Slama, Alex Ray, et al., “Training language models to follow instructions with human feedback,” Advances in neural information processing systems, vol. 35, pp. 27730–27744, 2022.arxiv.org/abs/2203.02155
- [4]Rafael Rafailov, Archit Sharma, Eric Mitchell, Christopher D Manning, Stefano Ermon, and Chelsea Finn, “Direct preference optimization: Your language model is secretly a reward model,” Advances in neural information processing systems, vol. 36, pp. 53279–53741, 2023.arxiv.org/abs/2305.18290
- [5]Yuanfu Wang, Pengyu Wang, Chenyang Xi, Bo Tang, Junyi Zhu, Wenqiang Wei, Chen Chen, Chao Yang, Jingfeng Zhang, Chaochao Lu, et al., “Adversarial preference learning for robust llm alignment,” arXiv preprint arXiv:2505.24369, 2025.arxiv.org/abs/2505.24369
- [6]Yichi Zhang, Siyuan Zhang, Yao Huang, Zeyu Xia, Zhengwei Fang, Xiao Yang, Ranjie Duan, Dong Yan, Yinpeng Dong, and Jun Zhu, “Stair: Improving safety alignment with introspective reasoning,” arXiv preprint arXiv:2502.02384, 2025.arxiv.org/abs/2502.02384
- [7]Maciej Świechowski, Konrad Godlewski, Bartosz Sawicki, and Jacek Mańdziuk, “Monte carlo tree search: A review of recent modifications and applications,” Artificial Intelligence Review, vol. 56, no. 3, pp. 2497–2562, 2023.
- [8]Sungyong Baik, Myungsub Choi, Janghoon Choi, Heewon Kim, and Kyoung Mu Lee, “Meta-learning with adaptive hyperparameters,” Advances in neural information processing systems, vol. 33, pp. 20755–20765, 2020.arxiv.org/abs/2011.00209
- [9]Hunter Lightman, Vineet Kosaraju, Yuri Burda, Harrison Edwards, Bowen Baker, Teddy Lee, Jan Leike, John Schulman, Ilya Sutskever, and Karl Cobbe, “Let’s verify step by step,” The Twelfth International Conference on Learning Representations, 2023.arxiv.org/abs/2305.20050
- [10]Yuxi Xie, Kenji Kawaguchi, Yiran Zhao, James Xu Zhao, Min-Yen Kan, Junxian He, and Michael Xie, “Self-evaluation guided beam search for reasoning,” Advances in Neural Information Processing Systems, vol. 36, pp. 4618–41650, 2023.arxiv.org/abs/2305.00633
- [11]Andy Zou, Zifan Wang, Zico Kolter, and Matt Fredrikson, “Universal and transferable adversarial attacks on aligned language models,” 2023.
- [12]Mantas Mazeika, Long Phan, Zhuang Yin, Andy Zou, Zifan Wang, Norman Mu, Elham Sakhaee, Nathaniel Li, Steven Basart, Bo Li, David Forsyth, and Dan Hendrycks, “Harmbench: A standardized evaluation framework for automated red teaming and robust refusal,” arXiv preprint arXiv:2402.04249, 2024.arxiv.org/abs/2402.04249
- [13]Liwei Jiang, Kavel Rao, Seungju Han, Allyson Ettinger, Faeze Brahman, Sachin Kumar, Niloofar Mershghallah, Ximing Lu, Maarten Sap, Yejin Choi, and Nouha Dziri, “Wildteaming at scale: From in-the-wild jailbreaks to (adversarially) safer language models,” 2024.arxiv.org/abs/2406.18510
- [14]Jason Wei, Xuezhi Wang, Dale Schuurmans, Maarten Bosma, Fei Xia, Ed Chi, Quoc V Le, Denny Zhou, et al., “Chain-of-thought prompting elicits reasoning in large language models,” Advances in neural information processing systems, vol. 35, pp. 24824–24837, 2022.arxiv.org/abs/2201.11903
- [15]Akifumi Wachi, Thien Tran, Rei Sato, Takumi Tanabe, and Youhei Akimoto, “Stepwise alignment for constrained language model policy optimization,” Advances in Neural Information Processing Systems, vol. 37, pp. 104471–104520, 2024.arxiv.org/abs/2404.11049
- [16]Wei Xiong, Hanning Zhang, Chengyu Le, Lichang Chen, Nan Jiang, and Tong Zhang, “Self-rewarding correction for mathematical reasoning,” arXiv preprint arXiv:2502.19613, 2025.arxiv.org/abs/2502.19613
- [17]Langqi Yang, Tianhang Zheng, Kedong Xiu, Yixuan Chen, Di Wang, Puning Zhao, Zhan Qin, and Kui Ren, “Harmmetric eval: Benchmarking metrics and judges for llm harmfulness assessment,” arXiv preprint arXiv:2509.24384, 2025.arxiv.org/abs/2509.24384
- [18]Liwei Jiang, Kavel Rao, Seungju Han, Allyson Ettinger, Faeze Brahman, Sachin Kumar, Niloofar Mershghallah, Ximing Lu, Maarten Sap, Yejin Choi, et al., “Wildteaming at scale: From in-the-wild jailbreaks to (adversarially) safer language models,” Advances in Neural Information Processing Systems, vol. 37, pp. 47094–47165, 2024.arxiv.org/abs/2406.18510
- [19]Morteza Jalali Alenijareghi, Fardin Ghorbani, Samira Keivanpour, Yuvin Adranian Chinniah, and Sabrina Jocelyn, “Proactive safety reasoning in human-robot collaboration in disassembly through llm-augmented stpa and fmea,” Robotics and Computer-Integrated Manufacturing, vol. 98, pp. 103162, 2026.
- [20]Xiang Dai, Sarvnaz Karimi, and Biaoyan Fang, “A critical look at meta-evaluating summarisation evaluation metrics,” arXiv preprint arXiv:2409.19507, 2024.arxiv.org/abs/2409.19507
- [21]Liamn Zeng, Wei-Lin Chiang, Ying Sheng, Siyuan Zhuang, Zhanghao Wu, Yonghao Zhuang, Zi Lin, Zhuohan Li, Dacheng Li, Eric Xing, et al., “Judging llm-as-a-judge with mt-bench and chatbot arena,” Advances in neural information processing systems, vol. 36, pp. 46595–46623, 2023.arxiv.org/abs/2306.05685
- [22]Zirui Liu, Jiatong Li, Yan Zhuang, Qi Liu, Shuanghong Shen, Jie Ouyang, Mingyue Cheng, and Shijin Wang, “am-elo: A stable framework for arena-based llm evaluation,” arXiv preprint arXiv:2505.03475, 2025.arxiv.org/abs/2505.03475
- [23]Mehran Kazemi, Bahare Fatemi, Hritik Bansal, John Palowitch, Chrysovalantis Anastasiou, Sanket Vaibhav Mehta, Lalit K Jain, Virginia Aglietti, Disha Jindal, Yuanzhu Peter Chen, et al., “Big-bench extra hard,” in Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), 2025, pp. 26473–26501.
- [24]Jiaqi Li, Xinyi Dong, Yang Liu, Zhizhuo Yang, Quansen Wang, Xiaobo Wang, Song-Chun Zhu, Zixia Jia, and Zilong Zheng, “Reflectove: Improving meta introspection of small llms by learning self-reflection,” in Findings of the Association for Computational Linguistics: ACL 2025, 2025, pp. 16948–16966.arxiv.org/abs/2505.16475



