Post-Training Large Language Models with LLMON

Introduction

Large Language Models (LLMs) process all tokens in a context window through the same attention mechanism, without any built-in distinction between executable instructions, passive data, and auxiliary context (Vaswani et al., 2017; Kaddour et al., 2023). This architectural uniformity means that any token can equally influence model behavior, whether it originates from a user instruction, a retrieved document, or an injected adversarial payload. The practical consequences are well-documented: prompt injection attacks that hijack model execution (Greshake et al., 2023; Perez and Ribeiro, 2022), ambiguous referential binding when multiple instructions coexist in context (e.g., “execute the second task”), and degraded reliability in compositional scenarios involving retrieval-augmented generation (RAG) and tool calling (Yao et al., 2022; Schick et al., 2023). These failure modes stem from a single root cause: the model has no mechanism to distinguish executable instructions from passive data at the token level. Current instruction-tuning approaches (Ouyang et al., 2022; Wei et al., 2022) train models to follow instructions implicitly, based on position, formatting conventions (e.g., “chat templates”), and statistical patterns, thus lack the mechanism to distinguish which instruction to execute when multiple candidates exist in a given prompt.

In this paper, we present a structured post-training methodology built on LLMON (Hind et al., 2026; LLMON Team, 2026) (LLM Object Notation), a hierarchical tagging framework that introduces explicit instruction-data separation and identifier-based execution binding to the LLM interface. Under LLMON, each instruction and data segment is wrapped in typed, uniquely identified blocks, and execution occurs only through explicit execution calls that reference specific instruction and data identifiers. This design steers the model toward a clear semantic principle where only explicitly invoked instructions are executed. Our contributions are:

1. We describe a scalable data pipeline that transforms instruction-tuning corpora (general, RAG, and tool-calling) into LLMON-structured training data, including distractor-augmented variants with up to three competing instructions and irrelevant data segments.

2. We conduct a post-training evaluation on two base models (Granite-4.1-3B-Base and

Granite-4.1-8B-Base) under full fine-tuning with 4.6B tokens, followed by reinforcement learning.

3. We introduce attention-based interpretability metrics: Instruction Focus Ratio (IFR), Data Attention Score (DAS), and Leakage (LEAK), to provide mechanistic insight into how structured training reshapes model attention.

Related Work

Instruction tuning and alignment. Instruction tuning (Wei et al., 2022; Ouyang et al., 2022) and preference-based alignment (Rafailov et al., 2023; Christiano et al., 2017) have become the standard post-training pipeline for modern LLMs. These methods train models to follow natural-language instructions but rely on implicit positional cues and chat templates (Hugging Face, 2025)-based formatting conventions, rather than explicit structural semantics. When multiple instructions appear in context, models lack a principled mechanism for disambiguation. Our work is complementary: LLMON-structured supervision can be applied on top of any instruction-tuning corpus to add explicit execution binding behavior.

Prompt injection and adversarial robustness. Prompt injection exploits the absence of privilege separation between instructions and data (Greshake et al., 2023; Perez and Ribeiro, 2022). Spotlighting (Hines et al., 2024) marks untrusted spans with in-band text delimiters at the prompt level, while StruQ (Chen et al., 2025a) and SecAlign (Chen et al., 2025b) make the boundary a training objective, fine-tuning on structured prompts and preference pairs in which injected instructions are present but must not be followed. We differ in two respects. First, the boundary is carried by reserved single tokens from a disjoint region of the vocabulary rather than by in-band delimiters or role names, so it cannot be formed by subword composition of ordinary content. Second, privilege attaches to an explicit binding rather than to a role: every span is individually addressable and authority follows from which pointer references it, so several same-role instructions can coexist with only one bound for execution.

Tool use and function calling. Modern LLM systems integrate external tools through structured generation (Schick et al., 2023; Patil et al., 2024; Yao et al., 2022). These approaches define tool schemas and train models to emit structured calls, but the instruction-data boundary within the prompt remains implicit. LLMON’s execution semantics are orthogonal and compatible with tool-calling pipelines: tool definitions, retrieved documents, and user instructions each occupy typed segments with distinct identifiers.

Structured generation and constrained decoding. Constrained decoding approaches [2] (Willard and Louf 2023; Zheng et al. 2024) enforce output structure at inference time through grammar-based token masking. Prompting frameworks (Khattab et al., 2023; Liu et al., 2026; POML) and programming-language abstractions (Dantanarayana et al., 2025) orchestrate LLM calls, and markup-based prompt languages impose document structure on the prompt, but none of these modifies the model’s internal representation of instruction-data boundaries. Unlike these inference-time or application-layer approaches, LLMON operates at the training data level, teaching models to internalize structural distinctions through supervised learning.

Using LLMON (LLM Object Notation)

This section describes how we leverage LLMON (Hind et al., 2026) to construct structured post-training data that teaches models explicit execution control. LLMON is a hierarchical tagging framework that introduces instruction-data separation and identifier-based execution binding at the token level, operating within standard autoregressive language modeling without architectural modification. We briefly summarize the notation relevant to our training methodology, then describe the data construction pipeline.

Notation Overview

LLMON defines six special delimiter tokens (“<|open|>”, “<|close|>”, “<|open_end|>”, “<|self_close|>”, “<|:|>”, “<|.|>”) that form a typed tagging system. The key structural elements relevant to training are:

Typed segments. All content is wrapped in identified and tagged blocks. Instructions carry role qualifiers (“instr<|:|>ID”), data segments hold passive content (“data<|:|>ID”), requirements hold output constraints (“req<|:|>ID”), and documents use nested paths (“doc<|:|>title”, “doc<|:|>text”). Tool definitions are flattened so that every leaf of the JSON schema becomes its own addressable sub-field (“tool<|.|>name”, “tool<|.|>parameters<|.|>arg<|.|>type”), and a tool result returns as a “data” block like any other input. A system block (“system<|:|>ID”) carries authoritative policy and persists across turns.

Execution binding. An execution block explicitly references which instruction to execute, which data to use as input, and where the output should be placed via identifier references as shown in Figure 1. This design enforces a clear semantic paradigm where multiple instructions can coexist in context without interference, as execution requires referencing a specific identifier regardless of position.

The privilege invariant. Execution binding yields the invariant our training targets: content reached through “exec<|.|>instruction” or “exec<|.|>system” is authoritative, whereas everything reached through “exec<|.|>input” (data, documents, tool schemas, and tool results) is material to be read, not a command to be followed, however it is phrased. Blocks that no pointer references are rendered into the context but bound to nothing. Privilege is therefore a property of the binding rather than of a block’s position or role name, and the delimiters that express it are reserved tokens occupying a disjoint region of the vocabulary, so the structure cannot be formed by subword composition of ordinary content.

Data Construction Pipeline

We develop an automated pipeline that transforms diverse instruction-tuning data into LLMON-structured training examples. The pipeline handles three distinct data modalities:

General instruction data. Single-turn and multi-turn conversations are serialized with per-turn instruction blocks, “exec” bindings, and assistant data blocks. Each turn receives unique identifiers, and multi-turn examples chain consecutive “exec” cycles.

RAG and document-grounded data. Retrieved documents are serialized as typed “doc” segments with nested “doc<|.|>title” and “doc<|.|>text” sub-fields. The “exec” block’s input field references both the user’s data and the relevant document identifiers, teaching the model to ground its response in explicitly referenced context.

Tool-calling data. Tool definitions are serialized using flattened path-based notation (e.g., “tool<|.|>function<|.|>name”, “tool<|.|>function<|.|>description”). Tool invocations produce a first “exec” cycle (user instruction → assistant tool call), followed by a tool-result data block and a second “exec” cycle (tool result → final assistant response).

Distractor Augmentation

The central innovation in our training data is distractor augmentation: systematically injecting competing instructions and irrelevant data into the context while keeping the “exec” block pointed at only the target. This directly trains the model to resolve referential bindings under interference.

For each base example we generate variants with $d \in\{1,2,3\}$ distractor instructions (sampled from the instruction pool, excluding the target) and $m \in\{0,1,2,3\}$ irrelevant data segments (sampled from other examples). The distractor instructions and data occupy their own identified blocks in the context, but the “exec” block references only the original target instruction and data, and the target output is identical across all variants of an example. Variants are sampled per base example, yielding 8 categories parameterized by whether target data is present and by the distractor and irrelevant-data counts. A model that keys on the “exec” block is unaffected by the decoys; one that keys on surface cues such as position or imperative phrasing is penalized by the loss.

We illustrate one such example with a multi-instruction and multi-data distractors execution in Appendix § 7.§ 1.

Experiments

Training Data

Following the pipeline of § 3.2, we assemble the LLMON-structured corpus from the sources below, totaling 4.4M examples and 4.6B tokens.

• General SFT data: 3.25M single-turn (2.79B tokens) and 59k multi-turn (120M tokens) examples drawn from the Granite-4.1 post-training SFT corpora.

• RAG data: 27k single-turn (45.5M tokens) and 8k multi-turn (17.8M tokens) retrieval-augmented examples from the same corpora.

• Tool-calling data: 451k single-turn (707M tokens) and 251k multi-turn (562M tokens) function-calling examples from the same corpora.

• Alpaca: 52k clean examples (Taori et al., 2023) (17.2M tokens) plus 103k distractor-augmented variants (39.4M tokens).

• Dolly: 15k clean examples (Conover et al., 2023) (3.3M tokens) plus 30k distractor-augmented variants (16.3M tokens).

• Prompt-injection data: 130k examples (211M tokens) in which an adversarial instruction is planted inside a payload the model is meant to read rather than execute, spanning general (40k), RAG (11k), and tool-calling (80k) modalities.

The general, RAG, and tool-calling examples teach the structural format across single-turn instruction following, multi-turn conversation, document-grounded QA, and tool invocation, and account for 93.6% of all tokens, which is what keeps capability close to the conventionally-tuned baseline. The Alpaca and Dolly expansions supply the distractor-augmented variants of § 3.3, expanding 67k base examples into 133k variants. The prompt-injection examples are the training-time counterpart of the evaluation in § 4.4: the injected instruction is rendered as a block that no execution pointer references, and the target output is the one that ignores it.

For the reinforcement-learning stages we use two further sources: approximately 169k prompts across five sequential policy-optimization stages spanning multi-domain reasoning, RLHF, knowledge calibration, and two mathematics boosters; and approximately 1k on-policy preference pairs for the final preference-optimization stage, constructed from each model’s own responses at agentic decision points where a tool result carries an injected instruction.

Models and Training

We post-train two base models from the same family and generation at different scales: Granite-4.1-3B-Base and Granite-4.1-8B-Base (IBM Granite Team, 2026; Granite Team, 2024). Both are base (pre-trained) checkpoints without prior instruction tuning, so the LLMON behavior is learned during post-training rather than layered on top of an existing chat format.

Supervised fine-tuning. We first install the six LLMON delimiters in the tokenizer as reserved single tokens by remapping unused vocabulary slots, which avoids growing the vocabulary or resizing the embedding matrix. We then run full fine-tuning, updating all parameters on the LLMON-serialized corpus: three epochs over 4.4M examples and 4.6B tokens at a sequence length of 4096 tokens, sharded across 8 × 100 GPUs with FSDP. We report the final-epoch checkpoint.

Reinforcement learning. Starting from the supervised checkpoint, we run five sequential Group Relative Policy Optimization (GRPO) stages against a reward model over the ≈169k prompts described above, which optimize end behavior directly rather than the token-level target. A final Direct Preference Optimization (Rafailov et al., 2023) (DPO) stage then trains on the ≈1k on-policy injection preference pairs ($\beta= 0.1$, one epoch, learning rate $5 \times10^{-7}$, effective batch size 32). This stage is placed last so that the general-capability stages, which carry no injection objective, cannot erode the boundary behavior installed earlier. Unless noted otherwise, results reported as “LLMON” refer to this final SFT→GRPO→DPO checkpoint.

Evaluation

Standard Benchmarks

To verify that LLMON post-training preserves general capabilities, we evaluate across seven categories, reporting the unweighted mean over the benchmarks in each: General (MMLU (Hendrycks et al., 2021), MMLU-Pro (Wang et al., 2024), BBH (Suzgun et al., 2023), AGI-Eval (Zhong et al., 2024), GPQA (Rein et al., 2023)); Alignment (IFEval (Zhou et al., 2023)); Math (GSM8K (Cobbe et al., 2021), GSM-Symbolic (Mirzadeh et al., 2025), Minerva Math (Lewkowycz et al., 2022), DeepMind Math (Saxton et al., 2019)); Code (HumanEval (Chen et al., 2021), HumanEval+ (Liu et al., 2023), MBPP (Austin et al., [1]), MBPP+ (Liu et al., 2024a), MultiPL-E (Cassano et al., 2023), CRUXEval-O (Gu et al., 2024)); Tool-Calling (BFCL v3 (Patil et al., 2025)); Multilingual (Global-MMLU (Singh et al., 2025), MGSM (Shi et al., 2022), INCLUDE (Romanou et al., 2025)); and Safety (6-task safety aggregate metric from Tulu3 (Zhao et al., 2026), AttaQ (Kour et al., 2023), SALAD-Bench (Li et al., 2024)). Each LLMON model is compared against the instruct counterpart at the same scale, trained on the matched chat-template corpus of § 4.1.

Indirect Prompt-Injection Benchmarks

Indirect prompt injection is the failure mode the privilege invariant is designed to prevent: a malicious instruction is planted in a data payload, such as a retrieved document or a tool response, rather than supplied by the user. We evaluate on two published benchmarks that place the injection on different surfaces. OPI (Liu et al., 2024b) injects into the data of classic NLP tasks and defines the five-attack taxonomy used throughout this evaluation (naive, escape, ignore, fake-completion, and combine), together with the attack-success metric. ASB (Zhang et al., 2025) moves the attack into an agent’s tool observations, which is the surface that matters for tool-using deployments; we report its observation-injection arm.

We report attack success rate (ASR), the fraction of adversarial prompts in which the injected instruction is followed; lower is better. For the LLMON models the injected payload arrives inside a “data” block reached through “exec<|.|>input”, so following it would require acting on a span the model was trained to read rather than execute; the instruct baselines receive the same payloads under conventional chat formatting.

Adversarial Red-Teaming

The two benchmarks above measure known, published attacks. Separately from them, and going beyond what a fixed benchmark can surface, our security team red-teamed both models with ARES (Nedoshivina et al., 2025), a robustness-evaluation harness that expands 808 seed prompts from five datasets (Purple (Bhatt et al., [3]), Gandalf (Pfister et al., 2025), and TensorTrust (Toyer et al., 2024) for direct attacks; Alpaca and a safety-policy set for indirect ones) into approximately 15k attack prompts by mutating each seed through eight attack strategies organized by attacker sophistication, from basic (asking directly) through intermediate (append, ignore, escape-separation, completion-real, append-and-prepend) to advanced. The advanced arm is a multi-turn variant of Crescendo (Russinovich et al., 2025), in which an attacker model generates each next turn and a judge decides whether the adversarial objective was met; we use DeepSeek-V3.2 (DeepSeek-AI et al., 2025) as the attacker, GPT-OSS (OpenAI et al., 2025) (120B) as the judge, and a budget of 20 turns. Results are reported as robustness ($1-\mathrm{ASR}$), so that higher is better, which is the convention of the harness; note that this is the opposite direction to Table 2.

Model	Training	OPI ↓	ASB ↓
Granite-4.1-3B	Instruct	100.0	99.2
	LLMON	0.1	5.6
Granite-4.1-8B	Instruct	96.3	52.8
	LLMON	0.02	3.1

Attention-Based Interpretability Metrics

Beyond output-level evaluation, we introduce three per-layer attention metrics designed to provide mechanistic insight into model behavior. These metrics quantify how attention from the final generated token is distributed across target and distractor spans. Let $A(S)$ denote the total attention mass assigned to a span set $S$. We define $\mathcal{I}_t$ and $\mathcal{I}_d$ as the target and distractor instruction spans, respectively, and $\mathcal{D}_t$ and $\mathcal{D}_d$ as the corresponding target and distractor data spans.

The Instruction Focus Ratio (IFR) measures the proportion of attention allocated to the target instruction relative to all instruction spans:

Similarly, the Data Attention Score (DAS) quantifies the proportion of attention assigned to the target data relative to all data spans:

Finally, Leakage (LEAK) is the share of all span attention that reaches distractor content, across both instruction and data spans. Writing $S$ for every labeled span in the prompt (the target and distractor instructions and data, together with the structural spans: the execution block, the delimiters, and the remaining prompt), LEAK is

These metrics are computed per layer across all attention heads (averaged over attention heads within each KV group for grouped-query-attention models). This enables layer-wise failure diagnosis: higher IFR and DAS values indicate stronger focus on task-relevant content, whereas larger LEAK values suggest increased attention allocated to irrelevant or competing information.

Results and Discussion

General Capabilities

Table 1 compares LLMON-structured post-training against the Instruct chat-template baselines at both scales under matched full fine-tuning. The objective here is parity rather than improvement: LLMON training is meant to install an execution-binding behavior, not to make the model better at mathematics or code. Averaged over the six capability categories, the 3B model is unchanged (66.7 both ways) and the 8B model loses 1.4 points (74.2 to 72.8), with mathematics at 8B the largest single regression (84.1 to 79.6). LLMON improves on Code and Multilingual at 3B. Safety improves at both scales, by 8.8 points at 3B and 6.2 at 8B, which is where the structural supervision is expected to show.

Model	Training	General	Alignment	Math	Code	Tool-Calling	Multilingual	Safety
Granite-4.1-3B	Instruct	57.91	82.30	75.19	64.19	60.80	59.89	80.89
	LLMON	57.16	81.02	73.25	65.45	60.71	62.68	89.64
Granite-4.1-8B	Instruct	64.95	87.06	84.09	72.38	68.27	68.68	84.19
	LLMON	63.31	86.69	79.63	71.01	67.93	68.24	90.36

Indirect Prompt Injection

Table 2 reports attack success rate on the two injection benchmarks of § 4.4. On text-level injection (OPI) the attack essentially stops working: ASR falls from 100.0 to 0.1 at 3B and from 96.3 to 0.02 at 8B. On agentic injection (ASB), where the payload arrives in a tool observation rather than in task data, ASR falls from 99.2 to 5.6 at 3B and from 52.8 to 3.1 at 8B. These are reductions of roughly 17× on agentic injection and three orders of magnitude on text-level injection, and they hold at both scales despite the 8B Instruct baseline already being markedly more robust on ASB than the 3B one. The mechanism is structural rather than a learned refusal: the injected instruction occupies a “data” block that no execution pointer references, so acting on it would require the model to execute a span it was trained to read. § 5.4 examines what this looks like inside the network.

Adversarial Red-Teaming

Independently of the published benchmarks of § 5.2, Table 3 reports the ARES red-teaming campaign of § 4.5, which uses its own seed datasets and attack strategies and separates results by where the adversarial instruction enters. Scores are robustness ($1 - \mathrm{ASR}$), so higher is better.

Model	Training	Indirect	Direct	Multi-turn
Granite-4.1-3B	Instruct	2.8	17.6	15.0
	LLMON	84.6	37.1	28.0
Granite-4.1-8B	Instruct	0.7	9.2	15.0
	LLMON	79.8	32.7	37.0

On indirect attacks the gain is large and is what the design predicts: robustness rises from 2.8% to 84.6% at 3B and from 0.7% to 79.8% at 8B, because the payload lands in a span that no execution block designates as an operable instruction. Direct and multi-turn attacks also improve, roughly doubling at both scales, but remain far weaker in absolute terms (32–37%). That asymmetry is expected rather than disappointing. When the adversarial instruction is the user’s own instruction there is no instruction-data boundary left to enforce, and an attacker who steers a conversation across twenty turns is not smuggling an instruction past the binding but building towards a request the model accepts on its own terms. The privilege invariant is therefore a defense against untrusted input, and should be composed with defenses of a different kind rather than relied upon alone.

Attention Interpretability Analysis

Both the injection reductions and the capability picture are consistent across the two scales tested, so the remainder of this section drills into how that robustness arises. Using the per-layer attention metrics defined in § 4.6, we analyze the Granite-4.1-3B pair (the LLMON-trained model and its Instruct counterpart) on distractor prompts in which several instructions, and in most cases several data segments, coexist while the “exec” block binds only one pair. We first report the per-layer behavior on a representative example to expose the mechanism, then aggregate over all examples to verify that the trends are not driven by ordering.

Where the binding lives. Both models are measured at the same layers, and the separation is largest in the late-middle of the stack rather than at the output. Table 4 reports the mean of each metric over the distractor examples at five representative layers. The LLMON-trained model holds a mean IFR of 86–87% at layers 25 and 28 against 26–38% for the Instruct model, with mean leakage of 6–9% against 35–45%. At the final layer both models collapse towards chance (IFR 28.0% vs 17.2%), which is why we report the span distribution at layer 31 rather than at layer 39: the execution binding is a mid-to-late-stack computation, and reading it off the last layer alone would understate it for both models.

Layer	IFR ↑ LLMON	IFR ↑ Inst.	DAS ↑ LLMON	DAS ↑ Inst.	LEAK ↓ LLMON	LEAK ↓ Inst.
25	86.5	25.8	59.1	38.0	8.6	44.9
28	87.2	37.7	62.2	36.6	6.3	34.9
31	63.8	15.3	53.9	37.2	12.2	40.6
34	75.8	26.3	57.9	35.6	6.4	47.2
39	28.0	17.2	47.0	40.2	13.7	39.7

Layer-wise Instruction Focus Ratio

Figure 2 shows the IFR computed at each transformer layer for both models on a representative distractor example, in which three instructions (“summarize”, “count the words”, “translate”) and two data segments coexist and the “exec” block binds only the word-counting instruction. The two models are indistinguishable through the first twenty layers, both fluctuating around 25–45%. They separate sharply from layer 22: the LLMON-trained model rises into a 55–89% band across layers 25–38, peaking at 89% at layer 28, whereas the Instruct model peaks at 62% and spends most of that range below 45%. The implication is that the binding is not a surface property of the prompt but a computation the network performs, and that LLMON training installs it in a specific region of the stack rather than shifting attention uniformly.

Layer-wise Distractor Leakage. Figure 3 presents the complementary LEAK metric across layers. Both models leak heavily in the first six layers, peaking above 50%, which is consistent with early layers aggregating context indiscriminately. From layer 7 the LLMON-trained model drops to a 5–25% band and settles below 15% after layer 25, while the Instruct model stays in the 20–50% range throughout and rises again in the last ten layers. Distractor suppression is therefore not a property of the input encoding (where the two models behave alike) but something acquired in the same region of the stack where instruction focus is established.

Attention span distribution. Figure 4 presents the per-span attention distribution and the three metrics at layer 31 for both models. The LLMON-trained model splits its span attention between the target instruction (I-T, 23.4%) and the target data it is bound to (D-T, 23.0%), with 35.6% on the prompt and execution structure and only 5.6% on the largest distractor instruction: IFR = 71.4%, DAS = 72.6%, LEAK = 9.9%. The Instruct model inverts this. Its single largest recipient is a distractor instruction (I-D1, 47.5%), with a second distractor taking 21.8% and the target instruction only 14.5%; target data attention is negligible. Its metrics record the failure directly: IFR = 17.3%, DAS = 22.1%, LEAK = 49.9%. The two models receive identical token sequences, so the difference is entirely in what the network does with them.

Layer-wise Data Attention Score. Figure 5 completes the picture with DAS. It follows the same shape as IFR: the two models track each other through the first twenty layers, then the LLMON-trained model rises into a 46-94% band across layers 23-38 while the Instruct model stays between 14% and 56%. All three metrics therefore separate in the same region of the stack, which is what

one expects if a single mechanism of resolving the execution binding is responsible for all three.

Token-level attention flow. Figure 6 traces how attention is allocated during generation, showing the correlation between each generated token and the KV-cache spans for the LLMON-trained model at layer 31. The target data span (D-T) activates precisely while the model is quoting the sentence it was asked to count, and falls away once it moves on to report the count; the target instruction span (I-T) carries weight at the opening tokens and again as the answer is formed. The distractor spans (I-D1, I-D2, D-D1) stay near zero throughout decoding. This token-resolved view complements the per-layer metrics: the model does not merely arrive at the right binding by the end of the forward pass, it holds it step by step as it produces the answer.

Conclusion

We presented a structured post-training methodology that uses LLMON’s explicit instruction-data separation and identifier-based execution binding to improve controlled execution in LLMs. Our evaluation on two Granite-4.1 base models demonstrates that: (i) LLMON-structured supervision reduces attack success rates on indirect prompt-injection benchmarks from 53-100% to 0.02-5.6%, without catastrophic degradation on standard capability measures; (ii) the effect holds at both 3B and 8B scale, and extends to adversarial red-teaming on indirect attacks, though direct and multi-turn attacks remain substantially harder; and (iii) attention-based interpretability metrics (IFR, DAS, LEAK) provide actionable per-layer failure diagnosis, revealing how structural training reshapes internal attention patterns. These findings suggest that explicit execution structure can be reliably taught to LLMs through standard post-training, offering a training-time complement to inference-time safety mechanisms.

Limitations

We discuss several important limitations of this work. First, our evaluation covers two model scales (3B and 8B) within a single model family and generation. While LLMON’s structural supervision is architecture-agnostic, it remains to be validated whether the same magnitude of improvement transfers to other pre-training corpora and architectures, and to larger scales where models may already exhibit stronger implicit instruction-following. Second, LLMON’s delimiter tokens increase context length by approximately 7-15% depending on the example complexity. We do not systematically quantify the impact on inference latency and throughput, though sequence packing at training time largely amortizes this cost. Third, the privilege invariant addresses indirect injection, where the adversarial instruction arrives as input. It offers far less protection when the adversarial instruction is the user’s instruction, since there is then no instruction-data boundary to enforce: red-teaming robustness reaches only 33-37% on direct attacks and 28-37% on multi-turn attacks, against 80-85% on indirect ones, and it is a learned behavior rather than an enforced guarantee: it substantially reduces, but does not eliminate, the chance that an instruction embedded in data is followed, and should be one layer of a defense rather than the only one. Fourth, while our capability evaluation includes multilingual benchmarks, the LLMON training corpus itself is predominantly English, and we have not characterized how tag overhead behaves on morphologically rich languages or on tasks with very long documents. Finally, our IFR/DAS/LEAK metrics provide correlation between attention patterns and output quality, but we do not establish a causal mechanism. Attention weights are an imperfect proxy for information flow in transformers, and the relationship between attention-level metrics and generation-level performance may not be monotonic.

LLMON Examples

Multi-instruction and Multi-data Distractors

Figure 7 illustrates a distractor example: three instructions and two data segments coexist, but the “exec” block unambiguously binds only one instruction-data pair. The model must learn to attend selectively to the referenced segments and ignore all others.