betaFeedback

WorksComputer Science

Nemotron 3.5 Bolt E4B 4GB: An Open and Efficient Small Language Model for Fast Conversational Agents

Abstract

We introduce Nemotron 3.5 Bolt E4B 4GB, an open and efficient small language model further post-trained from Gemma-4-E4B-it and optimized for speed and intelligence per GB of memory used for fast conversational agents. The model carries forward Gemma's strong conversational capabilities and further improves multi-turn tool use, instruction following, and input grounding for fast, local voice/device assistants, interactive personas, and embedded/Edge interfaces. The model combines natural conversation with multi-turn agentic capabilities to power voice/device assistants, interactive characters and robotic/IoT interfaces running on GeForce RTX, Jetson, and RTX/DGX Spark. Experiments on 21 benchmarks show that Nemotron 3.5 Bolt E4B 4GB achieves the highest average accuracy while generating the fewest output tokens in both reasoning and non-reasoning modes, compared with Gemma-4-E4B-it and Qwen3.5-4B. The model is released with a DFlash drafter. With DFlash, Bolt E4B achieves 2.74×2.74\times the per-user decode speed of inference without speculative decoding at concurrency~1, and 1.21×1.21\times the generation throughput of Gemma-4-E4B-it with MTP at batch size 1.

Introduction

Average benchmark score versus time to first token

Figure 1. Average benchmark score versus time to first token (TTFT). Scores are averaged across MultiChallenge, RoleMRC, BFCL-v3, IHEval, SysBench, and RelyToolBench. Local models use Q4_K_M quantization and are measured on RTX 5090 with input/output sequence lengths of approximately 3,000/50 tokens. Cloud reference results (score; TTFT): Gemini 3.8 Flash (77.1; 2,449.5ms) and Claude Sonnet 5.5 (76.2; 3,002.2ms).

Voice agents, interactive characters, and robotic interfaces need language models that can hold a conversation and take actions reliably. These models must follow instructions, use tools across multiple turns, and make sense of conversation history, application state, and tool results. For local deployment, they must also respond quickly and fit within limited memory.

We introduce Nemotron 3.5 Bolt E4B 4GB, an open small language model designed for fast conversational agents with a target memory footprint of approximately 4 GB. Nemotron 3.5 Bolt E4B 4GB supports reasoning and non-reasoning modes. The model can also generate concise conversational responses by enabling conversational_mode.

Nemotron 3.5 Bolt E4B 4GB is based on Gemma4-E4B-it (Gemma, 2026) and further trained using RLVR and RLHF through NeMo-RL [18] and NeMo-Gym [17]. Training covers conversational tool use, tool-call schemas, instruction following, instruction hierarchy, and output formatting. We use every training sample in both reasoning and non-reasoning modes. This choice emphasizes accuracy without explicit reasoning while preserving reasoning capabilities for tasks that benefit from them.

We evaluate Nemotron 3.5 Bolt E4B 4GB on 21 benchmarks spanning conversational proficiency, multi-turn tool calling, instruction following/control, context robustness, and state/action grounding. Across this diverse evaluation suite, Nemotron 3.5 Bolt E4B 4GB achieves the highest average benchmark score compared with Gemma4-E4B-it and Qwen3.5-4B in both reasoning and non-reasoning modes. It also generates fewer output tokens on average than either baseline in both reasoning and non-reasoning modes. These results demonstrate that Nemotron 3.5 Bolt E4B 4GB is a strong fit for fast conversational agents.

For deployment, we quantize the BF16 checkpoint to Q4_K_M with importance-matrix calibration. We also provide a DFlash parallel drafter to accelerate generation through speculative decoding. The model is available via HF and NVIDIA NIM.

Checkpoints

NVIDIA NIM

Data

Chat Template Design

We extend the chat template of Gemma4-E4B-it to support alignment between tool-use and spoken responses in conversational applications. When an assistant message contains text and tool calls, the template puts the text before the calls and their results. This order lets the assistant acknowledge a request or explain an action before it calls a tool. We keep the existing enable_thinkingenable\_thinking and preserve_thinkingpreserve\_thinking controls.

Conversational Mode

Nemotron 3.5 Bolt E4B 4GB also supports an optional conversational_modeconversational\_mode flag. When this flag is true, the template adds a system instruction to keep responses concise and without excessive text formatting, making it more suitable for spoken conversations. The instruction is added after an existing system prompt, or if there is no system prompt, this instruction becomes the system prompt. When the conversational_modeconversational\_mode flag is absent or false, no instruction is added to the system prompt.

Post-Training

Nemotron 3.5 Bolt E4B 4GB is further post-trained from Gemma4-E4B-it with multi-environment RLVR and RLHF. The collection of environments targets instruction following, multi-turn tool-calling and chat capabilities. The exact list of environments and link to the publicly released datasets is shown in Table 1.

EnvironmentsSource data
Tool-Calling🤗 Conversational Tool Use Pivot
Tool-call schema🤗 Function Calling Pivot
Instruction Following🤗 Instruction Following, SysBench, InverseIFEval, CFBench, MultiTurnChat, Calendar
Instruction Hierarchy🤗 System-User-Instruction-Hierarchy
Structured Outputs🤗 Structured Outputs
Free-form formatting🤗 Free-Form Formatting
GenRMUpdated version of 🤗 Nemotron-RLHF-GenRM-v1

Table 1. Environments and source datasets used in post-training for Nemotron-3.5-Bolt-E4B.

The RLVR and RLHF post-training is done using NeMo-RL and NeMo-Gym. We use a global batch size of 256 with 8 rollouts per prompt, a maximum generation length of 4096 tokens for reasoning samples, and 1024 for non-reasoning samples.

Non-Reasoning Focus

To emphasize non-reasoning accuracy in post-training, we use each sample in our RLVR/RLHF training blend with both reasoning (enable_thinking=trueenable\_thinking=true) and non-reasoning (enable_thinking=falseenable\_thinking=false) inference modes. We independently perform reward profiling for the datasets in both reasoning modes, and maintain a deliberate 2 : 1 non-reasoning to reasoning samples ratio in each training batch.

Verbosity Control

We observe that as RL post-training progressed, there was a considerable increase in verbosity for both reasoning and final answer tokens. We were able to control verbosity by increasing group-relative length penalties for reasoning and answer length in the GenRM environment, as well as disabling overlong filtering for all other environments. While these interventions were effective in controlling verbosity, we found that tightening and loosening the length penalties, along with enabling and disabling overlong filtering in a cycle as the training progressed helped improve accuracy while still maintaining lower verbosity.

SFT + RL vs. RL without SFT

In an attempt to improve tool-calling behaviors of Gemma4-E4B-it, we experimented with SFT using datasets from Nemotron-Post-Training-Datasets. We observed that we were able to improve multi-turn tool-calling with SFT and then further RLVR to a higher ceiling and with less RL training steps than with just RL-based post-training. However, SFT on data from diverse teachers resulted in a regression on conversational qualities and verbosity of Gemma4-E4B-it, for which it was used as a starting point. Hence, Nemotron-3.5-Bolt-E4B did not include an SFT stage and was post-trained purely with RL.

Post-Trained Model Evaluations

Benchmarks and Evaluation Setup

We evaluate Nemotron 3.5 Bolt E4B 4GB on a comprehensive suite of benchmarks covering conversational proficiency, multi-turn tool use, instruction following/control, context robustness, state/action grounding, and response conciseness. All evaluations for Nemotron 3.5 Bolt E4B 4GB and baselines are conducted with NeMo-Gym. For each benchmark, we estimate the number of repeats needed to limit score variation to 1–2 percentage points and evaluate each model using that number of repeats. The following sections describe our benchmark selection and evaluation methodology across these capabilities, and Table 2 presents the evaluation settings.

BenchmarkMetricReasoning ModeJudge/User ModelRepeats
Conversational Proficiency
MultiChallengeOverall ScoreOnJudge: GPT-4o16
RoleBench CustomWin % vs. Gemma4-E4B-itOffJudge: GPT-5.54
RoleMRCJudge Average w/o Multi-TurnOnJudge: GPT-4.18
EQ-Bench v3Rubric ScoreOffJudge: Ensemble14
Brevity BenchmarkImplicit - Average of Answer Quality & ConcisenessOnUser: Qwen3-235B-A22B-Instruct-25074
Multi-Turn Tool Use
BFCL-v3Average AccuracyOnN/A4
When2Call Multi-TurnOverall AccuracyOnJudge: GPT-5.24
ToolSandboxAverage AccuracyOnUser: GPT-4o4
Tau2-BenchAverage AccuracyOnUser: GPT-5.24
Instruction Following & Control
IHEvalConflict AccuracyOnN/A2
AgentIFInstruction Success Rate (ISR)OnJudge: GPT-5.14
StructEval-TAverage AccuracyOnN/A4
IFBenchPrompt Loose AccuracyOnN/A4
InverseIFEvalOverall AccuracyOnJudge: GPT-5.18
Context Robustness
RULER (64K)Overall AccuracyOffN/A4
LongMemEvalOverall AccuracyOnJudge: GPT-4o8
LongBench v2 (48K)Overall AccuracyOnN/A4
SysBenchInstruction Success Rate (ISR)OnJudge: GPT-5.14
State & Action Grounding
GaRAGeRelevance-Aware Factuality (RAF)OnJudge: GPT-5.14
RAGTruthF1OnN/A4
RelyToolBench w/ Native Function-CallingReal Pass Rate (RePR)OnJudge: GPT-5.18

Table 2. Evaluation configuration for the benchmark suite. Reasoning mode indicates whether reasoning is enabled (On) or disabled (Off). Judge and User identify the evaluator and simulated user models, respectively.

Conversational Proficiency. We evaluate conversational proficiency using EQ-Bench v3 [22, 21] for emotional understanding, empathy, and social reasoning, with an ensemble of Claude-Sonnet-5.5, Gemini-4-Argon, and GPT-6.1-Sol judges; MultiChallenge (Sirdeshmukh et al., 2025) for instruction retention, context tracking, and response consistency across multiple turns, judged by GPT-4o; RoleBench (Wang et al., 2024) for role-playing fidelity and character-consistent responses, judged by GPT-5.5; and RoleMRC [14] for role-conditioned reading comprehension and instruction following, judged by GPT-4.1. For RoleBench, we use a custom pairwise evaluation against a fixed Gemma4-E4B-it reference, with each response pair judged in both answer orders to reduce position bias. For RoleMRC, we report the average judge score excluding the multi-turn subset.

Response Conciseness. As part of conversational proficiency, we also evaluate response conciseness using an internal benchmark for brevity which covers personal assistant, customer service, and passage-based question-answering tasks. The benchmark compares response length, excluding reasoning tokens, with human-written reference responses, awarding high scores to responses that are no longer than the references. System prompts request concise answers through general instructions such as “brief” or “suitable for spoken conversation”, making conciseness an instruction-following task. Answer quality is assessed separately by an LLM judge to check that shorter responses preserve quality. We report the average of answer quality and conciseness scores. The benchmark will be publicly released in the future. We also use the results from this benchmark to determine the system prompt to be used for conversational_mode described in the chat template design section.

Multi-Turn Tool Use. We evaluate multi-turn tool use using BFCL-v3 [23, 15] for fundamental tool calling; τ2\tau^2-Bench [3] for task completion through tool use and dialogue, with GPT-5.2 as the user model; and ToolSandbox [13] for stateful tool use in conversational settings, with GPT-4o as the user model. We also evaluate When2Call-MT, our multi-turn extension of When2Call [27], which tests whether an assistant selects the correct response policy as a conversation develops, including when to call a tool, ask for clarification, refuse due to lack of tools, or respond directly. When2Call-MT uses GPT-5.2 as the judge and will be publicly released in the future.

Instruction Following & Control. We evaluate instruction following and control using AgentIF [25] for following complex instructions in agentic settings, judged by GPT-5.1; IFBench [24] for generalization to verifiable instruction-following constraints; IHEval (Zhang et al., 2025b) for resolving conflicting instructions according to the instruction hierarchy; InverseIFEval (Zhang et al., 2025a) for following counter-intuitive instructions, judged by GPT-5.1; and StructEval-T (Yang et al., 2026) for generating outputs that satisfy specified structural and formatting requirements. For InverseIFEval, we evaluate using the methodology as originally described, but use only English data (506 of 1012 total samples) and translate the judge prompt to English.

Context Robustness. We evaluate context robustness using LongBench v2 [2] for understanding and reasoning over long contexts, restricted to the subset with fewer than 48K tokens due to the model’s context limit; LongMemEval (Wu et al., 2025) for retaining and retrieving information across conversation histories, judged by GPT-4o; RULER [10] for retrieval and reasoning at a 64K-token context length; and SysBench [26] for following system instructions across varied user inputs, judged by GPT-5.1. For SysBench, we translate the benchmark data into English and use an adapted English judge prompt.

State & Action Grounding. We evaluate state and action grounding using GaRAGe (Sorodoc et al., 2025) for retrieved evidence-grounded answers, judged by GPT-5.1; RAGTruth [16] for detecting unsupported claims in retrieval-augmented responses, measured by span-level F1; and RelyToolBench (Xu et al., 2025) for reliable tool use under tool failures and misleading tool responses. We evaluate RelyToolBench with native function calling instead of the ReAct format and use GPT-5.1 as the judge. For GaRAGe, we follow the evaluation methodology as originally described and use our own implementation due to a lack of publicly available code.

Results

We show all benchmark evaluations in Table 3. Our model outperforms the competitive models on 15 out of 21 benchmarks: Conversational Proficiency (4/5), Multi-Turn Tool Use (1/4), Instruction Following & Control (5/5), Context Robustness (2/4), and State & Action Grounding (3/3).

Table 3 | Evaluation results for BF16 checkpoints. Evaluation metrics and reasoning settings are listed in Table 2. Bold marks the best score in each row. | | | |

BenchmarkNemotron-3.5 Bolt-E4BGemma4 E4B-itQwen3.5 4BNemotron-3 Nano-4B
Conversational Proficiency
MultiChallenge53.1042.0851.7635.30
RoleBench Custom67.0549.9553.6612.02
RoleMRC80.6374.6265.8147.00
EQ-Bench v339.4041.1732.9219.31
Brevity Benchmark82.5279.9975.3573.60
Multi-Turn Tool Use
BFCL-v368.5166.3970.9561.10
When2Call Multi-Turn63.3662.9037.8527.70
ToolSandbox68.7765.4369.5457.88
Tau2-Bench55.1038.5075.0629.20
Instruction Following & Control
IHEval84.7967.0460.5751.31
AgentIF25.8121.8920.4420.08
StructEval-T80.4079.2776.0163.50
IFBench63.7942.8458.9160.10
InverseIFEval46.8442.0030.9835.25
Context Robustness
RULER (64K)86.9986.5595.1591.51
LongMemEval72.1070.9566.8525.20
LongBench v2 (48K)50.5049.3459.9331.40
SysBench67.8865.8567.3460.88
State & Action Grounding
GaRAGe42.6134.7324.6625.18
RAGTruth77.5976.9775.6071.40
RelyToolBench63.1661.6043.3541.20

Table 3. Evaluation results for BF16 checkpoints. Evaluation metrics and reasoning settings are listed in Table 2. Bold marks the best score in each row.

Non-Reasoning Evaluations. We also conduct all benchmark evaluations in non-reasoning mode across models to validate the effect of focused post-training. The results are presented in Table 4. The results show a similar trend as reasoning, and Nemotron 3.5 Bolt E4B 4GB shows the highest accuracy on 14 out of 21 benchmarks.

BenchmarkNemotron-3.5 Bolt-E4BGemma4 E4B-itQwen3.5 4BNemotron-3 Nano-4B
Conversational Proficiency
MultiChallenge48.8140.7238.6023.27
RoleBench Custom67.0549.9553.6612.02
RoleMRC77.1872.4658.9146.11
EQ-Bench v339.4041.1732.9219.31
Brevity Benchmark78.7175.3975.2172.43
Multi-Turn Tool Use
BFCL-v364.6858.0866.0445.23
When2Call Multi-Turn51.2554.8046.0238.67
ToolSandbox61.5435.7252.1955.32
Tau2-Bench34.3425.9973.4626.78
Instruction Following & Control
IHEval66.9052.9534.2523.78
AgentIF23.7622.9821.2216.69
StructEval-T78.7676.0475.8410.38
IFBench53.3338.8334.3341.25
InverseIFEval30.2422.1322.9418.97
Context Robustness
RULER (64K)86.9986.5595.1591.51
LongMemEval60.2065.7965.0031.20
LongBench v2 (48K)41.0641.8951.7031.13
SysBench65.1264.0965.1255.28
State & Action Grounding
GaRAGe46.0437.3322.1923.61
RAGTruth67.3858.2065.0830.87
RelyToolBench61.2558.8719.4336.18

Table 4. Non-reasoning evaluation results for BF16 checkpoints. Bold marks the best score in each row, including ties.

Live Testing

Average benchmark score versus mean output token count across 21 benchmarks

Figure 2. Average benchmark score versus mean output token count across 21 benchmarks in reasoning and non-reasoning settings. Higher scores and fewer output tokens indicate higher quality and greater response conciseness, respectively. The horizontal axis uses a logarithmic scale.

Live Testing

Academic benchmarks only partially capture the skills required of conversational agents and typically assess individual capabilities rather than how they work together over a complete interaction. We therefore complement benchmark evaluations with live testing in interactive scenarios spanning virtual interactive characters, on-device voice assistants, and chat interactions. These tests are qualitative and examine whether the model can sustain coherent conversations, respond to changing user intent, environment feedback, handle ambiguity, and recover from mistakes or tool failures. GIFs in Figure 3 shows captures from some live-testing instances for Nemotron 3.5 Bolt E4B 4GB.

Reachy Mini running on Jetson Orin Nano 8GBBolt E4B controlling a blacksmith trader NPC

Figure 3. Reachy Mini running on Jetson Orin Nano 8GB. Cosmos 3 Edge [19] supplies real-time visual understanding of the world to Bolt E4B, which calls scripts from the robot’s animation libraries and powers its spoken responses. Bolt E4B controlling a blacksmith trader NPC. The NPC agent verifies correct payment for an item, then autonomously calls tools to check crafting recipes, collect ingredients, craft the item, and transfer it to the user. Examples of live-testing conducted with Nemotron 3.5 Bolt E4B 4GB. View the GIFs in our blog.

Quantization

We used llama.cpp to convert the Nemotron 3.5 Bolt E4B 4GB BF16 checkpoint to Q4_K_M with importance-matrix (imatrix) calibration. The calibration was conducted with approximately 1M tokens of RL rollouts from the training step that was selected as the BF16 checkpoint. On average accuracy, Q4_K_M with imatrix calibration not only outperforms vanilla Q4_K_M, but also IQ4_XS and IQ4_NL with imatrix calibration. Evaluation results for quantized models are shown in Table 5.

BenchmarkNemotron-3.5 Bolt-E4B imatrixGemma4 E4B-itQwen3.5 4BNemotron-3 Nano-4BQ4_K_M no imatrixIQ4_XSIQ4_NL
Q4_K_M model comparisonNemotron-3.5-Bolt-E4B Q4 variants
Conversational Proficiency
MultiChallenge55.6039.3652.2334.5351.1251.9152.12
RoleBench Custom71.2160.5152.3510.8665.2169.0769.86
RoleMRC79.1275.2462.6145.8480.3678.7975.65
EQ-Bench v338.1539.7732.0719.4537.4238.1938.28
Brevity Benchmark82.8478.4975.1770.9881.2981.3082.04
Multi-Turn Tool Use
BFCL-v367.3165.0370.5659.0764.3265.9567.68
When2Call-MT60.5061.1242.8835.9658.0660.6760.92
ToolSandbox74.1368.3174.5159.7870.2370.4469.88
τ2\tau^{2}-Bench52.3238.7261.7414.8946.9542.0548.74
Instruction Following & Control
IHEval82.4066.6162.8753.7583.5083.1983.65
AgentIF22.9520.0522.2421.2524.3325.3225.46
StructEval-T79.9977.2876.3816.5478.4481.4081.25
IFBench64.0042.6766.6758.5859.7959.0063.33
InverseIFEval45.2639.7522.0436.0242.6944.4743.08
Context Robustness
RULER (64K)85.7686.1495.3791.3182.9285.8486.88
LongMemEval69.9267.9576.6027.1564.3070.2072.80
LongBench v2 (48K)48.6745.7058.2837.4244.3747.3347.67
SysBench67.2265.6267.9060.4566.5568.0466.68
State & Action Grounding
GaRAGe40.7330.9131.7824.8038.8538.7439.95
RAGTruth77.8877.1774.0871.7576.9277.0678.24
RelyToolBench69.2967.1538.3438.9763.5064.1166.31

Table 5. Evaluation results for four-bit quantized checkpoints. The left block compares Q4_K_M checkpoints, with importance-matrix calibration (imatrix) used for Bolt. We use the Unsloth Q4_K_M GGUF checkpoints for Gemma4-E4B-it and Qwen3.5-4B.2 The right block shows additional Bolt variants: Q4_K_M without imatrix, IQ4_XS, and IQ4_NL. All benchmarks use reasoning mode except EQ-Bench v3, RoleBench Custom, and RULER (64K), which use non-reasoning mode.

Multilingual Evaluations

Since Nemotron 3.5 Bolt E4B 4GB was post-trained using English only, we tested if the model retains the original model’s multilingual capabilities. To test this, we evaluated Nemotron 3.5 Bolt E4B 4GB on WMT24++ for 36 languages and MMLU-ProX on 12 languages.

The results are shown in Table 6 and Table 7 respectively. While Nemotron 3.5 Bolt E4B 4GB retains the original model’s machine translation quality, it shows slight but consistent drops on MMLU-ProX. These results suggest that translation quality is preserved more consistently than multilingual knowledge and reasoning performance.

LanguageEnglish →\to X xCOMETEnglish →\to X chrFX →\to English xCOMETX →\to English chrF
ar_EG67.7 (-1.00)40.5 (+0.20)75.5 (+0.10)48.3 (-0.20)
ar_SA73.9 (-0.70)40.7 (+0.00)76.6 (+0.00)49.2 (-0.10)
bg_BG71.5 (+0.00)57.4 (+0.10)81.5 (-0.10)59.8 (-0.30)
cs_CZ66.0 (+1.60)49.8 (+0.20)82.2 (+0.10)59.1 (-0.20)
da_DK85.6 (-0.20)63.5 (-0.20)89.3 (-0.10)64.1 (+0.00)
de_DE90.9 (+0.20)58.6 (+0.00)90.1 (+0.00)58.8 (-0.10)
el_GR67.0 (+0.40)54.5 (+0.00)81.0 (+0.10)62.7 (+0.00)
es_MX84.6 (-0.20)66.6 (-0.50)89.3 (-0.10)66.4 (-0.20)
et_EE41.4 (+1.30)45.9 (+0.30)68.3 (-0.40)56.3 (+0.10)
fi_FI75.0 (+0.70)57.4 (+0.00)81.6 (+0.10)57.5 (-0.20)
fr_CA79.4 (-0.70)67.5 (+0.00)88.0 (-0.10)64.8 (-0.20)
fr_FR79.5 (-0.30)63.2 (+0.00)87.2 (-0.20)60.5 (-0.10)
hi_IN67.2 (+0.10)39.9 (+0.00)83.6 (+0.10)49.1 (-0.20)
hr_HR74.2 (+0.50)53.6 (+0.30)83.1 (+0.00)60.0 (-0.10)
hu_HU61.4 (+1.10)45.7 (+0.10)78.5 (+0.20)54.4 (-0.20)
id_ID83.8 (+0.00)63.2 (+0.20)89.0 (-0.10)57.9 (+0.10)
it_IT82.1 (-0.80)64.5 (-0.30)88.6 (+0.30)63.3 (-0.20)
ja_JP82.1 (+0.10)36.2 (+0.80)84.5 (+0.70)50.5 (-0.40)
ko_KR84.3 (+0.30)35.6 (-0.10)84.5 (+0.30)54.1 (-0.10)
lt_LT44.1 (+0.80)41.0 (-0.20)68.4 (-0.30)47.6 (-0.10)
lv_LV37.8 (+1.60)43.5 (-0.50)66.6 (-0.20)52.4 (-0.20)
nl_NL85.8 (+0.20)58.9 (+0.00)89.3 (+0.10)60.3 (+0.00)
no_NO87.6 (+0.20)67.0 (+0.10)89.0 (-0.10)66.7 (-0.30)
pl_PL78.3 (-0.10)48.5 (-0.20)84.9 (-0.40)56.9 (-0.20)
pt_BR85.0 (-0.10)64.5 (-0.20)89.0 (+0.00)62.5 (-0.10)
pt_PT83.8 (-0.20)61.1 (+0.00)88.2 (-0.40)60.9 (+0.00)
ro_RO73.3 (+0.00)59.7 (+0.00)82.2 (-0.10)63.6 (-0.10)
ru_RU78.7 (+0.20)51.5 (+0.10)83.4 (+0.00)53.0 (-0.10)
sk_SK59.9 (+1.40)45.4 (+0.40)80.8 (+0.20)56.0 (-0.10)
sl_SI58.6 (+1.70)48.2 (+0.60)78.6 (+0.10)58.0 (-0.20)
sv_SE86.7 (+0.60)63.2 (-0.10)89.4 (+0.00)65.1 (+0.00)
th_TH81.4 (-0.60)51.2 (-0.40)86.3 (+0.10)54.1 (-0.20)
tr_TR74.8 (-0.40)58.5 (+0.10)81.8 (-0.40)57.9 (+0.10)
uk_UA76.4 (-0.30)54.2 (+0.00)80.4 (+0.00)58.3 (-0.20)
vi_VN80.7 (-0.90)55.2 (-0.30)87.3 (+0.40)55.6 (-0.30)
zh_CN80.8 (-0.80)37.3 (+0.10)87.7 (+0.00)52.3 (-0.40)
zh_TW80.3 (-0.80)31.0 (-0.10)89.2 (-0.10)56.4 (-0.20)
Average74.4 (+0.20)52.5 (+0.00)83.4 (+0.00)57.7 (-0.10)

Table 6. Table 6 | WMT24++ translation results for Nemotron-3.5-Bolt-E4B BF16, measured by xCOMET and chrF (both ↑\uparrow). Both models use non-reasoning mode; reasoning mode shows similar trends. Parenthesized deltas are relative to Gemma4-E4B-it for the same language, metric, and direction. Green indicates improvement and red indicates degradation.

LanguageReasoningNon-reasoning
ar64.04 (-1.97)60.90 (-1.69)
de66.15 (-1.59)64.33 (-1.05)
en70.41 (-0.57)69.34 (-0.69)
es67.24 (-1.33)65.11 (-0.76)
fr66.99 (-1.30)65.01 (-1.08)
hi64.72 (-1.73)61.49 (-1.52)
it66.97 (-1.67)65.26 (-1.00)
ja64.49 (-1.87)62.17 (-1.64)
ko64.51 (-1.77)61.90 (-1.57)
pt66.59 (-1.49)64.70 (-1.56)
vi66.56 (-1.35)64.59 (-0.90)
zh65.17 (-1.61)63.32 (-1.12)
Average66.15 (-1.52)64.01 (-1.22)

Table 7. Table 7 | MMLU-ProX results (↑\uparrow) for Nemotron-3.5-Bolt-E4B BF16 with reasoning enabled and disabled. Parenthesized deltas are relative to Gemma4-E4B-it for the same language and mode. Green indicates improvement and red indicates degradation.

Multimodal Evaluations

Although the model was trained text-only, we found that the model retains the original model’s multimodal capabilities.

Audio understanding and spoken interaction. We evaluate Nemotron 3.5 Bolt E4B 4GB and Gemma4-E4B-it on two popular speech benchmarks: VoiceBench ([6]) and AudioMC ([9]). VoiceBench is a single-turn speech benchmark comprising subsets of benchmarks covering multiple-choice question answering (OpenBookQA, MMSU), open-ended question answering (CommonEval, SD-QA, AlpacaEval, WildVoice), reasoning (BBH), instruction following (IFEval), and safety (AdvBench). AudioMC is a multi-turn speech benchmark built on the text benchmark MultiChallenge, with an additional evaluation of voice editing that assesses a model’s ability to track information as users provide updates during a conversation.

Vision capabilities. In contrast, Nemotron 3.5 Bolt E4B 4GB exhibits slight but consistent degradation in vision capabilities. We attribute this degradation to misalignment between the vision encoder and the language model introduced by text-only training. However, the declines are not substantial on most vision benchmarks, suggesting that Nemotron 3.5 Bolt E4B 4GB still retains visual understanding capabilities of the original model. We attempted additional training of the vision encoder alone but observed no meaningful improvements. Restoring and further improving vision capabilities remain part of future work.

The results in Table 8 show that Nemotron 3.5 Bolt E4B 4GB improves upon Gemma4-E4B-it on both VoiceBench and AudioMC, with reasoning enabled and disabled. Both models support speech inputs of up to 30 seconds. In AudioMC, 23.36% of user turns exceed this limit, and 53.1% of conversations contain at least one such turn. Our reported AudioMC results are therefore subject to this per-turn input limit and do not reflect evaluation on the complete, unrestricted conversations. For the SD-QA subset of VoiceBench, we report the average of the GPT score and the PANDA score. We compute PANDA using qa_metrics v0.2.42, which showed better agreement with GPT-based judgments in our evaluation.

BenchmarkReasoning: Nemotron-3.5 Bolt-E4BReasoning: Gemma4 E4B-itReasoning: Qwen3.5 4BNon-reasoning: Nemotron-3.5 Bolt-E4BNon-reasoning: Gemma4 E4B-itNon-reasoning: Qwen3.5 4B
Audio
Audio MultiChallenge ↑25.22 (+6.19)19.03—19.25 (+2.21)17.04—
Voicebench ↑80.30 (+2.10)78.20—72.60 (+0.90)71.70—
CoVoST (BLEU) ↑36.14 (-0.81)36.95—37.61 (+0.59)37.02—
FLEUR (WER/CER) ↓10.16 (+0.38)9.78—10.43 (+0.26)10.17—
OpenASR (WER) ↓———9.28 (-0.25)9.53—
Vision
OmniDocBench 1.5 ↓0.42 (+0.21)0.210.170.47 (+0.23)0.240.20
MMMU-Pro Vision ↑42.02 (-5.61)47.6366.1339.08 (-7.10)46.1856.18
MathVision ↑46.25 (-2.04)48.29—43.72 (-0.36)44.0853.45
MedXpertQA Vision ↑26.85 (-1.35)28.2041.8523.10 (-0.65)23.7535.35

Table 8. Audio and vision benchmark results in reasoning and non-reasoning modes. Gemma4-E4B-it scores are measured in our evaluation setup1. Bold indicates the best score among models within each mode. Arrows indicate whether higher (↑) or lower (↓) scores are better; CoVoST excludes Chinese. Parenthesized deltas are Bolt minus the measured Gemma4-E4B-it score in the same mode; Green indicates improved performance and red indicates degraded performance, accounting for whether higher or lower scores are better.

Speculative Decoding

Nemotron 3.5 Bolt E4B 4GB is released with a DFlash [5] parallel drafter. The drafter proposes all draft tokens in one forward pass, which the target verifies in parallel; verification preserves the output distribution of the target [12, 4]. We measure drafting quality by the acceptance length (AL), the number of tokens generated per verification step, including the one contributed by the target (at most 8 for a draft length of 7).

Algorithm selection. Recent extensions of DFlash add a lightweight correction head that improves the mutual consistency of the drafted tokens. We compare DFlash with LiLiCorr (Rusanovsky et al., 2026), a state-of-the-art extension that jointly scores candidate tokens across all drafted positions. For Nemotron 3.5 Bolt E4B 4GB, LiLiCorr increases AL by about 5%. However, because each decoding step of such a small model is short, even a lightweight head adds non-negligible overhead, and the two drafters achieve comparable per-user decode speed at concurrency 1. We therefore adopt DFlash, which minimizes the drafting overhead.

Drafter and training. Our DFlash drafter is a five-layer Transformer with 360M parameters (excluding the token embeddings and LM head, which are reused from the target) and drafts seven tokens per step. Attention is bidirectional among the drafted positions and restricted to a sliding window of the most recent 1,024 positions. The drafter is conditioned on hidden states from five intermediate layers of the target. We train it in NVIDIA Model Optimizer (ModelOpt; NVIDIA Corporation, 2024–2026) by distilling the next-token distribution of the target at every drafted position. The training data comprise 3.79M conversations whose prompts are drawn from Nemotron-Post-Training-Dataset-v2 and whose responses are regenerated by the target at temperature 1.0, in both reasoning-enabled and reasoning-disabled modes.

Quantization. To speed up drafting on GPUs with lower memory bandwidth, typical of local deployments, we apply post-training NVFP4 quantization to the drafter using GPTQ [7] in ModelOpt. This reduces the weights read per drafting step, including the LM head, by about 3.6×, with at most a 2% reduction in AL. By default, activations remain in BF16 (W4A16); an alternative variant of the checkpoint also quantizes them to NVFP4 (W4A4), which yields additional speedup at higher concurrency on GPUs with native NVFP4 support. Table 9 reports the speedups of both variants on DGX Spark.

DrafterWeights (GB)ALSpeedup at concurrency 1Speedup at concurrency 16Speedup at concurrency 32
BF162.062.991.00×1.00×1.00×
NVFP4 W4A16 (default)0.582.991.13×1.05×1.01×
NVFP4 W4A40.582.931.11×1.09×1.06×

Table 9. Weights read per drafting step (including the LM head), AL, and per-user decode speedup of the NVFP4 drafters relative to the BF16 drafter on DGX Spark. AL is measured at concurrency 1.

Evaluation. Table 10 reports AL of the released drafter on MT-Bench (Zheng et al., 2023), the qualitative split of SPEED-Bench [1], and SPEED-Bench prompts with 32K-token inputs.

Benchmarktemp=0temp = 0temp=1temp = 1
MT-Bench4.013.65
SPEED-Bench, qualitative split3.883.50
SPEED-Bench, 32K-token inputs3.423.00

Table 10. AL of the released drafter (NVFP4, W4A16) with reasoning enabled, on one GB300 GPU at concurrency 16 with up to 4,096 output tokens. At temp=1temp = 1, the target samples while the drafter drafts greedily.

Inference Serving

NVIDIA NIM

We package the BF16 checkpoint of Nemotron 3.5 Bolt E4B 4GB as a downloadable NVIDIA NIM microservice built on vLLM 0.30.0 ([11]). The container image is published for amd64 and arm64 and ships a single BF16 profile on one GPU with the following defaults:

  • Speculative decoding. The DFlash ([5]) drafter (§ 8) is packaged next to the main weights and enabled by default with a draft length of 7, so no extra download or configuration is needed.

  • Reasoning and tool calling. Reasoning is on by default. The gemma4 reasoning parser returns the reasoning separately from the answer, and the gemma4 tool-call parser enables tool calling.

  • Context and modalities. The context length is 65,536 tokens. Text, image, and audio inputs are accepted.

  • APIs. OpenAI-compatible Chat Completions, Completions, and Responses endpoints, and the Anthropic-compatible Messages endpoint.

Three request-level controls are exposed through the chat template and the serving engine: enable_thinking turns reasoning on or off per request, thinking_token_budget caps the number of reasoning tokens, and conversational_mode requests brief, spoken-style replies for voice assistants. We validated the NIM on DGX Spark (GB10), RTX PRO 6000 and RTX PRO 4500 Blackwell Server Edition, and H100 80GB.

Speculative decoding speedup. We measure the effect of the packaged DFlash drafter by comparing two NIM builds that differ only in whether the model profile includes the drafter; all other serving settings are equal. We use SPEED-Bench ([1]) prompts from 11 categories (16 prompts per category on RTX PRO 6000 and 8 on DGX Spark), streaming Chat Completions requests with greedy decoding and max_tokens of 1,024, at concurrency 1 and 8. Per-user decode speed is 1000/ITL1000/\mathrm{ITL}, where ITL is the mean inter-token latency in milliseconds. Acceptance length (AL) is reported by vLLM’s speculative decoding metrics and includes the token produced by the target model. Table 11 reports means over the 11 categories from a single run per configuration.

ReasoningGPUConc.No drafterDFlashSpeedupNo drafterDFlash
OnRTX PRO 60001124.7341.22.74×2.74 \times5.32.0
OnRTX PRO 60008115.7313.72.71×2.71 \times5.92.2
OnDGX Spark120.171.63.56×3.56 \times31.79.1
OnDGX Spark823.366.32.85×2.85 \times28.710.1
OffRTX PRO 60001126.5302.02.39×2.39 \times3.41.4
OffRTX PRO 60008117.8273.32.32×2.32 \times3.71.6
OffDGX Spark120.162.53.11×3.11 \times20.46.4
OffDGX Spark823.457.22.45×2.45 \times18.67.0

Table 11. DFlash speedup in the Nemotron 3.5 Bolt E4B 4GB NIM on SPEED-Bench (BF16, one GPU, greedy decoding, max_tokens=1,024). Per-user decode speed is in output tokens per second; request latency is the mean end-to-end latency per request.

With reasoning on, DFlash reaches an AL of 3.8 and speeds up per-user decoding by 2.7×2.7\times on RTX PRO 6000 and 3.6×3.6\times on DGX Spark at concurrency 1. The speedup holds at concurrency 8 (2.7×2.7\times and 2.9×2.9\times). With reasoning off, the AL drops to 3.4–3.6 and the speedup to 2.32.3–3.1×3.1\times. Because speculative decoding shortens decoding rather than prefill, the gain in end-to-end request latency follows the decode speedup, while time to first token changes by less than 100 ms in all settings.

Serving overhead. In a matched comparison on DGX Spark against a standalone vLLM 0.30.0 server with the same model and drafter weights, chat template, and scheduler limits, the NIM delivered the same request throughput: per-run NIM-to-vLLM ratios ranged from 0.99 to 1.01 at concurrency 1 and 4.

Quantized Local Inference

We compare Nemotron 3.5 Bolt E4B 4GB with DFlash against Gemma-4-E4B-it with its MTP drafter3 on the coding subset of SPEED-Bench, using AIPerf with a batch size of 1. Both target models use Q4_K_M quantization, and the DFlash drafter uses BF16 precision. As shown in Table 12, Bolt E4B achieves 272.6 tokens/s on RTX 5070, compared with 225.9 tokens/s for Gemma with MTP, corresponding to 1.21×1.21 \times the throughput. On RTX Spark N1X, Bolt E4B achieves 124.4 tokens/s vs. 118.4 tokens/s, corresponding to 1.05×1.05 \times the throughput.

Model and decoderRTX 5070 (tokens/s)RTX Spark N1X (tokens/s)
Gemma-4-E4B-it + MTP225.9118.4
Nemotron 3.5 Bolt E4B 4GB + DFlash272.6124.4
Relative throughput1.21×1.21 \times1.05×1.05 \times

Table 12. Generation throughput on the coding subset of SPEED-Bench, measured using AIPerf with a batch size of 1. Both target models use Q4_K_M quantization; Bolt uses a BF16 DFlash drafter. Relative throughput is computed separately for each hardware platform against Gemma-4-E4B-it with MTP.

Conclusion

We presented Nemotron 3.5 Bolt E4B 4GB, an open small language model designed for fast conversational agents. Built on Gemma-4-E4B-it, we apply reinforcement learning to target multi-turn tool use, instruction following, and output formatting while retaining its conversational strengths. Training in both reasoning and non-reasoning modes, together with verbosity control, supports applications that need concise responses and reliable actions under tight latency constraints.

The release combines these training choices with practical deployment options. With DFlash, Bolt E4B achieves 2.74×2.74 \times the per-user decode speed of inference without speculative decoding in BF16 NIM serving on RTX PRO 6000 with reasoning enabled at concurrency 1. On the coding subset of SPEED-Bench, Q4_K_M Bolt E4B with DFlash achieves 1.21×1.21 \times the generation throughput of Q4_K_M Gemma-4-E4B-it with MTP on RTX 5070 at batch size 1.

Contributors

We thank the following people for their invaluable contributions to Nemotron 3.5 Bolt E4B 4GB.

Adi Margolin, Aditya Malte, Aishwarya Chhabra, Alex Hodson, Ameya Sunil Mahabaleshwarkar, Ambrish Dantrey, Amy Shen, Andrii Skliar, Annamalai Chockalingam, Anusha Pant, Ashwath Aithal, Ashwin Kumar, Besmira Nushi, Bhavesh Pawar, Bryan Catanzaro, Bryan Yu, Chantal Hwang, Chenhan Yu, Chris Alexiuk, Conner Currie, Cynthia Gao, David Mosallanezhad, Dina Yared, Ehsan Hosseini Asl, Eileen Long, Erick Galinkin, Felix Friedmann, Giedrius Burachas, Gorkem Batmaz, Grzegorz Chlebus, Guillermo Siman, Haifeng Qian, Hao Guo, Hayley Ross, Hrishith Thadicherla, Hui Wang, Huiying Li, Ike Nnoli, Irina Filatova, Iris Liu, Ji-Eun Lee, Jian Leong, Jian Zhang, Jiaqi Zeng, Joey Conway, Jonathan Cohen, Jose Vergara, Julien Veron Vialard, Kari Briski, Katherine Luna, Kedar Pandurang Potdar, Kevin Ning, Khalil Ben Khaled, Krzysztof Pawelec, Laura Dykes, Leili Tavabi, Lily Lee, Linda Liu, Liwei Ding, Maka Dong, Manoj Kilaru, Maria Bagdasarova, Matthew Lee, Meredith Price, Micah Schaffer, Michael Boone, Michael Fukuyama, Michael Hasper, Nancy Agarwal, Negar Habibi, Oleksii Kuchaiev, Oluwatobi Olabiyi, Pablo Ribalta, Payal Motwani, Pinky Xu, Prasoon Varshney, Pritam Biswas, Puhui Meng, Rameshwar Shivbhakta, Ran Zilberstein, Reka Kovacs, Richard Mazzarese, Rishabh Garg, Rita Fernandes Neves, Rochelle Woods, Ruby Ye, Ruvaid Virk, Ryan Hill, Saeid Motiiаn, Seph Mard, Shai Ben Nun, Shaona Ghosh, Sharad Yadav, Sudeep Sabnis, Suhas Hariharapura Sheshadri, Terry Kong, Thanh Do, Thinh Le, Udi Karpas, Viet Anh Trinh, Vinay Raman, Vinita Sharma, Warren Drones, Wenwen Gao, Whei Hsueh, Will Efstratis, Yoshi Suhara, Yunsheng Liu, Zhenming Wang, Zijia Chen, Zuhair Ahmed.

References

  1. [1]Talor Abramovich, Maor Ashkenazi, Izzy Putterman, Benjamin Chislett, Tiyasa Mitra, Bita Darvish Rouhani, Ran Zilberstein, and Yonatan Geifman. SPEED-bench: A unified and diverse benchmark for speculative decoding. In Forty-third International Conference on Machine Learning, 2026. URL https://openreview.net/forum?id=R12uQlCoQX.arxiv.org/abs/2604.09557
  2. [2]Yushi Bai, Shangqing Tu, Jiajie Zhang, Hao Peng, Xiaozhi Wang, Xin Lv, Shulin Cao, Jiazhen Xu, Lei Hou, Yuxiao Dong, Jie Tang, and Juanzi Li. LongBench v2: Towards Deeper Understanding and Reasoning on Realistic Long-context Multitasks, 2025. URL https://arxiv.org/abs/2412.15204.
  3. [3]Victor Barres, Honghua Dong, Soham Ray, Xujie Si, and Karthik Narasimhan. τ²-Bench: Evaluating Conversational Agents in a Dual-Control Environment, 2025. URL https://arxiv.org/abs/2506.07982.
  4. [4]Charlie Chen, Sebastian Borgeaud, Geoffrey Irving, Jean-Baptiste Lespiau, Laurent Sifre, and John Jumper. Accelerating large language model decoding with speculative sampling, 2023. URL https://arxiv.org/abs/2302.01318.
  5. [5]Jian Chen, Yesheng Liang, and Zhijian Liu. DFlash: Block diffusion for flash speculative decoding. In Forty-third International Conference on Machine Learning, 2026. URL https://openreview.net/forum?id=Oz335dV48X.arxiv.org/abs/2602.06036
  6. [6]Yiming Chen, Xianghu Yue, Chen Zhang, Xiaoxue Gao, Robby T. Tan, and Haizhou Li. Voicebench: Benchmarking llm-based voice assistants, 2024. URL https://arxiv.org/abs/2410.17196.
  7. [7]Elias Frantar, Saleh Ashkboos, Torsten Hoefler, and Dan Alistarh. GPTQ: Accurate post-training quantization for generative pre-trained transformers. In International Conference on Learning Representations, 2023.arxiv.org/abs/2210.17323
  8. [9]Advait Gosai, Tyler Vuong, Utkarsh Tyagi, Steven Li, Wenjia You, Miheer Bavare, Arda Uçar, Zhongwang Fang, Brian Jang, Bing Liu, and Yunzhong He. Audio multichallenge: A multi-turn evaluation of spoken dialogue systems on natural human interaction, 2025. URL https://arxiv.org/abs/2512.14865.
  9. [10]Cheng-Ping Hsieh, Simeng Sun, Samuel Kriman, Shantanu Acharya, Dima Rekesh, Fei Jia, Yang Zhang, and Boris Ginsburg. RULER: What’s the Real Context Size of Your Long-Context Language Models?, 2024. URL https://arxiv.org/abs/2404.06654.
  10. [11]Woosuk Kwon, Zhuohan Li, Siyuan Zhuang, Ying Sheng, Lianmin Zheng, Cody Hao Yu, Joseph E. Gonzalez, Hao Zhang, and Ion Stoica. Efficient memory management for large language model serving with pagedattention. In Proceedings of the ACM SIGOPS 29th Symposium on Operating Systems Principles, 2023.arxiv.org/abs/2309.06180
  11. [12]Yaniv Leviathan, Matan Kalman, and Yossi Matias. Fast inference from transformers via speculative decoding. In Proceedings of the 40th International Conference on Machine Learning, 2023.arxiv.org/abs/2211.17192
  12. [13]Jiarui Lu, Thomas Holleis, Yizhe Zhang, Bernhard Aumayer, Feng Nan, Felix Bai, Shuang Ma, Shen Ma, Mengyu Li, Guoli Yin, Zirui Wang, and Ruoming Pang. ToolSandbox: A Stateful, Conversational, Interactive Evaluation Benchmark for LLM Tool Use Capabilities, 2025a. URL https://arxiv.org/abs/2408.04682.
  13. [14]Junru Lu, Jiahzeng Li, Guodong Shen, Lin Gui, Siyu An, Yulan He, Di Yin, and Xing Sun. RoleMRC: A Fine-Grained Composite Benchmark for Role-Playing and Instruction-Following, 2025b. URL https://arxiv.org/abs/2502.11387.
  14. [15]Huanzhi Mao, Fanjia Yan, Charlie Cheng-Jie Ji, Jason Huang, Vishnu Suresh, Yixin Huang, Xiaowen Yu, Joseph E. Gonzalez, and Shishir G. Patil. BFCL V3: Multi-Turn and Multi-Step Function Calling. https://gorilla.cs.berkeley.edu/blogs/13_bfcl_v3_multi_turn.html, 2024. Version-specific benchmark announcement; released September 19, 2024.
  15. [16]Cheng Niu, Yuanhao Wu, Juno Zhu, Siliang Xu, Kashun Shum, Randy Zhong, Juntong Song, and Tong Zhang. RAGTruth: A Hallucination Corpus for Developing Trustworthy Retrieval-Augmented Language Models, 2024. URL https://arxiv.org/abs/2401.00396.
  16. [17]NVIDIA. Nemo gym: An open source library for scaling reinforcement learning environments for llm. https://github.com/NVIDIA-NeMo/Gym, 2025a. GitHub repository.
  17. [18]NVIDIA. Nemo rl: A scalable and efficient post-training library. https://github.com/NVIDIA-NeMo/RL, 2025b. GitHub repository.
  18. [19]NVIDIA. Cosmos 3: Omnimodal World Models for Physical AI. arXiv preprint arXiv:2606.02800, 2026. URL https://research.nvidia.com/labs/cosmos-lab/cosmos3/technical-report.pdf.
  19. [21]Samuel J. Paech. EQ-Bench: An Emotional Intelligence Benchmark for Large Language Models, 2024. URL https://arxiv.org/abs/2312.06281.
  20. [22]Samuel J. Paech. EQ-Bench 3: Emotional Intelligence Benchmark. https://github.com/EQ-bench/eqbench3, 2025. Benchmark repository; rubric-scoring evaluation.
  21. [23]Shishir G. Patil, Huanzhi Mao, Fanjia Yan, Charlie Cheng-Jie Ji, Vishnu Suresh, Ion Stoica, and Joseph E. Gonzalez. The Berkeley Function Calling Leaderboard (BFCL): From Tool Use to Agentic Evaluation of Large Language Models. In Proceedings of the 42nd International Conference on Machine Learning, volume 267 of Proceedings of Machine Learning Research, pp. 48371–48392. PMLR, 2025. URL https://proceedings.mlr.press/v267/patil25a.html.
  22. [24]Valentina Pyatkin, Saumya Malik, Victoria Graf, Hamish Ivison, Shengyi Huang, Pradeep Dasigi, Nathan Lambert, and Hannaneh Hajishirzi. Generalizing Verifiable Instruction Following, 2025. URL https://arxiv.org/abs/2507.02833.
  23. [25]Yunjia Qi, Hao Peng, Xiaozhi Wang, Amy Xin, Youfeng Liu, Bin Xu, Lei Hou, and Juanzi Li. AGENTIF: Benchmarking Instruction Following of Large Language Models in Agentic Scenarios, 2025. URL https://arxiv.org/abs/2505.16944.
  24. [26]Yanzhao Qin, Tao Zhang, Tao Zhang, Yanjun Shen, Wenjing Luo, Haoze Sun, Yan Zhang, Yujing Qiao, Weipeng Chen, Zenan Zhou, Wentao Zhang, and Bin Cui. SysBench: Can Large Language Models Follow System Messages?, 2024. URL https://arxiv.org/abs/2408.10943.
  25. [27]Hayley Ross, Ameya Sunil Mahabaleshwarkar, and Yoshi Suhara. When2Call: When (not) to Call Tools, 2025. URL https://arxiv.org/abs/2504.18851.

Paper details

vlnc:W_zp6pk95e
Version
v1 · latest
Published
Fresh — added to the Hub within the last 30 days
Added
License
CC BY 4.0
Usage

views · 0

downloads · 0

A view counts once per browser per day, a download each time the PDF is requested.

  • Computer Science

Not recorded for this paper

View statement
Looking for related papers…
Contents